Skip to content
System One

jev-rerank-bench

Benchmark comparing Jev against Cohere Rerank 4, ZeroEntropy zerank-2 and a chat-model baseline across 14 datasets, publishing every raw API response and bootstrap ranges on each gap.

Jev as a reranker. Equal-dataset nDCG@10: Jev rubric 0.692, Cohere Pro 0.691, ZeroEntropy zerank-2 0.682.
Image from github.com

anessbelbati/jev-rerank-bench on GitHub

Stars
7
Forks
0
Language
Python
License
MIT
Last push
23 Sep 2026

Read from GitHub on . Counts change daily.

How it uses Jev

# JEV_URL / JEV_MODEL let the same request builders run against a local server that speaks the same protocol
# (used for the open-weight Open-Jev adapters); JEV_GPU_RATE (USD per hour) then bills GPU seconds instead of tokens.
URL = os.environ.get("JEV_URL", "https://api.typesafe.ai/v1/systemone")
MODEL = os.environ.get("JEV_MODEL", "jev-latest")
RUBRIC = ["The passage is off-topic for the query.",
          "The passage is on a related topic but does not supply what the query asks for.",
          "The passage partly supplies the information needed to answer or verify the query.",
          "The passage fully supplies the information needed to answer or verify the query."]


def _headers() -> dict:
    key = os.environ.get("JEV_API_KEY", "") if os.environ.get("JEV_URL") else env("JEV_API_KEY")
    return {"Authorization": f"Bearer {key}", "Content-Type": "application/json", "Accept": "application/json"}

View in rerankers/jev.py

Open on GitHub

PrimitivesNoul

More like this

Use cases this is tagged with