Skip to content
System One

FirasSX914/Janus

Measures on your own labelled data where a small model's confidence stops paying off, then routes each decision to Jev or a larger model at the threshold it found rather than a default. On 500 Banking77 examples Jev was 77.8% accurate but 90.7% confident, and threshold 0.67 reached 80.2% accuracy for $0.1033.

FirasSX914/Janus on GitHub

Stars
2
Forks
0
Language
Python
License
MIT
Last push
18 Sep 2026

Read from GitHub on . Counts change daily.

How it uses Jev

for n, example in enumerate(todo, 1):
    run_date = datetime.now(timezone.utc).isoformat(timespec="seconds")
    started = time.perf_counter()
    response = client.system_one(
        state=example["text"], model=MODEL, questions=question
    )
    latency_ms = (time.perf_counter() - started) * 1000

    # choices[...] et jamais answers[...] : answers melange les types et
    # un NoulAnswer n'a pas de champ confidence.
    answer = response.choices[QUESTION_ID]

View in experiments/run_jev.py

Open on GitHub

Use casesIntent and model routingConfidence-gated actions

More like this

Use cases this is tagged with