Skip to content
System One

marcosmartinez/jev-acento

An independent, reproducible audit of how well Jev performs in Spanish, with a run of 19,200 calls across four datasets showing measurably lower accuracy when the same instructions are given in Spanish instead of English, plus a CLI to repeat the comparison on your own labelled data.

Calibration by arm
Image from github.com

marcosmartinez/jev-acento on GitHub

Stars
0
Forks
1
Language
Python
License
MIT
Last push
21 Sep 2026

Read from GitHub on . Counts change daily.

How it uses Jev

"typesafe": Provider(
    name="typesafe",
    base_url="https://api.typesafe.ai",
    api_key_env="TYPESAFE_API_KEY",
    # Always the versioned id, never the `jev-latest` alias: an alias can be repointed
    # between two passes of the same run. Bump this deliberately.
    model="jev-1.13.0",
    # Documented as 1200 rpm / 250k tokens per second, and explicitly subject to change
    # without notice. At roughly 800 input tokens per call, 1200 rpm is about 16k tokens
    # per second, so requests-per-minute is the binding limit, not tokens.
    default_rpm=1200,
    reports_version=True,
),

View in src/jev_acento/providers.py

Open on GitHub

More like this

Keep browsing