Skip to content
System One

zero-shot-ie-bench

Compares GLiNER 2.5, GLiFormer, GLiClass, Laya, Von and Jev on zero-shot information extraction and classification in one web UI.

Classification accuracy, mixed pool
Image from github.com

umstek/zero-shot-ie-bench on GitHub

Stars
1
Forks
0
Language
Python
License
MIT
Last push
23 Sep 2026

Read from GitHub on . Counts change daily.

How it uses Jev

Wire format follows @typesafe-ai/sdk 0.6.x: POST {model, state, questions}
to https://api.typesafe.ai/v1/systemone with a Bearer key. Questions:

    {"type": "choice", "instructions": str|obj, "criteria": {label: desc|None}}
    {"type": "score",  "instructions": ...,      "criteria": [desc, ...]}
    {"type": "noul",   "instructions": ...,      "criteria": optional}

Answers come back as {"answers": {name: {...}}, "usage": {...}}.

API key: set the TYPESAFE_API_KEY environment variable, or put
`TYPESAFE_API_KEY=...` in a `.env` file next to this module (gitignored).
Jev is a paid API; each `ask()` call is a request.

No third-party deps on purpose: urllib only.
"""

View in jev_client.py

Open on GitHub

PrimitivesChoice

More like this

Use cases this is tagged with