typesafe-ai/system-one-adapter-python on GitHub
- Stars
- 287
- Forks
- 43
- Language
- Python
- License
- MIT
- Last push
- 22 Sep 2026
Read from GitHub on . Counts change daily.
TypeSafe publishes this adapter so developers can compare Jev against an LLM on cost, speed and answers, using the exact same questions. The adapter mirrors the SDK’s system_one evaluation call, but instead of talking to Jev, it routes the same typed choice, score, and noul questions to OpenAI, Anthropic, Gemini, or a custom OpenAI-compatible endpoint, whichever provider is configured.
It stands in for a real client, so it handles the details each provider needs to return structured output reliably: OpenAI’s Responses API with strict JSON Schema, with custom OpenAI-compatible endpoints defaulting to Chat Completions; Gemini’s Interactions API with a JSON Schema response format; corrective retries that resend the conversation history when a provider’s structured output comes back malformed. Anthropic responses can hit an output-token limit on longer evaluations, and the adapter raises a clear error asking the caller to raise max_tokens or ask fewer questions rather than silently truncating. The provider SDKs themselves are optional extras, so installing only pulls in the ones actually used.
How it uses Jev
No direct call turned up in the code, so this is where it loads the SDK.
import httpx2
from pydantic import BaseModel, ValidationError
from pydantic_core import to_json
from typesafe_sdk import (
Answer,
ChoiceAnswer,
JSONValue,
Noul,
NoulAnswer,
Questions,
RetryPolicy,
Score,
ScoreAnswer,
TypeSafeAPIResponseValidationError,
TypeSafeError,
