Skip to content
System One

typesafe-ai-benchmark

Runs Qwen 3.8 27B on Cerebras and Jev side by side across seven synthetic workloads, plus a local Needle 3 evaluation on the same contracts. Records validated outputs, mistakes, latency, token usage and estimated cost.

Qwen on Cerebras vs. Jev — side-by-side benchmark demo
Image from github.com

iammrduncan/typesafe-ai-benchmark on GitHub

Stars
38
Forks
5
Language
TypeScript
License
MIT
Last push
19 Sep 2026

Read from GitHub on . Counts change daily.

How it uses Jev

export async function requestJev(plan: JevPlan, apiKey: string, signal: AbortSignal, endpoint = 'https://api.typesafe.ai/v1/systemone') {
  try {
    signal.throwIfAborted();
    const response = await fetch(endpoint, { method: 'POST', redirect: 'error', signal,
      headers: { authorization: `Bearer ${apiKey}`, 'content-type': 'application/json' }, body: JSON.stringify(plan.payload) });
    if (!response.ok) {
      await response.body?.cancel();
      if (response.status === 401) throw new Fault('provider_authentication_failed', 502);
      if (response.status === 429) throw new Fault('rate_limited', 429);
      if (response.status === 529 || response.status === 503) throw new Fault('overloaded', 529);
      throw new Fault('provider_unavailable', 502);
    }
    try { return decodeJev(plan, parseJson(await boundedText(response, signal))); }
    catch { signal.throwIfAborted(); throw new Fault('invalid_provider_output', 502); }
  } catch (error) {

View in packages/demos/lib/jev.ts

Open on GitHub

PrimitivesChoiceNoul

More like this

Use cases this is tagged with