Intent and model routing with System One models

The problem

Most requests arriving at an AI product don’t need the biggest model available. Someone asking where their order is needs a database lookup. Someone asking a policy question needs a specialist with the right documents loaded. Someone describing a situation nobody wrote a rule for needs a person. Sending all three to the same frontier model burns money on the easy ones and still gets the hard one wrong.

The usual fix is a router prompt: ask a cheap chat model which bucket a request falls into. That works until the router itself becomes the thing you’re debugging, because it answers in text, occasionally invents a bucket name, and gives you no signal when it was guessing. Jev replaces that with a typed answer. TypeSafe’s own use case map lists model routing as a job for Jev: build a custom router that chooses which model receives each prompt, set thresholds for your workflow, classify intent and domain, estimate difficulty and risk.

What the state looks like

State is what you want judged. Send the user’s message plus the small amount of surrounding context the routing decision depends on, and nothing else. A router that reads the whole conversation history is reading mostly noise, and Jev’s accuracy falls as unrelated material grows around the thing being judged.

{
  "message": "I ordered the wrong size and the return window closed two days ago, is there anything you can do?",
  "channel": "chat",
  "authenticated": true,
  "turn": 1
}

The questions you ask

Two questions, one call. One says what the request is, the other says how hard it looks.

from typesafe_sdk import Choice, Score, TypeSafeClient

client = TypeSafeClient()

questions = {
    "intent": Choice(
        instructions="The primary intent of this customer message",
        criteria={
            "order_status": "Asking about an existing order",
            "product_question": "Asking about a product before buying",
            "return_exchange": "Wants to return or exchange something",
            "complaint": "Unhappy with experience, wants resolution",
        },
    ),
    "complexity": Score(
        instructions="How complex is this request to resolve",
        criteria=[
            "Simple lookup or standard procedure",
            "Requires some judgment or multi-step process",
            "Unusual situation, edge case, or escalation needed",
        ],
    ),
}

response = client.system_one(state=message, questions=questions, model="jev-latest")

Intent is a Choice, a pick-one question over labels you define. You get back the winning label, a probability for every label that sums to 1, and a confidence value from 0 to 1 describing how concentrated that spread is. Because the labels are yours, the router can never return a bucket your code doesn’t handle. Up to 255 options fit in one Choice, which is more than most routing tables need.

Complexity is a Score, an ordered scale where you write one sentence per level. The returned number is the probability-weighted mean of the level positions counting from 0, so a request that sits between “standard procedure” and “needs judgment” comes back near 0.5 instead of being forced onto a rung. That in-between value is the useful part: it’s what separates a request a small model can finish from one that should go up a tier. The primitives guide has more on how the two shapes differ.

Decision policy

Jev hands back two readings. The router below is what actually spends money on a model call or a person’s time.

intent = response.answers["intent"]
complexity = response.answers["complexity"]

if intent.confidence < 0.5:
    return route_to_human_agent(ticket_id)

if intent.choice == "order_status":
    return lookup_order(ticket_id)            # no model call at all

if intent.choice == "complaint" and (complexity.score > 1 or complexity.confidence < 0.5):
    return route_to_human_agent(ticket_id)

if complexity.score > 1.4:
    return handle_with_llm(ticket_id, FRONTIER_MODEL)

return handle_with_llm(ticket_id, SMALL_MODEL)

The 0.5 floor is the pattern TypeSafe documents: below it, the model is telling you the request didn’t land cleanly in any bucket, and guessing costs more than asking a person. Above the floor, each branch sets its own bar according to what a wrong answer does. Order lookups are cheap to get wrong and get no model at all. Complaints get a second gate, because a low confidence complexity reading on an angry customer is exactly the case where automation goes badly.

The thresholds are yours, and TypeSafe says plainly that the right values depend on your domain and your traffic, so measure before you trust the numbers above. And the router only decides where a request goes: the downstream model still needs its own output checks, which is the job of LLM guardrails.

Community work exists here already. jev-router wires this shape into an LLM gateway.

When not to use this

Don’t ask the router to do arithmetic on its way to a decision. “Is this customer inside their contract term” is a date comparison, and Jev reads dates as text rather than as ordered quantities, so extract the parts and compare them in code. Same for anything that depends on counting prior tickets.

Indirection is the other trap. A question like “would the specialist model handle this better than the general one” is a question about a property of a property, and multi-hop reasoning costs accuracy. Ask directly instead: how complex is the request, what kind of request is it. Then let code map those two readings onto model names. And if the message itself might contain instructions aimed at your system, remember that Jev has no built-in defence against injected text, so screen the input before the router acts on it.

FAQ

Why not let the frontier model route itself?

It can, and it costs a full generation call plus its latency on every turn before any real work starts. A typed router is a small separate decision made before the expensive resource is touched. TypeSafe reports Jev latency of 70ms to 500ms, which is the company’s own measurement, so verify the saving on your own traffic.

How do I pick the complexity levels?

Write them as sentences describing states you can recognise in real traffic, then check them against tickets you already know the answer for. Two levels is the minimum and ten the maximum. Fewer levels give you cleaner separation; more give you finer gradations but blur the boundaries. Start with three or four and adjust once you can see the distribution.

Can one call route and triage at the same time?

Yes, and it usually should. Extra questions in the same request cost nothing in latency according to TypeSafe’s fan-out pattern, so a router can also ask the questions used for support inbox triage and hand the answers downstream. Your code ignores whichever answers turn out not to apply to the branch it took.

What if two intents are genuinely both present?

The Choice spreads probability between them and confidence drops, which your floor catches. If compound requests are common rather than rare, a Choice is the wrong shape: ask one Noul per intent instead, since each Noul is an absolute judgment and several can be high at once. TypeSafe documents that the two shapes answer different questions.

Examples in the wild