Goal
Decide which handler a support ticket goes to, and give that handler everything it needs, from one request to Jev. Two of the four questions only matter for one category each, so they are speculative: asked before anyone knows whether the answer will be used. TypeSafe’s own wording is that there is no speed cost for additional questions, and the reason to pay it upfront is the alternative. Four separate calls send the ticket four times and wait four times. This is the shape behind typed tool dispatch.
State shape
One state per request, and every question sees all of it. Send the ticket as an object so the subject and the body stay distinguishable, rather than gluing them into one string where the model has to guess which part is the complaint.
{
"subject": "Charged twice and now I can't log in",
"body": "Order #98423 was billed twice last Thursday. Since the update I also get a 500 error every time I sign in from Chrome on Windows."
}Questions
Four questions, three types, one call.
category is a choice: it picks one label from a list you define and returns that label plus a probability for every option and a confidence value from 0 to 1. Four options here, and the confidence value is what decides whether the router acts at all.
refund_requested and has_reproducible_steps are nouls, yes/no questions that come back as a single probability from 0 to 1 rather than a label. These are the speculative pair. The refund question only affects billing tickets and the reproduction question only affects bug reports, so one of them is discarded on every ticket. That is the point of the pattern.
severity is a score: an ordered scale you describe in words, with the answer coming back as the probability-weighted mean of the level numbers. Three levels here, numbered 0 to 2, so 1.55 means the mass sits between “degraded” and “blocked” and leans towards blocked. Jev takes from 2 to 10 levels. Severity is read on every path, because a blocked customer matters whatever the ticket turns out to be about.
Code
from typesafe_sdk import Choice, Noul, Score, TypeSafeClient
TICKET = {
"subject": "Charged twice and now I can't log in",
"body": (
"Order #98423 was billed twice last Thursday. Since the update I also "
"get a 500 error every time I sign in from Chrome on Windows."
),
}
QUESTIONS = {
"category": Choice(
instructions="What is this support ticket mainly about?",
criteria={
"bug_report": "Something is broken or producing errors",
"billing": "Charges, invoices, refunds, subscriptions",
"feature_request": "The customer wants new functionality",
"account": "Login, permissions, profile, security",
},
),
"refund_requested": Noul(
instructions="Is the customer explicitly asking for a refund or a credit?"
),
"has_reproducible_steps": Noul(
instructions="Does the customer describe steps that reproduce the problem?"
),
"severity": Score(
instructions="How badly is the customer blocked?",
criteria=[
"Cosmetic, nothing stops working",
"A feature is degraded, a workaround exists",
"Blocked, no workaround exists",
],
),
}
client = TypeSafeClient()
response = client.system_one(state=TICKET, questions=QUESTIONS, model="jev-1.13.0")
category = response.answers["category"]
severity = response.answers["severity"]
print(category.choice, category.confidence)
print(response.answers["refund_requested"].noul)
print(response.answers["has_reproducible_steps"].noul)
print(severity.score, severity.confidence)Decision policy
The call returns four judgments and no actions. Which handler runs, and whether one runs at all, is decided here. The 0.60 floor comes from TypeSafe’s routing pattern; yours depends on what a wrong route costs you.
category = response.answers["category"]
severity = response.answers["severity"]
refund = response.answers["refund_requested"]
repro = response.answers["has_reproducible_steps"]
if category.confidence < 0.60:
route_to_human(ticket_id, reason="category unclear")
elif category.choice == "bug_report":
if severity.score > 1.5 and repro.noul > 0.6:
escalate_to_engineering(ticket_id) # refund answer discarded
else:
add_to_bug_backlog(ticket_id)
elif category.choice == "billing":
route_to_billing(ticket_id, refund_likely=refund.noul > 0.7)
else:
route_to_queue(ticket_id, category.choice) # both nouls discardedSample output
Illustrative, not a recorded run. Choice probabilities sum to 1 with the named label highest, the score is the probability-weighted mean of the level numbers, and the two noul answers carry no confidence field.
{
"model": "jev-1.13.0",
"answers": {
"category": {
"type": "choice",
"choice": "bug_report",
"probabilities": {
"bug_report": 0.71,
"billing": 0.19,
"feature_request": 0.06,
"account": 0.04
},
"confidence": 0.68
},
"refund_requested": { "type": "noul", "noul": 0.31 },
"has_reproducible_steps": { "type": "noul", "noul": 0.83 },
"severity": {
"type": "score",
"score": 1.55,
"legend": {
"0": "Cosmetic, nothing stops working",
"1": "A feature is degraded, a workaround exists",
"2": "Blocked, no workaround exists"
},
"probabilities": { "0": 0.05, "1": 0.35, "2": 0.6 },
"confidence": 0.74
}
},
"usage": { "input_tokens": 296, "output_tokens": 61 }
}Pitfalls
- The saving is in the state, not the questions. TypeSafe’s parallel-questions cookbook ran 13 questions over a 54,000 character article and measured the batched call at 12.2 times cheaper than 13 single-question calls, because those 13 calls each re-send the article. On a two-line ticket the ratio is much smaller. Measure yours before claiming a number.
- Batching does not change any answer. Each question is scored on its own against the state, and the cookbook’s five repeats showed the same means and the same run-to-run spread either way. So a speculative question cannot contaminate a real one, but neither can you use one question to set up another.
- A speculative answer read on the wrong path is a real bug and an easy one to write. The ticket above returns 0.31 for a refund request while being routed as a bug report; nothing in the response marks that number as irrelevant. Only the branch structure of your code does.
- Do not pick a question count by guessing a limit. TypeSafe does not document a maximum number of questions per request. What is documented is the budget they share: 64k tokens per request in total, with state plus the longest single question capped at 32k.
- A ticket that carries two real complaints, as this one does, will split the category distribution. Confidence at 0.68 with billing sitting at 0.19 is the model reporting a mixed ticket accurately. The fix is a routing rule that can open two handlers, rather than a higher floor. The single-question triage recipe covers what that distribution looks like on its own.
FAQ
What makes a question speculative here?
A speculative question is one you send before knowing whether its answer matters. The refund question is meaningless on a bug report, the reproduction question meaningless on a billing ticket, yet both go out with the first request. Asking costs a few hundred tokens of question text. Not asking costs a second round trip.
Should I gate on the choice confidence or on the winning probability?
Use confidence for the routing gate. It summarises the whole distribution, so a two-way tie and a clean win are separated even when the top probability looks similar. The winning probability is the right number when you are multiplying values along a chain of decisions, which is what the taxonomy walk does level by level.
Does adding a fifth question slow the call down?
TypeSafe’s documented position is that additional questions carry no speed cost, since every question in a request is evaluated in parallel against the same state. The end-to-end latency figures they publish, 70ms to 500ms, are the vendor’s own measurements rather than an independent benchmark. Extra questions do add input tokens, so they are not free in money.