Hierarchical classification with System One models

The problem

Product catalogs, patent classifications, medical subject headings and file trees all have the same shape: thousands of leaves hanging off a handful of top-level branches. A choice question takes at most 255 options, so the whole taxonomy in one question is not on the table for anything of real size.

The tree gives you the way out. At any node, the question is only ever “which of these direct children”, and that is a small choice. Walk from the root and every step is a small decision made with the full document in view. TypeSafe’s hierarchical classification cookbook does this over the CPC patent scheme, the Shopify product taxonomy, MeSH biomedical subjects and a source-code repository.

Decomposing it this way buys more than a size workaround. You can count how often each node and edge gets traversed, and unit test a change to one branch without re-running the whole corpus. commit-miner applies the same tree-walking idea to repository history.

What the state looks like

The document stays the same at every level. Only the question changes, which is what makes the walk cheap to reason about: the model is never asked to remember where it has been.

{
  "listing_id": "SKU-44812",
  "title": "Suction cup cat hammock for windows",
  "body": "Holds up to 12kg. Four industrial suction cups fix it to any glass pane so your cat can nap in the sun. Machine washable cover."
}

Keep it tight. Accuracy drops as the state fills with material the decision does not need, so strip boilerplate before you start walking.

The questions you ask

One choice per sibling set, with the option keys mapped so you can reverse them afterwards.

from typesafe_sdk import Choice, RetryPolicy, TypeSafeClient

client = TypeSafeClient(
    model="jev-1.13.0",
    retry=RetryPolicy(max_retries=5, backoff_initial=1.0, backoff_max=20.0),
)

def choose(state: dict, labels: tuple[str, ...]) -> dict[str, float]:
    """Return the probability distribution over one node's direct children."""
    if len(labels) == 1:
        return {labels[0]: 1.0}
    keys = {f"c{i}": label for i, label in enumerate(labels)}
    response = client.system_one(
        state,
        {"child": Choice(
            instructions="Which direct child category best matches this document?",
            criteria=keys,
        )},
    )
    probabilities = response.answers["child"].probabilities
    return {label: probabilities[key] for key, label in keys.items()}

The useful part of that response is not the winning label but probabilities, the full distribution over the options, which sums to 1. That is what turns a walk into a search: it tells you how close the runner-up was.

Short opaque keys like c0 keep long category names out of the request and give you a clean mapping back. The primitives guide covers what else a choice answer carries.

Decision policy

Greedy takes the highest-probability child and throws the rest away, which means one early mistake is unrecoverable. Beam search keeps the best K paths alive and scores each one by the geometric mean of its edge probabilities.

BEAM_WIDTH = 3
EPSILON = 1e-9

def extend(candidate: dict, label: str, probabilities: dict[str, float]) -> dict:
    is_decision = len(probabilities) > 1
    product = candidate["product"] * (max(probabilities[label], EPSILON) if is_decision else 1.0)
    decisions = candidate["decisions"] + is_decision
    return {
        "path": candidate["path"] + (label,),
        "product": product,
        "decisions": decisions,
        "score": product ** (1 / decisions) if decisions else 1.0,
    }

Taking the nth root normalises for depth, so a shallow leaf and a deep one can be compared fairly. Every frontier path runs as its own question in the same request, and since there is no speed cost for additional questions, a beam of three costs about what a beam of one does in wall-clock time.

Then gate on separation: the top path’s score divided by the second path’s. A ratio near 1 means two branches fit the document about equally well, which is a review queue item rather than a classification. A large ratio means the walk found a clear winner. Your code owns the cut line and the review queue, and it should cap depth so a malformed tree cannot loop forever. The cookbook uses a beam width of 3 and a depth cap of 12.

Beam search is worth the complexity when the top of the tree is ambiguous. On TypeSafe’s four published examples, beam search with K=3 matched the expected leaf 4 times out of 4 while greedy matched 2 out of 4, with the recovered cases being the patent and the product taxonomy. That is four examples, not a benchmark, so measure your own tree first. The cascade recipe has the runnable version.

When not to use this

A flat label set under 255 options does not need any of this. One choice question answers it in one request, and every extra level is another place for an early error to send the walk down the wrong branch.

Trees whose branches are told apart by numbers rather than by meaning will struggle. Model years and version numbers are comparisons, not judgments, and Jev handles semantic distinctions far better than numeric ones. Route those levels in code and let the model handle the ones that turn on meaning.

Watch for nodes whose children overlap or contradict each other, since contradictory criteria are a documented failure mode. If two siblings could both be right, the probability splits and every path below inherits the doubt.

Multi-hop branches are a risk too. A node asking “which department owns the team that maintains this file” is several hops of indirection, so resolve it in code and ask about what you resolved.

A long document costs accuracy at every level, not just once, because the same state goes out at each step. Filter first, then walk.

FAQ

How many requests does one document take?

One per level for greedy, or one per level for beam search too, since the frontier paths go out as parallel questions in a single call. A twelve-deep taxonomy is twelve round trips. TypeSafe’s own measurement puts end to end latency at 70ms to 500ms per request, so a deep walk is seconds rather than minutes.

What if a node has more than 255 children?

Split it. Insert an intermediate grouping level in your own tree so no single question exceeds the documented limit, grouping siblings by whatever distinction reads most clearly in words. The same two-stage narrowing appears in structured extraction when a document yields more candidate spans than a choice can hold.

Should I send the child descriptions or just the names?

Send descriptions wherever the names are ambiguous, since criteria are read as an extension of the instruction and a bare label leaves the model guessing what it covers. Names alone are fine near the root, where the branches are obviously different. Deep in a taxonomy, where siblings differ by one qualifier, the description is doing most of the work.

How do I find where my errors come from?

Record the full distribution at every node, not just the chosen child. Then group your known misclassifications by the node where the path first diverged from the correct one. That usually points at one ambiguous sibling pair or one badly worded description, which you can fix and re-test without touching the rest of the tree.

Examples in the wild