Skip to content
System One

Walk a taxonomy with one choice per level

A taxonomy with hundreds of leaves does not fit in one question. This recipe walks the tree instead: one choice question per level, each asking only about the direct children of the node you are standing on. Multiply the winning probabilities, normalise for depth, and stop early when an edge looks weak.

Goal

Put a product description in the right leaf of a category tree that is too big to ask about in one question. The walk starts at the root, asks Jev which direct child fits, steps into the winner, and repeats until it runs out of children. Each step also hands back the probability it assigned, which is what lets you score the whole path afterwards and stop when the tree stops being sure. This is the cascade shape behind hierarchical classification.

State shape

State is the content being judged, and it stays byte-identical at every level. Only the question changes as you descend. Adding the parent node you are standing on gives the question something to anchor to, which matters once you are five levels deep in a tree whose sibling labels are near-synonyms.

{
  "product": "Three-layer waterproof shell jacket, taped seams, pit zips, helmet-compatible hood, 420g in a men's medium.",
  "parent": "Apparel"
}

Questions

One choice per level, which is a question that picks one option from a list you define and returns the winning label, a probability for every option that sums to 1, and a confidence value from 0 to 1. The options at each level are the direct children of the current node, never the whole taxonomy. Jev accepts up to 255 options in a single choice, so a wide level is fine, but a tree of 10,000 leaves still has to be walked.

Give each option an opaque key (c0, c1) mapped to the real label, and map it back yourself after the answer comes in. Category names in real taxonomies contain punctuation, numbers and duplicated words across branches, and keeping them out of the option keys means the label text lives only in the criteria descriptions where it belongs.

The number the walk uses is the probability on the winning option, not the confidence value. Confidence summarises the whole distribution; the edge probability is the specific quantity you multiply along a path.

Code

import math

from typesafe_sdk import Choice, TypeSafeClient

TAXONOMY = {
    "Apparel": {
        "Clothing": {"Outerwear": {}, "Shirts": {}},
        "Shoes": {"Boots": {}, "Sandals": {}},
    },
    "Sporting Goods": {
        "Outdoor Recreation": {"Climbing": {}, "Cycling": {}},
        "Athletics": {"Running": {}, "Swimming": {}},
    },
}
PRODUCT = (
    "Three-layer waterproof shell jacket, taped seams, pit zips, "
    "helmet-compatible hood, 420g in a men's medium."
)
FLOOR = 0.55

client = TypeSafeClient()
node, path, edges = TAXONOMY, [], []

while node:
    keys = {f"c{index}": label for index, label in enumerate(node)}
    response = client.system_one(
        state={"product": PRODUCT, "parent": path[-1] if path else "root"},
        questions={
            "child": Choice(
                instructions="Which direct child category best matches this product?",
                criteria=keys,
            )
        },
        model="jev-1.13.0",
    )
    answer = response.answers["child"]
    edge = answer.probabilities[answer.choice]
    if edge < FLOOR:
        break
    edges.append(edge)
    path.append(keys[answer.choice])
    node = node[path[-1]]

score = math.prod(edges) ** (1 / len(edges)) if edges else 0.0
print(" > ".join(path) or "unclassified", round(score, 3))

Decision policy

Two gates, one per edge and one for the finished path. The path score is the documented length-normalised form, product(edge_probabilities) ** (1 / decisions), which is a geometric mean: it compares a shallow leaf and a deep one fairly instead of punishing depth.

FLOOR = 0.55       # per-edge, checked during the walk above
PATH_FLOOR = 0.70  # geometric mean across the whole path
LEAF_DEPTH = 3     # this taxonomy's leaves sit three levels down

score = math.prod(edges) ** (1 / len(edges)) if edges else 0.0

if len(path) == LEAF_DEPTH and score >= PATH_FLOOR:
    assign_category(product_id, path)
elif path:
    # the walk stopped early: file it at the deepest node that cleared the floor
    assign_category(product_id, path)
    queue_for_review(product_id, path, score)
else:
    queue_for_review(product_id, [], score)

Sample output

Illustrative, not a recorded run. This is the root-level answer for the shell jacket, where c0 is Apparel and c1 is Sporting Goods. A jacket belongs in both branches of this taxonomy, and the split distribution shows it.

{
  "model": "jev-1.13.0",
  "answers": {
    "child": {
      "type": "choice",
      "choice": "c0",
      "probabilities": { "c0": 0.62, "c1": 0.38 },
      "confidence": 0.58
    }
  },
  "usage": { "input_tokens": 131, "output_tokens": 14 }
}

Pitfalls

  • A greedy walk cannot undo an early mistake. The sample above passes the 0.55 floor at 0.62, commits to Apparel, and never sees what the Sporting Goods branch would have offered three levels down. The cookbook’s answer is beam search: keep the best K paths, expand all of them at each level, and rank by the same geometric mean. On the cookbook’s four taxonomies, beam search with K equal to 3 matched all four expected leaves while the greedy walk matched two.
  • The per-level calls are sequential, because each question depends on where the last one landed. That is the one place this pattern gives up Jev’s parallelism, and it is why beam search pays for itself: the K frontier questions at a given level do run at the same time.
  • Edge probabilities from different levels are not interchangeable. A 0.62 out of two options and a 0.62 out of forty mean very different things, so a single global floor treats wide and narrow levels alike. Per-level floors, or a floor scaled by option count, track the tree better.
  • Sibling labels that overlap in meaning produce split distributions that look like uncertainty about the product when they are really uncertainty about your taxonomy. Contradictory or overlapping criteria are on Jev’s documented weakness list. Fixing the tree beats tuning the threshold.
  • Do not read the path score as a calibrated probability of correctness. It is a geometric mean of per-level numbers, useful for ranking paths against each other, not for reporting accuracy to anyone.

FAQ

Why multiply the edge probabilities instead of averaging them?

The product is the probability of the whole path under the model’s own per-level distributions, which is what you want when ranking one route against another. Raising it to the power of one over the decision count turns it into a geometric mean, so a deep leaf is not ranked below a shallow one by default.

How many levels can this walk handle before it gets expensive?

Cost scales with depth, not with the size of the taxonomy, because every call re-sends the same product text and only the option list changes. A twelve-level tree costs twelve times the state. Below about ten levels the plain product is fine; deeper than that, sum logs instead of multiplying to avoid losing precision.

Can I ask about two levels in a single call?

You can send both together and they run in parallel, but the second has to list options from every branch rather than the one you land in. That works where the grandchild set stays small. Fan-out pays off when questions are independent, as in one call carrying four questions and support triage.

Recipes using the same primitives