Skip to content
System One

Score a sales lead on four dimensions and weight them yourself

Asking a model how good a lead is gives you a number nobody can argue with or adjust. This recipe asks four narrow score questions instead, one per dimension, normalises each answer to a 0 to 1 range and combines them with weights kept in your code, so changing what qualifies means editing a constant.

Goal

Rank inbound leads without handing the ranking logic to a model. Jev answers four narrow questions about one lead, each on a scale you wrote out in words, and your code turns those four numbers into a single figure using weights you can change on a Tuesday afternoon. When a rep disagrees with a ranking, you can show them which dimension moved it. That traceability is the reason composite scoring beats one overall quality question.

State shape

Everything known about the lead, as an object, so each question can attach to the part that answers it. The four questions all see the whole object; none of them gets a private slice. Keep it to what a person would read before making the call, because unrelated CRM fields dilute the signal without adding anything.

{
  "company": "Northwind Logistics, 400 staff, freight brokerage",
  "last_email": "I run ops and I sign off on tooling under 50k. Our audit lands in February, so I need something in place before then."
}

Questions

Four score questions, where a score is an ordered scale you describe level by level and the answer comes back as the probability-weighted mean of the level numbers. Four levels each, numbered 0 to 3, which makes 3 the top index and score / 3 the normalised 0 to 1 value the weights multiply. Jev supports 2 to 10 levels.

The dimensions are budget fit, buying authority, urgency and product fit. Each one is written so that a person reading the level descriptions would sort the same lead into the same level. That is the test for a score rubric: if two of your colleagues disagree about which level a lead belongs to, the model will disagree with itself too.

Four questions rather than one “how good is this lead” score, because a single number cannot be audited or reweighted. With four, a lead that scores high on urgency and low on authority is visibly a different case from its mirror image, and moving authority from 0.20 to 0.35 is a one-line change with a predictable effect.

The answer also carries a confidence value from 0 to 1, which summarises how concentrated the distribution across levels was. A lead whose email says nothing about budget produces a flat distribution and a low confidence, and that is worth catching before the composite number hides it.

Code

from typesafe_sdk import Score, TypeSafeClient

LEAD = {
    "company": "Northwind Logistics, 400 staff, freight brokerage",
    "last_email": "I run ops and I sign off on tooling under 50k. Our audit "
    "lands in February, so I need something in place before then.",
}

RUBRICS = {
    "budget_fit": (
        "How well does the stated budget fit a 30k to 80k annual contract?",
        ["No budget mentioned", "Hinted at, no number",
         "A number below the range", "A number inside or above the range"],
    ),
    "authority": (
        "How much buying authority does the writer have?",
        ["Unclear role", "Individual contributor", "Influences the decision",
         "Signs the contract"],
    ),
    "urgency": (
        "How soon does the buyer need this in place?",
        ["No timeline", "Someday", "This year", "A dated deadline this quarter"],
    ),
    "product_fit": (
        "How closely does the need match audit-ready workflow tooling?",
        ["Unrelated", "Adjacent need", "Overlapping need", "Exactly this"],
    ),
}
WEIGHTS = {"budget_fit": 0.30, "authority": 0.20, "urgency": 0.25, "product_fit": 0.25}
TOP_LEVEL = 3  # four levels, indexed 0 to 3

client = TypeSafeClient()
response = client.system_one(
    state=LEAD,
    questions={
        key: Score(instructions=text, criteria=levels)
        for key, (text, levels) in RUBRICS.items()
    },
    model="jev-1.13.0",
)
for key in RUBRICS:
    answer = response.answers[key]
    print(key, answer.score, answer.confidence)
composite = sum(w * (response.answers[k].score / TOP_LEVEL) for k, w in WEIGHTS.items())
print(round(composite, 3))

Decision policy

The composite number does nothing on its own. These four lines of policy do, and the weakest of the four confidence values gets a veto, because a strong composite built on one dimension the model could not read is worth less than it looks.

HOT = 0.75              # straight to a rep
WARM = 0.50             # nurture sequence
CONFIDENCE_FLOOR = 0.5  # TypeSafe's worked example uses this floor; tune it on your data

normalised = {key: response.answers[key].score / TOP_LEVEL for key in WEIGHTS}
composite = sum(WEIGHTS[key] * normalised[key] for key in WEIGHTS)
weakest = min(response.answers[key].confidence for key in WEIGHTS)

if weakest < CONFIDENCE_FLOOR:
    queue_for_sales_review(lead_id, composite, normalised)
elif composite >= HOT:
    assign_to_rep(lead_id, composite)
elif composite >= WARM:
    add_to_nurture(lead_id, composite)
else:
    mark_unqualified(lead_id, composite)

Sample output

Illustrative, not a recorded run. Each score is the probability-weighted mean of the level indices, and the probability keys are level index strings starting at “0”.

{
  "model": "jev-1.13.0",
  "answers": {
    "budget_fit": {
      "type": "score",
      "score": 2.38,
      "legend": {
        "0": "No budget mentioned",
        "1": "Hinted at, no number",
        "2": "A number below the range",
        "3": "A number inside or above the range"
      },
      "probabilities": { "0": 0.04, "1": 0.14, "2": 0.22, "3": 0.6 },
      "confidence": 0.71
    },
    "authority": {
      "type": "score",
      "score": 2.59,
      "legend": {
        "0": "Unclear role",
        "1": "Individual contributor",
        "2": "Influences the decision",
        "3": "Signs the contract"
      },
      "probabilities": { "0": 0.03, "1": 0.07, "2": 0.18, "3": 0.72 },
      "confidence": 0.86
    },
    "urgency": {
      "type": "score",
      "score": 2.69,
      "legend": {
        "0": "No timeline",
        "1": "Someday",
        "2": "This year",
        "3": "A dated deadline this quarter"
      },
      "probabilities": { "0": 0.02, "1": 0.05, "2": 0.15, "3": 0.78 },
      "confidence": 0.84
    },
    "product_fit": {
      "type": "score",
      "score": 2.28,
      "legend": {
        "0": "Unrelated",
        "1": "Adjacent need",
        "2": "Overlapping need",
        "3": "Exactly this"
      },
      "probabilities": { "0": 0.05, "1": 0.12, "2": 0.33, "3": 0.5 },
      "confidence": 0.66
    }
  },
  "usage": { "input_tokens": 412, "output_tokens": 96 }
}

Pitfalls

  • Do not read an exact quantity back out of a score. The budget rubric asks which band the lead falls in, and 2.38 means the distribution sits between “below the range” and “inside it”. It does not mean 2.38 of anything, and TypeSafe’s jaggedness notes say plainly not to interpolate exact numbers from a score.
  • The urgency rubric’s top level says “a dated deadline this quarter”, and Jev reads dates as text rather than as ordered quantities. A lead writing “February” scores well here only because the rubric describes the shape of the answer, not the arithmetic. If you need to know whether a date is inside a window, pull the month out with a choice and compare it in code.
  • Weights that do not sum to 1 still work, they just make the composite unreadable against a 0 to 1 threshold. Keep them summing to 1 and change them relative to each other. The same normalise-then-weight step appears in scoring retrieved passages.
  • All four questions share one state, so a long CRM dump hurts every dimension at once. Accuracy drops as unrelated detail grows, and a lead record padded with page-view history is the common version of that.
  • Weighted averaging hides a zero. A lead with no budget at all can still clear the warm threshold on urgency and fit. Add a hard rule for the dimensions that disqualify outright, rather than trying to express that as a weight.

FAQ

Why divide by 3 instead of by the number of levels?

Levels are numbered by their position in the array, starting at 0, so four levels run from 0 to 3 and the highest index is 3. Dividing by 4 would cap a perfect lead at 0.75 and quietly shrink every weight. The pattern page writes this as dividing by the top level index, which is the level count minus one.

Can I mix these scores with a yes/no question?

Yes, and they travel in the same request at no extra round trip. Keep the two kinds of number apart in the arithmetic, though: a noul probability and a normalised score both live between 0 and 1 but mean different things, so averaging them together produces a figure with no interpretation. The fan-out recipe shows the mixed-type call.

How do I know whether my weights are right?

You cannot get that from the model. Score a few hundred past leads whose outcome you already know, compute the composite under several weightings, and see which ordering matches what closed. Keeping weights in code means that loop costs a re-run of the arithmetic, not a re-run of the calls.

Recipes using the same primitives