Skip to content
System One

RLCD explained: training a model to be calibrated

RLCD stands for Reinforcement Learning for Calibrated Decisions, the training method behind Jev. It rewards probabilities that match real outcomes rather than answers human raters prefer (RLHF) or answers a program can check (RLVR). TypeSafe has published the name and the goal, not the method.

Updated

What RLCD stands for

Reinforcement Learning for Calibrated Decisions. TypeSafe names it in the Jev launch post as the training method behind the model, and describes the target as “calibrated decisions: answers with epistemically honest probabilities on System One tasks.”

That phrase is doing the work. The goal is a model whose stated probabilities can be trusted as probabilities, rather than one that picks the right option more often.

What calibrated means

A model is calibrated when its confidence matches its accuracy. Take every answer it returned at 80% confidence. If roughly 80% of them turned out correct, the model is calibrated at that level. If 60% were correct, it is overconfident. If 95% were, it is underconfident, which sounds harmless and is not: your code will escalate cases it could have handled.

Accuracy and calibration are separate properties. A model can be right 90% of the time and useless for gating if it says 99% every time. A weaker model that honestly reports 70% when it is right 70% of the time is the one you can build a threshold on.

This is why confidence gating only works on top of a calibrated model. TypeSafe’s confidence docs suggest acting automatically at high confidence, confirming at medium, and routing to a human at low, with a worked example using 0.5 as the floor for genuine uncertainty and 0.9 as the bar for destructive operations. Those numbers mean nothing unless the underlying probabilities track reality.

RLHF, RLVR, RLCD

All three are reinforcement learning from a reward signal. What differs is what the reward measures.

RLHF, Reinforcement Learning from Human Feedback, rewards responses that human raters prefer. It is the method that made chat assistants pleasant to talk to, and it has a known side effect: models learn to sound confident, because confident answers rate better. Diogo Almeida, TypeSafe’s CEO, is listed on the company’s team page as a co-inventor of RLHF and a contributor to InstructGPT, which makes the pivot in objective a deliberate one rather than an accident.

RLVR, Reinforcement Learning with Verifiable Rewards, rewards outputs a program can check. Did the code compile, did the unit test pass, did the arithmetic come out right. It works where correctness is mechanically decidable and has nothing to say about how sure the model should have been.

RLCD, as TypeSafe describes it, rewards probabilities that match outcomes. A yes/no answer given at 0.8 should be right about 80% of the time across the whole distribution of such answers. Getting the answer right is necessary but not sufficient; getting the number honest is the objective.

Why a model trained this way looks different

Three consequences fall out of the objective, and they show up in Jev’s API.

The probability distribution is the primary output rather than a debug field. Every Choice answer returns a probability for each option, and every Score answer returns one per level. The chosen answer is derived from the distribution, not the other way round.

Confidence is computed, not claimed. TypeSafe describes it as “a statistic computed from the probability distribution the answer already gives you.” Nobody asked the model how sure it was. The shape of the distribution is the certainty measure, which removes the failure mode where a model asserts 95% because assertive text scores well.

Noul returns no confidence value at all, since a probability from 0 to 1 already is one. A second number would be redundant.

What TypeSafe has not published

No paper, no dataset, no training recipe, no calibration figure.

That last gap is the pointed one. The standard measure here is expected calibration error, the average gap between a model’s stated confidence and how often it is actually right, computed by bucketing predictions and comparing each bucket’s mean confidence to its accuracy. A vendor claiming calibration as its core differentiator could publish that number per task family. As of 18 September 2026, TypeSafe had not.

The weights are not released either, so nobody outside the company can measure it end to end on the real model. What you can do is measure it yourself on your own data, which is worth doing before you wire a threshold into anything that spends money.

Calibration does not fix the rest

TypeSafe’s own jaggedness page for Jev 1.13 is the useful corrective. Version 1.13 is weak at arithmetic and counting, “reads dates as text, not as ordered quantities,” degrades as unrelated content grows in the state, and does not treat input as hostile by default.

One entry on that list matters more than the others for anyone reasoning about calibration: the model gives no guaranteed mathematical relationship between related questions. Ask “is this urgent?” and “is this not urgent?” and the two probabilities need not sum to 1. Per-answer calibration is a statistical property across many answers. It does not impose logical consistency within a single call.

If you need that consistency, your code enforces it. Ask one question rather than its mirror, or reconcile the pair yourself. The build guide treats the whole jaggedness list as a set of design constraints rather than a bug report.

How to check it on your own data

Log every answer with its probability and, later, the true outcome. Bucket the answers by probability, say in ten bins. For each bin, compare the mean stated probability to the fraction that turned out correct. Plot the two against each other. A calibrated model sits close to the diagonal.

Do this per question type and per state shape. Calibration is not a single global property of a model, and a distribution that is honest on support ticket categories can drift badly on legal clauses. Self-consistency checks cover the related pattern of asking the same thing more than one way.

FAQ

Is there an RLCD paper?

No. As of 18 September 2026, TypeSafe had published the name Reinforcement Learning for Calibrated Decisions and a one-line description of the objective in its Jev launch post, with no paper, dataset, training code or evaluation methodology. Everything known publicly about RLCD comes from that post and the API’s behaviour.

How is RLCD different from RLHF?

RLHF rewards the answer a human rater prefers, which trains for persuasiveness alongside correctness. RLCD rewards probabilities that match observed outcomes, so a model gains nothing by sounding certain. The reward looks at the whole distribution across many answers rather than at any single response.

Does RLCD make Jev more accurate?

Calibration and accuracy are different things. RLCD targets honest probabilities, not a higher hit rate. A calibrated model tells you when it is unsure, which lets your code escalate those cases instead of acting on them. TypeSafe has published no accuracy or calibration figures for Jev.

Can I measure Jev’s calibration myself?

Yes, and it is the only way to get a number. Log each answer with its probability, record the real outcome, bucket by probability, then compare each bucket’s mean probability to its actual hit rate. Do it separately for each question type, since calibration varies by task.

Do complementary questions sum to 1?

No. TypeSafe’s jaggedness page for Jev 1.13 says there is no guaranteed mathematical relationship between related questions, so a yes/no question and its negation can both come back at 0.7. Enforce that kind of consistency in your own code if your logic depends on it.

Related guides