A worked version. Take every prediction a model made with confidence between 0.80 and 0.90, where the mean confidence in that bin works out to roughly 0.85. If the model was right on 62% of them, the bin’s gap is about 0.23. Repeat for each bin, weight by bin size, and the total is the ECE. A perfectly calibrated model scores 0, lower is better, and the figure only means something alongside the bin count and the dataset it was measured on.
The measure is standard in the calibration literature. The widely cited reference is Guo, Pleiss, Sun and Weinberger, “On Calibration of Modern Neural Networks” (2017), which showed that modern deep networks tend to be overconfident and that temperature scaling corrects much of it.
It matters for System One models because every answer ships with a probability. TypeSafe describes its training method, RLCD, as optimizing for “answers with epistemically honest probabilities on System One tasks”, and describes confidence as “a statistic computed from the probability distribution the answer already gives you”.
TypeSafe has not published an ECE figure for Jev, nor an RLCD paper or dataset, and no independent accuracy benchmark existed as of 2026-09-18. Measure it on your own labelled data if the number matters to you. See RLCD explained and calibrated probability.