Skip to content
System One

Hallucination

Hallucination is when a language model states something false in generated text. A System One model never writes free text, so it cannot invent a fact in prose. It can still be wrong: it can pick the wrong option or return a probability that does not match how often it is right.

The usual failure of a chat model is fluent invention: a citation that does not exist, an API method that was never shipped. Jev returns a typed answer drawn from options you defined, plus a probability distribution over them. There is no prose for a fabricated fact to hide in.

That removes one failure mode. It does not remove error. TypeSafe’s own jaggedness page for jev-1.13 lists nine things the model does badly, including reading dates as text rather than as ordered quantities, and no guarantee that P(A) and P(not A) sum to 1 across related questions. A Choice that comes back wrong at high confidence is a quiet failure, because nothing in the response looks off.

Diogo Almeida’s framing, as reported by agentpedia.codes: “I don’t think it’s fair to say a random forest ‘hallucinates.’” Fair enough as a point about vocabulary. It says nothing about accuracy.

Treat the probability as the thing to check rather than the wording. The jaggedness entry covers the documented weak spots, and confidence covers how to gate on a low number instead of acting on it.

Related terms