The usual failure of a chat model is fluent invention: a citation that does not exist, an API method that was never shipped. Jev returns a typed answer drawn from options you defined, plus a probability distribution over them. There is no prose for a fabricated fact to hide in.
That removes one failure mode. It does not remove error. TypeSafe’s own jaggedness page for jev-1.13 lists nine things the model does badly, including reading dates as text rather than as ordered quantities, and no guarantee that P(A) and P(not A) sum to 1 across related questions. A Choice that comes back wrong at high confidence is a quiet failure, because nothing in the response looks off.
Diogo Almeida’s framing, as reported by agentpedia.codes: “I don’t think it’s fair to say a random forest ‘hallucinates.’” Fair enough as a point about vocabulary. It says nothing about accuracy.
Treat the probability as the thing to check rather than the wording. The jaggedness entry covers the documented weak spots, and confidence covers how to gate on a low number instead of acting on it.