Play the video in the post belowPost on X
- Views
- 102,514
- Likes
- 1,356
- Reposts
- 104
Read from X on . Counts change daily.
Nick Khami argues that Jev’s probability-output behavior isn’t unique to TypeSafe’s model and can be reproduced on any open model with a bit of inference engineering, the low-level work of configuring how a model serves requests. To prove it, his team built a demo endpoint called deepseek-v4.1-flash-jev.
The mechanism relies on sglang, an inference engine, which offers a /v1/score endpoint alongside its normal /generate one. Given an input and a set of possible answers, the score endpoint forces the model to return a probability for each answer instead of streaming generated text, which is the same shape of output Jev gives. For DeepSeek specifically, Khami found one extra trick was needed: adding a closing “think” tag before the response to force a direct answer instead of a full reasoning trace, since reasoning first would work but Khami considers it too slow to be worth it.
Khami is careful to say DeepSeek v4.1 Flash run this way isn’t as good as Jev, but argues that applying the same approach to a larger model, given enough spare compute, would likely reach comparable or better decision quality. He adds, separately, that he thinks decision-making models kill all prospecting and sourcing work, and that he would have loved to have Jev or similar when recruiting for Mintlify.
