Sean Goedecke explains that Jev is fast because it is not autoregressive: it answers many questions in parallel in one forward pass instead of emitting a JSON blob token by token. The post puts Jev’s fastest responses at around 70 milliseconds and its slowest at 500. Goedecke’s example is the Doom demo, where the model gets a text version of the game state plus a set of choices and plays well in real time. The post calls fast structured output a possible new computational primitive, one that could open up AI uses that are not chatbots.
The main objection is that fast structured output is already available. With an ordinary language model you can prefill the response up to the answer and generate a single token restricted to the caller’s choices, batching several questions together. Goedecke reports a 2 to 3 times speedup from trying this with Qwen2.5-1.5B-Instruct, compared with structured output that is not prefilled. From that the post concludes Jev likely has no large technical moat, though a model trained only for structured output probably keeps an advantage.
Goedecke calls the “cannot hallucinate” claim a semantic dodge, since Jev can still pick the wrong option. The post also doubts Jev will ever be as smart as frontier language models, because it cannot use test-time compute.
