drinkmoonshine/parallel-constrained-decoding on Hugging Face
- Likes
- 33
Read from Hugging Face on . Counts change daily.
Parallel Constrained Decoding (Qwen2.5-1B-RLCD)
HuggingFace Space running a stock Qwen2.5-1.5B-Instruct 4-bit checkpoint that scores multi-field JSON schemas in one pass using KV-cache broadcasting and logit slicing, reporting a 5.6x to 7.0x latency cut on Apple Silicon. The README documents no reinforcement learning or fine-tuning despite the RLCD in the model name.
More like this
Ran Jev against an existing classifier eval that previously used Gemini 2.5 Flash Lite
Vercel CTO reports Jev saturated an existing classifier eval that had used Gemini 2.5 Flash Lite and ran about 6x faster. Original post; TypeSafe's quote-tweet is a separate item.
Malte Ubl (@cramforce)OpenRan Jev against an existing classifier eval that previously used Gemini 2.5 Flash Lite on x.com220k viewsSpeed vs Gemini 6x faster
[AINews] Jev: a System One Model that only decides/classifies/routes/scores
AI News daily roundup leading with the Jev launch, summarising RLCD and TypeSafe's 20-200x speed and 40-400x cost claims. Its recap of reactions notes posters using Jev as a structured classifier, judge and routing policy where text generation is unnecessary.
Introducing System One Models and Jev (Hacker News launch thread)
The 1,978-point, 513-comment launch thread. The top comment calls the speed comparison misleading, since Jev only returns structured output, and disputes the cannot-hallucinate claim because a valid type can still hold a wrong value. Others note ordinary LLMs can already be forced into structured output.
DSPy: programming, not prompting, language models
Declares typed input/output Signatures for LLM modules and optimizes the underlying prompts and weights against a metric. Its Signature abstraction is the closest widely-used open equivalent of Jev's typed-question interface. Star count is GitHub's rounded display figure.
Outlines: structured outputs for LLMs
Guarantees valid structured output during generation by constraining decoding to a grammar or schema. The mechanism HN commenters repeatedly cited as already covering Jev's cannot-produce-a-type-error guarantee on ordinary LLMs. Star count is GitHub's rounded display figure.
Instructor: reliable JSON from any LLM
Pydantic-based library that extracts typed, validated structured data from LLMs across providers with automatic retries. The established way teams get Jev-style typed answers today, at LLM latency and cost. Star count is GitHub's rounded display figure.
