Open source, self-hosted. There is no hosted API. You download the weights and run them on your own hardware. How open-source models are listed.
What H2O-Lightning-4B is
H2O-Lightning-4B is a decision model from H2O.ai, the open-source machine learning company in Mountain View, California. It is a fine-tune of Alibaba’s Qwen3.5-4B and ships as Apache 2.0 weights on Hugging Face. It answers typed questions about a record such as a document, a ticket, a policy or a conversation: Choice, Score and Noul, the same three shapes Jev uses. It also reads images that come with the record, such as receipts, screenshots and charts.
Each decision is one forward pass and one output token, so nothing is generated and the cost is input tokens only.
What it returns
Every answer carries a probability. Choice returns one for each option, Score returns an expected level with a distribution over the scale, and Noul returns the probability that a statement is true. It runs on unmodified vLLM 0.30.0 behind a small Python script that serves POST /v1/systemone, the same request shape Jev uses. H2O has not published a hosted API.
What it is good at
Speed and calibration for its size. On Benchmark Heaven’s JevBench it holds first place on the official composite among open-weights systems at 72.5, ahead of Quyet-1.0-Large at 71.4, with a calibration score of 90.0. Benchmark Heaven measured a median of 29 milliseconds a decision on its own GPU. H2O reports 205 of 231 correct on JevBench’s public items and 87.4% on 1,877 image questions it built from public datasets. Those are H2O’s own runs.
Microsoft’s launch post for Microsoft-Decision-1 scored it 77.2% accuracy and 91.8 calibration on a 36-benchmark suite Microsoft ran itself and did not name. Microsoft-Decision-1 scored 83.5% on that suite.
The Jev Decision Index (multimodalart’s community board on Hugging Face) has no row for it in the 0.3.1 file generated on 12 October 2026.
What it is not for
It trails the 31B models on raw intelligence: 60.0 on JevBench against 73.4 for Quyet-1.0-Large. In the Capability view, which leaves out speed and cost, it scores 75.0 against Quyet’s 81.7. On the public set it misses 25 of the 111 hard items. Counting objects in photos and picking the right element on an app screen are its weakest image tasks, at 63.3% and 65.0% in H2O’s own tests.
It needs a GPU you run yourself; the card measures a 32 GB card. The board scores v1.1, while the card’s current measurements use v1.2.3, so board figures may not describe the newest weights. If you want to flag uncertain yes/no answers, use the card’s routing mode, because the default reports every yes/no answer at 0.801 or higher.
Specifications
| System One Score | 90.6 from 1 of 3 boardsJevBench 90.6. How the score works |
| Question types | ChoiceScoreNoul |
| Max Choice options | 255 |
| Score levels | Not documented |
| Questions per call | Not documented |
| Total context | 40,960 tokens |
| State budget | Not documented |
| Rate limit | None. It runs on your own hardware. |
| Endpoint | POST /v1/systemone on your own server, through the h2o_lightning_shim.py script in front of vLLM |
| Licence | Apache-2.0 |
| SDKs |
The server runs with a 40,960-token context and answers HTTP 422 to anything longer. A question can have up to 255 options. A request can carry up to 4 images of at most 20 MB each, and each is scaled to at most 1,048,576 pixels. The shim reports every yes/no answer at no less than 0.801 on its chosen side unless you switch to the card's routing mode.
Versions
- h2oai/h2o-lightning-4b v1.1, 4 Oct 2026, The release Benchmark Heaven's JevBench board scores. The board lists the row as H2O-Lightning-4B v1.1. Release notes
- h2oai/h2o-lightning-4b v1.2.3, The release the card's current measurements and run commands pin. The tags v1.2, v1.2.1 and v1.2.2 sit between it and v1.1. The JevBench board has not re-scored it. Release notes
Use cases
What people use H2O-Lightning-4B for, one page per pattern.
Support inbox triage with System One models
Send a support ticket to Jev once with every question attached. Category comes back as a selected label, severity and frustration as numbers on scales you wrote, refund intent as a probability. Your code reads those values and decides what happens to the ticket.
Compliance verification with System One models
A compliance review is a fixed checklist run against a changing document. Jev answers the whole checklist in one request, returning a probability, a label or a rated level per item, so code can clear the clear-cut findings and send the borderline ones to a person.
Confidence-gated actions with System One models
Jev returns a confidence value from 0 to 1 alongside every Choice and Score answer. Your code treats it as a separate axis: act automatically when it's high, confirm or flag when it's middling, hand the decision to a person when it's low. Riskier actions get higher bars.
Intent and model routing with System One models
One Jev call reads an incoming request and returns its intent as a label plus a difficulty rating on a scale you wrote. Your router reads both numbers and picks the handler: deterministic code, a cheap model, an expensive one, or a human queue.
Benchmarks
Published scores, each linked to its source. A row marked vendor-run was measured by the vendor itself, so read it as one side's result. The compare hub sets these beside the other models.
| Benchmark | Metric | Score | Run by | Date |
|---|---|---|---|---|
| H2O model card benchmarks | Public JevBench items correct | 88.7 % | vendor-run | 12 Oct 2026 |
| H2O model card benchmarks | Accuracy on image validation sets | 87.4 % | vendor-run | 12 Oct 2026 |
| JevBench 1.7.46 | Composite Score | 72.5 score | Independent | 12 Oct 2026 |
| JevBench 1.7.46 | Capability Score | 75 score | Independent | 12 Oct 2026 |
| Microsoft 36-benchmark comparison | Average accuracy | 77.2 % | vendor-run | 9 Oct 2026 |
| Microsoft 36-benchmark comparison | Calibration score | 91.8 score | vendor-run | 9 Oct 2026 |
- H2O model card benchmarks, Public JevBench items correct: H2O ran JevBench's own command-line tool on the 231 public items: 205 correct. Easy 48 of 48, standard 71 of 72, hard 86 of 111. The public items are visible to every submitter, so this is not a held-out score.
- H2O model card benchmarks, Accuracy on image validation sets: H2O built 1,877 multiple-choice questions from 16 public datasets with human or source labels. Expected calibration error 0.017. Counting objects in photos scored 63.3% and picking a screen element scored 65.0%.
- JevBench, Composite Score: Row: H2O-Lightning-4B v1.1. First among open-weights systems on the official composite. Intelligence 60.0, Calibration 90.0, Speed 92.6, Cost 60.3. The board re-measures rows, so this can differ from earlier editions.
- JevBench, Capability Score: Row: H2O-Lightning-4B v1.1. The average of Intelligence and Calibration, with speed and cost left out.
- Microsoft 36-benchmark comparison, Average accuracy: Run by Microsoft on an undisclosed 36-benchmark suite, 147,137 questions. Microsoft-Decision-1 scored 83.5% on the same suite. The benchmark names are not given, so the figure cannot be reproduced.
- Microsoft 36-benchmark comparison, Calibration score: Run by Microsoft on an undisclosed 36-benchmark suite; 100 is perfect.