Play the video (opens in a new tab)Video on YouTube by AICodeKing
- Views
- 36,462
- Duration
- 8:21
Read from YouTube on . Counts change daily.
AICodeKing runs eight small playground requests with synthetic examples. A duplicate-charge support message goes to billing, with a 98% refund probability and 11% urgency. Rewording it to “I am not asking for a refund” drops the refund probability to 3%, so Jev handled the negation. A fake “system override” planted in a checkout-crash message did not flip the routing away from technical support. Asked to pick the current receipt address from a list of candidates, Jev chose the right one exactly as supplied. In an agent audit, it gave a 93% probability that an assistant’s “draft saved” claim was unsupported by a permission-denied tool result.
The most useful test is a failure the author set up on purpose. With only billing, technical and sales on offer, a question about cafeteria hours went to sales at 0.31 confidence. With an “other” option available it went to other. A restricted output stops Jev from inventing a category, but it does not make the chosen one correct. The author’s advice is to include an other or unknown option and a path for human review.
The author reports evaluation times of about 92 to 214 ms, not counting network, and 4,148 input tokens in total across the eight requests. The video also walks through Gregor Zunic’s Jev Ultrafast browser demo, where Jev picks the next action and a small separate model writes any text. The author says they did not reproduce that run.
