Jev Decision Model vs an LLM With Structured Output

Decision models are a class of AI model that read a situation and return one answer from a list you define, with a probability attached, instead of writing text. The Jev decision model, released by TypeSafe in September 2026, is the first one sold under that banner. This article compares Jev with what a large language model already does when it is held to structured output, separates the vendor’s measured claims from its marketing, and ends with a test for which questions belong to which model.

The comparison matters most where a system asks the same small question thousands of times a day, and a coding agent is the clearest case. Claude Code can run a PreToolUse hook before every tool call, and that hook stops the call by exiting with code 2 or by returning a “deny” decision (Claude Code hooks reference). Each check is a yes-or-no question: does this command expose a secret, delete data, send something out? In a video on using Jev in AI coding workflows, published on 2026-09-30, Cole Medin describes replacing his own guard, a long list of regular expressions, with Jev. By his measurements, which he has not published in detail, a check run through Claude Haiku took more than a second and cost about a tenth of a cent, while Jev answered in roughly a quarter of a second for a small fraction of that.

A Jev decision model answers typed questions, not prompts

Jev takes two inputs: a state, the situation in whatever unstructured form the application holds it, and typed questions about that state. OpenRouter, which serves the model as typesafe/jev-1.14, documents four question types (Jev on OpenRouter):

• Choice picks one option from a set and returns a probability for each option.
• Noul answers a yes-or-no condition with the probability that it holds.
• Score places the input on an ordered scale and returns a probability-weighted position.
• Box locates an object in an image, in version 1.14 only.

All the questions are answered in one call, and the answers come back as values a program can branch on without parsing. The same page says Jev produces no reasoning traces, no explanations and no free-form text. TypeSafe’s founder, Diogo Almeida, calls the intended use “smart if-statements”: classify, route, score, extract, branch (Introducing System One Models & Jev, 2026-09-15).

An LLM with structured output and Jev end at the same kind of value; only Jev skips the generated text and attaches a probability to every option.

The name System One, as the launch post says itself, borrows Daniel Kahneman’s fast, intuitive mode of thinking. A model that reasons in text before answering is the slow mode; Jev is meant to be the reflex.

Structured output already gives an LLM the same answer format

A typed answer, on its own, is not new. Anthropic’s structured outputs feature compiles a JSON schema into a grammar and constrains Claude’s sampling to it, so the response parses and its fields carry the declared types (Claude structured outputs). An enum of allowed answers is a supported schema feature, and that is all a Choice question is. The guarantee has documented exceptions, such as refusals and responses cut off by max_tokens.

An LLM can already answer “bug or feature” in a form code can use; Medin tested his guard with one before settling on Jev. What differs is the price, the latency and, above all, the probability attached to each answer.

The probability is the product, and LLM confidence is poorly calibrated

A probability lets software act alone above a threshold and escalate to a person below it, but only if it is calibrated: answers given at 90 percent confidence should be right about 90 percent of the time.

Language models are known to lose that property during post-training. OpenAI’s GPT-4 technical report plots calibration on a subset of the MMLU benchmark: the pre-trained model had an expected calibration error of 0.007, the post-trained model 0.074, and the report says post-training hurts calibration significantly (GPT-4 Technical Report, arXiv:2303.08774). A study of RLHF models including ChatGPT, GPT-4 and Claude found that confidence stated in words was usually better calibrated than their token probabilities, often cutting calibration error by about half (Tian et al., EMNLP 2023). A chat model’s probabilities are not trustworthy by default.

TypeSafe’s central claim is that Jev fixes this. The launch post says the model is trained with a method the company calls Reinforcement Learning for Calibrated Decisions, and promises that higher confidence means higher accuracy. It publishes no calibration curve, no calibration error and no description of how calibration was measured. For a product whose value rests on its probabilities, that is the largest gap in the public record.

TypeSafe’s speed and cost figures come from its own evaluations

The launch post is more careful with numbers than most of the coverage. It claims responses 40 to 200 times faster than frontier LLMs on “System One shaped” queries: 70 to 500 milliseconds for Jev against 3 to 329 seconds end to end for frontier models. The home page headlines 193.6 times faster and 444.6 times cheaper, figures the post traces to its workflow evaluations and places at the higher end of real-world gains. The cost range of 40 to 1,000 times that Medin repeats, and that Techstrong.ai attributes to TypeSafe, does not appear in the launch post.

The price list is easier to check. Jev costs $0.042 per million input tokens, and output tokens are free. The post sets that against $0.20 to $10 per million input tokens for frontier LLMs, with output about five times the input price. On input alone, Jev is roughly 5 to 240 times cheaper, before counting the tokens an LLM spends writing its answer.

Jev’s input price sits about 5 to 240 times below the frontier LLMs in TypeSafe’s comparison. Source: TypeSafe launch post, 2026-09-15.

The evaluation design matters as much as the headline. In the workflow tests, the reference answers were the averaged probabilities of GPT-6 Astra and Fable 5.1, so a correct answer means agreeing with two other models, not with ground truth. TypeSafe’s own team wrote the workflows, which the post concedes could bias the results, and it admits it cannot prove the price is not subsidized. On accuracy, it claims parity with existing LLMs on these tasks, not superiority. None of the coverage cited here reports an independent benchmark.

Purpose-built classifiers beating general LLMs is not a new result

A model built for decisions beating a general model at decisions is an old finding. In 2024, Martin Juan José Bucher and Marco Martini found that smaller fine-tuned models still outperformed large generative models prompted zero-shot on text classification, consistently and significantly (arXiv:2406.08660). A fine-tuned classifier is fast and cheap too, but it needs labeled data and a training run for every new question. Jev promises a classifier’s speed with a prompt’s flexibility, questions defined at call time and no training. Whether one general decision model can match per-task classifiers across arbitrary questions is what the missing benchmarks would have to show.

Where a Jev decision model fits in an AI coding workflow

Medin’s video is more useful for its catalogue than for its numbers: four points in AI-assisted software work where a workflow stops to ask a closed question.

• Guarding tool calls. His guard asks four questions of each call: are secrets exposed, is data destroyed, is data sent out, has the agent drifted off its task. The state is the tool name, its effect, its arguments and the working directory.
• Playing a game under test. At 30 or 60 frames per second, a model that writes text cannot choose the next move in time. Jev controls the character in a game Medin is building, after an LLM has written the harness that turns the game’s state into Jev’s input.
• Driving a browser. For end-to-end tests of a web app, Jev chooses which field to fill and which button to press, while an LLM prepares any text to type.
• Triaging work. In a workflow built with Archon, his open-source harness builder, Jev reads a GitHub issue, decides whether it is a bug or a feature, and picks the tier of model the next step needs.

His triage results are small and self-judged: all 12 runs matched his own call, and a similar classifier for pull requests got 15 out of 16 right, with Medin as the only judge. That is a sensible first trial, not evidence of accuracy.

The pattern he draws is the part worth keeping: an LLM prepares the decision by writing the state and the options, and an LLM acts on the result, fixing the bug a game run exposed or finding another route after a blocked call.

“Jev is never really the end of our workflow. It’s always just making parts of our workflows faster and more efficient.” - Cole Medin

TypeSafe’s comparison table agrees, leaving chatbots, copilots and coding agents to LLMs. It is the layering of an AI agent harness, where the model is one component next to the loop, the tools and the checks.

What a probabilistic guard still cannot promise

The guard is where the trade-offs bite hardest, because a decision model given a security job inherits every one of its limits.

• It cannot explain a block. Jev returns a probability, not a reason, so the only explanation a hook can pass back is the name of the question that crossed the threshold.
• A timeout lets the call through. The hooks reference states that a command or HTTP hook that times out does not block the tool call unless it is set to block on failure.
• Thresholds belong to one model version. OpenRouter warns that probabilities differ between models, so a cutoff tuned on Jev 1.13 must be rechecked on 1.14.
• Long inputs degrade the answers. Past about 32,000 bytes of text, Jev switches to a long-context mode TypeSafe calls slower and less accurate.
• The arguments leave the machine. A guard meant to keep secrets out of an agent’s context sends every tool call, arguments included, to a third-party API.

Only the first is specific to Jev; an LLM judge in the same hook has all the others. They are why AI agent permissions have to be enforced outside the model: a classifier, however fast, is a filter in front of the permissions, not a replacement for them.

Decide by the shape of the question, not by the model

A question suits a decision model when its answers can be listed in advance, when it is asked often enough that cost and latency add up, when a one-second delay breaks something, and when the program needs a probability to set a threshold. It suits an LLM when the answer is text, when someone has to read the reasoning, or when the question is rare enough that cost and delay do not matter.

Most workflows contain both kinds, which is why the sandwich is the realistic architecture. Before moving a decision to Jev, run the same questions through both models on a sample with known answers, compare accuracy and calibration rather than speed alone, and set the threshold on that data. TypeSafe has published its speed and its price; the calibration its argument rests on is still yours to measure.