Until this year we had two tools: deterministic code, which is exact but can't read a sentence, and LLMs, which read anything but return prose with no real confidence score. Decision models such as SubQ, Jev and OpenAI's new Decisions API sit in the middle. They read text, answer from a list you define, attach a calibrated confidence, and cost about 1/100th of an LLM call. Used together, the three can automate judgment in a regulated workflow without giving up control. This page runs all three on clinical-trial data work.
Numbers below come from the live runs on this page: SubQ Snap, Jev 1.13 and OpenAI GPT-6 Luna Decisions as decision models, GPT-5.5 as the LLM.
| Deterministic code | Decision models | LLMs | |
|---|---|---|---|
| Returns | An exact value | One option from your list, with a probability for each | Free text (or JSON it was asked to write) |
| Reads free text | No | Yes, and images for SubQ and OpenAI | Yes |
| Same input, same answer | Always | Typed and stable; small confidence wobble | No. Fast-mode eGFR came back 45.6 to 46.6 across 5 runs |
| Confidence | Not needed: exact | Per answer, usable as a threshold. At ≥ 0.8 every kids and phase answer in our benchmark was right but one | None you can trust |
| "I can't tell" | Raises a validation error | Yes, when you offer it as an option | Rarely; tends to answer anyway |
| Latency here | < 1 ms | 0.11–1.2 s median 0.2 s | 1.0–11.5 s fast mode to reasoning on |
| Price per 1M input tokens | ≈ $0 | $0.04–$0.10 output free | $5.00 + $30 per 1M output (GPT-5.5) |
| Part 11 audit fit | Full: versioned, validated, testable | Strong: log model, question, answer, confidence | Weak: log the full prompt and output, and review it |
| Best at | Edit checks, dates, windows, formulas, SAE clocks, CDISC rules | Triage, classification, routing, "is this already explained?", guarding LLM output | Writing queries and narratives, summarizing, choosing which tool to call |
Four everyday jobs from clinical-trial data management. Each one passes through all three engines, and each step is labeled with the engine that ran it and why it was chosen.
I tested accuracy techniques on a 220-trial benchmark from ClinicalTrials.gov, plus 90 fresh trials the models had never seen. Some worked, one didn't, and the ones that handle messy input are built into the fourth tab above.
Intervention type, 220 trials. Each point is a confidence threshold: how many trials clear it, and how often those are wrong.
Recommended next: the feedback loop. Log every human decision as a new labeled example, re-certify thresholds on a schedule, pin model versions such as jev-1.13.0, and rerun the suite before any upgrade. That loop is what turns a demo like this into a validated system an auditor can follow.
Default to the most precise engine that can do the job. Move right only when the job requires it.
If you can write the rule, write the rule.
Still the best we have. Validated once, right every time, and the only engine a regulator fully trusts.
If the answer is a choice, ask a decision model.
Never for math or dates. They choose; they don't calculate.
If the answer is words, use an LLM, and give it tools for every fact.
Never the final say on a fact or a regulated decision.
These are the moves the demo makes. They carry over to almost any workflow where a wrong answer is expensive.
Run every exact check before anything reads text. Only real exceptions reach a model, so the expensive engines see a fraction of the volume.
Ask a cheap model which engine should handle each item. Computable criteria go to code, judgment to a decision model, opinions to a person.
Code turns probabilities into actions. Auto-accept only when all models agree at 0.8 or higher, and send everything else to a person. The threshold is a validated setting.
Give the LLM tools for every number. Fast mode alone enrolled an ineligible patient 5 times out of 5. Fast mode with tools got it right, with a receipt for each value.
Before any generated text reaches a site, decision models check it: is it leading, does it leak an identifier? When the judges disagree, the draft waits for a person.
The same split works anywhere text arrives messy and decisions have to be defensible. Some places I would start.
Reviewing a flagged lab against its site comment, priced from this page's live run.