Compliance verification with System One models

The problem

Compliance work has a shape that repeats. There is a rulebook, there is a document, and someone has to go through the rulebook item by item deciding whether the document satisfies it. Classify the contract, find the missing clause, spot the prohibited claim, escalate anything that looks risky. That list comes straight from TypeSafe’s use-case map entry for legal and compliance, and it describes most of what a first-pass review actually involves.

The checklist is stable. The document is not. That asymmetry is what makes the work expensive when a person does it and repetitive when an LLM does it, because each question ends up as its own prompt with the same document attached again.

TypeSafe’s parallel questions cookbook measures exactly that. It runs a 13 item regulatory briefing over the GDPR Wikipedia article, roughly 54,000 characters, and reports that batching every question into one call came out 12.2x cheaper and 10.0x faster than one call per question, with no change in the answers. Those are TypeSafe’s own measurements on their own workload, recorded against jev-1.12 rather than the current jev-1.13, but the mechanism is easy to check: the document dominates each request, so sending it once instead of thirteen times is where the saving comes from.

What the state looks like

State is the content one request is judged against. Send the document plus the identifiers you will need in the audit record, and nothing else. Jev’s accuracy drops as the state fills with material the questions do not need, so retrieve the relevant sections in code before you send anything.

{
  "source": "https://en.wikipedia.org/?oldid=1363040264",
  "document_id": "POL-2026-0431",
  "text": "The General Data Protection Regulation (EU) 2016/679 is a European Union regulation on information privacy..."
}

Watch the ceiling. Jev takes 64k tokens per request, with 32k of that available to the state plus the longest question. A long agreement needs splitting by section, with the checklist run against each part.

The questions you ask

One request carries the whole checklist, mixing all three question types.

from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

client = TypeSafeClient()

questions = {
    "breach_72h": Noul(
        instructions="Must a personal data breach be reported to the supervisory authority within 72 hours?"
    ),
    "right_erasure": Noul(
        instructions="Does the document grant individuals a right to erasure of their personal data?"
    ),
    "instrument_type": Choice(
        instructions="What kind of EU legal instrument does this document describe?",
        criteria={
            "Regulation": "Directly binding law in all member states, no national implementation needed.",
            "Directive": "Sets goals that member states implement through national law.",
            "Treaty": "An international treaty between states.",
            "Recommendation": "Non-binding guidance.",
        },
    ),
    "compliance_burden": Score(
        instructions="How heavy is the compliance burden this document places on organisations?",
        criteria=[
            "Negligible: no meaningful obligations.",
            "Light: a few notices and disclosures.",
            "Moderate: documented processes and some dedicated roles.",
            "Heavy: records, impact assessments, officers, and breach procedures.",
        ],
    ),
}

response = client.system_one(DOCUMENT, questions)

Pick the shape to fit the finding. A noul, a yes/no question returning one probability between 0 and 1, suits “is this obligation present”. A choice suits a finding with named mutually exclusive outcomes, and gives you a probability for every option plus a confidence value from 0 to 1. A score suits anything graded, like how onerous or how severe, where you write the levels out in words and get back a weighted mean of the level numbers.

Adding questions is close to free, so the fan-out recipe is to include the speculative ones too and let code ignore whichever turn out not to apply.

Decision policy

Compliance is the wrong place for a model to have the last word. Jev produces findings; your code decides which findings clear, which get a note, and which reach a person.

CLEAR = 0.90
REVIEW_FLOOR = 0.60

verdicts = {}
for item in ("breach_72h", "right_erasure"):
    p = response.nouls[item].noul
    verdicts[item] = "satisfied" if p > CLEAR else "failed" if p < 1 - CLEAR else "review"

instrument = response.choices["instrument_type"]
if instrument.confidence < REVIEW_FLOOR:
    verdicts["instrument_type"] = "review"

The middle band is the point. TypeSafe documents a three-band policy: act automatically when confidence is high, confirm or gather more when it is middling, hand it to a person when it is low, with a worked example using a 0.5 floor and a stricter bar above 0.9 for anything destructive. Regulated work usually wants the bar higher than the docs’ defaults, and the docs say plainly that the right numbers depend on the domain. Store the probability next to the verdict so an auditor can see what the cut line was on the day. The same idea appears in confidence-gated actions.

When not to use this

Deadlines. Retention windows and “was this filed inside the statutory 30 days” are arithmetic over dates, and Jev reads dates as text rather than as ordered quantities. Ask it which month, day and year the document names, each as its own choice, then compare in code.

Counting is out for the same reason: “how many times does the agreement mention subprocessors” is a job for a search, not a judgment. Penalty caps and turnover percentages are numbers to extract, not to compute.

Deep indirection costs accuracy. “Does clause 8 contradict the definition in schedule 2 as amended by the side letter?” is several hops. Pull the relevant text together first and ask one direct question about it.

A document written to be misread is also a risk. Jev treats state as data rather than as hostile input, so a vendor’s own summary of their compliance position is an argument, not evidence. Point the questions at the operative clauses.

FAQ

Does batching questions change the answers?

No. Every question is scored on its own against the state, so an answer does not depend on what else is in the request. The parallel questions cookbook checked this by asking each question both ways several times and comparing the run to run variation. Most answers came back identical across all five repeats under either strategy.

How long a document can one request handle?

Jev accepts 64k tokens per request, with 32k available to the state plus the longest question, per TypeSafe’s models page. Long agreements need splitting in code, usually by clause or by section, with the checklist run against each part and the findings merged afterwards. Smaller states also read more accurately, so the split helps twice.

Can Jev quote the clause that failed?

Not directly. It is not trained to generate text, so it will not hand back a passage. Split the document into candidate spans in code, then ask a choice question whose options are those spans, which gets you a verbatim citation because the model only ever selects one. The structured extraction page covers that pattern.

Is this good enough to sign off a review?

Treat it as triage rather than sign-off. It clears the obvious passes, surfaces the obvious failures, and concentrates a reviewer’s attention on the uncertain middle. Keep the human approval step, keep the probabilities in the record, and sample the automatic passes periodically to check the thresholds still hold.

Examples in the wild