Human-in-the-Loop Credit Review: Designing the Analyst Override Workflow

Human-in-the-loop credit review works when the machine routes exceptions, not documents. Define four trigger classes — confidence, cross-source disagreement, exception rules and materiality — put the source page beside the extracted figure so a reviewer resolves in forty seconds, and log every override. With 100% of figures cited, the override record becomes your governance artefact.

Key facts

• Override capability is a written regulatory expectation in the EU. Article 14 of the EU AI Act requires that oversight measures enable a natural person "to decide, in any particular situation, not to use the high-risk AI system or to otherwise disregard, override or reverse the output", and separately to "remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias)". Creditworthiness assessment of natural persons is high-risk under Annex III point 5(b) (Regulation (EU) 2024/1689, Article 14; official text on EUR-Lex).
• US model risk guidance changed in April 2026, and it moved generative AI out of the model perimeter, not out of supervision. OCC Bulletin 2026-13 states that "Generative AI and agentic AI models are novel and rapidly evolving. As such, they are not within the scope of this guidance" (OCC Bulletin 2026-13, *Model Risk Management: Revised Guidance*, 17 April 2026; Federal Reserve SR 26-2). Which means the question an examiner asks is not "is it in your model inventory" but "who reviewed the output, and can you show me".
• Buying the tool does not move the accountability. Interagency third-party guidance expects risk management "commensurate with the bank's risk profile and complexity as well as the criticality of the activity supported by the third party" (OCC Bulletin 2023-17, 6 June 2023). The override log is how you demonstrate you actually did it.
• Commercial credit is not auto-decisioned anywhere near the top of the market. The FDIC found that "28 percent of all large banks would auto-approve a small loan, only 11 percent would auto-approve a medium loan and a mere 1 percent would auto-approve a large loan" (FDIC, *Small Business Lending Survey*, Section 3: Loan Underwriting and Approval). The loop has a human in it whether or not you designed one.
• YuSight cites 100% of figures with one-click source verification, which is the mechanic that makes the arithmetic in this post work: a reviewer who can see the source page beside the extracted value resolves an exception in tens of seconds rather than minutes.

What should actually trigger a human review?

The default design — "flag everything the model is unsure about" — produces a queue nobody can clear. There are four distinct trigger classes, they fire for different reasons, and they should be tuned separately.

Two design rules that separate a working queue from a decorative one:

Route the exception, not the document. The unit of human work is a flagged field with its context — not a 180-page PDF with a note saying "please check". A reviewer handed eleven flagged fields clears them in minutes. A reviewer handed the file re-does the file, and you have bought a licence to reproduce your manual cost.

Never let a confidence score suppress an exception rule. A model that is 0.99 confident the DSCR input is 1.04 does not make a 1.04 DSCR acceptable. Confidence is about reading; exception rules are about policy. Conflating them is the commonest architectural error in these systems, and it is the one that produces a finding.

Where this sits in the wider automation programme — and why the ceiling on any extraction-only deployment is arithmetic rather than opinion — is set out in the pillar, underwriting automation in commercial lending.

What must be on the reviewer's screen?

The whole economics of human-in-the-loop rests on the time it takes to resolve one exception. Forty seconds and the queue is affordable. Four minutes and it is three full-time staff. Nothing else in the design changes that number as much as the screen does.

A reviewer resolving an extracted-figure exception needs, without navigating away and without a search:

1. The extracted value, in the context of the line it belongs to.
2. The source page image, with the figure highlighted — the single highest-value element. If the reviewer has to open a PDF and find page 47, you have added a minute per exception before they have thought about anything.
3. The alternatives the system considered, where there were any. "Read as 4,182 — also present on the page: 4,128 (prior year column), 41,820 (thousands)." This turns a judgement into a choice.
4. The confidence signal, stated plainly. Not a raw number nobody calibrates against. "Low confidence: the page is a scan at 180 dpi and this column is partly obscured."
5. The comparison the exception is about — prior year, the other source document, or the policy threshold that was breached.
6. An edit box that writes an audit entry, with a reason code and free text, and no separate save step.

The last one is not a nicety. If accepting the machine value is one click and overriding it is a four-field form, you have designed a system that quietly rewards rubber-stamping. Make the two paths cost the same.

The extraction technology underneath this — and why citation-level provenance is a different capability from accuracy — is unpacked in OCR vs IDP vs LLM extraction and in how to stop an AI credit memo tool from hallucinating numbers.

Worked example: tuning the confidence threshold

This is the calculation nobody shows, and it decides whether your programme works.

Setup. 200 commercial files a month. 180 material extracted fields per file, so 36,000 fields a month. Extraction accuracy 95.2%, so 4.8% of fields — about 1,727 — are wrong before any review. From a labelled sample of the same 36,000 fields, the errors are not spread evenly across confidence bands:

Now set a single global threshold and read off the consequences. Assume a well-designed review screen at 40 seconds per exception.

Review hours = fields reviewed x 40 seconds ÷ 3,600 Below 0.95 : 7,200 x 40 = 288,000 s = 80.0 hours Below 0.99 : 14,400 x 40 = 576,000 s = 160.0 hours

The marginal cost of each tightening:

0.80 → 0.90 : +2,160 fields, +475 errors caught, +24 hours = 3.0 minutes per extra error caught 0.90 → 0.95 : +3,600 fields, +288 errors caught, +40 hours = 8.3 minutes per extra error caught 0.95 → 0.99 : +7,200 fields, +144 errors caught, +80 hours = 33.3 minutes per extra error caught

Moving from 0.95 to 0.99 costs 80 additional reviewer-hours a month — half a full-time analyst — to catch 144 more errors. That is a real trade, not an obviously wrong one, but it should be made deliberately and by someone who owns the credit risk, not by whoever configured the tool.

Now the same arithmetic with a badly designed screen, at 4 minutes per exception:

Below 0.95 : 7,200 x 4 minutes = 28,800 minutes = 480 hours/month = about 3 FTE

Identical thresholds, identical error outcome, six times the cost. The threshold debate is usually a screen-design debate wearing a disguise.

Tiering by materiality beats any single global number

Not every error matters equally. Suppose 20 of the 180 fields per file actually drive the outputs — the DSCR inputs, total debt, interest expense, the leverage numerator and denominator. That is 11.1% of fields, or 4,000 of 36,000 a month. Apply a tight threshold there and a loose one everywhere else:

Decision-driving fields : 4,000 fields, review below 0.99 → 40% reviewed = 1,600 fields Everything else : 32,000 fields, review below 0.90 → 10% reviewed = 3,200 fields Total reviewed : 4,800 fields (13.3%) x 40 s = 53.3 hours/month Errors, decision-driving : 4,000 x 4.8% = 192. Caught at the 0.99 rate (95.0%) = 182, escaped 10 Errors, everything else : 32,000 x 4.8% = 1,536. Caught at the 0.90 rate (70.0%) = 1,075, escaped 461 Total escaped : 471 Material errors escaped : 10

Compare with the flat 0.95 threshold: 80 hours a month, 230 total escaped errors, of which roughly 11.1% — about 25 — are material.

Flat 0.95 : 80.0 hours, 230 escaped, ~25 material escapes Tiered : 53.3 hours, 471 escaped, 10 material escapes

Two-thirds of the review effort, and 60% fewer material errors escaping. The tiered design looks worse on the vanity metric — total escaped errors doubles — and is substantially better on the metric that decides whether a facility is mispriced. Publish both, and make sure your credit committee understands which one you are optimising.

How should an override be recorded, and why does the record matter more than the override?

An override that is not recorded is indistinguishable from a defect. Every override should write, atomically with the edit:

• What changed — field, prior machine value, new value, source document and page.
• Who changed it — user, role, authority level.
• When — timestamp, and the version of the extraction model or ruleset in force at the time.
• Why — a reason code from a short controlled list, plus free text. Controlled lists that run past about eight options get answered with the first one every time.
• Whether the change propagated — which ratios, covenant tests and memo paragraphs recomputed as a result.

That last element is what makes it a governance artefact rather than a log line. An examiner asking "who validated the output" — the question OCC Bulletin 2026-13 and SR 26-2 leave standing after generative AI falls outside the model perimeter — is asking to trace a number in an approved memo back through every hand that touched it. What that trace has to contain is set out in what examiner review requires from an AI-drafted credit memo and in what questions bank examiners ask about AI underwriting.

Two rules that are worth writing into policy:

Nobody needs permission to override. The moment an analyst has to raise a request to correct a machine reading, the process has two failure modes: they stop correcting, or the file stalls. Both are worse than a wrong override that is logged and later reviewed.

Machine values are never silently re-applied. If a re-run of extraction would overwrite a human correction, the human correction wins and the conflict is surfaced. Systems that quietly re-extract over an analyst's edit lose the credit team permanently, and they deserve to.

How does override data feed back into the system?

The override log is the only labelled dataset you will ever get for free, and most lenders never read it. Four uses, in order of how quickly they pay:

1. Recalibrate thresholds quarterly. Override rate per field class tells you where the tight threshold is wasted and where the loose one is bleeding. This is the table above, rebuilt from live data instead of a sample.
2. Find the systematic defect. If 60% of overrides on one field come from one document format, that is an extraction fix, not a reviewer-training problem. A handful of format-specific defects usually account for most of the volume.
3. Publish the override rate as a metric on the tool, not on the analyst. The moment override rate becomes an analyst performance metric, it goes to zero and you lose the signal along with the safety.
4. Watch the overturn rate. Of overrides sampled and re-checked by a senior reviewer, what proportion were themselves wrong? A rising overturn rate means reviewers are pattern-matching rather than checking — the earliest available warning of rubber-stamping.

What are the failure modes?

Alert fatigue. A queue of 400 items a day gets triaged by scanning, not reading. The tell is a resolution-time distribution with a spike under five seconds. Fix it by tightening on materiality rather than loosening globally — the tiered arithmetic above is the mechanism.

Rubber-stamping. Accept is one click, override is a form, and the analyst has eleven files to close. This is a design defect, not a discipline problem, and it is measurable: track the accept rate on flagged fields. An accept rate above roughly 90% on a queue that was supposed to be uncertain means either your thresholds are far too loose or nobody is looking.

Thresholds so tight the queue becomes the new bottleneck. The 0.99 row above: 40% of every field reviewed by a human, which is not automation with review, it is manual work with extra steps. If your exception rate is above roughly 20% of fields, you have not deployed a tool, you have hired a proofreader.

Reviewing everything at go-live "to build trust". Understandable and self-defeating. It produces a fatigued team, no threshold data worth having, and a rework spike that gets read as a quality failure. Review a stratified sample intensively instead, and tighten from evidence.

Confidence scores nobody has calibrated. A 0.94 that does not mean 94% is a number the whole design rests on and no one has checked. Build the band table before go-live, not after the first finding. If you are building the extractor yourself rather than buying one, calibration is your job too — the cost of that is worked through in build vs buy for credit analysis AI.

How do you build the override workflow? Six steps

1. Label a sample. Take 30 recent files, have a senior analyst mark every material field as correct or incorrect against the source document, and record the machine's confidence for each. This is two days of work and everything downstream depends on it.
2. Build the band table. Fields, share, error rate and errors per confidence band — the first table in the worked example above.
3. Classify fields by materiality. Which 15 to 25 fields drive DSCR, leverage, drawing power and the risk grade? The ratio definitions worth pinning down first are in credit analysis ratios.
4. Set two thresholds, not one. Tight on the material set, loose elsewhere. Compute the review hours and the material escape count for both, and have the credit head sign the trade-off.
5. Design the screen against a stopwatch. Time ten reviewers on ten exceptions. If the median is above 60 seconds, fix the screen before you touch the thresholds.
6. Instrument the log and read it quarterly. Override rate by field, by format, by reviewer; accept rate on flagged fields; overturn rate on a sampled re-check.

FAQ

How should analysts review AI-generated credit analysis?

By exception, not by re-performance. The machine flags the fields it is unsure about, the ones that disagree across sources, the ones that break a policy rule and the ones that drive the decision. The analyst resolves those with the source page on screen beside the extracted figure, and leaves the rest alone.

What should trigger a mandatory human review?

Four things: a confidence score below the threshold for that field class, a disagreement between two sources that exceeds your tolerance, any deterministic policy exception, and any field that drives DSCR, leverage or the risk grade regardless of confidence. Policy exceptions should never be suppressible by a high confidence score.

How are overrides recorded for audit?

Every override writes the field, the prior machine value, the new value, the source page, the user and role, the timestamp, the model or ruleset version in force, a reason code, free text, and what recomputed downstream. That record, not the override itself, is the artefact an examiner will ask to see.

Should analysts need approval to override the machine?

No. If correcting a machine reading requires a request, analysts stop correcting or the file stalls, and both are worse than a logged override that turns out to be wrong. Sample and re-check overrides afterwards instead — that gives you the safety without the bottleneck.

What is a reasonable exception rate?

Somewhere between 5% and 15% of material fields for most commercial credit files, though it depends entirely on your document mix and your thresholds. Above roughly 20% you are doing manual work with extra steps; below about 3% you should check whether anything is being caught at all.

How do you know reviewers are actually reviewing?

Track three numbers: the accept rate on flagged fields, the distribution of resolution times, and the overturn rate when a senior reviewer re-checks a sample of overrides. A cluster of resolutions under five seconds and an accept rate above 90% together mean the queue is being scanned, not read.

Does a human-in-the-loop design satisfy regulators?

It is necessary, not sufficient. Article 14 of the EU AI Act requires the ability to disregard, override or reverse the output for high-risk systems, which includes creditworthiness assessment. In the US, model risk guidance now places generative AI outside its scope, which shifts the question to general internal control — who reviewed the output, and can you show the record.

Does the override queue slow underwriting down?

It should not, if it is built on exceptions rather than documents and the screen is designed properly. A reviewer clearing eleven flagged fields at forty seconds each spends seven minutes. The same reviewer handed the whole file back for checking spends two hours, and that is the design most implementations accidentally choose.

How often should confidence thresholds be recalibrated?

Quarterly, from your own override log rather than a fresh labelled sample, and immediately after any change to the extraction model or a material shift in document mix. Thresholds set once at go-live drift into either an unusable queue or silent errors, and you will not know which.

Key takeaways

• Route exceptions, never documents. The unit of human work is a flagged field with its context.
• Four trigger classes — confidence, cross-source disagreement, policy exception, materiality — tuned separately. A confidence score must never suppress a policy exception.
• The reviewer screen decides the economics. At 40 seconds an exception the queue costs 80 hours a month; at 4 minutes the identical queue costs three FTEs.
• Tiering thresholds by materiality beat a single global number in the worked example: 53.3 hours instead of 80, and 10 material escapes instead of 25.
• The override record — field, values, source page, user, timestamp, model version, reason, downstream recomputation — is the governance artefact. The override itself is not.
• Publish override rate as a metric on the tool, never on the analyst, or the signal disappears.
• Watch accept rate, resolution-time distribution and overturn rate. Those three catch rubber-stamping before an examiner does.

See the audit trail an examiner would see — [book a live demo](https://yuverse.ai/yusight).