How Do You Stop an AI Credit Memo Tool From Hallucinating Numbers?
You do not stop it by prompting. You stop it architecturally: extract figures from the source document rather than generating them, attach a page-level citation to every number, compute ratios in code instead of in the model, and reconcile across independent sources. The model drafts language. It never invents a figure.
That distinction is the whole answer, and it is the only version of the answer a credit committee or an examiner will accept. YuSight's position on this is deliberately narrow: 100% of figures in a YuSight CAM carry a citation with one-click source verification. Not that the system cannot be wrong — that every figure it produces is checkable in one click, by a reviewer who does not trust it.
Key facts
- In the FAITH benchmark of intrinsic tabular hallucination on S&P 500 annual reports, the best-performing model scored 97.0% on direct lookup but fell to 80.0% on multivariate calculation; several models scored 0% on multi-step numerical reasoning (Zhang et al., 2025). Extraction and arithmetic fail at completely different rates.
- SR 26-2 / OCC Bulletin 2026-13 (17 April 2026) superseded SR 11-7 and states plainly that "Generative AI and agentic AI models are novel and rapidly evolving. As such, they are not within the scope of this guidance" (Federal Reserve, OCC). There is no safe harbour. Your own controls are the control.
- A staged extraction pipeline — image pre-processing, OCR, then a compact vision-language model — achieved 8.8x higher field-level accuracy than feeding the whole document to a large VLM in one pass (Jin et al., 2025). Architecture beats model size on this task.
- At 98% field-level accuracy across 40 spread fields, only 44.6% of documents come out entirely clean (0.98^40). Per-field accuracy and per-memo accuracy are not the same number, and vendors quote the flattering one.
- YuSight's CAM Generation module produces a memo where every figure is traced to source document and page, spreads stay analyst-editable, and full version history records who changed what.
Why do language models invent numbers in the first place?
A large language model predicts the next token given the tokens before it. That is the entire mechanism. There is no separate arithmetic unit, no lookup table, no internal check that a number it is about to emit corresponds to anything in the world.
The consequence matters more than the mechanism. When the model writes "₹4.20 crore", it produces that string by the same process it uses to produce "the borrower operates three plants" — by selecting what is statistically plausible in context. A correct figure and an invented one are generated identically. The model has no internal signal distinguishing them, which is why it reports both with the same fluent confidence. Asking it how sure it is returns another prediction, not a measurement.
Two properties follow, and both are load-bearing for credit work.
Plausibility is the failure mode, not absurdity. A hallucinated figure is not random. It sits in the right order of magnitude, matches the format of surrounding numbers, and often makes the spread more internally consistent than the truth. That is exactly what makes it survive review.
Arithmetic degrades with steps. The FAITH results show the pattern clearly: reading a number off a table is close to reliable; combining several numbers into a derived figure is not. A ratio is a multi-step numerical operation. Asking a model to compute DSCR is asking it to do the thing it is worst at.
What are the failure modes that actually bite in credit work?
Seven, in rough order of how often they reach a memo.
1. Outright invention. A figure that appears nowhere in the file. Contingent liabilities are the classic: the notes to accounts disclose them in prose, the model is asked for a number, and it produces ₹1.80 crore because a number was expected there.
2. Wrong-entity attribution. A real figure, correctly read, from the wrong company. A borrower group files audited accounts for the operating company, a property-holding SPV and a trading arm. Turnover of ₹142 crore is genuine — it belongs to the trading arm, and the facility is to the operating company at ₹38 crore.
3. Wrong-period attribution. A real figure from the adjacent column. Comparative FY24 figures sit inches from FY25 in every audited P&L. A model reading a two-column table under time pressure picks the wrong one, and the memo now trends FY23–FY24–FY24.
4. Correct extraction, wrong derivation. Every input right, the output wrong. The model reads PAT, depreciation and interest correctly, then computes DSCR with finance costs in the denominator instead of interest on term debt only — or simply adds wrong. This is the mode a citation does not catch, because each cited input verifies clean.
5. Fabricated citation. "Source: audited financial statements, page 47." Page 47 is the auditor's signature page. A citation generated as text is as inventable as the figure it supports. Only a citation produced by the extraction step — a page and a coordinate box — is evidence.
6. Silent unit and scale error. The statement header reads "₹ in lakhs". The system treats it as millions. Every figure inflates 10x, uniformly, so every ratio still reconciles and nothing looks wrong. The same happens with thousands separators in Gulf and European formats: 1.234.567 read as 1.234.
7. OCR misread that propagates cleanly. A scanned bank statement renders 8 as 3, or drops a digit from a closing balance. Downstream everything is arithmetically correct on top of a wrong input. Average bank balance, utilisation percentage, turnover reconciliation — all consistent, all wrong.
Modes 2, 3, 6 and 7 are the dangerous ones, because each produces a memo that is internally coherent. Coherence is what reviewers use as a proxy for correctness, and these four defeat it.
Worked example: how one wrong figure flips a DSCR
A mid-market manufacturer. Term loan renewal. Covenant threshold: DSCR ≥ 1.35x.
DSCR = (PAT + Depreciation + Interest on term debt) ÷ (Interest on term debt + Scheduled term debt repayment)
As drafted — PAT pulled from the FY24 comparative column instead of FY25:
Line | Figure (₹ cr) |
|---|---|
PAT (FY24, wrong period) | 4.20 |
Depreciation (FY25) | 2.60 |
Interest on term debt (FY25) | 1.85 |
Scheduled repayment (FY26) | 3.60 |
Numerator = 4.20 + 2.60 + 1.85 = 8.65 Denominator = 1.85 + 3.60 = 5.45 DSCR = 8.65 ÷ 5.45 = 1.59x → passes, with headroom
Corrected — PAT FY25 as audited, page 14:
Numerator = 1.95 + 2.60 + 1.85 = 6.40 Denominator = 5.45 DSCR = 6.40 ÷ 5.45 = 1.17x → fails the covenant
One line item, moved one column. Nothing else in the memo changes. The ratio table still foots, the commentary still reads well, and the recommendation is wrong. A reviewer catches this only by opening page 14 — which is a five-second act if the figure is cited, and a forty-minute act if it is not.
Which controls actually work, and which only sound like they do?
Ordered by strength.
- Extraction, not generation. The number in the memo is lifted from the document by a component whose only job is to locate it. The language model narrates around figures it is handed; it is never asked to supply one.
- Page-and-coordinate citation on every figure. Produced by the extraction step, not written by the model. A reviewer clicks the number and lands on the highlighted region of the source page.
- Deterministic computation. DSCR, TOL/TNW, interest coverage, working capital cycle — all computed in code from cited inputs, with the formula visible. The model does no arithmetic.
- Cross-source reconciliation. Declared turnover against bank statement credits against GST/VAT returns. Bureau outstandings against the borrower's own debt schedule. Disagreement is the signal; it surfaces as an exception, not a silent averaging.
- Confidence thresholds with human routing. Low-confidence fields — poor scans, handwriting, unusual layouts — are flagged and queued for an analyst rather than filled in with a best guess.
- Diff and version history. Every change to a spread or memo is recorded with author and timestamp. A reviewer sees what the system produced, what the analyst changed, and why.
Against that, the controls that do not work:
Claimed control | Why it fails |
|---|---|
"We prompt it to be accurate" | Instructions bias output distribution; they do not create a verification step. The model still has no way to know a figure is wrong. |
Temperature 0 | Removes sampling randomness, so the same wrong answer comes back every time. Deterministic ≠ correct. |
Model self-checking | The check is generated by the same mechanism that produced the error, and inherits the same blind spot. It reliably approves its own hallucinations. |
"99% accurate" with no denominator | Accurate at what — field, document, memo? On which document mix? Against whose ground truth? Without a denominator the number is unfalsifiable. |
Human review as the only control | Reviewers approximate correctness by coherence. The four worst failure modes produce coherent output. |
What should a lender test in a proof of concept?
Do not accept the vendor's benchmark. Build your own, on your own files.
Sample. 30–50 borrower files minimum, stratified to your actual mix: clean audited PDFs, scanned and photographed statements, at least one multi-entity group, at least one file with a qualified audit opinion, and your two most common regional statement formats.
Ground truth. Two analysts spread the same files independently. A third resolves disagreements. Freeze the result before the vendor sees anything. Where your two analysts disagree, that field is ambiguous — record it as such rather than forcing a call.
Two accuracy numbers, always. Field-level accuracy is the vendor's number. Document-level accuracy — the share of files with zero errors — is yours. At 40 fields and 98% per field, 44.6% of documents are clean and 55.4% contain at least one error. Both statements are true simultaneously. Ask which one the vendor quoted.
Weight by consequence. An error in a covenant input matters more than an error in a segmental note. Score the fields that reach a ratio separately.
Test the tail deliberately. Feed the worst scan you have. What matters is not whether the system gets it right — it is whether the system knows it did not, flags the field, and routes it to a person. A tool that fails loudly is safer than one that is right more often but fails silently.
Where does YuSight actually stand on this?
Plainly: YuSight can make mistakes. Any system reading a photographed bank statement can. What YuSight will not do is put a figure in a memo that a reviewer cannot check.
Figures are extracted and traced to source document and page. Every figure in a YuSight CAM carries a citation with one-click source verification. Ratios are computed from those cited inputs. Spreads remain analyst-editable, and version history records every change. Document Intelligence maps documents to the right borrower entity in multi-entity structures — the control that addresses failure mode 2 directly. The Repayment Tracker compares bank statement conduct against bureau data, which is reconciliation, not a second opinion from the same model.
The claim is checkability, not infallibility. On this topic, any stronger claim is a reason to distrust the vendor making it.
Related reading
FAQ
How do I stop an AI credit memo tool from hallucinating numbers?
Require that every figure comes from an extraction step with a page-level citation, that ratios are computed in code rather than by the model, and that low-confidence fields route to a human. Prompting and temperature settings are not controls.
Can AI write a credit memo?
It can produce a complete, structured, well-evidenced draft in well under an hour. It cannot make the credit decision, and it should not be the last thing that touches the memo before committee.
How accurate is AI at extracting data from financial statements?
On clean tabular data, leading models read individual figures at roughly 92–97% accuracy in published benchmarks; accuracy falls sharply on derived calculations and on poor scans. Insist on a per-field number, a per-document number, and the document mix behind both.
Does credit memo software replace underwriter judgment?
No. It removes the extraction, spreading and drafting hours. Purpose, structure, risk-mitigant reasoning and the recommendation itself remain judgement calls that carry a named person's accountability.
Does AI write the credit memo, or does the analyst?
Both, in sequence. The system drafts and cites; the analyst edits, challenges and signs. If your version history cannot show which parts the analyst changed, you have a governance gap regardless of accuracy.
How do we keep an auditable trail when AI drafts our credit memos?
Store the source documents, the citation for every extracted figure, the formula used for every computed ratio, and a full version history with author and timestamp. An examiner's question is almost always "where did this number come from" — you need to answer it by clicking, not by searching.
What should banks look for in AI credit analysis software?
Source-level citations on every figure, deterministic ratio computation, entity mapping for group structures, cross-source reconciliation, confidence flagging, and an audit trail that survives examiner review. Ask what happens when the system is unsure — the answer tells you more than any accuracy claim.
How often should a bank audit memos for drift?
Sample monthly during the first two quarters, then quarterly once the error profile is stable, with an immediate re-test after any model or pipeline change. Document extraction drifts when document formats change, so re-test whenever a major statement format in your book changes.
Key takeaways
- Hallucination is a property of next-token prediction, not a bug to be prompted away. A plausible wrong number is produced exactly like a correct one.
- The four failure modes that reach committee — wrong entity, wrong period, wrong scale, propagated OCR error — all produce internally coherent memos. Coherence is not evidence.
- Extraction with page-level citations, deterministic ratio computation, cross-source reconciliation, confidence routing and version history are controls. Prompting, temperature 0 and model self-checking are not.
- SR 26-2 and OCC 2026-13 explicitly leave generative AI out of scope. Your institution owns this control environment.
- Test on your own files, with your own frozen ground truth, and demand both field-level and document-level accuracy.
See your first CAM in 30 minutes — [book a live demo](https://yuverse.ai/yusight) and click any figure in it back to the page it came from.
