How Accurate Is AI at Extracting Data From Financial Statements? A Benchmark Study

Published benchmarks put top systems near 96 overall on OmniDocBench, 0.967 ANLS on DocVQA and 95 TEDS on table structure. None of those numbers describes a credit file. They are measured on academic papers and biomedical journal tables, with metrics that barely penalise a lost decimal point. Below: what each benchmark measures, and why the score does not transfer.

YuSight publishes 95.2% extraction accuracy against a manual benchmark, and the honest reading of that number is at the end of this post — after the evidence that makes every such number, ours included, hard to compare.

Key facts

• YuSight measures 95.2% extraction accuracy against a manual benchmark — a field-level figure against analyst-produced ground truth, not against any public dataset. It is not comparable to a TEDS or an ANLS, and the section at the end of this post says exactly what it does and does not cover.
• The general document benchmark is nearly saturated, on the wrong corpus. OmniDocBench's leaderboard shows top systems scoring above 95 overall with table TEDS above 92 (OmniDocBench, OpenDataLab). The benchmark spans 1,651 PDF pages across 10 document types — of which financial and research reports are one type among academic papers, textbooks, magazines, newspapers, slides, exam papers and handwritten notes.
• The dominant table metric under-penalises exactly the errors that matter in credit. Horn & Keuper find TEDS correlates with human judgement at only r = 0.68 (their LLM-based alternative reaches r = 0.93), and state that "string-based metrics consequently assign low scores driven by harmless representational variation, while the few-character errors that fundamentally alter the table's meaning are barely reflected" (arXiv:2603.18652, March 2026).
• On finance-specific tasks the numbers collapse. In FinBalance, which asks models to build a cited double-entry ledger and balance sheet from source-document bundles, the highest exact-balance-sheet score across six models was 46% (Tumpati et al., arXiv:2606.15949, June 2026). In FinSheet-Bench, the best model scored 82.4% overall but fell to 48.6% on the largest spreadsheet, against 86.2% on simple files (Ravnik et al., arXiv:2603.07316, March 2026).
• Reading and calculating fail at completely different rates. FAITH, built on 2,406 answerable spans across 453 S&P 500 companies' annual reports, reports the best model at 97.0% on direct lookup and 80.0% on multivariate calculation, with most models "at or near 0.0%" on multi-step numerical reasoning (Zhang et al., arXiv:2508.05201, August 2025).
• Architecture beats model size on this task. A staged pipeline — pre-processing, multilingual OCR, section retrieval, then a compact vision-language model — reached 8.8x higher field-level accuracy than passing whole documents to a large VLM, at 0.7% of the GPU cost and 92.6% lower end-to-end latency, on SMB financial documents (Jin et al., arXiv:2510.23066, October 2025).

What do the published benchmarks actually measure?

Nine benchmarks get quoted in this market. Here is what each one is, on what corpus, with what metric — and the specific reason its score does not carry over to a credit file.

Two things stand out from that table.

First, the benchmarks that are nearly solved are not measuring your problem. OmniDocBench, DocVQA, PubTabNet and olmOCR-Bench measure whether a page can be read and laid out. That capability is largely there. It was never the hard part of spreading.

Second, the benchmarks that *are* measuring your problem are not close to solved. FinBalance's 46% ceiling and FinSheet-Bench's collapse from 86.2% to 48.6% as spreadsheet complexity rises are the numbers a credit team should be looking at, and they are two orders of magnitude further from "solved" than a 95 on OmniDocBench suggests.

Why does a 95 on OmniDocBench not mean 95 on your credit file?

Three reasons, in descending order of size.

Corpus composition. OmniDocBench's own design admits this — it publishes six page-level attributes (PDF type, layout type, language, fuzzy scan, watermark, coloured background) and six table-level attributes (language, frame type, merged cells, coloured background, formula content, rotation) precisely so scores can be read per stratum. A blended overall is a weighted average of somebody else's document mix. If your intake is 30% phone photographs of stapled statements and the benchmark's is 3%, the headline number was never about you.

Task shape. Parsing a page to markdown and producing a chart-of-accounts-mapped spread are different tasks separated by several steps: entity assignment, period attribution, line-item mapping, sign convention, scale, and continuation across pages. A benchmark that scores the first says nothing about the last five. OCR vs IDP vs LLM extraction walks through where each of those steps breaks.

Metric shape. This is the one nobody checks, and it is worth its own section.

What does TEDS forgive?

TEDS is 1 − (tree edit distance ÷ the larger node count of the two trees), with cell contents compared by normalised string edit distance. That construction has a property that should stop a credit analyst cold: the cost of a content error is bounded by one node, and diluted by the size of the table.

Work it through on the Horn & Keuper example — 1.12 extracted as 112, a 100x error:

• Normalised Levenshtein between 1.12 and 112: one deletion, over a maximum string length of 4 → cost 0.25
• A modest financial table — 20 rows × 5 columns plus headers and structure nodes — has roughly 200 nodes
• TEDS penalty: 0.25 ÷ 200 = 0.00125, i.e. the score falls by about 0.125 of a percentage point

A hundred-fold error in a revenue line costs about a tenth of a point of TEDS. Meanwhile, a parser that recovers every digit correctly but represents a merged header cell differently from the ground truth can lose several points. That is what the r = 0.68 correlation with human judgement is measuring, and it is why Horn & Keuper report overall scores across 21 parsers on 451 tables ranging from 2.10 to 9.55 out of 10 — "parser choice can largely determine whether extracted tables are usable" — using an LLM-based semantic scorer rather than TEDS.

The generalisation: document-AI metrics were designed for document AI, not for accounting. They optimise for representational fidelity, and they are indifferent to the sign, the decimal point and the scale — the three things that determine whether a spread is safe.

What do the finance-specific benchmarks report?

Four are worth reading in full. Here is what each one establishes.

FAITH ([arXiv:2508.05201](https://arxiv.org/abs/2508.05201)) separates reading from computing on real annual reports. Best model at direct lookup: 97.0%. Best at multivariate calculation: 90.0%, with the same-family model that led on lookup dropping to 80.0%, and one widely used model at 30.0%. Most models score at or near zero on multi-step reasoning. The lesson for spreading: never accept one accuracy number that spans reading and computing. These are different failure surfaces, which is why ratios should be computed deterministically in code from extracted operands rather than asked for in a prompt — the argument made in full in how to stop an AI credit memo tool from hallucinating numbers.

FinSheet-Bench ([arXiv:2603.07316](https://arxiv.org/abs/2603.07316)) uses synthetic portfolio data modelled on real private-equity fund structures — fund dividers, multi-line column headers, the layout pathologies of a real workbook. Twenty-four evaluation files. Best model 82.4% (roughly one error every six questions); three others between 80.2% and 80.4%. And the finding that matters: on the largest file — 152 companies across 8 funds — average accuracy fell to 48.6%, against 86.2% on simple files, consistently across every model tested. Accuracy degrades with document size, and it degrades steeply. A benchmark run on single pages will not show you this.

FinBalance ([arXiv:2606.15949](https://arxiv.org/html/2606.15949)) is the closest published analogue to spreading, because it requires citation and internal consistency, not just extraction. Models get a bundle of source documents and must produce journal entries, a balance sheet, and inconsistency classifications. Results across six models: highest BSexact of 46%, and — the finding to take to a vendor meeting — four of six models showed a 26–41 percentage-point gap between the balance sheet they reported and the one obtained by replaying their own journal entries through a deterministic ledger. A system can produce an internally inconsistent answer and present it confidently. That is the case for computing the output from the cited operands rather than asking for the output.

FinVerBench ([arXiv:2605.29586](https://arxiv.org/html/2605.29586v1)) goes at verification: given a set of statements from SEC 10-K XBRL filings for 43 S&P 500 companies, can a model tell whether they are internally consistent? Under the original guided-checklist prompt on unrounded data, nine of fourteen model runs produced 95–100% false positives on clean statements. On a realistic rounded variant, a calibrated configuration achieved 79.0% recall at a 0% false-positive rate, versus 100% recall on the unrounded variant. The paper's own conclusion is the useful one: rendering choices — rounding, presentation — materially move measured performance, and the task is "calibrated judgment under incomplete observability" rather than arithmetic checking.

Field-level versus document-level: the compounding arithmetic

Every number above is a per-item score. A spread is not one item. Take three years of balance sheet, P&L and cash flow with the notes you actually use — call it 90 fields.

Under naive independence, the probability a spread comes out with zero errors is p⁹⁰:

The working for the 99% row: ln(0.99) = −0.010050; × 90 = −0.904530; e^(−0.904530) = 0.4047. So 40.5%.

Two conclusions, and they pull in opposite directions.

Independence flatters the pessimism. Extraction errors cluster — one bad scan produces eight errors in one document while thirty others come out clean — so real document-level accuracy is materially higher than p⁹⁰, and no formula gets you from one to the other. The clustering factor is a property of your files.

But benchmark scores are not field-level accuracy either. A TEDS of 94 is not "94% of cells correct"; as shown above, it is a structure-weighted similarity in which a 100x numeric error costs a tenth of a point. An ANLS of 0.967 is not "96.7% of answers right"; it is average fuzzy string agreement. Neither can be raised to the 90th power and turned into a document-level expectation. Anyone who does that arithmetic on a benchmark score is compounding a metric that was never a probability. The full treatment of what field-level and document-level accuracy do and do not predict is in extraction accuracy vs straight-through rate.

Why vendor accuracy claims are not comparable to each other

Benchmarks publish six things. Vendor claims publish none of them. That asymmetry is the whole reason a vendor number and a benchmark number cannot be placed on the same axis, and why two vendor numbers cannot be placed on the same axis either.

There is also a structural problem worth naming: several of these benchmarks are now saturated or near-saturated, and a saturated benchmark stops discriminating. The gap between 95.2 and 96.3 on OmniDocBench tells you far less than the gap between 46% and 60% on FinBalance would. When you evaluate vendors, ask which benchmark they report and what its headroom is.

A protocol for generating your own number

The eight-step vendor bake-off protocol is set out in extraction accuracy vs straight-through rate. This is the complementary job: building the benchmark itself, so the number you generate has the properties the published ones do.

1. Define the unit before you define the metric. Decide whether you are scoring characters, fields, spread lines or complete spreads. Write it down. Every subsequent argument traces back to this choice.
2. Build a stratified corpus from your own book — 300 to 500 documents. Stratify on the dimensions that actually move the score: native PDF vs clean scan vs photograph; audited vs provisional; single-entity vs group; language and script; number of distinct layouts. Record the stratum counts. You will report per stratum and never blend.
3. Establish the human ceiling first. Two experienced analysts spread 30 of the documents independently. Their disagreement rate is the maximum achievable score. Publish it alongside every vendor result. Horn & Keuper's α = 0.77 among annotators, and FinBalance's double-review design, exist for exactly this reason.
4. Write the normalisation rules before the test. Sign conventions, scale, currency, period labelling, chart-of-accounts mapping, what counts as one field. Otherwise every scoring dispute becomes a definitional argument settled after the results are known.
5. Score reading and computing separately, as FAITH does. Run a direct-lookup set (is this cell's value correct) and a derived set (is DSCR correct given correct operands). Reporting one blended number hides the fact that these fail at completely different rates.
6. Add an internal-consistency replay, as FinBalance does. Take the system's own extracted operands and recompute its own reported totals in code. Any gap between what it reported and what its inputs imply is a defect the accuracy score will not show — and it was 26 to 41 points wide for four of six models in the published benchmark.
7. Stratify by document size, as FinSheet-Bench does. Score short documents and long documents separately. A 48.6%-versus-86.2% split by size is invisible in a blended average, and your worst files are your long ones.
8. Re-run quarterly against a frozen set. Models change under you. A number generated once is a number about a system that no longer exists.

Then run the same protocol against your incumbent manual process. It has an error rate too, and almost nobody measures it.

Where does our own 95.2% sit?

YuSight states 95.2% extraction accuracy, validated against a manual benchmark. Placed against everything above:

• It is field-level, on a manual benchmark — figures spread by analysts, compared field by field against platform output. It is a comparison against human work rather than a public dataset, which makes it more relevant and less reproducible than a benchmark score.
• It is not comparable to a TEDS, an ANLS or an OmniDocBench overall, because those are not field-level accuracies. Nor should it be compared to another vendor's percentage without both parties disclosing the six columns in the table above.
• What we do not publish alongside it, and you should ask for: the field list and count, document mix by type and scan quality, market split, abstention rule, and the corresponding straight-through and escaped-error rates.
• What it does not measure at all: entity assignment, period attribution and chart-of-accounts mapping — the steps between reading a page and having a usable spread. Those are handled by classification and validation upstream (see multi-entity document mapping) and by deterministic checks downstream, and they need their own tests.

The defensible position, applied to us as much as to anyone: a single accuracy percentage without a stated corpus, metric definition, field list and abstention rule is marketing. The number that should decide a purchase is the one you generate on your own files. And the property that makes a spread safe is not the percentage at all — it is that every figure is citable back to its page and every total is re-derived in code, so the errors that remain are visible. That argument is made in full in financial spreading with citations.

FAQ

How accurate is AI at extracting data from financial statements and bank statements?

On reading a clean page, very accurate — top systems are above 95 on general document-parsing benchmarks. On turning that into a correct, mapped, internally consistent set of financials, much less so: the closest published benchmark to that task, FinBalance, reports a best exact-balance-sheet score of 46%. Reading is largely solved; spreading is not.

How is extraction accuracy measured?

It depends entirely on the benchmark. TEDS compares table trees, ANLS compares answer strings with fuzzy matching, CDM scores formulas, BSexact requires an exact balance sheet match. These are not interchangeable and none of them is a simple "percentage of numbers read correctly", which is why a vendor percentage and a benchmark score cannot be compared.

Which benchmark is closest to what a credit team actually does?

FinBalance, because it asks for a cited ledger and balance sheet built from a bundle of source documents and checks internal consistency. FAITH is the best decomposition of reading versus calculating on real annual reports. Neither uses scanned SME accounts, which is the gap for lenders.

Why is TEDS a bad metric for financial tables?

Because a content error is diluted across the whole table. Extracting 1.12 as 112 — a hundred-fold error — costs roughly a tenth of a percentage point on a mid-sized table, while a benign difference in how a merged header is represented can cost several points. That is the mismatch behind its 0.68 correlation with human judgement.

What role does AI play in modern tax return spreading?

Reading and classifying, mostly. Fixed government forms are the one case where deterministic template extraction still wins outright — the layout is issued, not authored. The work AI adds is routing, cross-document reconciliation and flagging the mismatches, not re-reading a form that a template already reads at very high precision.

Does accuracy get worse on longer documents?

Yes, and steeply. FinSheet-Bench found average accuracy of 86.2% on simple files and 48.6% on its largest — 152 companies across 8 funds — consistently across every model tested. If your benchmark uses single pages, it will not surface this, and your real files are the long ones.

Should I believe a vendor's published accuracy number?

Believe that they measured something. Without the corpus, the metric definition, the field list, the ground-truth source and the abstention rule, you cannot tell what. Ask for all five, ask what the human ceiling on their ground truth was, and treat any refusal as the answer.

What accuracy is good enough for a credit decision?

There is no threshold, because accuracy is not the safety property. What makes a spread safe is that errors are caught: footing and balance checks in code, cross-statement ties, and every figure citable to a page. A 96% system that flags its own doubts beats a 99% system that quietly makes an inconsistent column foot.

How big should my own test set be?

300 to 500 documents, stratified to mirror your real intake including scan quality and document length. Size matters less than stratification — a 1,000-document sample of clean native PDFs will give you a confident number about a population you do not lend to.

Conclusion

Three things to take away:

1. The benchmarks that are nearly solved are measuring page reading, not spreading. OmniDocBench, DocVQA and PubTabNet run on academic papers, industry archives and biomedical journal tables. The finance-specific benchmarks — FAITH, FinSheet-Bench, FinBalance, FinVerBench — report much lower numbers, and those are the ones describing your problem.
2. The metrics forgive the errors credit cares about. TEDS penalises a 100x numeric error by about a tenth of a point and correlates with human judgement at r = 0.68. ANLS gives partial credit for a near-miss. Neither is a probability and neither can be compounded into a document-level expectation.
3. Generate your own number, with a stated corpus, a stated metric, a human ceiling and a consistency replay. Score reading and computing separately, stratify by document size and scan quality, and re-run quarterly.

YuSight's Document Intelligence classifies and validates tax returns, bank statements, financial statements and bureau reports, maps each to the right borrower entity, and hands cited figures to financial spreading, where ratios are computed in code from traceable inputs. Extraction accuracy is measured at 95.2% against a manual benchmark, at field level — and the point of this entire post is that you should test that on your own documents rather than take it from this page. For a like-for-like vendor comparison run on your files rather than anyone's brochure, see Ocrolus vs Docsumo vs YuSight.

Upload a messy document set and see it classified — book a live demo.