When an intelligent document processing system starts returning wrong values, the first instinct is usually to blame the underlying model. But in practice, most extraction errors we trace back to their source turn out to be layout problems, not model problems. The model is reading accurately. It is reading the wrong thing.
Understanding why requires a brief look at how field extraction actually works on document layouts that vary across vendors and formats.
How Extraction Actually Fails
A modern IDP system uses a combination of OCR to convert document pixels to text, and then a field extraction layer that uses spatial context, label proximity, and semantic signals to identify which text belongs to which field. The extraction layer is not just doing keyword matching. It is making spatial inferences: "the number to the right of the text 'Invoice Total' is probably the total amount."
This works well on documents where field positions are predictable. It breaks down in a few specific scenarios:
Multi-column layouts where numeric fields are close together
A common invoice layout has unit price, quantity, and extended amount in adjacent columns. If the column headers are small or positioned far from the data, the extraction layer can associate the wrong value with the wrong field. You get the unit price extracted as the total, or the quantity pulled as the subtotal.
This is particularly common with invoices generated by older accounting software that renders tables without clean column separation. The OCR layer reads the text correctly. The extraction layer mis-assigns it.
Rotated or skewed document scans
Documents that arrive as scans of physical paper rather than digitally generated PDFs introduce deskew challenges. A document scanned at 2-3 degrees of tilt puts spatial coordinates slightly off. Fields that should align horizontally in the extraction layer's view of the document do not. The system may associate a vendor tax ID with the adjacent address field instead of the correct label.
The fix here is not model retraining. It is pre-processing: deskewing and normalizing the document before extraction runs. Most IDP systems have this capability but it needs to be enabled, and the threshold settings matter. Too aggressive a correction on a document that is not actually skewed introduces its own artifacts.
Inconsistent label language across vendors
One vendor writes "Net Amount." Another writes "Balance Due." A third writes "Amount Owing." A fourth puts no label at all and relies on column position. All four mean the same thing. An extraction model trained on a large corpus handles this reasonably well across common field types. It struggles more with niche terminology specific to certain industries or geographies.
We are not saying that label inconsistency makes accurate extraction impossible. It means that the confidence threshold on those fields needs to be set conservatively, and that the human-in-the-loop review queue needs to include low-confidence extractions for exactly these fields rather than auto-approving everything above a fixed score.
Confidence Thresholds: The Underused Control
Every IDP system produces a confidence score alongside each extracted value. This score reflects how certain the system is that it identified the right field and read the right value. The most common configuration mistake is treating this as a binary pass/fail at a fixed global threshold, typically something like 0.85.
A better approach is field-specific thresholds. The "invoice total" field is consequential. You want a high confidence threshold for it, say 0.92, because a wrong total causes a real downstream problem. The "vendor address" field matters less for processing purposes. A lower threshold is acceptable there.
Field-specific thresholds reduce the human review queue without increasing the error rate on fields that matter. Teams that run a flat global threshold end up with one of two problems: either they set it high and review a lot of documents that did not need review, or they set it low and pass errors on important fields.
Line-Item Extraction Is a Different Problem
Header-level extraction (total, vendor name, date, invoice number) is well-solved by current IDP systems for most standard invoice formats. Line-item extraction is harder.
Line items require the system to identify table structure, associate each row across potentially variable column counts, and handle multi-line descriptions that span rows. Invoices from construction subcontractors, professional services firms, and custom manufacturers are particularly challenging because their line item descriptions are long and sometimes wrap mid-row.
The practical implication: if your use case requires accurate line-item extraction for PO matching or cost-code allocation, test your specific document corpus before committing to an extraction pipeline. Header-level accuracy on a sample set does not predict line-item accuracy on the same set. They use different extraction mechanisms and have different failure modes.
Tools like Nanonets and similar document AI platforms publish headline accuracy figures, but those are typically measured on clean, digitally generated documents. Your real vendor set, especially in distribution, construction, or any industry with many small suppliers, will include a meaningful fraction of scanned documents, fax conversions, and PDFs generated from legacy accounting software with inconsistent table rendering. Test on your actual data, not on benchmark documents.
Practical Steps to Improve Accuracy Without Retraining
Before concluding that a model needs retraining or that a different platform is required, these configuration and process adjustments address the majority of accuracy issues we see:
Enable deskew and image normalization. Most extraction errors on scanned documents disappear when pre-processing is properly configured. Check whether your pipeline is normalizing DPI (300 dpi is the standard target for accurate OCR), correcting rotation, and handling contrast before extraction runs.
Set field-specific confidence thresholds. Define which fields are consequential for your workflow and set higher thresholds specifically for those. Route low-confidence extractions on those fields to human review rather than letting them pass automatically.
Build a vendor exception list. Identify the three to five vendors whose invoices consistently produce extraction errors. Treat those as known layout variants. Whether that means custom field mapping, manual review routing, or a pre-processing step that normalizes their specific layout, addressing the top offenders by name usually solves a disproportionate share of your accuracy problem.
Log field-level rejection reasons, not just document-level outcomes. If your review queue shows "invoice failed extraction" without indicating which field triggered the review, you cannot diagnose patterns. Field-level logging lets you identify whether errors cluster on specific fields, specific vendors, or specific document types. That diagnosis determines the right fix.
When You Actually Do Need Model-Level Attention
Some accuracy problems genuinely are model-level rather than configuration-level. The clearest signal is when a field that appears consistently across your document corpus, with clear labels and consistent positioning, still extracts incorrectly after pre-processing and threshold tuning are addressed. That suggests the model has not seen enough examples of your specific field vocabulary or layout pattern to generalize well.
At that point, custom extraction field training, available in most mid-tier IDP platforms, is the appropriate next step. The training typically requires 50-200 annotated examples of the specific field, which is a meaningful investment in annotation time. It is worth doing for high-volume, high-consequence fields. It is not worth doing for edge-case fields that appear in a small fraction of your document volume.
The sequencing matters: pre-processing, then threshold tuning, then vendor-specific handling, then model training. Most teams skip to model training first because it feels like the technical solution. In practice, the operational fixes in the earlier steps resolve the majority of issues faster and with less effort.