Help for owners
The AI that reads my invoices and forms gets fields wrong
Short answer
AI extraction is probabilistic: it can read a total, date or supplier wrongly and still look confident. Errors cluster in poor scans, unusual layouts, tables and handwriting. Do not let extracted values post straight to accounts. Add checks, a review step for doubtful documents, and a measured error rate.
Does this sound like your situation?
- A total or tax amount does not match the document.
- Dates are swapped between day and month.
- The supplier name is read from the wrong line.
- Line items are merged, split or missing.
- It works on clean PDFs and fails on photos or scans.
Why does this happen?
Bad input in, bad reading out
Skewed photos, faint prints, stamps over text, fax-quality scans and handwriting are hard to read. The system guesses, and a guess can look like a clean value.
Layouts it has not seen
A supplier changes their template, or a document has multiple columns or nested tables. The system picks the wrong number for the field.
Formats that are ambiguous
A date written 03/04 or a number written with a comma as the decimal separator can be read two ways unless the country convention is stated.
The output is not checked
A model can return a reply that is well formed but wrong. OpenAI's documentation notes that a schema fixes the shape of the output, and that business rules the schema cannot express still have to be validated by your own application.
Confirmed by developers.openai.com
Nobody measures the error rate
Without a set of documents with known correct answers, 'it seems fine' is the only evidence, and bad cases only surface when an accountant finds them.
Technical detail: The model's reply breaks JSON.parse
What can I do about it today?
- Stop auto-posting to your booksIf extracted values go straight into accounting or payments, route them to a person for approval first. This is reversible and removes the main risk.
- Collect twenty documents it got wrongKeep the original file, what the system read and what is correct. Group them by supplier and type.
- Check the easy cases yourselfCompare the total, tax and date on a sample against the paper. A pattern, such as one supplier or one scan source, usually shows quickly.
- Improve the inputAsk suppliers for PDF invoices instead of photos where you can, and scan flat and straight at normal quality.
- Tell reviewers what to look atTotals, tax, dates, bank details and supplier name are the fields where a mistake costs money.
When should I bring someone in?
- Wrong amounts have already been paid or booked.
- You process a large volume and cannot review everything by hand.
- Several document types, languages or layouts are mixed together.
How do I handle it?
- Build a test set of real documentsYour examples with the correct values are the yardstick. Every change is scored against them.
- Add validation rulesLine items must add up to the total, tax must match the rate, dates must be plausible, and required fields must be present.
- Send doubtful ones to a personDocuments that fail a rule or look unusual go to a review screen with the original beside the extracted values.
- Use the format rules the tools supportWhere the provider offers a way to force a defined output structure, I use it, and I still check refusals, truncated replies and business rules in my own code.
- Report the error rateYou get a simple count of right, wrong and reviewed documents, so you can decide how much to trust it.
The price depends on what I find, so I quote in writing after a short discovery call. Tell me what is happening and I reply within 24 hours.
What do owners ask about this?
Is a better OCR the answer?
Sometimes the reading step is the weak point, sometimes the interpretation. A test set tells you which, before you spend on a change.
Can it ever be fully automatic?
For some clean, repeated document types with strong validation, a high share can pass without review. The risky ones should still be checked.
Is there a page on automating supplier invoice entry?
Yes, the help page on automating supplier invoice data entry covers the build. This page is for when the extraction is already running and wrong.
Sources
How was this page made?
I drafted this page with AI assistance from the official documents listed under Sources. Found a mistake? Tell me.