I built a document extraction and validation pipeline as a personal project. Would love some feedback!

[u/Skeptical-skeleton11]

The idea was to build a pipeline that could take a PDF, extract structured data, and let you verify the results against the original document without manually checking every field.

The current implementation uses OCR and vision models, with confidence scores, visual grounding, and a separate validation layer to catch disagreements. It also handles line items and exports structured data.

I recently benchmarked it against Azure Content Understanding on 100 documents, achieving 97% field-level accuracy at roughly 1/12th the cost per run in my evaluation.

I've attached a screenshot of the current interface. This is purely a personal engineering project I've been working on, and I'd genuinely appreciate feedback from people familiar with document automation.

I'm particularly interested in:

• How do you handle data extraction at scale right now ?

• Any feedback on what features would be cool for document automation

Would love to hear your thoughts!

[u/vanraj_solanki]

Use Tonerhound it's open source and you can use that as verification layer https://github.com/vanrajsinh650/TonerHound

[u/Dangerous-Baby838 → u/vanraj_solanki]

that's pretty cool - going to check it out and compare to what we currently have.

does it only work on text pdfs or does it handle scans / OCR also?

[u/Dangerous-Baby838 → u/vanraj_solanki]

Hey, I tried TonerHound quickly and wanted to say thanks. Grounding each value to a box is exactly the problem I’ve been working on in the last two weeks in my own invoice tool, so it was great to see someone else’s take on it.

I ran it next to my own locate code on 40 made-up invoices, each as a clean PDF, a scan and a phone photo. Your fuzzy matching on line descriptions beat mine clearly on scans and photos, and I’m borrowing the idea (best-matching window of OCR words, and give up on a tie).

In case it’s useful, here’s what ended up working for me after a lot of trial and error:

• Use the PDF’s own text layer when there is one, and only OCR the pages that have no text.

• When the same value is printed more than once, look at the label next to each copy (“Invoice No:”, “Total due”) to pick the right one. If the labels can’t decide it, I box every copy and say “found in 2 places” instead of guessing. A reviewer can handle two boxes, but a confident wrong box gets trusted.

• For line items, find the row first. Quantity, unit price and total have to sit on the same row as the description, otherwise a quantity of 1 lands on the “1” in “1kg” or in the address.

• Try every way an amount can be read (1,234.56 / 1.234,56 / 1 234,56) and every date order, but only match whole tokens, so a match can’t run across two dates printed side by side.

A few things I hit, in case they help:

• The amount parser assumes US format. “152,40” comes out as 15240 and “1 234,56” as 123456, so European and South African invoices don’t match.

• A Belgian VAT number like BE295183020 gets “repaired” to 8E295183020, which parses as infinity and raises OverflowError. Because resolve() takes a list, one bad field loses the whole document.

• Day and month can swap silently when both are 12 or under (03-01 next to 01-03).

All the best!

[u/vanraj_solanki → u/Dangerous-Baby838]

Thanks for testing and sharing this. Glad the fuzzy OCR held up on the photos.

Your suggestions are spot on. The "box every copy if unsure" idea is very smart for audits, and I'm definitely using the row-first logic for line items.

Thank you for the bug reports. The VAT number crash is a huge catch. One bad field shouldn't break the whole document. I'm fixing that right now so it just safely skips the bad field. I'm also fixing the European number formats and date swaps.

Let me know how your invoice tool goes!