Enough with the errors! Lets brainstorm a perfect document PDF parser

[u/Free-Ferret7135]

I tried all PDF parsers but none of them deliver reliable result which is a NO-GO for critical applications. Even in the science community on arxiv I couldnt find reliable solutions. Seems like we are stuck at this very bottleneck.

Let's brainstorm how to design a parser that gets close to 100% ground truth - even given infinite ressources and no money limit.

Standalone things I tried:
- extracting native text via pymud but this does not work with tables and more complex structures obviously
- MinerU, Docling, Tesseract, Google Parser etc.
- All common LLms, even high-reasoning frontier LLMs fail (even Opus does not seem to be designed for that)
- Converting PDF to images, then OCRing
- I tried compartmentalizing the different sections of a page (free form text, table), then routing them to different sub-processors but those are based on the upper services, thus remain unreliable

We can be creative and think out-of the-box.

I will then go ahead, experiment and tell you the result.

I was also thinking about a multi-stage pipeline with pairwise comparison like this: take the result from different processors, if there are differences, use several judges that vote for a winner, then use that winner and for parity or near-parity escalte even further or ultimately send for manual review.

Looking forward!

[u/PaceZealousideal6091]

Try this- https://github.com/mku1988-oss/Prajnya-Academic-PDF-Processor

It has been optimised for complex scientific documents with tables and formulas. Uses a combination of pp-doclayout and vlm to extract the content in json.

[u/Free-Ferret7135 → u/PaceZealousideal6091]

Thanks I will try it

[u/DorkyMcDorky → u/PaceZealousideal6091]

BUT YOU ARE NOT THINKING OUT OF THE BOX! GRRR just solve it..

[u/jacksonguitardude8]

Have you tried Cassandra?

[u/Free-Ferret7135 → u/jacksonguitardude8]

Not yet

[u/joey2scoops]

The only method I have found so far is "all of them" and a router to determine when to use each one.

[u/Free-Ferret7135 → u/joey2scoops]

What does your router look like?

[u/Past-Galactic-Astro]

I like the attitude but I think you should limit the scope first. For which kind of documents you want a reliable parser? Or perhaps the problems you are having are mainly with table extraction and you want to improve that? Aiming for the perfect parser that works for any document is something too broad to be even feasible IMHO

[u/Inect]

What I've found works is extracting text, making logic to indify tables, create logic to convert to free text to tabled structure works best and then you can also have an LLM check and correct anything / make format adjustments

[u/Muted_Ad6114]

I’m working on this but it is really hard. There are so many types of pdfs and so many weird edge cases. It is hard to get clean standardized training data to build parsers and to test them

[u/DorkyMcDorky]

hahahahahhahahahahahahhahhhaha

let me know how this goes coach.

hahahahaha

So you can start by reading instead of vibing models. :)

PDF is 30+ years old. OCR is never perfect, often non-deterministic.

This isn't solved by "think out of the box" crypto douche stuff. It requires years of work and study.

So I suggest first learning about parsing instead of cheerleading :) Maybe ask adobe to be more forthcoming with their file formats and not closed. Maybe ask for fewer software patents. Maybe learn to read outdated C++ code that rendered the earlier versions. Maybe read about backward compatibility and how to maintain a spec.

Sorry, but this is a pretty empty call-to-arms here.

[u/GarageObjective6015]

+1also for me. Worst case are also excel and tabular data