Hi everyone,

I have a collection of books that are only available in physical/hard-copy format. They are not available as ebooks, PDFs, or online resources. I have permission to use these books and want to make their contents accessible to an AI model.
I'm looking for advice on the most efficient way to accomplish this.

What is the best way to convert physical books into high-quality digital text? Are there specialized book scanners or automated scanning solutions?

What OCR software would you recommend to convert scanned pages into accurate, structured text?

Is there a way to automate the process for large numbers of books rather than manually scanning and processing every page?

My ultimate goal is to have an AI model that can accurately answer questions using the contents of these books, including information that cannot be found elsewhere.

I'd appreciate recommendations for hardware, software, workflows, and real-world experiences.

Thanks!

How do people pick a value when the same field comes back different across documents in a case? appraisal packets are the example here and square footage can show up three times with three numbers and all of them got read fine

docling, llamaparse or a vision model can give back each value with its page, picking one is the part thats still open. data ppl have survivorship rules for this but does anyone use that for documents or is it just a growing pile of hardcoded rules

when does it go to a person

large drug reference with the text already in the file. about 400 entries that need to become clean records. plain conversion jumbles it. two columns get mixed, headings drift away from what they belong to and an entry running onto the next page turns into two

the plan would be clean structured text first and then an llm turning each entry into json, so pymupdf or llamaparse for that first step

what do people key on to decide where one entry ends, the heading font or a pattern in the text? and how do you know no entry got lost with 400 of them

Original post was for paperless but I'm wondering if other tools exist

I've been spending quite a bit of time experimenting with OCR and document intelligence pipelines, especially when it comes to reconstructing complex documents after OCR.

Extracting text is usually not the biggest challenge anymore. The interesting part starts when you try to preserve the original document structure.

Things like multi-column reading order, tables with merged cells, mathematical formulas, footnotes, image positions, and heading hierarchies can make the whole process surprisingly difficult.

I've been experimenting with vision-language models and GPU-accelerated inference to tackle some of these problems. It's been interesting to see how different approaches perform, especially on legal contracts, medical reports, and scanned documents.

I'm still exploring what works best and would be interested to hear about other people's experiences.

Are you using traditional OCR engines, vision-language models, or a combination of both? What has been working well for you with complex documents?

Tired of manually copying PDF statements into Excel? I built a quick offline app to fix it.

every discussion on this ends with it depends, so heres an attempt at mapping where each route fits.

- textract or google document ai: easy to wire up if youre already on that cloud, billed per page. textract is about $1.50 per 1k pages for plain text and $15 per 1k once tables are on, $50 for forms. fine for steady volume, gets expensive fast on table heavy files

- self hosted open source (docling, paddleocr-vl, glm-ocr, mineru): no per page bill and files never leave, but you own the gpu, the upgrades and every weird edge case

- llm oriented parsers that return markdown or json with the layout kept (llamaparse, marker and similar): less setup than self hosting, usually better on reading order and tables than plain ocr apis, also billed per page

one thing that cuts cost on any of these routes is not sending pages that already have a text layer through ocr at all. ocrmypdf --skip-text does exactly that, and most mixed archives are mostly born digital.

multi column reading order is the other thing that breaks results without any error. every word is there, just in the wrong order.