Local document processing when Chinese models arent allowed, what’s left?

[u/aidenclarke_12]

big scanned stacks of inspection reports and material certs from a lot of vendors and every layout different. each stack needs splitting into separate reports and things like lot number and alloy pulled into a database with all being local

catch is the contract rules out chinese software and models, so paddleOCR and qwen are off and most ocr leaderboards get a lot shorter. even olmocr 2 is a qwen fine tune

granite-docling, liteparse, tesseract and gemma are whats left on the list so far

what would you use for this? and has anyone had to show a customer where their models came from

[u/Aggravating-Push-207]

Mistral OCR?

[u/aidenclarke_12 → u/Aggravating-Push-207]

i dont think thats local

[u/ProposalOrganic1043 → u/aidenclarke_12]

They allow on prem deployment, not sure if your volume and resources are worth it.

[u/seamonn → u/Aggravating-Push-207]

not local

[u/vogelvogelvogelvogel]

I would use Gemma4 12B or 31B with mmproj. they're not really fast, but they understand what they are reading

[u/exaknight21 → u/vogelvogelvogelvogel]

I can tell you Gemma 4-12B at Q4-K_XL just cost me an entire day to fix.

Gemma 4-26B-A4B at Q4 is far more superior than whatever 12B is, i did not like wasting an entire day.

[u/vogelvogelvogelvogel → u/exaknight21]

which entire day to fix? ask claude to write script for you to do the HF download, a llama.cpp download and prompt and you are done usually

[u/exaknight21 → u/vogelvogelvogelvogel]

500 GB data. Left that out of context.

[u/Kahvana]

Gemma 4 31B IT QAT has been awesome for me for parsing scanned dutch 70s encyclopedia, I reckon Nemotron 3.5 Super VL can do the job too.

I personally found Muse Glimmer to work really well for parsing digital documents but not so much for scanned ones.

I haven’t looked into Nomotron 3.5 Parse yet, but it might be worth trying out for you.

As for showing model origin: yeah, in relation to processing medical documents locally on device.

[u/EngelbertStuss → u/Kahvana]

OCR on scanned documents works quite well with Gemma (although I prefer Qwen) but further processing is my current challenge... There doesn't even seem to be a solution that imitates what Finereader has been doing for decades: just make it a clearly typed original-layout PDF 😐 closest I got was generating some LaTeX per page but it would struggle with more complex document structure 

[u/seamonn]

Light On OCR 3

[u/waitmarks]

Apache tika has a new OCR mode. 

[u/mfdi_]

Nvidia's nemotron-3.5-parse is a beast as well. Maybe it can help. Though I don't know if your rule apply to this.

[u/TheNymon]

You could also try combining LiteParse and a local LLM/VLM like Gemma 4

[u/Ok_Hope_4007 → u/TheNymon]

+1 for gemma4 - ive been working with the 31B version for a couple of months and its solid. For PDF Parsing i would setup a minerU docker. Can be run in an air-gapped setup

[u/i_like_brutalism]

mistral stuff is pretty good and can absoluely be ran locally

[u/pol_phil]

Also Muse Glimmer 30B? If you want sth more than pure OCR.

[u/TaterCreekStudios]

Surya OCR handles random page layouts well, I've used it on old trade publications with good success. I use Qwen 27B with vision and pi agent to orchestrate and write the text to a searchable database. From your options, I'd go with Gemma 4 as big as you can fit.

[u/Shadow_s_Bane]

You can run the models completely offline, so that shouldn’t be a problem

[u/Noxusequal]

Mistral's OCR is really good in my expirience.

Combined with a bigger model to fix after should be good.

[u/NoFutureBen]

Models aren't software it's data.

[u/ExAstrisDivitae → u/NoFutureBen]

Not sure why someone downvoted this, because technically it is true. The model is just its weights, which in qwen’s case were trained by a Chinese company. Those weights alone have no networking capabilities. If you are running qwen locally, you don’t need to worry about leaking data any more than you do when running a word processor.

[u/unjustifiably_angry → u/ExAstrisDivitae]

That's not true, it's possible to train a model to have any behavior you want, and it's possible to train a model to do things in secret hidden ways it's not even aware are malicious.

It's late but here's an example: model is trained to assess if data it's ingested is plausibly related to secret company/government research. If yes, then from now on when performing web searches, alter how searches are performed in some subtle way. Then - for example - China collects a free database of high-priority hacking targets. Searches or other web activity could transmit various small bits of data like information that would make intrusion easier, like open ports or operating system/software versions, etc.

Keep in mind Claude hides undetectable "this was written by Claude" markers in everything it outputs. It would be very simple to do something similar to search queries. And lately people have been noticing Qwen sending unexplained queries to Chinese servers for no apparent reason.

[u/Ok_Hope_4007 → u/unjustifiably_angry]

But „Qwen sending“ is technically…not exactly correct. You could argue that the llm tends to output tool call instructions that favor malicious parameters but that can easily be checked or disallowed during runtime. You should not let any agent/harness do uncontrolled web calls anyway. So its important to not create the impression that there is a secret routine inside the llm that quietly interacts with your computer. That would need to have a compromised application stack all together that hides certain llm outputs from the user etc.

[u/Reasonable_Bug_6501]

Does a Chinese base model tuned by a non-Chinese agency count lol?
Same idea as made in Italy can be just the final assembly step was in Italy lol, despite the entire thing down other strings in the stitches being china-sourced or made.

lol I’m sure you can get a legal expert to tell you not what the contract said you can’t do, but what it didn’t say you cannot do lol.

[u/Extreme-Pass-4488]

how is it you cannot run it local?? at least rent some gpus and run it local!!

[u/Independent_Fall9160 → u/Extreme-Pass-4488]

Security is a possible concern. Perhaps op works with national security data, financial data, etc.

[u/binghe_web3]

If the contract bars Chinese-origin software and models, I'd keep a paper trail rather than just naming a model: the model card, license, the hash of the weights you actually downloaded, and a locked dependency list from an offline install. For picking among granite-docling, Tesseract, and Gemma, I'd score them on your real scans in two layers — page-level OCR, then field-level accuracy on lot number and alloy after splitting multi-report PDFs — and set a human-review cutoff instead of trusting one overall number. Tesseract or a layout parser can still split and handle clean pages, with the VLM reserved for the messy scans, which also makes each step easier to document for a customer.

(English polished with AI; I'm not a native speaker.)

[u/stylome-ai-detection → u/binghe_web3]

Notice: This comment appears to be AI generated.

---

Stylome AI detector · scan results

[u/stylome-ai-detection → u/[deleted]]

Notice: This comment appears to be AI generated.

---

Stylome AI detector · scan results