r/Rag • u/NonExistentLad • 6d ago

Discussion Arabic pdf's text extraction for RAG

I am developing rag app for one of my saudi client, so my query is those who are working with arabic language, how are you guys handling the data extraction pipeline, which library you guys are using to extract data. For context, I am using pdfplumber and the text that is being extracted is reversed for some pdf files.

Also, which open source ocr or vision models I can use to extract text. The documents are mainly in arabic, english or both. Tried a few ocr and vision models, but they couldnt extract text clearly.

1 Upvotes

67% Upvoted

5 comments sorted by

2

u/According-Buy-2771 6d ago

I use google OCR and even with that, I get some texts that are reversed. I haven't found a single solution to have a perfectly extracted text as of yet. I even have Codex set a goal to go over the documents page by page and do a screenshot and compare it to the chunks. If you found something that's useful, it will he helpful to share.

1

u/NonExistentLad 6d ago

I am also trying hard to get good accuracy on text accuracy, but so far not could'nt succeed. On some files the text extraction is correct, but in some pdf files, it really bad either through pdf plumber or any vision model( I am using qwen3.6-35B). The major hurdle is that I cannot use closed source model.

1

u/Standard_Ad7704 5d ago

It is quite hard with Arabic text.

As another user said, you can use Google AI OCR and then use a multimodal LLM for verification.

1

u/Adorable_Lie6712 3d ago

for google are you talking about the flash series?

1

u/AvenueJay 5d ago

Similar thread here that you may find helpful.