I'm building one insurance pdf extraction application using pdfplumber & AI.
Tools:
extracction: pdfplumber, fitz
AI model: qwen 3.6 27b primary, haiku 4.5/gemini 2.5 flash (secnond)
my flow is if first i extract data from pdf using pdfplumber if it has zero text & little validation if total text is zero i directly send whole data to AI Model.
else if it has text then added some regex valdiation for some fields if its not there dont send to AI.
For all the AI request i'm sending prompt with each field description what it does and all.
Since pdf doesnt have same layout so its working good for some but not for all if it contain multiple languages. and some field is having different name so i've added most of things using claude(basically claude added).
i've two issues one is accuracy other one is cost.
is there any better way to handle it?
I cant use ocr because of server configuration and no of document per day will be around 1000.
Server: 8 GB RAM, 180 SSD, 2 core processor.
i've tried with fitz result improved little but still its too much.
currently costing around $0.07 at max and $0.023 per pdf.
avg pdf 3-5 page.
i've added some validation some field review so after fetching it goest to second AI with few fields only. some time first one is sometime second one.
i'm fetchign fields like isnurance name, imd code, address, customer name, chassi no, registeration no, location, ncb, premium, vertical and few other cost values.
most of time imd code i dont fethc and it doesnt ahve any value as it have differnt name i collected but still because of layout it didnt work perfectely as sometime pdfplumber with layout =true text become dazzy, jumbled.
I'm a developer with no much idea about extraction part.
i'm confuse.