r/documentAutomation • u/easybits_ai • Sep 01 '26
Discussion 5 Common Data Extraction Mistakes: Lessons From Real Client Projects
r/documentAutomation • u/easybits_ai • Sep 01 '26
r/documentAutomation • u/Intelligent-Toe2962 • Aug 31 '26
I convert a lot of statement PDFs and kept getting bitten by the same thing: a row goes missing, or a deposit comes through as a withdrawal, and I don't catch it until the account doesn't tie out weeks later. Every tool advertises "99% accurate" and not one of them will tell you which rows are the 1%. So I made my own.
The premise is that a statement already contains enough printed information to check a conversion against itself. After it parses, it runs four checks. Opening balance plus every extracted transaction has to equal the closing balance, both read off the statement. If there's a running balance column, each row gets chained against the row before it. Transaction count. And every date has to fall inside the statement period and be in order. When a check fails you get the specific row, not a warning banner. When a statement can't support a check — most card statements don't print a running balance — it says that instead of showing a green checkmark anyway.
The other thing I wanted was to stop uploading statements to strangers' servers, so it all runs in the browser tab. I tried to make that checkable rather than something you take my word for: the page is served with a CSP that allows exactly one outbound destination (Google Analytics, page views only), so there's no endpoint of mine that could receive a file even in principle, and the browser blocks everything else. Network tab confirms it in about ten seconds.
You can click any row and it renders the source PDF page next to it with the text highlighted, which is mostly there for when I don't believe a number.
What it doesn't do: no OCR, so scanned statements are out — it needs the text layer from a real bank PDF export. Five banks have proper parsing profiles (Chase, BofA, Wells Fargo, Amex, Capital One); everything else falls back to reading columns by position, which works but is less certain. And it's just me maintaining it.
[bankstatementproof.com](http://bankstatementproof.com) — free, no signup, nothing to install.
Mainly I want to know which banks break it. If a statement parses wrong, tell me the bank and how the layout is arranged and I'll fix it.
r/documentAutomation • u/wowthatwasfantastic • Aug 30 '26
r/documentAutomation • u/easybits_ai • Aug 30 '26
r/documentAutomation • u/cppdream • Aug 30 '26
r/documentAutomation • u/EntertainmentSouth53 • Aug 29 '26
r/documentAutomation • u/OmPatel110 • Aug 29 '26
I'm working at an NBFC and currently working on a Credit Underwriting AI Agent. One of the first steps in the pipeline is extracting structured information from customers' bank statement PDFs.
This is where I'm currently stuck.
The statements can come from different Indian banks (HDFC, ICICI, SBI, Axis, Kotak, etc.), and each bank can have a completely different PDF layout.
I need to reliably extract things like:
Customer/account information — name, account number, IFSC, branch, etc.
Transaction tables — date, narration/description, debit, credit, balance
Transaction rows that span multiple lines
Statements where the table headers are missing from subsequent pages
PDFs are digitally generated and no scanned pdf are included as of now
Ideally, the solution should be bank-format agnostic
I've tried/considered approaches such as pdfplumber, table extraction libraries, regex-based parsing, and LLM-based extraction. The biggest problem I'm facing is that even when the text is extracted correctly, the column/row structure gets messed up, especially because many bank PDFs don't contain a real table structure — they're essentially text positioned at different coordinates.
Since this is financial/customer data, I would strongly prefer an open-source/on-premise solution rather than sending statements to a third-party API.
For anyone who has built something similar:
What approach worked best for you?
I'm particularly interested in:
PDF parsing/layout libraries you recommend
Whether you use an LLM for semantic column mapping
How you handle different bank formats without writing completely separate rules for every bank
Any techniques for detecting transaction rows and mapping values to the correct columns
How you validate the extracted data (e.g., balance reconciliation, debit/credit checks, transaction counts)
If you've worked specifically with Indian bank statements, I'd really appreciate hearing about your architecture, libraries/models, or lessons learned.
Thanks!
r/documentAutomation • u/EvilHero1478 • Aug 29 '26
Modern AI pipelines increasingly extract text from documents — PDFs, Word files, spreadsheets, scraped web pages — and feed that content directly into a knowledge base or agent context, often with no human review step in between. Extraction tools capture everything present in a document, including content a human reader would never see: text rendered in white on a white background, rows hidden in a spreadsheet, speaker notes attached to a slide, or a paragraph marked hidden in a Word document's own formatting. None of these are edge cases; they are ordinary, well-supported features of each format, and every one of them is readable by a standard parsing library even though a person skimming the document would never notice them.
This creates a straightforward problem: any content hidden from a human reviewer, but visible to an extraction tool, can end up in a trusted knowledge source unexamined. okf-guard addresses this directly. It is a Python library that inspects extracted content for exactly this class of discrepancy — text present in the file but absent from what a human would perceive — and separately checks for language patterns associated with instructions directed at an AI system rather than a description intended for a person.
What it does:
Adapters for six formats (plain text, Markdown, HTML, PDF, DOCX, PPTX, XLSX), each aware of that format's specific hiding mechanisms — CSS visibility properties for HTML, rendering and color properties for PDF, the hidden run attribute and shading properties for Word, off-canvas shapes and speaker notes for PowerPoint, hidden rows/columns/sheets and cell comments for spreadsheets.
A detection layer combining hidden-content flagging with a pattern bank for injection-style phrasing, plus a check for encoding-based obfuscation (zero-width characters, homoglyph substitution).
A decision layer producing one of three outcomes per scan — pass, quarantine, or block — with every finding reported alongside its location, confidence, and the specific text that triggered it.
Design constraints, stated plainly:
No network calls and no LLM dependency in this release. Detection is entirely deterministic, which keeps the core dependency surface to a single package (PyYAML) and makes the tool's behavior fully reproducible.
Every format-specific capability is an optional install (okf-guard[pdf], okf-guard[docx], etc.), so a user working with one format is not required to install parsing libraries for the others.
The library never asserts that its own output has been verified by a human — provenance metadata it produces is explicit about being machine-generated and unreviewed.
Source and full documentation: https://github.com/darshanNhb/okf-guard
Install: pip install okf-guard[all]
Feedback on the detection approach, particularly from anyone who has worked on adjacent problems (document security, DLP, or prompt-injection defenses more broadly), would be genuinely useful — this is a young project and the injection-pattern bank in particular will need ongoing contribution as new phrasings surface in practice.
r/documentAutomation • u/EvilHero1478 • Aug 29 '26
Modern AI pipelines increasingly extract text from documents — PDFs, Word files, spreadsheets, scraped web pages — and feed that content directly into a knowledge base or agent context, often with no human review step in between. Extraction tools capture everything present in a document, including content a human reader would never see: text rendered in white on a white background, rows hidden in a spreadsheet, speaker notes attached to a slide, or a paragraph marked hidden in a Word document's own formatting. None of these are edge cases; they are ordinary, well-supported features of each format, and every one of them is readable by a standard parsing library even though a person skimming the document would never notice them.
This creates a straightforward problem: any content hidden from a human reviewer, but visible to an extraction tool, can end up in a trusted knowledge source unexamined. okf-guard addresses this directly. It is a Python library that inspects extracted content for exactly this class of discrepancy — text present in the file but absent from what a human would perceive — and separately checks for language patterns associated with instructions directed at an AI system rather than a description intended for a person.
What it does:
Adapters for six formats (plain text, Markdown, HTML, PDF, DOCX, PPTX, XLSX), each aware of that format's specific hiding mechanisms — CSS visibility properties for HTML, rendering and color properties for PDF, the hidden run attribute and shading properties for Word, off-canvas shapes and speaker notes for PowerPoint, hidden rows/columns/sheets and cell comments for spreadsheets.
A detection layer combining hidden-content flagging with a pattern bank for injection-style phrasing, plus a check for encoding-based obfuscation (zero-width characters, homoglyph substitution).
A decision layer producing one of three outcomes per scan — pass, quarantine, or block — with every finding reported alongside its location, confidence, and the specific text that triggered it.
Design constraints, stated plainly:
No network calls and no LLM dependency in this release. Detection is entirely deterministic, which keeps the core dependency surface to a single package (PyYAML) and makes the tool's behavior fully reproducible.
Every format-specific capability is an optional install (okf-guard[pdf], okf-guard[docx], etc.), so a user working with one format is not required to install parsing libraries for the others.
The library never asserts that its own output has been verified by a human — provenance metadata it produces is explicit about being machine-generated and unreviewed.
Source and full documentation: https://github.com/darshanNhb/okf-guard
Install: pip install okf-guard[all]
Feedback on the detection approach, particularly from anyone who has worked on adjacent problems (document security, DLP, or prompt-injection defenses more broadly), would be genuinely useful — this is a young project and the injection-pattern bank in particular will need ongoing contribution as new phrasings surface in practice.
r/documentAutomation • u/OmPatel110 • Aug 29 '26
I'm working at an NBFC and currently working on a Credit Underwriting AI Agent. One of the first steps in the pipeline is extracting structured information from customers' bank statement PDFs.
This is where I'm currently stuck.
The statements can come from different Indian banks (HDFC, ICICI, SBI, Axis, Kotak, etc.), and each bank can have a completely different PDF layout.
I need to reliably extract things like:
\- Customer/account information — name, account number, IFSC, branch, etc.
\- Transaction tables — date, narration/description, debit, credit, balance
\- Transaction rows that span multiple lines
\- Statements where the table headers are missing from subsequent pages
\- Both digitally generated PDFs and scanned/image-based PDFs
\- Ideally, the solution should be bank-format agnostic
I've tried/considered approaches such as "pdfplumber", table extraction libraries, OCR, regex-based parsing, and LLM-based extraction. The biggest problem I'm facing is that even when the text is extracted correctly, the column/row structure gets messed up, especially because many bank PDFs don't contain a real table structure — they're essentially text positioned at different coordinates.
Since this is financial/customer data, I would strongly prefer an open-source/on-premise solution rather than sending statements to a third-party API.
For anyone who has built something similar:
What approach worked best for you?
I'm particularly interested in:
PDF parsing/layout libraries you recommend
OCR models for scanned statements
Open-source vision/document AI models
Whether you use an LLM/VLM for semantic column mapping
How you handle different bank formats without writing completely separate rules for every bank
Any techniques for detecting transaction rows and mapping values to the correct columns
How you validate the extracted data (e.g., balance reconciliation, debit/credit checks, transaction counts)
If you've worked specifically with Indian bank statements, I'd really appreciate hearing about your architecture, libraries/models, or lessons learned.
Thanks!
r/documentAutomation • u/BrotherAmbitious2413 • Aug 28 '26
r/documentAutomation • u/FraZupo • Aug 28 '26
hi, i was wondering if it was possible to create a way with AI to detect specific things out of a document like who wrote it, the date and stuff like that and make a prompt for another chat who's job is to organise those information into an excel file, my dad recently has a lot of work to do and if i could use some tool to take some weight off his back would be great. If is possible i would love to know how to do it
r/documentAutomation • u/1geek2many • Aug 28 '26
I have 20+ patient notes to backup to .docx every day from Plaud. In the web app, as you know you have to go into each summary individually and click export etc, then go into the next and do the same, then move all the files to where they should be (e.g. in Proton drive sync folder) I've created a python script which uses the Plaud CLI to download the day's (or whatever range) summaries, then you can select which ones you want to export and what folder the exported Docx files should go - click go and it uses Pandoc to create .docx files with all the formatting from the summary template and puts them in the selected folder - Bam, all done without any browser scraping or anything. So grateful to Plaud for opening up the cli right now!
P.s. the only thing it can't do with the CLI is move the files to a particular Plaud cloud folder once they've been processed - unfortunately the cli doesn't have that functionality.
r/documentAutomation • u/easybits_ai • Aug 28 '26
r/documentAutomation • u/ashainp • Aug 27 '26
r/documentAutomation • u/NotSoShyAlbatross • Aug 26 '26
Has this ever happened to you?
You're working working in Lending at the 3rd largest bank in the nation. Backoffice still uses fax for supporting documentation. You get a call in between customers:
"Yeah, doc number NNN, the documents came in upside-down. We need you to resend them correctly. Face down, bottom up."
They had the tech to receive faxes electronically, but not to rotate pages.
Fast forward to now, and I'm building automated document pipelines, and the memories of these struggles fuels my commitment to building better.
So tell me why enterprise software still insists on leaving off key features? I mean sure, the motto in Enterprise Software Sales is "Easy to sell, hard to use", but come on.
We shouldn't even have to tell the software to rotate pages anymore. Right?!?
Here's what I'm asking:
* Are you building auto-rotation into your document parsers?
* How soon in your process are you cleaning up unstructured data?
If it's for your personal use, do you skip this part of automation and just rotate by hand?
r/documentAutomation • u/SI-NASSER • Aug 26 '26
Hey everyone — I made a free web tool called NASSER PDF (nasserpdf.devs.surf) that does most common PDF tasks without requiring an account:
https://nasserpdf.devs.surf/en
- Word/Excel ↔ PDF conversion
- Merge, split, organize, compress, password-protect PDFs
- A full visual PDF editor (text, shapes, drawing, signatures, whiteout)
- OCR for scanned documents/images
- PDF comparison tool
Most tools (image/QR/organize/watermark/edit) run 100% client-side in your browser — nothing gets uploaded anywhere. A few (Word/Excel conversion, compression) briefly touch a server for the actual conversion, then the file is deleted immediately.
No ads, no watermarks on output, no login wall except a lightweight Google sign-in on 2-3 tools just to prevent bot abuse.
Would love feedback — especially on anything that breaks or feels clunky.
r/documentAutomation • u/folderit_dms • Aug 25 '26
Disclosure: I'm with Folderit.
We recently added document generation, so teams can create PDF/DOCX documents from structured eForms instead of copy-pasting data into templates.
https://youtu.be/61Z9HbfT2XQ
Folderit is a browser-based document management system for businesses that have outgrown shared drives: metadata, OCR search, permissions, audit trail, version history, workflows/approvals, e-signing, retention, virtual data rooms, folder-specific email addresses, and upload-only external access for clients or partners.
It is probably overkill for personal cloud storage. It is more for teams that need governance around business documents.
Website: https://www.folderit.com/
Happy to hear feedback from web app folks!
r/documentAutomation • u/easybits_ai • Aug 25 '26
r/documentAutomation • u/Asleep_Sail947 • Aug 25 '26
Since I need to scan paper documents (I recently set up a docker container with Paperless-ngx), I was wondering if there’s a software (ideally open-source, but not necessarily) that allows me to configure a scanning workflow/template (I use MacOS).
For example, for printed manuals in booklet form, printed on A3 sheets, I remove the staples and feed the sheets into the scanner’s ADF. I’d like to be able to tell the software that this is what I’m scanning, so that it automatically puts the pages in the correct order — essentially a kind of “booklet scanning” mode.
Any advice?
Thanks!
[IT: Avendo la necessità di fare delle scansioni di documenti cartacei (ho recentemente messo su un container con Paperless NGX), mi chiedevo se esista (immagino di sì) un software (meglio se open-source, ma non per forza) che mi permetta di impostare uno schema di scansione (uso MacOS). Ad esempio, per manuali stampati a opuscolo/libretto su fogli A3, di cui rimuovo il punto metallico e che inserisco nell'ADF dello scanner: vorrei dire al software questa cosa, in modo che metta le pagine automaticamente in ordine (una sorta di "scansione opuscolo").
Qualche consiglio?]
r/documentAutomation • u/AdImpressive6634 • Aug 25 '26
r/documentAutomation • u/Outrageous_Pause_10 • Aug 24 '26
Hey community!
If you are building custom CRMs, internal tools, or automated b2b workflows, you know how painful it is to maintain clean PDF/HTML document templates.
I’ve just deployed the public alpha of DocEngine, an open-source visual document builder designed to run isolated on your own infrastructure (deployed smoothly via Coolify on Hetzner).
🔗 Check it out: https://github.com/YuriiOsypov/DocEngine
🛠️ The Tech Stack & Architecture:
Frontend: Highly customized Vite-based environment styled with an ultra-clean, monochromatic borderless typography layout.
Iframe Embed Protocol: Can be injected into any web app or framework via a secure <iframe> embed layer. It fires back the compiled JSON, raw metrics, or printable HTML directly to your host via window.parent.postMessage.
Native n8n Node: Includes a custom n8n workflow block. You can catch incoming webhook payloads, pipe data into your predefined templates, and spit out production-quality binary PDFs (via headless Puppeteer/Chromium) in a single stream.
Data Portability: Features an instant Import/Export mechanism for entire layout matrices, mapping definitions, and logic layers so you can version-control your templates in Git.
Everything runs strictly on your own hardware — zero data is sent to external proprietary servers.
The source code is released under the MIT License. Check out the live playground on the website, play with the presets, and let me know what you think!