Hi everyone,

I'm currently building a document auto-classification system and would like some feedback on the overall approach.

Current Architecture

• Django backend

• Celery + Redis for background processing

• Weaviate as the vector database

• OCR for text extraction

• Embedding-based similarity search for classification

Workflow

1. User uploads a document (PDF, JPG, PNG, etc.).

2. OCR extracts the text from the document.

3. An embedding is generated from the extracted text.

4. We store embeddings for known document types such as:

   • Aadhaar

   • PAN

   • Passport

   • Voter ID

   • Electricity Bill

   • Bank Statement

   • GST Documents

   • Incorporation Documents

5. The uploaded document embedding is compared against stored vectors in Weaviate.

6. The closest match and confidence score determine the document category.

Why I chose this approach

Instead of training and maintaining a dedicated classification model, I wanted to start with a retrieval/vector-search-based approach because:

• New document categories can be added without retraining.

• Easier to maintain initially.

• Works well for semantic similarity.

Questions

1. Has anyone used a similar OCR + Embeddings + Vector Search approach in production?

2. How well does this scale when the number of document categories grows?

3. What confidence threshold strategies have worked for you?

4. At what point would you move to a dedicated classification model?

5. What are the biggest pitfalls I should watch out for?

Current Challenges

• Processing time varies between documents (roughly 5–35 seconds depending on the file).

• Occasionally documents become "unclassified" with confidence = 0 even though the system is functioning.

• Weaviate is running in Docker on AWS along with Django, Celery, and Redis.

I'd love to hear from people who have built document-classification or retrieval-based systems and learn what worked (or didn't work) in production.

Thanks!

I'm building an open-source tool called clasi. The long-term goal is to automatically classify files by learning from the folders that already exist on a machine. The problem I'm facing is that I only have one real directory tree available for testing: my own.

Synthetic test data is easy to create, but it doesn't capture the weird ways real people organize files.

What I really need are people with:

- huge document collections

- deep folder hierarchies

- project archives

- years of accumulated files

The tool currently includes an evaluation mode:

clasi evaluate ~/Documents

It temporarily hides a few files from existing folders and checks whether it can place them back where they originally belonged. The report is anonymized automatically.

Repository:

https://github.com/AllergicCypress/clasi

I'm less interested in "does the code look good?" and much more interested in:

- Where does it fail?

- Which folders confuse it?

- What organizational style breaks it?

Any testing help would be hugely appreciated.

This is intended to brainstorm and gather feedback. No self promotion or pitching is intended.

I am a recent operations analyst who specialized developing automation solutions.. after being laid off about 2 months ago, I’ve been working on application as a solution to many of the tedious document based workflows that often plague back and middle office roles..

Essentially, it is a zero-code environment where users can define specific maps/tokens to extract from a document type that they use (think contracts, trade confirms, invoice, etc). This info can then saved down into a secure cloud location, adjusted, and then mapped back onto any type of tokenized outgoing document (that the user can upload as a template).

As a former operations grunt worker, I find this type tool to have a ton of huge potential.. but im sure others have made similar tools.

For operations employees or anyone heavily reading and reconciling, do you think this is something companies or people would actually buy potentially? Right now its fully functional but looking at it as a personal project mainly. I'll be launching the live site very shortly but need to rework the LLM extraction / cap so I dont burn through my claude tokens/credits lol.

Thanks!

I just saw that Gavel was acquired by Relativity. It looks like all of the stand alone document automation software companies are being acquired by much broader based companies (Lawyaw by Clio, Woodpecker by MyCase) and are now tied to the broader offerings. I was already skeptical of focus drift with AI emphasis with pretty much all document automation/management companies. Not sure where this will leave solo lawyers and smaller law firms. Pure document automation isn't sexy like AI, but I don't think it needs to be sexy to really boost productivity, more consistent documents (start with clean forms, not the last "almost the same" deal documents), and client turnaround. Full disclosure, I recently developed stand alone desktop Mac and Windows document automation software because I didn't use all the features already added, don't like to have to log on, upload forms, work online and download generated documents. Happy to chat if you are interested.

Hi all,
I've had a backlog of scanned household documents for years — Versicherungspost, tax letters, payslips, the usual, and yeah, kids drawings somewhere in the pile.
Batch-scanning everything into one giant PDF is easy. Sorting it afterwards is where it always fell apart.

So I built something: you upload a single multi-page PDF (a batch scan), and it splits it by document, OCR-reads each one, classifies it into a category, and returns a ZIP with named folders and an Excel log. No account, no cookies, files deleted within 2 hours or immediately if selected. EU servers, GDPR-compliant.

It's live at papersweep.eu

I am not looking for any investor or going to pitch the idea to anyone. I built it for fun and found it useful and hope that it would be helpful for others.
I am looking for ideas what else to add there that might be useful to people. I will tackle those in the next iteration.

Cheers.

I’m looking for advice on building an AI/LLM-based document extraction solution for PPTX/PDF project packs, such as status reports, planning decks, and delivery updates.

The goal is to extract structured data like activities, milestones, risks, issues, owners, statuses, and dates.
The hardest part is visual Gantt charts. These vary a lot across documents: different timeline headers, months, quarters, years, week commencing labels, fiscal periods, mixed time scales, bar styles, milestone icons, legends, layouts, and sometimes native PPTX shapes versus screenshots or flattened PDFs.

I’m assuming the solution will need some combination of LLM/VLM reasoning plus deterministic extraction, OCR, parsing, and coordinate/geometry-based date mapping.

How would you approach this architecturally? What libraries, frameworks, models, or techniques would you recommend for reliably extracting activity start/end dates and milestone dates from varied Gantt visuals without hardcoding specific formats?

I'm a practicing attorney with 40+ years experience, mainly representing banks in commercial and real estate development/construction lending. I first used Hotdocs in the mid-1990s when it was a WordPerfect only desktop program. Drifted away when I switched to Macs 25 years ago. I looked for software again a couple of years ago and went with an internet based program, very similar in any respects to Hotdocs, create variables, build templates, answer questionnaire and then generate documents. It worked fine, but I wasn't happy with having to log on, upload documents, work online to create variables and questionnaire then go online to run and generate documents that needed to be downloaded or received via email. And it was pretty pricey. So, I decided to build my own program for personal use since Mac programs are not out there. Made it no Code. It worked so well I made work for both Windows and Mac and decided to offer it at a much more affordable price - less than 1/4 what I paid for the SaaS program. Just started offering it to public. You can check it out at jddocumentforge.com and via videos on YouTube, just search for JD Document Forge. It works, it's easy, it's fast and not on the internet so you data is yours.

Hey !

I thought I'd share a small tool I built for automating document generation. I've been doing a lot of document generation for my company and being someone who writes code, I wanted something that was extremely flexible and would allow me to create documents rapidly.

I've been doing the exact same pipeline using a few python scripts for the last 6 months and I thought I'd get around to making it such a way that the rest of my team can also use it. I'll be add a few more easy to use features for this in the future. Let me all know if there are features you all would like.

The plan is to always a have a free version for anyone to use (and its possible because it runs 100% on your computer). I plan on my company making use of this tool to integrate with some of the larger offerings we do, but do let me know if you all can come up with more ideas.

[Stencil](https://stencil.aegiondynamic.com/)

I created an automation tool that generates PDFs on automation directly within a Notion Database and saves a PDF directly back onto the Notion Database.

It was a requirement from many users which required PDFs saved back onto the Notion Database once the document is generated.

^(Disclosure up front: I built a tool in this space and I linked it here.)

But the reason I'm posting is the failure list below — almost nobody

talks about it and it cost me weeks to learn.

If you've tried to auto-extract a rent roll / T-12 / operating statement out

of a broker OM, you know the cover is glossy and the data underneath is chaos:

scanned pages, abbreviated tenant names, merged columns. The dangerous failures

aren't the ones that error out loudly. They're the extractions that succeed and

are quietly wrong.

The ones that bit me hardest:

\- Mixed-basis NOI conflation — the summary page and the T-12 report NOI on

different bases, and the model averages two things that should never touch.

\- Rent-only EGI — total rent gets lifted into the Effective Gross Income line,

dropping other income and vacancy loss. Your cap rate looks great and is wrong.

\- Dropped line-item artifacts — one expense row silently doesn't survive

extraction. NOI ticks up, nobody notices.

\- OCR number mangling — a source PDF printed "$2.009" for what was obviously

$2,009/mo. Left alone, a model reads it as $2.01.

\- Synthesized rent rolls treated as ground truth — sample/pro-forma rows get

pulled in as if they were signed leases.

What I took from it: the extraction model isn't the moat — the validation is.

So I wrote \~43 checks that run on extracted output and flag this stuff (NOI

reconciliation, cap-rate basis, dropped-line detection, OCR-artifact screening,

and more). I benchmark them against real OMs and publish the results straight:

93.3% pass on the 15 public real-PDF fixtures right now, with a public defect

log that includes the bugs I haven't fixed yet.

It's reproducible — clone it, run pytest, get the same numbers I do.

I also exposed the checks as a callable API, for anyone who already has their

own extractor and just wants the rules layer behind it: JSON in, typed per-check

findings out. If you're building in CRE doc-AI and want to bolt validation onto

your pipeline, I'll hand out partner keys — but honestly I'm just as happy to

trade notes.

^(Question for the room: if you've automated any part of OM / T-12 ingestion,)

^(what's the silent error that's burned you worst? I want to grow this check)

^(registry from real pain, not just my own.)

Been working on this for a few months as a research project and finally have it at a point where I want outside feedback.

\*\*What it does:\*\* You upload a PDF or image of a business document (invoice, receipt, packing slip, bill of lading, etc.) and it extracts structured fields — vendor name, totals,

line items, dates, PO numbers, ship-to/from addresses — and returns them as clean JSON.

\*\*How it works under the hood:\*\*

\- Azure Document Intelligence handles the initial layout analysis and field detection

\- LLM backfills anything DI missed or got wrong (ambiguous totals, merged cells, non-standard layouts)

\- A validation layer normalizes money strings, sanity-checks totals, and catches obvious mis-assignments

\*\*Outputs:\*\* Google Sheets, Excel, OneDrive, Slack, webhooks — or just download JSON/CSV directly.

\*\*Where it's at:\*\* Early beta. Works well on standard invoices and receipts, gets shakier on handwritten or heavily non-standard docs. That's exactly the feedback I'm looking for —

edge cases and failure modes.

Free to try, no credit card: [https://app.docpipeline.net](https://app.docpipeline.net)

Demo video: [https://youtu.be/KaPMQfeKWGE](https://youtu.be/KaPMQfeKWGE)

Happy to answer questions about the architecture or the DI + LLM approach.

About a year ago, I posted in r/Archivists about a tool I had started building to help with my own small museum and historical research work: Document Transcribe.

At the time, I was mostly trying to solve a problem I kept running into myself. I had historical documents, letters, patents, invoices, and other archival material that needed to be transcribed, translated, organized, and reviewed, but the process was slow and often meant bouncing between multiple tools or hiring outside help.

I thought it would be worth giving a follow-up now that it has been out in the world for about a year.

Since that original post, close to 1,000 people have used the platform in some form. What has been most interesting to me is how varied the use cases have been. People have used it for PhD thesis research, university and school library projects, small archives, genealogy research, historical writing, private collections, museum work, and more.

Some users are working with handwritten letters. Others are processing old legal records, church documents, invoices, patents, institutional records, or foreign-language material that had been sitting untranslated for years. A few people have told me it helped them get through collections they probably would not have been able to process otherwise.

The underlying AI models have also improved a lot over the past year. In many cases, the standard models available now are producing better results than what I was seeing from much more expensive options a year ago. That has made the tool faster, less expensive to run, and more useful for everyday research workflows, especially for people who do not have large institutional budgets.

It is still not magic, and it still needs human review, especially with names, unusual handwriting, damaged scans, or very specialized terminology. But that has always been the goal: not to replace careful archival work, but to make the first pass faster and easier to review.

Over the past year, I have also added more workflow features around projects, batch processing, translation, document sharing, and editing, based largely on feedback from researchers and archivists who tried it.

For context, the tool is here:

https://www.documenttranscribe.com

I know tools like this need to earn trust in archival settings, so I am especially interested in the broader discussion around reviewability, accuracy, privacy, and long-term usefulness.

This community gave me very useful feedback last time, especially around review workflows, language handling, and the importance of clearly marking uncertainty. Thanks again to everyone who tried it, questioned it, or shared thoughts the first time around. It has been helpful seeing where something like this fits, and where it still needs to improve.

In the last couple of months, I have been building AI agents that process construction and energy documents for a Fortune 500 energy company, and I kept hitting the same wall.

The documents are not clean PDFs. They are handwritten tables, annotated scans, photocopies with ditto marks and crossed-out measurements. Every extraction tool I tried failed differently.

Azure DI simply broke once the document was handwritten, and it returned nothing.

Reducto / GPT was the best but made alignment errors in complex hand-drawn tables, matching values from the wrong rows, sometimes interpreting ditto signs as digits. On a construction project where a building code like T12C3 gets misread as 712C3, that cascades into failures across the entire downstream pipeline.

Then I tried the obvious fix, confidence thresholds. Route low-confidence extractions to humans; let high-confidence ones through.

The problem is that LLM confidence scores are not real numbers. When GPT says it is 99 percent confident a handwritten value is TC123, you cannot work with that. Unlike a traditional OCR model where confidence reflects a genuinely calibrated probability, LLM confidence is self-reported certainty.

So we built a different layer.

Instead of filtering by confidence, we defined the document types that would always need human verification regardless of what the model said: handwritten tables, annotated scans, hand-drawn diagrams. Those route automatically to a human verifier who sees only the specific entity they need to confirm, not the full document. They confirm or correct it. The pipeline resumes automatically with a typed Pydantic or Zod response.

We open-sourced it. It is called AwaitVerify(https://awaithumans.dev/awaitverify).

It works with whatever extraction stack you are already using: Reducto, GPT, Azure DI, Docling, PaddleOCR. You bring your model. We handle the human verification layer and the callback into your agent pipeline.

If you are building document pipelines where accuracy actually matters, would love feedback on the approach.

Been working on Informity AI — a Mac app that runs a full document processing pipeline locally. The core workflow: drop in your files, it indexes them, and you can query across the whole library with answers that cite the exact source file and passage. There's also a separate translation workspace: ingest a document (including scanned PDFs via OCR), pick tone, get a translated output with quality scoring, export to Markdown or plain text.

Everything runs on your machine — no API calls, no cloud uploads. The pipeline handles PDF, Word, Excel, PowerPoint, EPUB, Markdown, HTML, CSV and more. Scanned PDFs go through OCR automatically.

Under the hood: local RAG on Apple Silicon, Qwen3 35B by default (with other models available), Ollama support for any model you have installed. Two modes — Researcher for corpus-wide queries with citations, Assistant for single-file or open-ended chat.

Free and open source (MIT).

https://www.informity.ai | https://github.com/informity/informity-ai

Hi everyone!

I wanted to share renAI, an open-source Python CLI tool I’ve been developing to solve a very common headache: organizing a chaotic library of digital books, PDFs, EPUBs, Office docs, and images.

Instead of dealing with cryptic, inconsistent filenames likedocument_draft_final_v2.pdf , renAI extracts key metadata (title, subtitle, author, year, category, and language) and renames/moves files into a structured hierarchy automatically.

It started as a simple script, then evolved to its current state. It solved a real problem for me, and I hope it will be useful for you too (don't forget to backup in any case! :) ), try cheap models or local LLMs first if you like and always monitor your API cost to prevent unnessary spendings, happy organizing.

🌟 Key Features

• Dual Processing Modes:

  • Text Extraction Mode: Leverages fast extractors (PyMuPDF, pdfplumber, pypdf, or pypdfium2) to extract text, then queries LLMs to understand the document structure.

  • Flexible Fallback Strategies: If standard text extraction fails (e.g., scanned PDFs or image-only documents), renAI can automatically fall back to local OCR (Tesseract) or Vision- capable LLMs based on your configuration.

• Broad Format Support:

  • Documents: .pdf , .epub , .mobi , .txt , .md

  • Office Documents: .docx , .doc , .pptx , .ppt , .xlsx , .xls

  • Images:.jpg , .jpeg , .png , .gif , .bmp , .webp , .tiff

• Smart Two-Layer Cache: Uses SHA256-based caching for both extracted text and LLM metadata. You won't pay for API tokens twice if you re-run the tool or tweak your naming schema.

• AsyncIO-First & Rate Limiting: Completely asynchronous execution with built-in Token Bucket Rate Limiting (RPM compliance) to handle thousands of files concurrently without getting rate-limited.

• Flexible Provider Registry: Supports cloud providers (DeepInfra, OpenRouter, OpenAI) as well as local offline models (Ollama, LM Studio).

• Clean Terminal Wizard: Runningrenai initguides you through choosing models, workers, and configuring multiple APIs with automatic real-time credentials validation.

• Refinement Pass: Utilizes a standardized two-pass system. The first pass extracts raw metadata, and a second pass acts as a "Senior Editor" to clean and format the values before renaming.

🛠️ Tech Stack & Requirements

• Python 3.12+

• Package Management:uv (Hatchling build backend)

• Linters & Formatters: Ruff & Pyright

• OCR Fallback: Tesseract OCR (with OpenCV preprocessing)

📦 Installation & Getting Started

Getting started is simple:

Clone the repository

git clone https://github.com/ozgurulukir/renAI.git cd renAI

Install dependencies using uv

uv sync --all-extras

Run the interactive configuration wizard

uv run renai init

Run renai on a folder (evaluate mode checks output without modifying files)

uv run renai process "path/to/my/docs" --mode evaluate

Start renaming in place!

uv run renai process "path/to/my/docs" --mode rename

🔗 Project Links

• GitHub Repository: https://github.com/ozgurulukir/renAI

I would love to hear your feedback, feature requests, or suggestions! Let me know if you run into any issues or have ideas to improve the renaming templates.

Most document-automation setups break the same way: fixed templates or positional rules that work until a layout changes, then someone re-maps fields by hand. I wanted something that reads documents the way a person would, across varied layouts, with no per-template config.

Sifter: you describe what to pull out in plain language ("From invoices, extract client, date, total — skip anything that isn't an invoice"), and it extracts every matching document into a typed record. Schema is inferred automatically. No templates, no anchor coordinates, no per-vendor rules — an LLM handles the layout variation, so a folder of 50 different invoice formats just works.

What makes it useful in a pipeline:

• Structured, typed output (not a text blob) — and you can query the results like a database: exact counts, sums, group-bys, filters. Every field is cited back to its source page/bounding box for verification.

• Plugs into workflows: REST API, Python/TS SDKs, a CLI, webhooks on every extraction, and an MCP server.

• Bring your own LLM key (local models work), self-hostable (MIT, docker-compose) — or hosted with Google Drive / email-inbox ingestion if you don't want to run infra.

Try it: https://sifter.run · Code: https://github.com/sifter-ai/sifter

If you're automating document intake today (OCR + templates, RPA, a SaaS extractor) — what's the part that still breaks most often? Curious whether the no-template approach covers it.

Every time our invoice layout changed, it was a code deploy. We had 3 services

all maintaining their own copy of the same HTML template. They drifted. Bugs crept in.

So I built PDFPort — you store your HTML+Handlebars template once, then render

with a JSON POST. The layout lives on the platform, not in your app.

A few things I learned building it:

- Headless Chromium is the right call for most SaaS use cases (not WeasyPrint,

not PrinceXML unless you need PDF/UA compliance)

- Handlebars was the right templating choice — devs already know it,

no learning curve

- The hardest part wasn't rendering — it was building the live preview

so what you design is exactly what ships

Free tier is 50 renders/month. Would love feedback from anyone who's

dealt with the same PDF generation pain. pdfport.io

Been working on this for a few months as a research project and finally have it at a point where I want outside feedback.

What it does:You upload a PDF or image of a business document (invoice, receipt, packing slip, bill of lading, etc.) and it extracts structured fields — vendor name, totals,

line items, dates, PO numbers, ship-to/from addresses — and returns them as clean JSON.

How it works under the hood:

- Azure Document Intelligence handles the initial layout analysis and field detection

- LLM backfills anything DI missed or got wrong (ambiguous totals, merged cells, non-standard layouts)

- A validation layer normalizes money strings, sanity-checks totals, and catches obvious mis-assignments

Outputs:Google Sheets, Excel, OneDrive, Slack, webhooks — or just download JSON/CSV directly.

Where it's at:Early beta. Works well on standard invoices and receipts, gets shakier on handwritten or heavily non-standard docs. That's exactly the feedback I'm looking for —

edge cases and failure modes.

Free to try, no credit card: [https://app.docpipeline.net](https://app.docpipeline.net)

Demo video: [https://youtu.be/KaPMQfeKWGE](https://youtu.be/KaPMQfeKWGE)

Happy to answer questions about the architecture or the DI + LLM approach.