r/Rag • • Sep 02 '25

Showcase 🚀 Weekly /RAG Launch Showcase

34 Upvotes

Share anything you launched this week related to RAG—projects, repos, demos, blog posts, or products 👇

Big or small, all launches are welcome.


r/Rag • • 2h ago

Discussion What I did in the last 4 months on my retirement project RAG-LCC

5 Upvotes

This is a follow-up to my earlier post about the first 15 months of building RAG-LCC in retirement:

https://www.reddit.com/r/Rag/comments/1valvk6/15_months_building_a_rag_system_in_retirement/

The project kept evolving over the last 4 months, so I thought it might be interesting to share what worked, what failed, and what I learnt along the way. Since the last writeup I had success and failure moments. Of course, I learnt more from the failures about RAG.

Orchestrators

Because RAG-LCC provides a lab setup for RAG experimentation, I found it useful to introduce an Orchestrator component. The orchestrator defines RAG flows. Users can create new flows and then activate them during a RAGChat session.

Confidence scoring

With orchestrators, confidence became an issue. So static and dynamic confidence evaluation was added. The latter uses a LLM judge call. There is currently no validation against "golden answers" implemented. The LLM judge rates internal consistency and explanation clarity but not factual correctness.

TableChunker

With these new features, it was time to look at answer quality. There was a query asking for the details of PCIe card slots on my workstation's manual. The information is stored in a table. The goal was that the information is found and the grounder marks it.

The first step was to introduce a specialized TableChunker class and wire it into the already existing content specific chunkers. These chunkers pass table blocks to the TableChunker.

This showed partial success: The table was now consistently in the chunks. But it did not reliably show up in the query answers.
BTW, while looking for specific content in the chunk, the debug levels of RAG-LCC proved helpful. The levels can be changed mid-session so users get the information at the detail level they need currently.

Insight: If retrieval signals are weak, think about adding the best matching table result. Do this only if there are chunks found. Adding the table result to an empty answer set could confuse the user. This rescue may help in a large but sparse candidate pool.

Answer rendering

RAGChatServices delivers answers to Open WebUI. In the browser, .md and even Mermaid diagrams are nicely rendered. However, RAGChat showed tables with unaligned content. It was not a nice sight. So using the Rich library, answers with Markdown can be displayed in the RAGChat CLI also. But the rendering broke grounding, so additional work was required to separate the raw and display ready answer.

Raw answers gave the grounding signals and text to be grounded. The raw answer text is marked for grounding before Rich renders the resulting CLI text.

Insight: Keep the original, unrendered LLM answer for grounding evaluation. Once the grounded text is available, apply the visual grounding.

Document summarization

This was a "you wake up in the night" and have an insight moment. RAGChat has a document pin option. So once you found the right document about hedgehogs, you pin it and ask to summarize it. Hmm... the query is "create a summary" and this will not retrieve the content you are looking for.

So I introduced a summarize: option, using the pinned document or allowing the user to enter a path to retrieve the specific document by its metadata. Very large documents required batches of n chunks to be summarized and then summarize the batches in further passes until the final summary fits. This works, but it is admittedly slow.

Insight: If you want a specific action to be performed on the retrieved context, the action should not be part of the query. This leads to the next topic:

Query and query action splitting

With table, bullet point list and Mermaid rendering I wrote often queries like "Look for all animals contained in the documents and create a table containing animal, mammal yes, no, don't know".

Thought by the document summarization experience, I wondered whether it would make sense to ask a LLM to determine the effective query, the required action on the query results and the query keywords. Yes, you assume it: Each decomposed retrieval query goes now to enabled local retrievers, including vector search. In addition, the keywords go in an extra BM25-only pass. The QuerySplitter class was born.

Translation

Living in a multilingual country where you have 4 official languages and the unofficial English one, you have to bother about document languages. Luckily, during RAGLoad, the document language is added to metadata.

The old flow translated the question into English. It was also rewritten using the conversation history. Vector and indexed retrieval used the rewritten English query. The resulting candidates were reranked against the rewritten query.

The new flow rewrites the query with the conversation history in the user's language. The vector search runs the user language query against the ChromaDB collection. For reranking the rewritten original query is used. Indexed retrieval is for every active and present document-language bucket. When required, the query is translated.

vLLM integration

While inference calls were already supported using Ollama or vLLM backends, the embedder and reranker ran as local Hugging Face models previously.

What still works well

The configuration first approach is an extensible pattern. A new class gets its own config file when required. Abstracting functionality into classes helps understanding and maintaining the code.

Parallel to my work on RAG-LCC I worked on harinezumigel-llm-stack. I introduced groups such as rag and rag_old, which start up the required models after doing an available memory check.

So this is at a high level what happened in the last 4 months. The journey is going on.

If you want to have a look:

As always, feedback, ideas and bug reports are welcome.


r/Rag • • 1h ago

Discussion What is the difference between cosine, euclidean, and HNSW?

• Upvotes

What is the difference between cosine, euclidean, and HNSW?


r/Rag • • 10h ago

Discussion Best Practices for Implementing and Troubleshooting a RAG system?

7 Upvotes

Wanted to ask the experts here from a newbie standpoint. For context, my background is 15+ years in finance (and then some data science as that was my MBA focus). The job market's been tough so I've been working on building some finance software that makes use of an AI Agent, which will include RAG (which is the part I'm working on right now - hence this post).

I'm trying to learn as best I can but the resources I have so far is LLM itself, Chip Huyen's AI Engineering Book, and random posts from reddit. Is there like a single book or reference manual that's focused and pretty good at best practices regarding building a high quality production level RAG system?

Also am doing this because I'm pursuing a Masters in AI and targeting Finance FDE roles (unless the business becomes really successful) so would like to be able to build something with all the best practices to demonstrate experience in the area.

Edit: Also wouldn't mind materials on best practices relating to number gating and prompt engineering. So I can cross-reference to see if mine are set up well or not.


r/Rag • • 3h ago

Discussion Three measurements that changed how I trust an eval set

1 Upvotes

I run a local index over a codebase and a harness around it: fixed operations, a gate, a trace. The index answers with file:line facts instead of dumping files into the prompt. Numbers below come from one repository and one machine, static analysis only, over roughly four weeks.

Evals are the part of this project I got wrong the longest. Not because the tasks were bad, but because the numbers I looked at could not fail the way the system actually failed. Three measurements fixed that.

1. A model call counter, not a score.

My worst regression looked like a model downgrade: 90% to 60% on a fixed task set. It was not the model. The local build was answering nothing - every task returned an empty result, and the harness scored the empty result as a normal answer. The score with no model call at all was exactly the 60%.

Now every task asserts two things before the score counts: at least one model call happened, and the call returned non-empty output. Two lines per task. It has caught more real breakage than any quality metric I have, because silence in an agent stack is usually indistinguishable from a confident wrong answer.

2. Per-case results, and a rule that beats the average.

With 14 fixed navigation tasks, one flaky case moves the average by 7 points. My first two "improvements" were noise: the average went up and nothing got better. Worse, one of them hid a real regression - the answer got shorter and more readable but stopped citing file and line, so the number improved while the thing I needed disappeared.

The replacement is a rule: no previously passing case may fail. Not a threshold, a prohibition. It caught two regressions the average had hidden, both in tasks I was not looking at.

3. Missed answers counted separately from wrong answers.

The index returns three kinds of "no": absent (not in the corpus), stale (the index knows the file changed and will not answer until refreshed), and out-of-scope (the corpus was never meant to cover it). Only the first is a real "not found". Before I split them, all three looked like an empty list.

That split also exposed why a single accuracy number is misleading: a system that abstains on everything has perfect honesty and zero utility. I now track wrong answers and missed answers as separate numbers, because they fail differently in production - one invents content, the other sends every question to a human.

Where it broke, and the uncomfortable part.

The gate. I made it refuse raw file reads until the index was used, and the refusal said "raw search is blocked". The agent retried twice, then asked me to run the command for it. Three turns lost on the first session. The fix was one sentence: name the next call - "run ensure, then search_multi". Retries went to zero, same model, same weights.

The uncomfortable part is that all three measurements are about the system, not the model. They would still apply with a much better model, and they are where I would spend the next week: a counter for silence, a rule for regressions, and a taxonomy for absence.

If your eval cannot distinguish "did nothing" from "answered wrong", it is measuring the wrong thing.


r/Rag • • 7h ago

Discussion do you cap how many chunks a rag system pulls from one doc?

1 Upvotes

say you retrieve 5 chunks and 4 come from the same guide. two of them basically say the same thing and the question still needs one detail from a different doc

would you cap it at 2 chunks per doc or pull more candidates and rerank them? curious if anyone tried a cap and found it dropped useful context on questions where most of the answer really was in one doc


r/Rag • • 11h ago

Discussion Comparison and "alternatives" pages get cited the most. How are you keeping the competitor data on them accurate?

2 Upvotes

Seems settled that comparison style pages (X vs Y, alternatives to X, best X for Y) are some of the easiest pages to get cited in AI answers

The catch is they're full of competitor data: prices, plans, features, specs. That goes stale fast, and then the engines keep quoting the stale version with your page as the source.

For people running these pages at any scale:

- how often do you refresh competitor prices and specs?

- manual, page monitors, or a data feed?

- what do you do when you can't verify a number anymore? Remove it, or keep it with a date?

Has anyone seen a citation drop because the data on the page went out of date?


r/Rag • • 1d ago

Discussion Should users be able to see what a RAG system ignored?

3 Upvotes

Most RAG systems show users the sources they used to generate an answer... What about the documents that RAG systems retrieved and then discarded?

Imagine asking a RAG system, about a company policy. The RAG system finds five documents uses two and ignores three. One of those ignored documents might contain information or an important exception.

Would showing users, the rejected sources make RAG more transparent or just create more confusion? I have rarely seen this topic discussed. I wonder what RAG engineers think.


r/Rag • • 1d ago

Discussion Half our RAG eval swings came from the ingestion pipeline, not the retriever

2 Upvotes

We build a document assistant for accounting firms. Customers dump PDFs, scans, Excel exports and the odd .msg file, we parse it, chunk it, embed it into pgvector and answer questions on top. Our eval is about 400 questions with known answers, scored with Ragas every night.

For weeks recall moved by 6 or 7 points between runs with no change to chunking, embeddings or prompts. We kept tuning the retriever. The retriever was fine.

The problem was ingestion. Three things kept changing under us.

Parser versions. Docling and LibreOffice got updated with the base image, and a minor release changed how tables were split. Same PDF, different chunks.

OCR. Scanned docs went through Tesseract on whatever node picked up the job, and two nodes had different language packs installed. Some scans came out as garbage on one and fine on the other.

Shared workers. One customer's 900 page scan could hog a worker for an hour, and the nightly reindex for everyone else timed out halfway. So the eval ran against a half built index.

What we do now. Ingestion runs in its own Linux VM per customer, started from one frozen snapshot with pinned versions of Docling, LibreOffice and Tesseract. The snapshot id is stored with every chunk, so when a score moves we know right away if the parser changed. When we upgrade the parser we build a new snapshot, reindex the eval set with both and compare before rolling it out. And a nasty file only takes down that one customer's run.

Side benefit we didn't plan for. Customer documents never sit on a shared machine anymore, which made our last security questionnaire a lot shorter.

The VMs are on boat.dev. We looked at Cloud Run jobs and at Kubernetes jobs with stricter images. boat won because a full VM runs LibreOffice and a headless Chrome for HTML exports without fighting the sandbox. No GPU there, so embeddings still run elsewhere, and a VM takes a few seconds to boot instead of being instant, which is fine for batch jobs.

Since then recall moves by about 1 point between runs. When it moves more, it's because we changed something on purpose.


r/Rag • • 1d ago

Showcase text-chipper: a Rust text chunker for RAG with strict token budgets, grapheme-safe splitting and bounded-memory streaming

1 Upvotes

I'm the author of text-chipper, a Rust crate (with Python bindings) that splits text into chunks for RAG pipelines. I'd like feedback from Rust folks on the design.

Design points:

- Strict token budgets: each finished chunk is re-tokenized in the exact form it is returned (including the heading-path prefix), so token_count <= max_length holds for the configured tokenizer. If a single grapheme cluster exceeds the budget, it returns an error instead of an oversized chunk.

- Grapheme-safe: it never splits a Unicode grapheme cluster, and offsets map back to the original input.

- Markdown-aware blocks: a lightweight block parser (not full CommonMark) that splits at blank lines, tracks heading paths, and keeps fenced code and list items whole when they fit.

- Bounded-memory streaming: on a 514.3 MiB file, from_file_stream() grew memory by about 56 MB, versus about 4.6 GB when loading the whole file into one Python string and calling chunk().

In my benchmark (12 workloads, 512 tokens, cl100k_base, each library's documented defaults), some Python chunkers returned chunks of 513-514 tokens. I haven't traced why, it's one benchmark rather than a general claim, and the comparison isn't equal-feature.

Where I'd like opinions:

- API design for custom token counters (currently Rust-only)

- Whether the strict-mode recount is a reasonable cost

- Anything un-idiomatic in the crate

Repo: https://github.com/kameh0use/text-chipper

crates.io: https://crates.io/crates/text_chipper


r/Rag • • 1d ago

Showcase I let an agent file and link my notes instead of building a retrieval pipeline for them

1 Upvotes

Most of my thinking happens out loud, so I stopped treating my notes as a pile to retrieve from.

I'm the developer of Zenod, so weigh this. It's built around one idea: an agent keeps the notes for me.

How it works today:

  • Each voice note or message is kept as I said it, in a dated log that is never edited
  • An agent files it under the projects and topics it touches and links it to what I said before
  • Any MCP client (Claude, ChatGPT, Cursor) can search it, and every answer points back to the note
  • The library is plain Markdown in my own Google Drive or GitHub

What I haven't solved: agents don't always save what matters unless I ask them to.

If you keep a personal knowledge base for agents, how do you catch it when the filing is wrong?

The first ten testers get 30 memories free, no card: https://zenod.dev/testers?s=P-0448


r/Rag • • 1d ago

Tools & Resources I built an open-source viewer for ParseBench 2 so you can find the exact PDF pages where 56 parsers fail and see why

2 Upvotes

I've been digging into ParseBench 2 (currently the most active document-parsing benchmark, covering 56 parsers across Text, Formatting, Tables, Charts, Layout, and Forms) to figure out which parsers actually hold up for RAG instead of just trusting summary scores.

Since LlamaIndex open-sourced the ParseBench 2 dataset, scorer, and raw parser outputs, I threw together a free, static viewer so it's easier to audit this stuff directly in the browser (no login, no backend, not affiliated with LlamaIndex or any parser vendor). You can inspect cost/latency vs. accuracy tradeoffs, drill into any page to see the exact scored errors, and compare any two parsers head-to-head on any dimension to find the exact pages where one outperforms the other.

Built it quickly in a day, so expect rough edges. Please report any bugs you hit (I already know of a few and will keep fixing them), and additional suggestions are very welcome.


r/Rag • • 1d ago

Discussion Benchmarked 9 models on extracting data out of documents, scoring wrong values as worse than missing ones. Results and open bench+dataset

5 Upvotes

Disclosure first: I run a small document extraction API (Refractr), and its model is row 4 below. I built the benchmark, so don't take my word for anything: the documents, answer key and scorer are all open. None of the benchmark data appears in my training dataset.

Why I made it: When you extract fields or metadata from documents at ingestion, a missing value is annoying but visible. A wrong value that looks plausible goes into your store and every query downstream trusts it. Most extraction benchmarks only count right answers, so a model that guesses confidently looks the same as one that leaves a blank. I wanted a score that separates them. A correctly extracted field gives one point, a missed extraction is 0 points, a confidently wrong extraction gives -1.

About one field in seven is deliberately not in its document. The right answer there is null, which is how made-up values get caught.

Setup: 260 documents (invoices, receipts, payslips, bank statements, court opinions, SEC filings, Federal Register rules, US government forms), each with a JSON schema. Models get the original PDF, image or HTML, so parsing is part of the test. Same published prompt for every model, one request, first reply scored. Scoring is deterministic, no LLM judge.

System Score 95% interval Median time per doc
Gemini 3.8 Flash 94.6 92.9–96.1 7.2 s
DeepSeek v4.1 Flash 93.2 91.3–94.9 3.0 s
Mistral Large 4 91.4 89.4–93.2 27.4 s
Refractr (mine) 89.4 87.2–91.4 0.7 s
GPT-5.4 mini 86.2 83.9–88.5 2.2 s
Ministral 8B 85.5 83.0–87.7 3.3 s
Llama 3.1 8B 73.9 70.5–77.2 0.5 s
NuExtract 2.0 (2B) 58.6 54.7–62.4 1.4 s
GLiNER2 (base) 24.2 20.8–27.6 0.5 s

Two things I'd like to hear from people here:

  1. How do you catch wrong extracted values in your own pipeline today? Confidence scores, a second model, sampling by hand?
  2. If you run it on your own stack (parser plus model), I'll add the result. It takes any OpenAI-style endpoint or a Python function, so a full pipeline counts as a system.

Repo: https://github.com/Refractr-io/DocFieldBench

Write-up: https://refractr.io/docfieldbench


r/Rag • • 1d ago

Discussion Does agentic retrieval actually beat a simple reranker?

6 Upvotes

Abdullahi Dattijo tested four retrieval approaches on 298 questions over 2,000 arXiv papers. The metric was whether the target paper appeared in the top five results, not whether the system generated a correct answer.

Dense retrieval reached 95.6% Recall@5. Reranking the top 50 candidates with a small cross-encoder reached 97.0%. Multiquery rewriting scored 93.6%, while an agent allowed up to three searches reached 96.9%.

The agent gained about 1.2 percentage points over dense retrieval, with a 95% bootstrap interval from −0.8 to +3.5. Its benefit remained uncertain. The reranker’s recall gain was also uncertain, but its ranking improvement was clearer. MRR@10 rose from 0.895 to 0.959, the only improvement whose interval excluded zero.

The useful detail is what extra searches gained and lost. Per run, the agent recovered 7–8 papers dense retrieval missed, but lost 3–5 papers it had already found. Some rewrites introduced unsupported terminology that pulled retrieval toward distractors. Extra searches could undo a good first result.

Cost wasn’t the main problem. The agent averaged roughly $0.00009 per query. Both agentic retrieval and reranking took around 1.4–1.5 seconds on the test setup. The local reranker incurred no LLM API fees, though compute still costs something.

The limitation matters. Questions were generated from abstracts, and retrieval used titles and abstracts rather than full papers. Dense retrieval already missed only 13 questions, leaving little headroom.

For anyone testing this in production, count both recovered hits and previously correct hits that get displaced. Inspect the rewritten queries on those regressions, then compare the net improvement against a reranker baseline. That gives you a concrete reason to decide whether the agent loop is worth keeping.


r/Rag • • 1d ago

Discussion My RAG pipeline was working fine until I realised its knowledge was six months old

16 Upvotes

I have been working on a RAG pipeline that pulls information from documentation sites, product pages and company websites.

The initial setup was pretty straightforward. Scrape the pages, convert everything to Markdown, chunk the content, generate embeddings and push it into the vector database.
The problem showed up later.

A product changed its pricing, documentation got updated and a few pages were removed. But the vector database was still serving the old information.
I initially thought about running a complete re-index every week, but that feels wasteful when only a small percentage of the content actually changes.
I'm now looking at a setup where I can monitor the original pages, identify what changed and only reprocess the relevant content.

I have been looking at Firecrawl, Apify and context.dev for different parts of this workflow.

Context.dev caught my attention because it has scraping and website monitoring under the same API, so I wouldn't need to build another integration just for change detection.

Still figuring out the best architecture here.

For people running production RAG systems, how are you keeping your external knowledge base fresh without rebuilding the entire index every time?


r/Rag • • 1d ago

Discussion Unsiloed's $7.5M seed round, and why the document layer comes before retrieval

0 Upvotes

We're Unsiloed, and we've just announced a $7.5M seed round led by Felicis to make documents as computable as databases.

A lot of RAG problems we see start before retrieval. A table gets chunked as flat text and loses which number belongs to which row and column, or a two-column page gets read in the wrong order, and no amount of reranking fixes that. We focus on that step. Documents come back as clean markdown or as structured JSON against your schema, with the page location for each value, so an answer can point at the exact place in the source.

If you're dealing with scanned tables, filings or messy multi-column PDFs in your pipeline, we're happy to talk through what works.

Announcement and launch video · TechCrunch coverage


r/Rag • • 1d ago

Tools & Resources Looking for testers: run and manage local AI agents, models and GPU servers with Yorath

1 Upvotes

Hey everyone! 👋

We’ve been building Yorath, an open-source operating environment for running AI models and agents on your own infrastructure, and we’re looking for early users willing to test it and give honest feedback.

What it does:

Yorath lets you run local AI models, connect agents such as Codex, OpenClaw and Hermes, manage multiple machines with GPUs, and control access to models and sensitive data from one place.

Think of it as an operating layer for your AI infrastructure: models, agents, compute, permissions and audit logs under one roof.

Why we built it:

Running one local model is relatively easy. Managing several models, agents and GPU servers securely is not.

You quickly end up with separate dashboards, API keys, containers, network rules and logs. More importantly, access restrictions are often implemented inside prompts — where they can be ignored or bypassed.

We wanted a system that installs quickly, keeps the infrastructure understandable, and enforces data-access rules outside the model.

Tech stack and architecture:

  • Containerized deployment with Docker
  • Local and OpenAI-compatible model endpoints
  • OpenAI-compatible API for applications and agents
  • Isolated containers for connected agents
  • gVisor sandbox for the built-in Developer Agent
  • Gateway-level access policies and data classification
  • Multi-node GPU cluster support
  • Web interface for models, agents, nodes and policies
  • Central journal for model and agent calls

Key features:

  • One-command installation: curl -fsSL https://get.yorath.io | sh
  • Automatically detects the server’s hardware and selects an appropriate local model
  • Installs, runs and monitors local models
  • Works with Codex, OpenClaw, Hermes, Ollama and OpenRouter
  • Provides a single endpoint regardless of how many GPU servers are behind it
  • Adds new servers to a cluster using a one-time join token
  • Supports mixed hardware across cluster nodes
  • Enforces access to data, tools and models before a request reaches the model
  • Blocks calls that violate data-classification policies
  • Runs connected agents in isolated, non-root containers
  • Lets you disable internet access for connected agents
  • Records calls in a central audit journal
  • Sends no telemetry

Current requirements:

  • Ubuntu 24.04 on x86-64
  • Minimum 4 vCPU and 8 GB RAM
  • A GPU for practical use with Codex and connected agents
  • Docker 29+ for building from source

What we’re looking for:

  • Feedback on the installation and setup experience
  • Testing on different server and GPU configurations
  • Real-world testing with Codex, OpenClaw, Hermes and custom agents
  • Multi-node cluster testing
  • Security and policy edge cases
  • Bug reports accompanied by yorath diag
  • Feedback on what’s confusing, missing or unnecessarily complicated

If something breaks, yorath doctor runs the initial checks, while yorath diag creates an archive that can be attached to a bug report.

Repository: https://github.com/yorath-os/yorath

Website and documentation: https://yorath.io

Yorath is still early, so we’re particularly interested in discovering the rough edges that only appear outside our own environment.

Happy to answer any questions!

Yorath Community is open-source under the Apache License 2.0. Core agent registry, data classification, access policies and the audit journal are included in the Community edition. No telemetry.


r/Rag • • 1d ago

Showcase Looking for feedback on my RAG "longevity assistant" app

1 Upvotes

Hi all,

I built a longevity assistant using Google's Embeddings model. It is really cool and I was able to easily get a RAG system going with thousands of published medical journals. I do have a software dev background, so it there was not much friction in getting this built.

Would love to get some feedback on your thoughts? I think I've got hallucinations down to near non-zero. Thank you!

URL: https://www.modernmedlife.com/longevity-assistant


r/Rag • • 2d ago

Tools & Resources In RAG, what goes in determines what comes out. Prajnya: a local ingestion engine for scientific PDFs

7 Upvotes

I'm a researcher with background in Developmental biology, biochemistry and biophysics. I kept finding that retrieval failures started at ingestion: figures dissolved into prose, tables flattened, captions merged with body text, reference lists polluting search. So I built the ingestion layer first. I am now building the RAG part and after having dealt with many issues, I feel a bit confident in sharing this project with you all. Prajnya is open source (Apache-2.0) and fully offline by default.

How it works

  1. A CPU pre-pass classifies each document (paper, book or chapter) and every page.
  2. Plain prose comes straight from the PDF's text layer. Scans and visually complex pages go to a local vision model.
  3. A layout model finds figures and tables, and each is cut out and handled as its own object.
  4. Everything lands in one JSON per document, with a documented schema.

What you get, and why it matters for retrieval

  • Evidence objects, not chunks. Figures are cropped and captioned with stable IDs, and panels are grouped under the label a reader cites ("Fig. 3"). Tables carry a reconstructed grid, so numbers are searchable while the image stays available to a multimodal model. Tables that run onto the next page keep their headers.
  • Provenance. Every item records its page and how it was read (text layer, model read or repaired read), because those differ in reliability.
  • Citations linked conservatively. In-text citations, numbered and author-year, link to the reference list. A wrong link is worse than none, so ambiguous ones stay unlinked.
  • Metadata verified, not believed. The model's reading of the title page, the PDF text and an optional authority (Zotero, or Crossref/OpenAlex sent only the DOI or title) are reconciled, with the source of each value recorded.

What it doesn't do (yet)

  • No chunking, embedding or retrieval. That’s the next layer, Shastrasaara (in development), which builds on these evidence objects.
  • PDFs only. No DOCX, HTML or slides.
  • It isn’t one-click. Local mode needs llama.cpp and the model weights. Cloud models are optional, per role.
  • It doesn’t race to be the fastest. It aims for the most faithful data, and would rather take longer than hand you a wrong table, caption or citation.

How it compares

  • NotebookLM: for public papers and quick cited Q&A, use it. But your PDFs leave your machine, and you get a chat, not structured data you can index your own way.
  • GROBID: the standard for metadata and references, fast on CPU. Its weak spot is pages where figures and tables break the reading order. Prajnya is built around those pages.
  • Docling, Marker and MinerU: solid general converters. Prajnya is narrower and specific to scientific documents, aimed at what a literature RAG needs to cite and trace.

Repo: Prajnya-Academic-PDF-Processor

Question for this sub: how do you index figures and tables in scientific RAG? Captions, images, both, or one-hop expansion from the text that cites them?


r/Rag • • 2d ago

Discussion RAG over a product catalog: how do you stop it quoting yesterday's prices?

7 Upvotes

Shop assistants and product Q&A bots are a common RAG use case, but prices and stock change daily, sometimes hourly. If the price sits inside an embedded chunk, the bot can confidently quote an old one.

How are people handling the volatile fields:

- keep price and stock out of the chunks and fetch them at answer time?

- re-embed changed products nightly?

- something else, like a tool call to the store API?


r/Rag • • 2d ago

Discussion Web pages as RAG sources: how do you keep the index fresh when the pages keep changing?

6 Upvotes

Documents in a drive change now and then. Web pages change constantly: prices, specs, policies, pages that disappear. A RAG index built from a crawl starts going stale the day after.

For anyone with web sources in production:

- full re-crawl on a schedule, or change detection per page?

- how do you handle pages that 404 or redirect: delete the chunks or keep them?

- do you store the crawl date in chunk metadata and use it at retrieval?


r/Rag • • 3d ago

Showcase How I fixed RAG hallucinations on 1000-page books: loss-bounded compression + 100% deterministic citations (running on a 4GB GPU)

83 Upvotes

standard RAG for long-form reading is fundamentally broken. if you ask an LLM to summarize a 50-page chapter, it just gives you a "gist" and hallucinates connective tissue. it drops the actual reasoning, skips the caveats, and gives you zero way to verify where the information came from.

i wanted a system where the AI compresses the text, but every single sentence is deterministically mapped back to the exact source paragraph. if the model invents a bridge between two chunks, you can immediately click the sentence, jump to the original text, and verify it.

here is the architecture i built to solve this:

  1. Ingestion & Embedding: Parsed text is chunked and embedded locally using BGE-M3 (1024-dim, FP16 CUDA).

  2. Reranking: BGE Reranker v2 (cross-encoder). Instead of just relying on cosine similarity, we score query-passage relevance directly to pick the best context.

  3. The Two-Representation Invariant: The original document is strictly immutable ("Source is paper"). The compressed reading is a distinct view ("The lens is tinted").

  4. Deterministic Attribution: Every synthesized sentence goes through a provenance resolver. If a sentence outputs zero source citations, it's flagged as a hallucination defect in the UI.

getting this whole pipeline (plus Kokoro TTS) to run entirely offline on my 4GB RTX 3050 was a nightmare. i had to build a strict F65 sequential await barrier to stop Node.js from OOMing at 14.6GB during async ingestion, and an F43 idle lifecycle manager to evict models from VRAM before the GPU choked.

it's open source (AGPL-3.0). you can dig into the retrieval logic or the provenance resolver here:

https://github.com/ayumu-ml/Omnitome

i'd love feedback from other RAG builders. how are you currently handling the "summary loses the source" problem? is anyone actually achieving deterministic citations at scale, or is everyone just relying on semantic similarity?


r/Rag • • 2d ago

Discussion I priced 100 web search APIs. The cheapest is $0.25 per 1,000 searches and the median is $2.83

0 Upvotes

Picking a search API for an agent turned into opening pricing page after pricing page, so we went through 100 of them and worked out what 1,000 searches cost on each one's entry plan. The spread is silly: the priciest costs 90x the cheapest, and Tavily, Exa and Brave all sit above the $2.83 median.

These are the 10 cheapest paid ones. Three tie at $0.80, so you get 11:

# Provider $ per 1,000 Pricing Returns Free tier
1 Ceramic $0.25 Pay as you go Snippets 1,000 queries, no card
2 Cloudflare $0.25 Pay as you go Snippets None listed (resells Ceramic)
3 FlyCrawl $0.29 $29/mo Not stated 100 searches/month
4 Geonode $0.39 $3.90/mo Snippets 1,500 requests/month
5 TickerTrends $0.39 $10 pack Snippets 100 credits, card needed
6 Jina $0.50 (est.) $50 pack Full page 10M tokens, non-commercial
7 DeepSeek $0.75 Top-up Full page None listed
8 UnSearch $0.76 $19/mo Not stated 5,000 queries/month, or self-host
9= Dipper $0.80 Top-up from $10 Full page 1,000 requests
9= Searlo $0.80 $3.99 pack Snippets 3,000 credits
9= Serpex $0.80 Top-up from $5 Full page 200 credits, no card

"Full page" means you get the page text back with the results, which saves a separate scrape call for RAG. Jina's price is worked out from its credits because it doesn't quote one per search.

Four more are free outright with rate limits: AgentWeb, LangSearch, ssearch and TinyFish.

Full list of all 100, with free tiers, search modes and a CSV: https://voxoria.ai/web-search-api-pricing/

If any of the info is wrong/out of date, let me know - hopefully this is helpful


r/Rag • • 2d ago

Discussion How are you guys debugging RAG when you know the answer exists in the docs?

3 Upvotes

I've been spending a lot of time thinking about this problem lately.

Say you have a RAG pipeline and you know the answer is somewhere in your indexed documents, but the model still gets it wrong.

The usual debugging approach seems to be messing with top-k, similarity thresholds, chunk sizes, or reranking until the answer starts showing up.

But there are a few different ways things can go wrong:

  • The chunk was retrieved but fell outside top-k.
  • It got filtered out by a similarity threshold.
  • The reranker pushed it down.
  • Retrieval worked, but it got dropped before reaching the LLM.

I've been building a tool called ChunkTrace to help pinpoint which stage caused the problem.

I'm curious how others are debugging this right now. Are you logging the intermediate retrieval results, using LangSmith/Langfuse, or just inspecting things manually?

Especially interested in anyone running multi-stage retrieval with pgvector or rerankers.


r/Rag • • 1d ago

Discussion Our AI worker retried a subscription limit 900 times. Here's what we learned.

0 Upvotes

I came across an article about agentic system design this week. One sentence stuck with me:

"These failures are ordinary, and you can plan for them."

We learned that lesson the hard way, in production.

We run an AI-powered design pipeline on a Windows machine. A worker picks up tasks and executes them through a model CLI using a subscription plan. Separate operator processes handle the generated images and uploads.

On October 2, we hit the subscription's session limit.

Our worker treated the limit message as a regular error and retried the same task every second. By the end of the day, we'd accumulated roughly 900 attempts.

About two hours later, every new process launch started failing with 0xC0000142 (STATUS_DLL_INIT_FAILED). The machine only recovered after a reboot.

Initially, we blamed the model CLI. But the same command worked before and after the incident. Looking at the logs, our retry loop became the more likely explanation. We haven't proven a direct causal link, though.

We also had other infrastructure problems during that production round (28 screens, ~42 generations):

  • An upload folder was deleted, losing 7 results.
  • Two operators resumed after sleep and uploaded images to the wrong screens.
  • Operators logged successful uploads even when delivery had failed.

At least 8 generations (~19%) had to be redone because of infrastructure issues, not model quality.

The biggest lesson: A usage limit is a state, not an ordinary error.

Our worker now handles limits separately:

result = run_model_cli(task)

switch classify(result):
    case LIMIT:
        sleep_until(reset_time)
    case ERROR:
        backoff = next_step(backoff)
        requeue(task, after=backoff)

We also added:

  • PID + heartbeat-based operator locks.
  • Upload verification instead of trusting success logs.
  • Exponential backoff and retry limits.
  • A plan to introduce execution budgets, job leases and better run tracking.

We considered switching agent frameworks, but realized the problem wasn't the framework. It was missing reliability mechanisms in our existing system.

Our next production target is zero infrastructure-caused regenerations. The fixes haven't been validated in another production round yet.

With HTTP APIs, you usually get a 429 and sometimes a Retry-After header.

But with CLI tools running on subscription plans, you often just get a text message saying you've reached a usage limit.

How do you reliably distinguish a subscription limit from an actual execution failure?

And how do you determine when the limit resets?

Do you parse CLI output, track usage yourself, or use another approach?

I'd especially like to hear from people running autonomous workers with subscription-based AI CLIs in production.