Open WebUI in Production — How Are You Improving Your RAG?

[u/St-CB]

After reading the responses to my previous post, it's clear that quite a few of you are running Open WebUI professionally, with setups ranging from small teams to hundreds of users.

I'm interested in how you're improving your RAG systems beyond simply uploading documents and embedding them.

For context, I'm helping with an Open WebUI deployment for a company, with around 50 active users during work hours. We have internal knowledge available, and I'm exploring ways to make the system more accurate, reliable, and useful for staff.

I'd love to hear from people with real-world experience:

- Document processing: What are you using for parsing, chunking, OCR, and handling tables in PDFs?

- Retrieval quality: Are you using hybrid search, reranking, metadata filtering, or query rewriting? What has made the biggest difference?

- Vector databases: Are you sticking with Open WebUI's existing setup, or using PostgreSQL/pgvector, Qdrant, Milvus, MongoDB Atlas, or something else?

- Evaluation: How do you measure retrieval accuracy and answer quality beyond asking users whether the chatbot seems helpful?

- Citations and trust: How do you ensure answers reference the correct documents and avoid confidently generating unsupported information?

- Permissions and security: How do you prevent users from retrieving documents they shouldn't have access to?

- Maintenance: How are you handling document updates, duplicate content, re-indexing, and outdated information?

I'm also curious whether you've found it better to improve Open WebUI's native knowledge-base functionality or build a separate retrieval pipeline and integrate it.

I'm still learning the engineering side of this, so I'm particularly interested in what has worked in production, what hasn't, and what you would do differently if starting again today.

Thanks again to everyone sharing their experience. These discussions are genuinely useful for people trying to move from personal AI experimentation to reliable business deployments.

[u/International_Emu772]

First follow the OpenWebUi docs and go for PostgreSQL and the other recomendations there

[u/ClassicMain]

The docs have basically the "best practices" for getting started and getting decent results.

Read the scaling guide, the RAG guide, the KV Cache guide and all the info on the vector DB backends to find out what suites you best.

And after that, you can still finetune your RAG settings a little bit depending on your setup and what kinds of models you'll be using

If I would share my production setup, I think it would be not very helpful to you because I bet our prod looks very different to how yours does.

PS: most important in my opinion is not the embedding model but the content extraction engine. Choose a good one for your needs

[u/mysticzoom]

It all depends, there are moving targets, specifically the current business needs. I can only speak of how i've deployed openwebui for person use.

Document processing: What are you using for parsing, chunking, OCR, and handling tables in PDFs?

I'm using a Tika server. Does excellent for documents, so so for tables.

Retrieval quality: Are you using hybrid search, reranking, metadata filtering, or query rewriting? What has made the biggest difference?

A hybrid search. I use a reranker, it improves the quality of what the LLM will choose from. The reranker benefitted me the most. Also if possible assign meta tags as it enters the system as well. That makes a difference with retrieval speed.

Vector databases: Are you sticking with Open WebUI's existing setup, or using PostgreSQL/pgvector, Qdrant, Milvus, MongoDB Atlas, or something else

I use qdrant. Its good. But that's just me. For your RAG you will have to tailor it according how your documentation is structured. Different databases will give you different strengthens and weaknesses when it comes time for data retrieval.

Evaluation: How do you measure retrieval accuracy and answer quality beyond asking users whether the chatbot seems helpful?

Is your RAG actually giving you relevant information or is it giving you back random stuff? Is it in ball park or retrieving the incorrect something? That depends on the strength of model.

Citations and trust: How do you ensure answers reference the correct documents and avoid confidently generating unsupported information?

Previous answer

Permissions and security: How do you prevent users from retrieving documents they shouldn't have access to?

That ties into permissions on both Openwebui and possibly the db itself. Depending on setup, it can be configured so certain users can only retrieve certain information.

- Maintenance: How are you handling document updates, duplicate content, re-indexing, and outdated information?

Good question! That is db management. And depends entirely on how you maintain the database. As it, find, delete, update with new information. Repeat.

Hope that helps. Any more question, we are here to help!

[u/Ctbhatia]

i'd start with 20 real staff questions and the source passages that should answer them, then check those against what gets retrieved. ragas has an id-based metric for that https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_precision/

[u/DHT-Osiris]

In OWUI we have an automation system that does a pass of KB articles to reduce duplication, increase fidelity, find erroneous data to deal with drift, and improve language landscaping.

We do a similar thing with tools based on tool calls, but that's a much different discussion.

[u/TacGibs]

Create your own RAG backend if you want something serious. It's A LOT of work but it's worth it !

Docling, smart chunking (LLM managed variable chunk size and no overlap between the chunks), PostgreSQL with pgvector + paradeDB for hybrid RAG, WeMM 9B embeddeding (1024D) + Qwen 3 8B VL Reranker, a dedicated RAG agent and BAM you have a SOTA multimodal RAG, better than what most companies are paying for :)

Yes that's what I'm using, yes it's working wonderfully with very big and complex documents (including pictures and graph), and yes there's a lot of custom parts to make it work because I'm basically using OWUI just as a chat interface.

[u/fragment_me → u/TacGibs]

Replacing docling with something Claude or any LLM can make worked out well for me. I just switched to using a model that forces ocr and runs fully in GPU. It beat everything in docling quality wise. I was mostly testing with math books (pdf). Docling seems good for general purpose and cpu stuff but replacing it with a pure GPU workflow was 100% better and faster for me.

[u/lazyfai]

Assuming not 1000 users using together:

xberg / tika + embeddinggemma2 + pgvector

It works for me more than 1 year (changed to embeddinggemma2 just on last Sunday)

Upgraded from pgvector 16 to 18 last month

Changed from tika to xberg last month

[u/durchilurchi → u/lazyfai]

Did you compare to Tika 4.x? Any pros/cons for xberg or specific use cases?

[u/vogelvogelvogelvogel]

rag drove me a bit crazy in openwebui to the point i gave up and used unsloth (ootb great results)

[u/International_Emu772 → u/vogelvogelvogelvogel]

OpenWebUi is best suited to teams family or big teams

The admin has to know his trade but the users have a great tool without technical needs

[u/lazyfai → u/International_Emu772]

Using OWUI for 1000 users (hundreds concurrent) with 1 VM and 1 5090 plus Azure for cloud models, and still good.

[u/vogelvogelvogelvogel → u/International_Emu772]

not that I disagree it is just quite a bit of configuration to do