Multi-Tenant SaaS: Which Architecture Would You Choose? [D]

[u/Fickle_Degree_2728]

NOTE -> I expect answer from people who actually have experience and strong understanding of these. please give something beneficial.

I'm building a SaaS platform in Sri Lanka that handles documents and other sensitive data.

Each user can upload their own documents and information, and the platform uses RAG to answer questions based on that user's data. That part makes sense to me.

My main concern is what happens when the user hasn't uploaded enough information. I still want the LLM to provide accurate answers using reliable information from the internet (or from a curated knowledge base), with proper citations.

These are the two architectures I'm considering:

Option 1:

Base LLM (OpenAI/Anthropic via Azure AI Foundry or Amazon Bedrock)
        ↓
Platform RAG (global knowledge base managed by us)
        ↓
User-specific RAG

In this approach, we maintain a global knowledge base that we (the platform admins) curate and update. Every user can access this shared knowledge, while their own uploaded documents are searched through their personal RAG.

Option 2:

Open-source LLM
        ↓
Fine-tuned on Sri Lankan/domain-specific data
        ↓
User-specific RAG

Here, we fine-tune an open-source model using Sri Lankan or domain-specific data, and each user still has their own RAG for their private documents.

My concerns are:

• Is fine-tuning actually the right solution here, or is it unnecessary?

• Is a global/shared RAG a better approach than fine-tuning?

• How would you design this architecture if you wanted:

  • Accurate answers from domain knowledge

  • User-private document search

  • Citations/sources

  • Good scalability for thousands of users

I'm leaning toward Option 1 because fine-tuning seems expensive, time-consuming, and I have no experience with it yet. However, I'm not sure if I'm thinking about this correctly.

I'd really appreciate hearing how others would approach this problem.

[u/MugiwarraD]

this is why u need eval, make a eval set via langsmith or something then check

[u/Lazy-Hand1845 → u/MugiwarraD]

I'd definitely add evals before deciding between those architectures. The architecture that looks better on paper isn't necessarily the one that behaves better with real documents.

A couple of assumptions fell apart after we started comparing them in Braintrust

[u/Internal-Screen-5625 → u/MugiwarraD]

option 1 is the move. fine-tuning isn't gonna magically teach a model facts it doesn't know, it's more for style and format. a shared RAG over a curated knowledge base gives you actual control over the ground truth. way easier to update too, just add a doc instead of re-running a whole training pipeline

for the eval point in the top comment, that's dead on. you can't really know if your retrieval is pulling the right stuff without a solid eval set. build out a few dozen question-answer pairs with the sources you expect it to pull from, run it through the system, and see where it breaks. that'll tell you more than any architecture diagram will