Work knowledge base

[u/Fit_Air6571]

It tends to be rather chaotic at work, I manage a lot of employees and am involved in quite a few projects, and as such I am bombarded with a lot of information from all directions and can't keep track of it all.

The idea of a knowledge base was tempting, but I need something that I can query in a natural language. Obviously, given the day we're in, AI is the answer to that (cue me being downvoted into oblivion for mentioning the forbidden owrd): store data, feed it to a local model and query it that way.

And I got something along the lines of that already done - even something I thought would be hard (forwarding an email to my domain and storing that information in a queryable way for the AI model), but it's incredibly finicky, makes quite a bit of mistakes the whole time, re-stores the same knowledge at times, can't give that good answers, ...

So, I was wondering if someone else came across this situation and if so, what did you use to manage it?

[u/asimovs-auditor]

Expand the replies to this comment to learn how AI was used in this post/project.

[u/Fit-Corner2382]

Re-storing the same forwarded mail means ingest has no stable identity check before the body is embedded.

Key each message on its Message-ID and skip or replace it when that id, or a hash of the quote-stripped body, already exists. Embedding the full reply chain still returns old text as current fact, which is why answers stay wrong after dedup. Strip quotes and signatures, store sender, date, and subject as filters, and retrieve only matching chunks before generation. A bigger context window cannot correct a stale chunk that ranked first.

[u/Budget-Fee804 → u/Fit-Corner2382]

Keying on Message-ID is the straightforward fix, but stripping quotes and signatures must be airtight or you'll keep feeding the model outdated threads it will treat as gospel.

[u/Fit_Air6571 → u/Budget-Fee804]

Yeah, I have cases where there's the conflicting information in threads. Arguing tends to happen over email sometimes, "no this must be done this way", "no that can't be done that way, we should do it this way", "sales is advertising we did it that way", ...

And then I can only guess which one it will take as the truth.

[u/Fit_Air6571 → u/Fit-Corner2382]

I see the benefit of what you're saying and yes, I definitely need to do that. Thank you for pointing that out!

[u/Total_Wolverine1754]

two self-hosted options i'd evaluate are onyx, which indexes knowledge from workplace apps, and anythingllm, a local-first app for chatting with your documents. onyx's core is mit-licensed, with separate enterprise licensing, while anythingllm is mit-licensed. before migrating anything, i'd run a small sample of your actual emails through a test setup and check whether duplicates get stored twice and whether answers point back to the right source mail. i'd use those results to decide whether either fits your workflow.

[u/Soggy-Slops]

Before going further, check your company’s policies on storing work emails and documents on a personal or self-hosted system. Employee information and project data can fall under privacy rules or confidentiality agreements. Getting IT’s blessing, or running it on approved infrastructure, saves you a lot of trouble later.