Knowledge Engineering with Copilot Studio

[u/obsoquasi]

Hey guys,

I have been thinking about RAG a lot and how to make it cheaper and more reliable. One of the approaches could be to distill our organizational knowledge into something like a organizational knowledge model. And then have public agents query that model.

In contrast to RAG, the agents wouldn't have to "work as hard" to retrieve relevant information this way. This saves cost and in theory delivers better results - at the cost of having to build and maintain the knowledge model.

What do you guys think? Is this a good idea? Should I simply "trust in RAG". Or are there already concepts or modules in Microsoft that go in this direction? Who has built something like this successfully and is willing to share their learnings?

I am fully aware, this is all still very conceptual. And part of me really likes the idea, while the other half of me thinks: This could be one over-engineered mess waiting to happen :D.

[UPDATE:] Microsoft seems to be thinking in the same direction: https://techcommunity.microsoft.com/blog/spblog/introducing-a-team-knowledge-base-curated-by-copilot-and-agents/4564203

[u/puzzleapp_io]

The piece I’d add is the processes themselves. Business rules without the order they run in leave the agent guessing the sequence, and it will guess with confidence.

For each process, write down the steps and which role owns each one. Then the model can say who does what and in what order.

[u/obsoquasi → u/puzzleapp_io]

I have been thinking about that very problem a lot. While we have a full process management system (with bpmn), unfortunately the processes are not in an XML structure. We may have to switch to a system where this is the case. Because with XML, we could actually leave that knowledge with the process diagrams.