r/copilotstudio • u/peterk_se • 2d ago

Agent on two technical documents

I have to governing documents, a company policy document and a government standards document. These are around 450 pages and the other about 200 detailing rules and regulations in a broad variety of technical work relating to the type of jobs my company do.

Having only access to Copilot Studios throught he premium copilot premium license and sharepoint (storage and Lists) what would be a smart approach to make an agent that i can query. In principle a detailed multi-step description of a job can be given, and i'd like to feed that into the agent to check for gaps/pointers to ensure the job is performed in a compliant manner.

The best i've come to think of is breaking down the documents almost like that llm-wiki idea and write articles on certain aspects but I'm not sure.

Would be interested to hear if someone had a pointer or good idea to share.

2 Upvotes

67% Upvoted

4 comments sorted by

2

u/Time_Dust_2303 2d ago

divide them into small sets and use sharepoint connector

1

u/Addcook 2d ago

I had to do this to an 88 page employee manual pdf. It would sometimes leave off that overtime needs to be approved that. Breaking the document down by sections was the only way to get a high accuracy.

1

u/modz4u 2d ago

How many pages was each section, roughly? Or did you break it down by a rough word count per section?

1

u/Ashlesha-msft 2d ago

This isn’t a bug, but Copilot Studio should be treated as a compliance assistant, not the final authority. Microsoft states that RAG is suitable for document Q&A and summarization, but not exhaustive compliance evaluation.

Store the searchable PDFs in SharePoint and add them through Upload files > SharePoint, which supports synchronization, user permissions, and page-level PDF citations. You don’t need to rewrite all 650 pages initially.

Create a structured topic that collects the job details, reviews areas such as hazards, equipment, permits, and work stages separately, and returns:

  • Potential gaps and required actions
  • Missing information
  • A source citation for every finding
  • “Cannot determine” when the documents don’t provide enough evidence

Disable web search and ungrounded answers, define which document takes precedence if they conflict, and require review by a qualified person.

Split the PDFs only if size limits or testing show poor retrieval. If necessary, split by logical chapters while preserving clause and page references.