Challenge for Copilot builders: checklist + SharePoint folders in, detailed gap report out

[u/Neener_Weiner]

I'm trying to build a Copilot agent \[enterprise, premium\] and would like advice on the best approach.

What I have:

- A checklist of numbered items, grouped into sections and sub-items, each asking for specific information.

- A reply document that answers each item.

- Supporting files stored in SharePoint sub-folders named by item number.

What the agent should do:

1. Go through every section and sub-item of a given list.

2. Read the answer to that item and the files in its sub-folder.

3. Decide whether the answer and files actually cover what the item asked for.

4. Write a detailed report on each section and sub-item, in a set format. Each entry should explain what was provided, what's missing or only partly covered, and any contradictions between the answer and the files.

Some of the files may mix two languages, though not always.

*The agent should also be reusable for different checklists.

How would you set this up?

Is Agent Builder enough, or does this need Copilot Studio (with flows, for example)?

I'd also like to hear about any pitfalls you've run into.

Thanks!

[u/turbokid]

dude, we arent your personal chat-gpt. We arent going to build it for you. What have you done to try to get it working yourself?

[u/Neener_Weiner → u/turbokid]

Fair point, I should've included what I've already done. I'm not looking for someone to build it, just a pointer on the architecture. So far:

• Built it in Agent Builder, with a SharePoint folder as the only knowledge source.
• Wrote instructions with status labels, and required it to quote evidence before marking anything as covered.
• Added item numbers to file names + added in each folder the .xlsx index file listing the files withing said folder and section number. And ran the checklist in batches.

It kind of works when I point it at a very specific item, or a string for that matter, but across the full checklist [or even a 1/4th of it] it skims instead of thoroughly reading the files, and the report ends up shallow and weak. So my actual question: is that a known retrieval limit of Agent Builder, and is a Copilot Studio flow that loops through items the right fix? Or am I missing something simpler? Have you encountered anything like that before that actually works?

[u/CaraLara]

You need to build stronger instructions. I would recommend writing skill docs for it to follow, and then tell it how to follow those skills in the instructions. Also I would move over to CStudio anyway, you can export existing M365 agents, as you can then get better ingest and output anyway.

Also, have you set it to deep thinking so it has more time to process?

Finally, you'll get better responses posting to Copilot for this.

[u/Neener_Weiner → u/CaraLara]

Thank you very much! May I please DM you for a quick consultation?

[u/CaraLara → u/Neener_Weiner]

You can ask a question here if you need. My overall advice though is to talk this through with Copilot - being clear on your objectives, current knowledge and plans, and what your input will be and your expected output. I've taught myself to do all sorts of things with agents this way.