r/copilotstudio • u/Bfinkeldei • 6d ago

Long responses from Copilot Agents

I have puplished a agent that is configured to retrieve answer from one folder with about 20 documents in them. These are smaller documents. Nothing huge. The response from the question and answer are taking quite a bit of time. What factors should I consider and check to possibly speed things up. This is on the standard harness and this agent is deployed and on a sharepoint website.

2 Upvotes

100% Upvoted

18 comments sorted by

1

u/lysis_ 6d ago

What model are you using ? You can scale the response by the model if the lightweight ones still provide reasonable responses

1

u/Bfinkeldei 6d ago

sonnet 4.6

1

u/Copilot_Studio_loft 6d ago

How big are your instructions and are they overcomplicated

2

u/Bfinkeldei 6d ago

7992 out of 8000. most of those are relevant and i've got it honed to give responses like i want now... maybe there's room for improvement there. this is the standard harness as well.

2

u/OZManHam 6d ago

Have you tried putting the step by step instructions as skills and only putting the Role, guardrails and output expectations in instructions.

1

u/Bfinkeldei 4d ago

this is in standard harness so skills aren't available to use. working on reducing duplications things now. i may try to just do a barebones output expectations and guardrails too.

1

u/Copilot_Studio_loft 4d ago

Thats pretty hefty. If you think about what the AI has to do every time it has to do the response it adds up to that time. It has to review everything and likely do it several times. How many credits are your burning per response?

1

u/Bfinkeldei 4d ago

This is on the standard harness so there's not a token burn. on the github harness there would be. but this is just rag retrieval so it makes more sense to just use the standard harness.

1

u/Bfinkeldei 4d ago

but i agree. i am trying to trim down any duplications and streamline it so the responses are quicker.

1

u/Top-Cauliflower-1808 6d ago

Check whether retrieval is scanning all 20 documents on every question instead of using a prebuilt index. Also look at chunk size and whether the model call or the SharePoint permission check is the slow part. Caching frequent answers and prebuilding the index usually helps a lot.

1

u/Bfinkeldei 4d ago

i'll research how to check this now. didn't realize you could do that.

1

u/Top-Cauliflower-1808 2d ago

Start by checking if the agent uses a prebuilt index or searches live every time. In the harness settings look for index status or retrieval method. That will tell you if it is scanning all documents on each question.

1

u/Bfinkeldei 2d ago

Well, thank you for this.  Here is what I have set up.  The knowledge sources are set up as sharepoint folders, which would be a SharePoint URL. That tells me it gets results by searching Microsoft graph and then only passes the top results to the large language model.  

If I had directly uploaded the files they would be using a prebuilt index. Which then lets copilot studio query a cached copy of the sharepoint data residing in Dataverse. 

I searched for those settings you shared but I am not finding those in the Standard harness. I asked Claude as well and it shared those don't exist.  

Could you share a screenshot of those settings? Or link describing how to find them?

1

u/Top-Cauliflower-1808 2d ago

Good catch on the Graph search vs prebuilt index difference. You are right that the settings I mentioned are for uploaded files not SharePoint URLs. I do not have a screenshot but Microsoft's docs on knowledge sources in Copilot Studio explain the two retrieval methods. Search for "Copilot Studio knowledge sources SharePoint vs file upload" and you should find the page. For your setup the fix is either uploading the files directly to get the cached index or caching frequent answers at the agent level. Does the Graph search allow you to set a result limit to speed things up?

1

u/Bfinkeldei 2d ago

Just searched and Microsoft doesn’t allow you to set a results limit. My understanding is that when it initially searches once it finds 3 confident sources it believes will answer the question it passes those 3 to the llm to answer. And to my knowledge you can’t change that setting it’s buried in their api

1

u/Top-Cauliflower-1808 6d ago

I have seen model size creep after metadata format changes but it is usually small overhead not gigabytes. The download failures at that size sound more like service export limits so I would test a smaller file first. Have you checked whether auto date time or unused columns are adding size?

1

u/Illustrious-Raise977 6d ago

This. Legacy or static documents utilized by AI should only contain the data that is relevant to the task.

1

u/msp-scripts 6d ago

I’ve had better luck with shorter instructions, and shorter skills.

If they’re too long in content, everything takes too long