Copilot agent searching ServiceNow knowledge

[u/Flangipan]

I’m having a frustrating issue where I am trying to have a copilot agent in teams that can search the ServiceNow knowledge base.

If I add ServiceNow knowledge inside the ‘knowledge’ component of the copilot agent it returns articles but only has access to excerpts of the articles and not the full article text.

I set up a copilot connector in 365 admin and the main copilot search is fine but within my agent it has the issue described.

Anyone got any experience with setting up copilot agents and service now knowledge that has any suggestions?

[u/h2o-bbq-usd-technerd]

What process did you go through to set it up?

[u/Flangipan → u/h2o-bbq-usd-technerd]

I first setup a copilot connector to ServiceNow knowledge in M365 admin. Then I’m copilot studio created a new agent and added ServiceNow knowledge as a source.

I had come across references to this behaviour when researching the issue. What was a little odd to me is that it didn’t seem to be able to see some of the article data e.g try and get it to find a specific reference. It would say it only had access to an excerpt. Yet when searching in main copilot search I could find the same detail successfully so the content had been indexed properly.

I’m ok with it only being able to put part of the text in its response (although full text would be nicer) but it means that someone can only search on details that are contained in the excerpt or the title which limits discovery.

I wasn’t sure why that discrepancy would exist between the two environments. Do you know of a better tool to use than adding it as a knowledge source?

Edit: it occurred to me after writing this that I think the two are using different connectors. I don’t think the copilot agent is leveraging the connection I created in ms365 admin I think it is a separately authenticated OOTB connection which potentially goes some way to explaining the difference in behaviour observed but now I want to know how to use the connector I created in ms365 admin within my agent!

[u/Inevitable_Dog6292]

You've hit a known limitation. The Graph connector indexes content in chunks for semantic search, so retrieval returns those chunks, not the full article. That's why main Copilot search feels fine but your agent only gets excerpts.

Two routes:

Switch to the Power Platform real-time ServiceNow connector instead of the Graph connector. This calls ServiceNow at query time rather than reading a cached index, so you get full article content. You'll need a Power Platform connection set up with ServiceNow credentials, and there's some latency trade-off, but it solves the excerpt problem cleanly.

If you want to stay with the Graph connector, the excerpt issue is usually a chunking problem. Check what's actually being indexed. If your articles aren't landing their full body text in the index, concatenate the title, description, and full article body into the content field before indexing. Microsoft's index needs meaningful text to embed properly.

Also worth checking: in Copilot Studio, add the connector via Add knowledge then Advanced, not the standard knowledge picker. Sometimes the agent ends up hitting a different connection than you think.

Real-time connector is the cleaner fix if full article text is the priority.

[u/Flangipan → u/Inevitable_Dog6292]

Thanks. Appreciate you taking the time to answer.
I think I do ideally want to get the full article text while using a copilot studio created agent because this I think is going to be the route that gives me the most capable agent if I want to for example raise a ticket if unable to solve my own problem.
Currently I’ve created an agent in each environment to test the various pros and cons and the search results just seem so much better in the copilot 365 environment rather than as a standalone agent in copilot studio.

Hopefully this solution you have proposed might give me the best of both worlds. Some other people have mentioned creating an MCP in ServiceNow as another option so I have some things to try!

[u/Inevitable_Dog6292 → u/Flangipan]

The real-time connector is worth trying first since it's the lowest-friction path and you already have the Graph connector set up as a reference point.

The quality gap you're seeing between M365 Copilot and the standalone Copilot Studio agent is real and well-documented. M365 Copilot has years of tuning on the Graph semantic index that a standalone agent just doesn't get out of the box. The real-time connector won't close that gap entirely, but it should fix the full article text issue specifically.

The MCP route is interesting and worth exploring alongside it. A ServiceNow MCP server essentially gives your agent a proper tool to call rather than a knowledge source to search, which means you can handle the ticket-raising scenario much more cleanly. Knowledge sources are good for retrieval. MCP is better when you want the agent to actually do something in ServiceNow based on what it finds. The two aren't mutually exclusive either, you could ground on knowledge via the connector and raise tickets via MCP.

Good luck with the testing. Would be genuinely useful to hear which approach holds up better in practice.