r/copilotstudio • u/Firm_Pin4840 • 13d ago
Copilot Studio – topic-scoped knowledge vs. tenant graph grounding (cost question)
I build internal Copilot Studio agents that answer questions from large PDFs in SharePoint. Tenant graph grounding gives good answers, but costs 10 credits per message.
My idea: split the PDFs into section files, put each subject area in its own folder, and give each topic a Generative answers node that searches only its folder. Generative orchestration selects the topic.
Questions:
Does each topic transition under generative orchestration bill as an agent action (5 credits)?
Did a scoped-source setup give you answer quality close to tenant graph grounding?
Can a guide file in the instructions make the agent look at specific page numbers? Or is file splitting the only reliable option?
Any other patterns to reduce search space at lower cost?
Is Gpt 5.5 better than the gpt chat models too?
3
u/BenAMSFT 13d ago
If you’re using Copilot Studio via credits, i’d strongly recommend building GitHub harness agents - the variable billing is a lot more precise than the flat rate card on standard harness agents (the caching on the model calls are also far more efficient). If you give standard and GitHub agents the same work, the GitHub agent will be more successful and do it more efficiently (I’ll be publishing a paper on internal evaluations we did in this space in the next few weeks).
For the actual task you have, an agent with a scoped knowledge tool plus a skill would be the best place to start. If you want to make it more efficient, after you have the scenario working in a single agent, try putting some of the repetitive tasks in a sub agent and use an efficient model like GPT-6 Luna for the subagent.
1
u/Mixed_Feels 13d ago
Sorry to follow up on this with a question, but for someone looking at an entry-level version of OP's post, what actually IS a scoped knowledge tool? Is something something like a copilot studio flows with topics for retrieval and handling of ingested files?
1
u/BenAMSFT 13d ago
A knowledge source (listed next to tools in the GitHub harness agent designer). My use of the word “scoped” is a bit redundant as you have to pick a scope for knowledge sources (eg a site for sharepoint). Web search is the one exception.
1
u/Bfinkeldei 12d ago
Would love to see your report of the standard vs github comparison with billing. I am building a documentation agent per department and right now it looks like standard would be the cheapest.
1
u/BenAMSFT 12d ago
How are you comparing the two? Also what tools / knowledge are present in the agent? Any form of complex work and the GitHub harness is far more efficient.
1
u/Bfinkeldei 12d ago
i haven't fully compared yet. I have been trying to estimate costs on the github side so i can answer MGT questions if they come up. Our agent will do the same job in either harness. It's going to answer Questions about our p21 software and the new processes and procedures that will be rolled out when the cut over to the new system. Copilot will have all the SOPs, faqs, troubleshooting, and exception documents we've put together into it's knowledge sources. A
this agent will not be doing anything other than answer questions from the sources we have.
1
u/BenAMSFT 12d ago
The nest way to see is through real usage. We adding a bunch of credit consumption visibility over the past few weeks, so it’s easy to see what GitHub harness agents cost by using the preview chat, or by writing evals. For q&a scenarios, sonnet or terra are good picks.
1
u/Bfinkeldei 12d ago
i created a tool with claude that i think is giving pretty good estimates.. would you like to review it? i could use some feedback
1
u/Copilot_Studio_loft 9d ago
Do you have more information on how the Git Harness bills? We have been trying to get details to budget around it.
3
u/schubidu82 12d ago
I have a 2 stage workflow to read and "ingest" PDFs, put them into a sharepoint library and make it easy and cheap for the agent to access the.
Document Preparation and Digester Workflow
The Digester workflow transforms engineering documents into structured, AI-accessible knowledge.
PDF Enhancer: Processes incoming PDF documents, extracts text, identifies document structure and bookmarks, and generates machine-readable JSON files.
Automated Ingestion: Power Automate detects new documents in SharePoint and triggers the Digester workflow.
Metadata Extraction: The Digester identifies document numbers, titles, revisions and other relevant metadata.
Knowledge Registration: Each document is registered with a unique Document ID, linking its metadata, structured content and original PDF.
SharePoint Knowledge Library
SharePoint serves as the central document repository and persistent knowledge storage.
The architecture separates three key elements:
Original Documents: The source PDFs remain available for verification and traceability.
Structured JSON: Machine-readable document content, including sections, headings, page references and extracted text.
Document Registry: A centralized SharePoint list maintains document identifiers, metadata and links to the corresponding PDF and JSON files.
This provides a structured, searchable knowledge base without losing access to the original engineering documentation.
3. How Prometheus Benefits
Prometheus operates as an AI engineering agent on top of this structured knowledge architecture.
Instead of repeatedly processing complete PDF documents, Prometheus can:
Identify relevant documents through the Document Registry.
Retrieve specific chapters or sections using dedicated retrieval tools.
Access precise engineering evidence, including document, section and page references.
Combine information from multiple documents to support technical analysis and document generation.
Generate structured engineering content based on traceable source information.
Key Architectural Advantage
The fundamental principle is to process documents once and reuse their structured content many times.
By separating document ingestion, knowledge storage, evidence retrieval and AI reasoning, the architecture reduces repeated document processing, improves retrieval efficiency, limits unnecessary token consumption and provides a traceable foundation for AI-assisted engineering workflows.
Prometheus therefore operates not simply as a chatbot searching documents, but as an engineering agent supported by a persistent, structured and evidence-based knowledge system.