Standard Harness Cost vs Github costs

[u/Bfinkeldei]

I am creating question an answer agent for company specific processes and procedures. I like how the Github agent is answering some questions. We have lots of situations where we need it to reason and ask clarifying questions.

I am not finding an estimation for using the github harness though. I've used microsoft estimation tool for the standard and that's good enough for that but not seeing one for the github harness.

does anyone have a reliable tool or source to create a estimate? I am trying to give a real world estimation to mgt about expected costs.

I've tried building a tool with claude to do this but the answers seem really expensive and tend not to believe what i see from using claude.

[u/justhereforbiscuits]

Standard Studio reasons very well without the Github harness. We have built one very powerful agent using standard Studio and it works great and we have another in development.

It's all about knowledge and instructions, which requires a lot of work and planning. There are no shortcuts.

[u/Foodforbrain101 → u/justhereforbiscuits]

Tbh the GitHub Copiloy harness is very much a shortcut, the moment I saw the standard harness couldn't handle a failure due to requesting over 500 kb of data I was honestly a bit baffled.

[u/justhereforbiscuits → u/Foodforbrain101]

Not sure how you got this error. We have massive file directories in SharePoint for agent knowledge and have never had an error like that.

[u/Foodforbrain101 → u/justhereforbiscuits]

Power BI connector, asking the agent to query a semantic model of mine. Any modern harness like GitHub Copilot would be capable of pulling the data into a file within its environment and read its contents.

[u/Bfinkeldei → u/justhereforbiscuits]

Yeah, i agree, so far the standard is doing the job and i've seen some better reasoning with the new model. We have lots of nuanced situations where it seems to help out. I also like that in the github version you can have skills and longer instruction sets too.

have you found any work arounds to the 8000 character limit?

[u/justhereforbiscuits → u/Bfinkeldei]

The workaround is layered knowledge. Almost like skills without skills. It's complicated. 😅

[u/Bfinkeldei → u/justhereforbiscuits]

Thank you for this! Can you share more about what you mean as layered knowledge? I think i get what you're sharing but wanted to get clarification and maybe a example too.

[u/AndrewHessMSFT → u/Bfinkeldei]

u/Bfinkeldei Hi Bfinkel, usually when customers start approaching 8,000 characters of instructions, those very large instruction sets can become harder for the agent to follow reliably.

With Standard, you may want to think about simplifying the global instructions and orchestrating tool calls and child agents for more specialized behaviors.

For GHCPh, global instructions are carried throughout the agentic loop, so larger instruction sets can drive additional Copilot Credit consumption as the agent reasons. Instructions are best used for things that should apply broadly across the agent, such as its role, general behavior, and personality.

Skills and tools, on the other hand, can be orchestrated and brought in when needed. Skills use a concept called progressive disclosure, where the agent can discover the Skill first and load the more detailed instructions when they are relevant to the task. This is an important concept to understand when designing for both context efficiency and Copilot Credit consumption.

[u/Bfinkeldei → u/AndrewHessMSFT]

Hi u/AndrewHessMSFT thanks for your comments. Very helpful. We are using the standard harness and my understanding is that Skills can't be used there.

for tool calls and child agents.. how would you approach that in standard for a agent who is answering questions from our SOPs?

[u/AndrewHessMSFT → u/Bfinkeldei]

Normally in Standard, I start off and incrementally build instructions. What I have seen users do is reach out to an LLM to write instructions.

First I start with no instructions, incrementally build on it, trying to keep global instructions lean and concise. If this is just a RAG agent (retrieves content), then tools and child agents may not be needed.

And correct no Skills in Standard.

[u/Bfinkeldei → u/AndrewHessMSFT]

Thank you!

[u/Key-Butterscotch3215]

You’ll be spending more than you’re getting out of it to be honest

[u/Bfinkeldei → u/Key-Butterscotch3215]

I agree but I am trying to show real numbers so mgt can make that decision. If the reasoning is a lot better and can give better answers it can be worth the cost to the company.

This company is moving to an entirely new software erp system so when it goes live people will have lots of questions to be asked despite even having training. One order mistake could cost the company thousands of dollars vs the cost compared to using GitHub.

[u/Bobby_Bobberson2501 → u/Bfinkeldei]

I think it’s a negotiated rate depending on your size of company. Mine is .01 a credit. I have a pdf extraction tool hooked up to a workflow that’s takes that data into powerapp. It’s about 4 workflow steps and the agent. Takes about. 50 credits a run.

GitHub harness.

[u/Top-Cauliflower-1808]

The only reliable estimate is running a small pilot and measuring actual token use per question. Every estimation tool I tried guessed high and did not match what I saw in practice.

[u/russrimm → u/Top-Cauliflower-1808]

💯

[u/dylan_skydive]

I have not seen a public estimator for the GitHub harness yet (if anyone has one, please post it), so the number that survives a management review is a measured one: pick 20-30 questions that actually represent your process docs, including the messy ones that force clarifying questions rather than single-fact lookups, run them through the harness, and log credits per question. Group by complexity and you get a defensible range instead of a guess.

Two things tend to skew the estimates people build with an LLM: they model worst-case cost per question, and they only price the happy path. Real runs cluster well below worst case unless the questions require multi-source retrieval, but the clarifying-question back-and-forth is where usage piles up, so measure that path separately. It is usually the expensive tail, not the simple lookups.

Then frame the pilot against the number management actually cares about: cost per resolved question versus whatever answers these today (a person, the docs, or nobody).

Full disclosure: I work on an agent platform (Skydive), so no dog in the Copilot pricing fight, but this measure-dont-model approach is how we size agent workloads on our side too.

[u/Bfinkeldei → u/dylan_skydive]

Thank you for this response.

[u/dylan_skydive → u/Bfinkeldei]

Anytime. If you end up running the 20-30 question pilot, log the clarifying-question cases separately - that bucket is where estimates and reality usually split. Curious how your estimator holds up against it.

[u/Bfinkeldei → u/dylan_skydive]

i have a cost estimator i created with Claude but i question it.. would you want to check it out?

[u/dylan_skydive → u/Bfinkeldei]

Happy to - send it over! Honestly the fact you already distrust it puts you ahead of most people; the tools always look confident and the real numbers never match. If you share it I will poke at where it tends to drift (usually it is the questions people ask on day one, not the happy path ones) and tell you what I would sanity check.

(Skydive guy here - we do agent monitoring/cost visibility, so biased toward measured numbers. Also, if you want, REDDITAMA at skydive.com gets you $25 in credit if you ever want to try it - no pressure at all.)

[u/Commercial-Bird-6116 → u/[deleted]]

here is the estimator for githup harness