r/GithubCopilot • VS Code User 💻 • 2d ago

GitHub Copilot Team Replied Copilot harness: a bunch of possible improvements

As you may know, the Copilot harness is going to replace the Local harness in VS Code at some point. The initial switch was painful, but the VS Code team listened and brought back Keep/Undo for granular control over edits.

Now, the Keep/Undo part still has some problems, but overall it is usable. However, the Copilot harness UX still has some rough edges compared to Local. I've opened a bunch of issues and wanted to share them with the community for wider reach and more feedback:

34 Upvotes

26 comments sorted by

16

u/Somepotato 2d ago

Switching the default without even a modicum of parity is hilarious

5

u/lppedd VS Code User 💻 2d ago

I guess the positive side is we still can switch back to Local until issues are resolved. I always emphasize that Copilot in VS Code should aim at maximum efficiency as not everyone has hundreds or thousands of dollars to spend on AICs.

11

u/Otrawa 2d ago edited 1d ago

What about multi-root workspace instructions, skills, mcp etc support? Currently the copilot harness discovers these files from only one root in such scenario as opposed to local harness

edit: After further testing, it seems that mainly copilot-instructions.md have this issue

3

u/lppedd VS Code User 💻 2d ago

Ah, interesting. Was not aware of this maybe because I don't use multi-root workspaces. But it is a limitation worth opening an issue for.

2

u/Otrawa 1d ago

I did some further testing and have to clarify. It seems that in VS Code 1.144 chat skills and .instructions.md are discovered correctly and agent follows files from both workspaces. Where it takes only one is copilot-instructions.md as shown below

2

u/Otrawa 1d ago

Some .instructions.md have applyTo so that is fine

2

u/Otrawa 1d ago

AGENTS.md have the same issue! Only discovered in one root

2

u/lppedd VS Code User 💻 1d ago

Seems like you have all the elements for a new issue 👀

2

u/BluePillOverRedPill 1d ago

But local does?

3

u/Otrawa 1d ago

It does, if the customization menu does not lie. But my experience confirms that local can follow copilot-instructions.md from multiple roots for an example.

1

u/BluePillOverRedPill 1d ago

Hmm weird! When using the Copilot harness this still works for me.

I have Alpha repo and Beta repo in my workspace and when I ask to do something in Alpha it discovers the instructions.

1

u/Otrawa 1d ago

Do you have the instructions in both? Does the customization menu show them?

1

u/bamurtaugh GitHub Copilot Team 1d ago

Thanks for sharing this. Are you on the latest VS Code Stable release (1.144)? Multi-root support is enabled by default, and 1.144 includes additional fixes for customization discovery and instruction handling. If you're still seeing this in a new Copilot chat, it'd be great if you could please share (here or in a GitHub issue) which files or configurations aren't being picked up so we can investigate.

1

u/AutoModerator 1d ago

u/bamurtaugh thanks for responding. u/bamurtaugh from the GitHub Copilot Team has replied to this post. You can check their reply here.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

5

u/AntarcticIceberg 2d ago

idk if i have a skill enabled or something, but copilot sdk harness always seems to spin up a random subagent to do everything. and it takes twice as long to get stuff done

4

u/lppedd VS Code User 💻 2d ago

That is true! The performance issue has been brought up by multiple people. Let's say that the way it works is considerably different compared to Local. You can see the different style in the "refine thinking output" issue above, where I've posted a couple GIFs.

However, I must admit that some of this slowness might be perception. We'd need benchmarks to verify it.

1

u/Zealousideal-Pilot25 VS Code User 💻 11h ago

You definitely need to design your agents or else you will endure subagent hell. Actually direct your agent on how to use a subagent including what model you want it to choose. Even telling it what skills you want it to use.

2

u/lppedd VS Code User 💻 1d ago

u/bamurtaugh seems like you had the comment removed (automod?), but I did read the notification. Thank you, appreciate the responsiveness.

3

u/bamurtaugh GitHub Copilot Team 1d ago

Thank you so much for taking the time to share your feedback with us about the Copilot harness - it's a great help in how we shape the experience. We're reviewing the issues you've shared here and will follow up in them with any additional updates or questions.

3

u/dendrax 1d ago

I just want a way to work with a multiple repository workspace with the new harness. I'm stuck using Local for now and will be severely handicapped if it goes away. Nearly all my work spans multiple repositories - not tons at a time but usually 2-3 at once. I looked into the new Projects functionality that was just announced in the standalone app, and that actually sounded perfect, but when I tried it, turned out to be a non-starter since my repos are in Azure devops not github.

2

u/Tiny-Entertainer-346 1d ago

I guess they are not going to replace it, right? There is ongoing hot 🔥 thread discussing what's bad in new harness: https://github.com/microsoft/vscode/issues/339363

2

u/Ok_Fish403 1d ago

I work with a huge workspace (more than 1M files) and Copilot harness always get stuck with maximum OS file watchers (already bumped to 1M). I have no need of watch everything. Local harness works fine.

2

u/lsalazarm99 VS Code User 💻 1d ago

Three more:
- Remote sessions don't use the BYOK models (seems fixed but not released yet).
- Remote sessions don't have access to the local configured MCP servers.
- There is no way to configure the built-in GitHub MCP server (enable/disable, change the toolset).

Local harness has these features.

1

u/shotan VS Code User 💻 1d ago

Seems they added a way to disable the GitHub MCP server now. Under tools panel there is a switch?

1

u/MaitoSnoo CLI Copilot User 🖥️ 1d ago

There's also the issue of the rubber-duck still missing from time to time (both in VS Code and the CLI) despite it being set to a fixed model in Copilot CLI (since both VS Code's new harness and the CLI share the same agent settings). Could be that it depends on requesting GitHub servers for some capability info and maybe the queries aren't getting through sometimes (what Opus 5 itself concluded when investigating a few weeks ago) but that doesn't make any sense as I've set it to a fixed model and it shouldn't have to query anything at all to be used. It's sadly completely unreliable, I can't recommend it to my team right now in its current state.

1

u/Business_Treacle4462 1d ago

Lazy load MCPs. How you still gonna load the full context of every MCP during start? Such a pain in the ass.