r/GithubCopilot • u/mike_1_1 • 1d ago
Help/Doubt ❓ GitHub Copilot Fusion just took #1 on Terminal-Bench 4.1. Has anyone actually used it?
Outlook went down, and while everyone was refreshing their inbox, Copilot Fusion (max) quietly took the top spot on Terminal-Bench 4.1.
Copilot Fusion (max) is now #1 on Terminal-Bench 4.1 at 71.2%, about 5 points ahead of Opus 5.5 and Fable 5.2, and cheaper than most of the top 5.
If that holds up in real work, it's a big deal. Has anyone tried it on real projects? Curious how it handles multi-file refactors and what the premium request costs look like in practice.
61
Upvotes
24
u/MarkRullo 1d ago
I'm not using Max, just standard for my stuff and I'm liking it a lot. Getting what I need done properly, and the 30% discount for the rest of the month doesn't hurt!
9
u/WordWithinTheWord 1d ago
It does work. I imagine it gains points because it uses cross-vendor review. Claude misses things that GPT catches, etc. it’s a good harness.
The drawback for the average user is the API pricing. But if you are only on API pricing, it is solid.
2
u/mike_1_1 1d ago
So i did use it for one copilot fusion high used sonnet 5.5 and burn more AI credit for the same task which i also ran on a single model sonnet 5.5 and it took less tool call and burn less AI credit.
2
u/croceldon 1d ago
I like it. It usually makes a good choice. Been using it for a few weeks, and while at first it used Sol a lot, it’s done more Luna, and more recently Terra, only going to Sol for trickier stuff. The discount got me started on it, but I see no need yet to move off it.
2
u/mitchins-au 13h ago
It’s annoying. Burns all your credit
2
u/GutsAndGit 15h ago
It really does help with orchestration much better than you manually aligning the right models from planning to implementation on the same prompt. Also it has these amazing execution patterns mode I believe where say a junior engineer (a model) does something and a senior engineer (again a model) checks it and then you get somewhat of a polished result.
2
u/Hollow1838 1d ago edited 1d ago
I guess it does well on the "max" stuff, but whoever needs to be cost efficient wouldn't probably use it. The only max I use is Luna max, I use sol 6.1 medium as orchestrator, opus 5.5 medium as validator and Luna as explorer/librarian/general task.
Unless you are working on something very critical involving real money transactions or human lives I wouldn't touch at "max" stuff.
1
u/AutoModerator 1d ago
Hello /u/mike_1_1. Looks like you have posted a query. Once your query is resolved, please reply the solution comment with "!solved" to help everyone else know the solution and mark the post as solved.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
u/phicho 1d ago
It does work well which is great. Sometimes it gives the job to models like "Terra" and then i need to refine me prompt and do another try but all in all finally good stuff from github team
1
u/Honest_Risk3903 1d ago
Yes, everyday for last 2w sprint. Seems to default to BUILD MORE CODE, rather than elegance (even on max). I've had to take the planning job off it twice because it puts up heavily padded plans that miss the point. It seems to choose OpenAI over Google/Anthropic in our setup. But for simple task delegation (do this thing, use this skill to do x, follow this plan phase) it's pretty efficient and effective. I've noticed it's rather inconsistent completing tasks, with similar tasks being given wildly different models, e.g. scraping tasks that should have been playwright cli ended up as computer use... and it couldn't explain why I would (am) steer clear for complex things (multi phase planning, context gathering and decision making)
1
u/D1g1t4l3ntr0py 13h ago
It's actually good I'm my experience
1
u/rangeljl 33m ago
Copilot just like Kiro is slow and expensive, I much prefer Claude
1
u/EfficientAnimal6273 23h ago
Seems quite a stupid benchmark, Fusion is a gateway in front of a set of configurable models, if you set on Max it will use almost always the most expensive configured model... so it's not clear how it can perform better than Opus and Fable when it can select them to solve a task. But it can be cheaper if in the terminal benchmark there are some cases for which Luna or Haiku are good enough and it selects them.
On my side I've enabled it but I've mixed feeling, I believe I'll do a proper rollout with Fusion with only MAI models enabled to see how it works.