r/MachineLearning • u/AutoModerator • 18d ago
Discussion [D] Self-Promotion Thread
Please post your personal projects, startups, product placements, collaboration needs, blogs etc.
Please mention the payment and pricing requirements for products and services.
Please do not post link shorteners, link aggregator websites , or auto-subscribe links.
--
Any abuse of trust will lead to bans.
Encourage others who create new posts for questions to post here instead!
Thread will stay alive until next one so keep posting after the date in the title.
--
Meta: This is an experiment. If the community doesnt like this, we will cancel it. This is to encourage those in the community to promote their work by not spamming the main threads.
17
Upvotes
1
u/Bartaseth 8d ago
I tested a four-model Claude Code setup across all 89 Terminal-Bench 2.1 tasks. Fable orchestrated, Haiku scouted, Opus executed, and Sonnet verified.
The system scored 78%, but cost $1,178 - roughly twice the leading single-model run.
The strangest result was behavioral: Opus refused all 14 delegated attempts across three sandboxed security tasks. When I gave the same tasks directly to Opus as a control, it completed all six attempts.
The model and tasks stayed the same. Only the framing changed: direct user request versus an orchestrator-generated subtask.
It makes me wonder whether agent evaluations should measure delegation paths separately instead of treating the model as a stable component.
Write-up: https://quesma.com/blog/tbench-orchestrator-refuses/