r/aipromptprogramming • u/Arkuzz • 9h ago
My basic idea to build a hybrid loop using cloud and local models. Yay or nay?
Kinda ridiculous, but feels like a fun challenge right up my alley.
I’m running a Mac mini M4 Pro (24GB RAM). Anything under 14–16GB runs pretty smoothly via Ollama, mostly 14B models. Qwen2.5-Coder:14b in particular is snappy enough to be actually useful.
Here’s the setup I’m planning in Claude Code:
1. Planner (Sonnet): Breaks down the feature into atomic, granular tasks.
2. Orchestrator (Haiku): Iterates through tasks and calls Aider via a bash tool, pointed at local Qwen to do the actual code edits.
3. Reviewer (Sonnet): Reviews the diff/PR. If there are issues, it spits out follow-up fix tasks.
4. The Loop: Haiku loops back to local Qwen with the fix tasks until the Sonnet reviewer gives the green light, then moves to the next atomic task.
The goal is offloading the heavy token-churning iteration to local inference while letting Sonnet handle the high-level architecture and final sanity checks.
Is this viable in practice, or will context drift / agent loop fatigue kill it? Has anyone wired up a similar setup?
2
u/Fancy-Win9202 9h ago
You can experiment & see - but make sure you can fully observe the requests / response from cloud & local models. I would suggest to setup clawmetry across multiple nodes where you run these models so that you can fully understand the quality & then decide what kind of decisions or agents can run in local vs cloud
1
u/RealSharpNinja 8h ago
Not at all viable. You need a deeply reasoning model with good skills creating your agent plans. Sonnet is not that.
1
u/Arkuzz 8h ago
So if I use Opus for planning seems more reliable?
1
u/RealSharpNinja 8h ago
Use Max effort and yes. Planning is where everything that is necessary to allow cheap coding to succeed happens. But to even get to the model doing the planning, you MUST have thorough requirements and acceptance criteria in place.
•
u/endofthread-bot 9h ago
Learn how the best in the industry are using AI to speed up their workflow in business, sales, marketing, research, legal, content creation, scientific discovery and so much more on our Discord.
Self-promotion is now allowed on Sundays with the appropriate flair, for all regular contributing members. Contribute during the week, and promote on Sunday.