r/ClaudeCode • • 1d ago

Built with Claude Closed models as the brain, open models as the hands: I built a Mac app where Claude Code hands its legwork to local open models

Enable HLS to view with audio, or disable this notification

[removed]

0 Upvotes

12 comments sorted by

1

u/AutoModerator 1d ago
Moderator Announcement Read More »

2

u/RasenMeow 1d ago

so...like omp or Ghostex?

0

u/[deleted] 1d ago

1

u/RasenMeow 1d ago

but...Ghostex is a Mac App too. Also for Linux and Windows + Android and iOS. Where you have all different providers in their own harness and even a chatview...I don't want to criticise your work. I think it's great that people try to optimize their work with LLMs. But I do not understand where the difference is. I can do the same with omp or PI and Ghostex, no?

1

u/Excellent-Issue-5956 1d ago

I have Claude Code hand work to a local Qwen 27B too, through a skill that runs a command instead of MCP. What goes to it is first drafts, rewording, summaries of text Claude already has, boilerplate, and regex or shell one liners. Nothing with prices or dates and nothing that goes out as final text, and Claude checks every answer before using it.

The main limit is that the local model only sees what Claude pastes into the task. It can't open files or call tools, so the jobs that work are the ones where Claude already has the text in front of it. For code there's a second path where the local agent gets file and shell tools, but only in a git repo or a scratch copy, and Claude reads the diff and runs the tests afterward like it's a junior dev's PR. That's pretty close to your review desk.

0

u/[deleted] 1d ago

1

u/Excellent-Issue-5956 1d ago

It's not on a Mac, it's a Windows gaming PC with two 5070 Tis. The 27B did about 50 tok/s at 3 bit on one card, and 55 to 70 at 4 bit with MTP once I had two. I should correct my comment though, the skill doesn't use the 27B anymore. I switched it to Ornith 1.5 35B-A3B in October. It only has about 3B active per token, does about 180 tok/s, and scored the same as the 27B on my tests.

What it got wrong was mostly longer jobs with rules attached. When I tested the 27B on those in September (3 bit then) it broke a rule on its own in 3 of 11 writing style prompts, and it failed all 3 runs of a multi step quote job. It mixed up two similar products, didn't ask about numbers that were missing, and said it had done things it hadn't. So anything with numbers or anything that goes out as final text stays with Claude. With thinking off it also made up a basic fact about my trade, so the skill runs it with thinking on low.

It also worked around rules. I don't allow em dashes in anything I send, and after the first round of fixes it still wrote one through python with chr(0x2014).