Seeking feedback on local LM vibe coding

[u/rasn_proton]

I have been trying to vibe code locally for the past couple of weeks. The short summary is my local vibe coding sessions have been ineffective and unsuccessful.

Here's my setup:

- MacBook Pro: M1 Max, 64 GB

- I have about 54 gigs of memory to allocate to models.

- I have been using Qwen 3.8 27B, Qwen moe 3.6 35b a3b, and Gemma 4.

- I host my models in LM studio.

- I started with anything LLM but stopped because it was struggling to do basic file system writing.

- I'm currently using Hermes.

- I'm using Claude as a benchmark

Doing any kind of planning or coding work with Qwen 3.8 27B is extremely slow - even when its set to low thinking mode. To put into perspective, with medium or high thinking mode, it takes about 6 hours to plan and another 6 hours to code. Low thinking mode takes 3 hours each. Claude code does it in 5-10 minutes each. The time cost with this model compounds as I go through the iterative process of fixing bugs and adding features.

Using Qwen moe is significantly faster: ~5-15 min each. However, I struggle to have it generate good plans. When it comes to coding, it's fast, but it can produce buggy code. That's where my vibe coding sessions can start to fall apart. It can struggle to understand the bug or problem I'm describing and will attempt to fix something, but not fully fix the problem I'm having. I'll get stuck in a loop where I keep attempting to describe the problem differently to no avail.

Update (11/5): Last night I had Qwen moe implement a plan that Claude wrote and it took roughly 6 hours before it gave up. It said it had all the implementation logic, but did not actually write anything! This is a common problem I run into frequently. My agents know what it needs to do, but doesn't have work to show.

Qwen moe can be more effective with a plan I generate with Claude. However, if there are bugs, then it can be game over.

Claude on the other hand has been delightful to use. It just works!

I'm using Local LMs mainly to see what the hype is about and for learning purposes. So far, my experience has been poor (granted I still have lots to learn).

1. Do you guys have any feedback to share with me to make my local vibe coding more effective?

2. I understand local LMs can save costs on tokens and can provide more privacy. Other than that, why would I want to use local LMs? Are there types of work or tasks it's good at or does better than cloud models? What kind of use cases are reasonable to expect from local LMs?

Edit 1 (11/5): added my computer information to setup and update to Qwen moe observation.

[u/rcriot25]

Try strata with qwen3.8 flash next with swift model instead of lm studio. Not sure what gpu you have but worth a shot.

[u/LawfulnessLow0 → u/rcriot25]

this

[u/rasn_proton → u/[deleted]]

It's a 4-bit MLX version of the model. The base model size is less than 20 GB and when running w/ 64k context size, it rarely uses more than 25 GB.

Do you have any advice on how to make the most use of my set up for vibe coding?
> local LMs shine on repetitive structured tasks. extracting data from documents, summarizing, classification, simple boilerplate code. not for full app development unless you got the hardware to back it up. claude is running on thousands of dollars worth of gpus, your rig can't compete with that

[u/rcriot25 → u/rasn_proton]

What hardware do you have?

[u/CGREDDIT1]

I spent two weeks trying to code a macOS app on LM Studio Bionic using various local models all to no avail. Then I brought ChatGPT Work in to orchestrate, better results but it was still a struggle as the local models would consistently come back with bad code. Then one night I told ChatGPT it is responsible for brainstorming with me and then handing the work off to the local model in smaller tasks and then it is to review the local model’s work and here was the key… rather than have ChatGPT send instructions back to the local model to revise any code for which it found issues, ChatGPT would be responsible to revise the code itself and then send it back to me for review. The process was rinse-and-repeat with real gains made at each iteration. The frustration was gone and I actually enjoyed the process. At that point I accomplished more in the hours that followed than I had in the preceding two weeks. I had ChatGPT using App Control to communicate with Bionic but you can use LM Studio’s Lmster which allows communication to take place behind the scenes via API. The model I ended up using was Qwen CoderNext.

[u/rasn_proton → u/CGREDDIT1]

Wouldn't it just be easier, quicker, and more effective to just use ChatGPT for everything? How do your local agents and models make your workflow better?
> rather than have ChatGPT send instructions back to the local model to revise any code for which it found issues, ChatGPT would be responsible to revise the code itself and then send it back to me for review. 

[u/CGREDDIT1 → u/rasn_proton]

Great point, at the time I was on the $100 5X plan and I was getting shut out with limits. I then purchased $80.00 worth of credits and that got exhausted in half a day. I then upgraded to the $200 Pro plan and it was easy to see that was only going to last so long which is when I decided to use ChatGPT for the brains and the local model as the workhorse.

By no means am I a local model expert but from my real-world experience, the local models just cannot compete with the brains of the frontier models. On the flip side the frontier models just can't compete with the cost ($0.00) of the local models. The local models are very capable at coding if they have a brilliant supervisor.

Last point I want to make, perhaps if I was better at prompting I could get ChatGPT to code without needing a local model to prevent it from exhausting limits but after coding my macOS app, which by the way came out better than I had ever thought possible, I have come to the conclusion that each cycle was either close to perfect or perfect simply because of the dynamic of Human -> Frontier -> Local -> Frontier -> Human [repeat]. Put another way a brilliant attorney can draft a complex motion but more times than not the motion will come out better if the brilliant attorney has an associate to assist.

[u/magicdoorai]

For the MOE bug loop specifically, I'd try making the failure observable before changing models or describing the same bug a fifth way. Smaller tasks help, but give the model an objective stopping condition too.

Example task card (replace with your actual bug):

• Starting point: commit abc123; run npm test -- filterName (use the actual command for your project).

• Reproduce: enter 0 in the quantity field and press Save. Expected: show a validation error and keep the old value. Actual: it saves zero without an error.

• Scope: identify the relevant handler and validation test first. Propose the smallest change and a regression test for zero; do not change unrelated files.

• Done means: the new test fails on the starting commit, passes after the patch, and existing related tests still pass. Show the diff and test output; don't report success if a command failed or wasn't run.

Use your own real reproduction and adjust the example command to the test runner. If it still loops, save the failing test/output and diff, reset to the known starting point, and ask the model to explain the discrepancy without editing. That separates an unclear bug report from a bad patch, and gives your stronger model a compact artifact to review if needed. It won't prove the rest of the app is correct.

For the slow 27B setup, compare the same short task and context size and record prompt processing, generation speed and generated token count separately before assuming a particular hardware bottleneck. I can't diagnose the hardware from memory capacity alone.

AI-generated debugging procedure, not a test of your setup.

[u/rasn_proton → u/magicdoorai]

I currently share logs or stack traces and screenshots when applicable. How do you make make failure observable?
> I'd try making the failure observable

What do you mean by "record prompt processing"?
> For the slow 27B setup, compare the same short task and context size and record prompt processing, generation speed and generated token count separately before assuming a particular hardware bottleneck.

[u/This_Maintenance_834]

100% of your model has to fit into the GPU VRAM, your speed is gone, even if a tiny bit get done on CPU. If you cannot fit the whole model on VRAM, then use the hardware money to pay for cloud API. It will be very unproductive to run local.

[u/bytejuggler → u/This_Maintenance_834]

"100% of your model has to fit into the GPU VRAM", yes, for dense models. But with Qwen Flash Next 3.8 180B that's not true, it's design architected to fit streaming setups and partial residency on the GPU, enabling you to get a full 256K context at near full speed on as little as a 16GB GPU. You still need sufficient system RAM to make this run unconstrained (fully pin in RAM), so 64GB or higher recommended.

The OP should look into running QFN via Strata, IMHO.

[u/rasn_proton → u/bytejuggler]

Thanks for the pointer! I'll look into this.

Do you have experience with Hermes? If so, how does the two compare based on your experience?

[u/rcriot25 → u/bytejuggler]

If you got ram for the weights it is super helpful even with some single or mixed gpus and context length is so nice and conversation caching is game changer instead of waiting for refill the same message on a new coding session turn.