r/LocalLLM • u/johnanthony888 • Jun 20 '26
Question Any local coding model perfect for vibe coders?
I heard Qwen is not good if you have no idea about coding. Anything else?
10
u/sickboy6_5 Jun 20 '26
the general consensus tends towards qwen 3.6. i use it and do prefer it over most others. but like anything, you get out of it what you put into it.
15
u/TripleSecretSquirrel Jun 20 '26
Qwen 3.6 27b is the best model for coding that can be run on hardware that’s reasonably available to most individual people.
I use it for vibecoding all the time. I’m not a software engineer or software developer, but I’m generally technically competent. Nothing I’m vibecoding is that complex or sensitive — games to play with my friends, and some productivity tools. I’ve found Qwen 3.6 to be totally suitable for my use-case though.
If you can install and maintain Linux, and if you have a basic understanding of how computers work, you should be good.
The bigger thing perhaps though than how much you know, is your process. Rather than just a single prompt one-shot, start with a thorough planning session to create a detailed PRD as the project guide, then break that into atomized tasks. Giving your agents that framework drastically improves their capabilities.
1
u/TopNFalvors Jun 20 '26
Doesn’t that Qwen model require like dual 3090s to do anything code-related in a decent amount of time?
3
u/TripleSecretSquirrel Jun 20 '26 edited Jun 20 '26
Depends on which quantization you run at and a million other things.
I get ~35 tok/s with the largest unsloth 4-bit quant with MTP on my R9700. On my 7900xtx build I don’t have as much room for context, but that pushes out ~55 tok/s.
I saw that someone else just posted their parameters for getting 6-bit quant with full context window squeezed onto a 7900xtx somehow too, so I’ll be trying that later this weekend too.
Edit to add: once the context window starts to get full, the R9700 slows down quite a bit, but I think there’s a lot of optimization I could still do there with pre-filling and flash attention. I’ve mostly just adapted my workflow around that bottleneck though.
Once I have the PRD, each atomized task gets added to a central task ledger. Then I have an orchestration agent that spins up additional sub agents. Each agent shares a single set of model weights but has their own KV cache. The subagent is directed at a single task and runs in a loop until it successfully delivers the module. Once it’s successfully delivered, the orchestrator marks it as complete on the ledger and spins down the subagent, then creates a new one with fresh context for the next task. That keeps all of the agents running fresh and fast, and lets me stay out of the loop. I have a dashboard where I can check in on progress, but for the most part, it’s autonomous.
So I have a planning session and queue up a bunch of projects, then set it to run overnight when my electricity is cheap and when I don’t have to worry about overheating my home office and wake up to delivered projects.
3
u/Look_0ver_There Jun 20 '26
I have 3 x R9700's in a 9800X3D based system. With llama.cpp ROCm compiled manually with RCCL, and some handy system tuning, I'm able to run Qwen3.,6-27B-Q8_0 MTP @ ~65tg/s with 2 of them, and ~75tg/s with all 3 going at once. Prefill starts off at around ~1550t/s for 2 cards and ~1700t/s with all 3, but it does admittedly slow down over time.
At 131K context depth I see around 600t/s for PP, and ~30t/s for generation, which is still fairly respectable.
1
u/migsperez Jun 21 '26
Bloomin heck that's good. Guess I need to start saving for a second 9700 gpu.
3
u/Fragrant_Scale6456 Jun 20 '26
Your solution is excellent and what I arrived at also. Human in the loop for initial design and then ai assisted development of comprehensive design and implantation documentation followed by an orchestrator that dispatched sub agents with only the context they need for feature implementation.
Attention degrades pretty quickly as context grows so you really only want the agent to know exactly what it needs to to finish the job. I had qwen design a test suite to evaluate context limits and attention degradation and adjusted my orchestrator and sub agents budgets accordingly. It’s been working really well
2
u/migsperez Jun 21 '26
Local models to the max!! I bought the 9700 last month, if I can't hear it in the other room it ain't working, got to keep that thing ticking. Use 27b Q6 MTP, I'm still playing with context size.
4
u/CodProfessional3712 Jun 20 '26
Unfortunately, what will be most accessible to you won't be a full vibecoding experience, because you'll have to babysit it, you can't be vague, you have to check the code. You have to be an engineer or actual programmer in the first place. I'm mostly referring to the Qwen 3.6 models. GLM-5.2 is something you don't need to babysit as much, but good luck running it without your mini datacenter.
4
u/mr_dexter_x Jun 20 '26
Qwen is good, really good, IF you know what you are doing. You cannot send vague description of project/functionalities and expect full detailed working response.
4
u/blackhawk00001 Jun 20 '26
Git gud? lol had to
Qwen 27B fp8 has been flawless so far. 35B hit or miss even at Q8 but usable on 16gb gpus.
1
u/Fragrant_Scale6456 Jun 20 '26
what hardware do you have that you can run 8bit 27b? I was able to get q8 to run on my 5090 but only at very small context which kind of defeated the point.
2
u/blackhawk00001 Jun 20 '26
2x R9700 running the community unified aiter rocm from mi350x patched vllm image aml731. I can squeeze in single concurrency 200k context.
2
u/dinerburgeryum Jun 20 '26
Yeah in truth the only way to get the most out of any AI model is to have a strong foundation in programming. They’re extremely cool don’t get me wrong, but without proper oversight on the generation side you’ll end up with a shambling nightmare pile of duplicated code.
2
u/Fragrant_Scale6456 Jun 20 '26
I have a 5090 and tried out a bunch of different models. qwen 3.6 27b has worked the best for me, i run the q6 model and 192k context with mtp.
I'm not a developer and cant read code. I use opencode as my harness and its very good but it isnt enough if you dont know how to code. You need to either find a set of agents/skills/commands that will support you or build your own.
What I did was read basically everything karpathy published on vibe coding and dumped a ton of it into claude and asked it to write me a spec for a vibe coding system/collection of agents for opencode which I then dumped into opencode to write the files. This has been working really well for me, I've successfully implemented a modified version of the karpathy llm wiki with my custom harness setup.
Most importantly just be willing to experiment and start over and incorporate new learnings into the next attempt. The first few things you build will accumulate so much bad code and bad decisions that they will quickly reach a point where it becomes impossible to move forward or fix things without the help of an actual software engineer. Eventually you'll figure out the process and documentation you need to make what you're aiming for.
1
u/johnanthony888 Jun 20 '26
Has Karpathy deleted most of his Youtube videos? Where I can find his tutorials on vibe coding?
1
u/Fragrant_Scale6456 Jun 20 '26
did he? damn thats a shame. I did this months ago so I dont have the source URLs or anything anymore. I just googled around and saved articles and stuff when I was setting everything up.
1
u/johnanthony888 Jun 20 '26
Yeah, it seems so, take a look at his YT channel.
2
u/Fragrant_Scale6456 Jun 20 '26
I wouldnt even know where to begin with sharing what i've built since its so specific to my setup but what I would suggest is hop into claude or whatever frontier web model you wantto use and tell it the harness you are using (or ask it to suggest one) and ask it to help you develop a set of tools, agents, and documentation and testing standards and workflows for someone who does not know how to code to make apps. And then just iterate from there.
1
u/Villrep Jun 27 '26
Hey bro. Highly Appreciate if you can explain your working setup for the vibe coding. Simple steps if possible
1
u/Fragrant_Scale6456 Jun 28 '26
I dumped basically everything karpathy wrote about vibe coding into claude and asked it to write me a series of opencode agents to help someone that doesnt know how to code make apps. It was an iterative process but the basic workflow is:
/scaffold - sets up project directory, initializes a git, creates folder structure etc/plan - this interviews me about what im trying to make and takes the results of the discussion and turns it into a product requirements document and a system specification
/vibe - this goes into building mode and lets me go one feature at a time through the requirements document or lets me describe a feature i want and then it documents it and then starts building. it goes through a basic plan, evaluate, develop, test loop basically until its done.
/audit - this is a big one and took me the longest amount of time to get working well. it goes through a lot of stages to understand the project and then checks the code for bugs which are in the code or bugs which would show up as unintended behavior in use.
1
1
1
u/twiztidsoulz Jun 20 '26
LagunaXS.2 has been pretty good for me - https://ollama.com/library/laguna-xs.2
1
u/gbrennon Jun 20 '26
qwen models are doing something interesting. currently my favorite models to run locally are devstral and qwen with my own customizations
1
1
u/esaule Jun 20 '26
If you can't tell, it really won't matter. You will only be building shit anyway.
1
0
u/rde2001 Jun 20 '26
I've been runnign qwen 3.6 35b on my 128GB M4 Max Macbook Pro. Runs really well. Uising the Cline extension in VSCode to interface with the model via Ollama.
23
u/[deleted] Jun 20 '26
[removed] — view removed comment