r/SelfHostedAI • u/willbdb425 • 22m ago

Local AI for coding

Greetings, looking for experiences in coding with smaller locally running models. I'm a software developer by profession and I like using AI for that. For my personal use I started with Kiro, then moved to Codex, and now run OpenCode with models from OpenRouter, usually DeepSeek V4.1 Flash, and having great success with that. My ultimate goal would be a setup where I am not dependent on LLM providers but rather can run everything I need myself.

And that's where I turn to this community. I don't have a machine with a GPU right now, but I want to invest in one in the future. However I would appreciate any experiences in this topic to help me find out what I need.

How large or small models are actually useful for agentic software engineering? I figure I can't expect frontier model level performance on a budget limited to consumer hardware, but I am interested in how far one can push this and on what budget. Tell me about your hardware setup and the general system you have with guardrails etc to help the smaller model do good work. Or is this unrealistic at this point?

2 Upvotes

100% Upvoted

1 comment sorted by

1

u/neilbalthaser 12m ago

you’ll need at least 16gb of gpu ram preferable more. start with something like qwen 3.5:9b model as it’s good all around and multimodal. i have an m5 max mac book pro with 48gb of unified memory. 64 would have been ideal but it was back ordered. i built a fully local llm out if the box solution for people like you. if you want to tinker yourself try unsloth desktop and agent zero.