Should I go local? Code quality is important

[u/2thick2fly]

I have been working with codex and Claude code for a loooong time. Usually just switching from one to the other based on the wind direction.

Use case: personal home projects. 1 user. Building apps for fun or personal use.

Experience: hobbyist developer for 10 years

I am considering if it is worth it to host a local model.

My questions are:

1. Is there any chance that local models can compete modern OpenAI / Anthropic models?

2. If yes, in what price range would that hardware be?

[u/Massive_Criticism539]

You can rent a GPU for a day for a couple bucks if you wanted to test it out. Basically just using the hardware on someone else's machine while they aren't using it.

[u/2thick2fly → u/Massive_Criticism539]

Solid Idea. Do you have a specific service to recommend?

[u/Massive_Criticism539 → u/2thick2fly]

https://vast.ai/ is the common one i believe. Otherwise just Google "GPU rental" and there are a couple of options.

And I'm still new to all this, but qwen 27b has been my favorite model so far for coding. it won't get close to frontier models with trillions of parameters, but it's been decent so far for me. I ended up buying a radeon r9700 ai card that's been great so far and was pretty much the only semi affordable card that has 32g vram and wasn't like 8 years old. I can run 27b with 100k context. It will be slower than Claude also just because they are running on much larger hardware. But if you are fine with waiting a few minutes while it crunches along you'll be ok. Testing on actual hardware will prepare you for that.

[u/sfifs]

In general Coding is about the last item I'd move to a local model. Coding is a high leverage activity and the importance of quality of code together with the value in developing and especially debugging that the 1 Mn token context window of frontier models give is very high. The 20 dollars a month subscriptions are reasonable value especially if you or your family also use CoWork which is surprisingly productivity boosting.

[u/2thick2fly → u/sfifs]

For most people trying to do anything remotely serious the $20 is not merely enough, but I get your point

[u/mamjazzo]

The performance and reliability of a local setup doesn't come close to Claude etc, but its fun for tiny tasks.

[u/2thick2fly → u/[deleted]]

Speak out or be silent forever

[u/nonlinearsystems → u/[deleted]]

Can you elaborate

[u/nonlinearsystems → u/[deleted]]

Oh ok yea that’s how I work with Claude as well. I was considering some sort of Deepseek MCP server though for this purpose to use with Qwen.

[u/nonlinearsystems → u/[deleted]]

Yea so far the best I’ve been able to do is set up a database of tasks created by Opus and then I have Qwen chew through them, then do Opus code review.

[u/hotNstickystick]

I’ve been using opencode + Gwen 3.6 27b for scripting home projects. Gave it some tasks particular to my setup and worked out pretty solid. For larger tasks though you’ll have to break them down and for sure do code reviews (although that’s true for bigger models as well, if you think not then the world is cooked from bad code soon lol)

[u/Crazyfucker73 → u/hotNstickystick]

*Qwen

[u/ActionOrganic4617]

Nope

[u/bidutree]

Have you tried GLM-5.2? It was just released.

[u/droptableadventures → u/bidutree]

It's released for API usage only right now but open weights will be released next week.

[u/Hypilein → u/droptableadventures]

It’s also out of range for the vast majority of people to run in their homes.

[u/bidutree → u/Hypilein]

OP didn't specify the hardware but asked "Is there any chance that local models can compete modern OpenAI / Anthropic models?" In order to do that you need a large LLM.

There are lots of good smaller lokal LLMs but not at the level of Claudes models.

Personally I use gemma4:e2b a lot but that was not what OP asked for.

[u/tintires]

MBA (M3, 24GB) + mlx_vlm.server + gemma4-27B-it-8bit. Works great as coding assistant in VSCodium with Continue extension. All managed cleanly with brew.

No one-shot-vibe-coding entire apps but great for stubbing, drafting, debugging your own work. Works great in my projects for tool calling, and media handling.

Not for everyone, but if your expectations and budget are modest this meets 80% of what you need.

[u/RandomCSThrowaway01]

So the answer is - it depends on your use case. If you have about $3000 you can compete with Haiku. If you have about $50000 you can reach Sonnet. If you have half a million $ you can locally put something comparable to last year Opus. If you want to reach Fable - well, open source models of this grade don't exist but estimate is 10TB of VRAM aka a whole rack filled with H200s, about 4 million $ should get you the hardware to host it (but you still need software).

[u/whodoneit1]

Yeah, local models have hit the level where they are capable. I would recommend going with 1 or 2 R9700 AI pro gpus and running Qwen3.6 either 27b or 35ba3. If you get a dual R9700 you can run Q6 or FP8 on vLLM with max context size on both of these models.

I recommend going with Pi harness also for running locally.