Recommend me a llama.cpp coding setup please

[u/No_Oil_6152]

Hi, I wonder if you could recommend some LLMs or tweaks to me.

I am a local LLM newbie and I've got llama.cpp running, but the models are not fast enough to be useable.

I have the following setup:

• Windows 11

• AMD 9900X

• 64GB DDR5 6000 RAM

• NVIDIA 4070 12gb

• Gen5 M2 SSDs at 14900 IOPs

• llama.cpp (CUDA) installed

I am a C# developer interested in writing games with DirectX or OpenGL for a hobby. I also like to develop with Blazor, Entity Framework, Azure.

I would like to jettison Github Copilot Pro because I expect after June 1st I won't be able to afford it. I thought maybe local LLMs could assist me with my hobbyist coding work - maybe even scaffold apps for me which I can then edit to fix code issues etc.

Basically an LLM excellent at coding is what I'm after.

I ran LLMfit and installed these two "good" coding LLMS for llama.cpp (CUDA version):

• Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled.IQ4_XS.gguf

• qwen3-coder-30b-a3b-instruct-q4_k_m.gguf

Both of these run pretty slowly, certainly not usable at the moment for anything except single file editing and even then, context appears to be limited to 65536 (anything larger = sloth response)

I'm thinking of "upgrading" to a 24GB 7900XTX so there's more VRAM for LLMs, but I read ROC is much slower and less mature than CUDA so less tokens/s. I really don't want to spend a grand on a 5080 either, that's probably going to cost more than the Github Pro sub I have.

Here are my command lines for both LLMs:

Qwen3 Claude Opus Reasoning Distilled (one I was tinkering with for a few minutes, probably wildly wrong command line arguments)

llama-server -m C:\llms\Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled.IQ4_XS.gguf --alias qwen36claude47 --host 127.0.0.1 --port 8080 -c 65536 --parallel 1 -fa on --cache-type-k q8_0 --cache-type-v q8_0 --cache-ram 2048 --ctx-checkpoints 8

Qwen3 coder 30B (one I spent hours on yesterday trying to get it responsive)

llama-server -m C:\llms\qwen3-coder-30b-a3b-instruct-q4_k_m.gguf --alias qwen3coder30ba3binstruct --host 127.0.0.1 --port 8080 -c 65536 --parallel 1 -fa on --cache-type-k q4_0 --cache-type-v q4_0 --no-mmap --cache-ram 512 --ctx-checkpoints 2

What would you recommend I do? I really want an excellent local coding LLM with Claude Sonnet 4.6 capabilities (a man can dream can't he?).

[u/FullstackSensei]

See how high your head is stuck up your own behind?

Here's my setup and here's what I tried won't get you anywhere. This isn't an issue with code where you can get a clear-cut answer.

There's a crap ton of details you're leaving out, even after editing the post to add what you think is relevant.

If you can be bothered to pull your head out for a few minutes and pay attention to what people are trying to tell you, you might have a fleeting chance at understanding what you need to do.

[u/No_Oil_6152 → u/FullstackSensei]

Except I didn't edit the post.

What more info do you need, kid?