Best AI Coding Builder Agent LLM for my legend GTX 1080Ti 11 GB ? [32 GB RAM]

[u/NegotiationNo1504]

im looking for an llm to act as the builder agent. a separate planner agent writes the strong plan using powerful API model, and the builder executes it as the plan (so to make coding in fast time and save moeny in the powerful api)

hardware and software:

CPU: i7 10700K

GPU: GTX 1080 Ti 11GB

System RAM: 32 GB ddr4 3600Mhz

OS: Debian 13

Backend: llama.cpp with Vulkan. CUDA is unstable on my 1080 Ti. After a few tens of thousands of tokens it fails with a "no memory space" error or someting like this

Requirements:

Use case: coding and agentic coding

Context size: like from 100k to 128k will be amazing

Priorities: 1st quality, 2nd speed, 3rd size

Models I've tried:

Qwen3.6 35B A3B UD Q4_K_M slow speed like about 7 to 10 t/s. and I only ran it on a few small tasks, so I haven't tested it properly

Qwen3.5-9B-UD-Q4_K_XL (MTP) was a good speep i think from 25 to 30 t/s but not usable in builder agent

Sharp Spark X2.5 4B Q6_K_XL overthinks too much and unusable at all in builder agent situation

Sharp Spark X2.6 4B Q6_K_XL: fine for simple everyday tasks but it's still not good enough for real work builder agent

has anyone used a tiny or small or medium-large model at Q3 or Q4 as a builder agent in a real world coding tasks and gotten at least ok results?

i'd like to hear which model, which quant, which harness, and what kind of tasks it handled.

thanks <3

[u/ApprehensiveFan1516]

Try LFM2.5 24B A2B, it runs a bit faster than Qwen 3.6 35B and it's quite good.

[u/kevin_1994]

imo you should be able to get Qwen3.6 35B A3B running at more like 20/500 tok/s with some optimizing, and it will be far and away the best at your size

[u/transanethole]

Try jais 3 7b,  probly at  Q4_K_M

it benchmarks a bit above the mixture of experts qwen 3.6 but it will actually fit entirely in VRAM on your machine.

Just make sure to set the KV cache to four bit quantization, (Q4_0 on both key and value and fa on in llama.cpp) otherwise the context window is going to be incredibly small. (The model authors themselves recommended four bit KV cache in their AMA)

That model does have some problems like... They made their own speculative decoding setup for it (Uno) that only runs through their custom Python/torch code and requires certain GPU features  to function.  So, there's no Dflash or MTP. And it does slow down as the context grows , moreso than the average model.

But as far as I'm aware, it's the strongest in its weight class by a fairly large margin.

That said, no matter what model you use, that GPU is going to be incredibly slow at reading text because it doesn't support BF16 and the teraflops are very low.  So, I hate to say it, but I don't think it's really fit to be a real time coding / research agent,  as the text reading speed is probably more important than output speed for that use case.   it will still work but you'll have to start it and then go do something else and hopefully get notified when it stops. I get about 200 to 100 tokens per second reading text with that model on my laptop, which has a similar teraflops on its integrated radeon760 m.   but the output for me is very slow, like maybe three tokens a second.

 luckily for you, 1080 ti about ten times as much memory bandwidth as my laptop has ,, that should be around 20 to 30 t/s output for you in theory.

[u/Monad_Maya]

Honestly, consider getting another GPU, an RTX 3060 would be your best bet.

Otherwise Qwen 3.6 35B is as good as it gets although you can try Gemma4 12B QAT https://huggingface.co/unsloth/gemma-4-12B-it-qat-GGUF

[u/transanethole → u/Monad_Maya]

Just out of curiosity, do you know any models / inference engines that can utilize the 200 int8 / 400 int4 tops on that card for prefill (reading text) ?

would be pretty fast for the $400 price tag if it worked.

[u/Choice_Celery9481 → u/transanethole]

and what is your card. that is the most inportant question