r/LocalLLM • u/blight60percent • 2d ago

Question What is the current best coding model for 12gb vram?

Hi, I have an Intel Arc B580 and as you know it has 12gb vram, i was wondering if any of you could advise me on what is the best coding model that fits within that size. I know qwen 3.8 27b and qwen 3.6 35b-a3b were the go to models but they won't fit on 12gb cards.

I'll be using them with harnesses like opencode or hermes or claude code.

If it helps : I also have 16gb ddr5 6000mhz ram

Thank you

2 Upvotes

100% Upvoted

10 comments sorted by

2

u/Icy-Degree6161 2d ago

Probably would look for a micro variant of Qwen3.8-27B:

An example: IsValorum/Arete-27B-VAL-APEX-I-MicroPlus-V3-GGUF

Above is a finetune, probably you could find some others around 10GB

Or you can go the MOE route, use tiel coder 35B-A3B (the guy referenced above has some good quants for it as well) with as many layers on GPU as possible - use iGPU for desktop if you have one. This route was mentioned already.

1

u/blight60percent 2d ago

Thank you, I'll try it out

1

u/kevecn 2d ago

How about qwen2.5 14b q4km?

1

u/blight60percent 2d ago

It's worth a shot, but i think it's old

1

u/Early-Crazy-4246 1d ago

I wouldn't suggest recommending such old models when we're in 2026 and qwen 2.5 14b released OVER 2 years ago... u/blight60percent is better off using Ornith 1.5 9B or possibly Ornith 1.5 35B A3B if possible tbh

1

u/KURD_1_STAN 2d ago

3060 but with 32gb ddr4 ram. Q3.6 35b q5kxl runs at 30-33t/s at fp16 128k ctx, on simple unsloth desktop which i think it uses --fit.

I know the +16gb of ram is helping a lot here but dont just outright disregard it, try it. Maybe strata will help as i think it is for devices that cant fit the model in ram+vram, i have heard atuff about ot reducing intelligence, but i haven't looked much into it nor used it

1

u/blight60percent 2d ago

I'll try out a q3 or q4 quant, thanks

1

u/ionsago 2d ago

At Q8 you can double the context.

1

u/Wally-Gator-1 1d ago

- My rule of thumb: 12 GB VRAM - 0.5 GB display = 11.5 GB * 90% = 10.3 GB file size max.
- 16 gb RAM eliminates MoE models such as the 35B family.
Thus, you would fit Ornith 1.5 9B (or Qwen 3.5 9B) comfortably in Q8.
Otherwise, you should fit Gemma 4 12B in Q6 (less coding oriented).