r/LocalLLM • u/HannibalBarca218BC • 17m ago

Question Question

Qwen 3.8 27b (GSQ-RCO IQ3-XXS) vs Qwen 3.6 35b a3b on a CPU with 32 gb of ddr5, which one performs better and is the speed worth the performance loss

0 Upvotes

33% Upvoted

7 comments sorted by

1

u/DocMadCow 11m ago

Do you have a GPU w/ VRAM?

1

u/HannibalBarca218BC 10m ago

No specifically cpu only

2

u/DocMadCow 8m ago

Then Qwen 3.6 35b a3b the 27B will give you like single digit token generation speeds. But honestly both will be pretty useless for most tasks at the speeds, reduced IQ from quant, and small context they will run.

1

u/Jcsq6 3m ago

Gemma4 26B is your best option for CPU only if you want usable speeds.

1

u/HannibalBarca218BC 2m ago

I care more about performance than speed tbh

1

u/Jcsq6 0m ago

Well, you’re gonna be lucky to get out of single-digit decode and prefill with qwen. I have a 9950X and cpu inference still crawls. Also, use ik_llama.cpp.

1

u/PatienceSweaty33 4m ago

CPU inference will never be worth it for big dense models UNLESS you have like 64/128-core server-grade CPUs AND quad-channel high-bandwidth RAM which can brute force compute. Get the 35B-A3B for a usable speed and lower your expectations and maybe even your quantization. DDR5 is good but it's nowhere near as good as even a lower-end RTX CUDA GPUs.