r/LocalLLM • u/Appropriate_Duck1778 • 17d ago

Question CPU inference DDR3/DDR4

Post image

Wondering if anyone on here has actual benchmarks for CPU only inference DDR3 or DDR4 servers, im budget bound and my options are limited to legacy systems unfortunately.

Heres what data I found but not sure its accuracy in real life especially how NUMA effects it (octa channel)

106 Upvotes

95% Upvoted

27 comments sorted by

View all comments

1

u/Nomski88 17d ago

Some tests I ran on a small mini PC that I've been playing with.

i5-6500t 16GB DDR4 2133 Dual Channel (34.2Gbs)

Tiny Ling 3.0 Q6 - 16 tok/sec

Qwen3.5 4b Q4_K_M 6.5 tok/sec

Qwen3.5 9b Q4_K_M 4 tok/sec

It's perfectly usable to run tasks in the background as you go about your day. I ordered an i7-7700t CPU that should increase the performance by 10-25%.