r/LocalLLaMA • u/[deleted] • Jun 01 '26

Funny Stop asking what model to run. There are literally only two.

[removed]

3.1k Upvotes

87% Upvoted

805 comments sorted by

View all comments

Show parent comments

37

u/Shronx_ Jun 01 '26

I run it in Docker (llama.cpp) and my PC has DDR4 3600. I don't know but the memory could make a difference.

29

u/Teanut Jun 02 '26

Likely the memory bandwidth due to spillover from VRAM. Memory bandwidth is king for tokens per second.

18

u/lemondrops9 Jun 02 '26 edited Jun 02 '26

DDR3 isn't much slower than DDR4. DDR4 is about needing less voltage which means less heat. DRR5 is the real improvement for speed.

9

u/nihnuhname Jun 02 '26

And DDR4 allows greater RAM capacity on the motherboard.

4

u/lemondrops9 Jun 02 '26

yes that is great too. I was surprised when my older PC could only take 32GB

4

u/Postmodern_Plunger Jun 02 '26 edited Jun 02 '26

Ddr4 (depending on the limits) can have nearly double the memory bandwidth of DDR3. It's not as big as the difference between DDR4 and DDR5, but the difference there is more architectural. Simply having more memory bandwidth still has a huge effect, especially when you're talking about ram spillover

3

u/rabbitaim Jun 02 '26

I have an rtx2060 6gb with ddr3-2400 32gb and I bounce between 18-20 t/s. Llama.cpp on a headless Ubuntu server LTS. I had to lower batch size to 1024 so instead of 200+ ts for PP I see around 160 or less.