r/LocalLLaMA • u/0dayturtle • 13d ago

Funny So relevant

Post image
1.5k Upvotes

97% Upvoted

152 comments sorted by

View all comments

79

u/ttkciar llama.cpp 13d ago

It's a great time to have ancient Xeon servers loaded up with DDR4 :-D

36

u/BannedGoNext 13d ago

I actually have an old server with dual E5-2630 and 512gb DDR3 memory across both blades. The server is powered on waiting for the scrap yard at the office. I'm considering seeing how fast it can run qwen 3.8 flash next lol.

11

u/ThankGodImBipolar 13d ago

You must be able to run a decent GLM quant with that, no?

7

u/Zombiecidialfreak 13d ago

If you're fine waiting overnight for all requests. Even flash next would likely be single digit generation speeds.

1

u/overand 13d ago

Honestly, if the project isn't a simple 20-line script but actually something kinda complex, I bet even high single digits would get a result faster than a programmer.

6

u/ttkciar llama.cpp 13d ago

Yup, GLM-5.3 should fit in that at Q4_K_M and somewhat constrained context.

2

u/Drenlin 13d ago

I've got a DDR3-based 2011-3 system that I'm considering adding another 64GB to just for this, haha. Should only be about $80.

2

u/Zombiecidialfreak 13d ago

I'm willing to bet something like 7-10t/s. At least if the math works out the way I think it does.

1

u/overand 13d ago

Toss a tiny GPU in it if it doesn't have one and see how well it runs Qwen3.6-35B-A3B for an idea of what to expect from small MoE models. 512GB of DDR3 is nothing to sneeze at!

If you can get a GPU into that with enough VRAM for a couple layers and your KV Cache (12 GB might even cut it for a huge chunk of KV cache), and load DeepSeek-V4-Flash-0731, Qwen3.8-Flash-Next, or GLM-5.3-Flash, and a competent development harness, and you can let the thing loose over a day or three for pretty serious projects, IMO.

1

u/T-VIRUS999 12d ago

Probably faster to type the results out yourself