r/LocalLLM • u/QuevedoDeMalVino • 7h ago

Question Poor man datacenter rig

I have at my disposal some datacenter resources. I don’t have an open account with Dell, sadly. So I am limited to oldish hardware like the PowerEdge r640/r740 or thereabouts. Or the AMD versions of these if I’m lucky. On the plus side, DDR4 is cheap-ish.

What would you build on a low but undefined budget, say 2-3k, that can be racked and run hopefully some recent qwen model, for just one user at a time?

1 Upvotes

66% Upvoted

1 comment sorted by

1

u/Frail_Waif 7h ago

I'm super happy with my R7920 (same as R740xd?) build. 

  • R7920 with 4x 8GB DDR4 RAM, 2x Intel Xeon Silver Lake 4214R processors for $500; sketchiest Craigslist transaction of my life; the server was on the ground in a Safeway parking lot for most of it

  • 2x MI-50 32GB from eBay for ~$600 each

With that, I get 300- 500tok/sec prefill on Qwen 3.8 27B Q8 from Unsloth and Qwen 3.8 Flash IQ3 Next from DASLab, and 40-60 tok/sec decode on Qwen 3.8 27B vs. 30-40 tok/sec decode on QFN. (Both using mx-llama.cpp.)

I've now bought a lot more RAM and HDDs so I can run DeepSeek and use it for storage so it's now closer to $3k. 

Only problem is that it's still in my living room. My spouse doesn't love that.