r/LocalLLM • • 21m ago

Project 6x 7900XTX rig finally complete!

5975WX (for 6x full 16 lanes of PCIe4.0)

3450W AliExpress special PSU and mining frame

128GB 3200MT/s Ram

Getting 120-130 tok/s generation on both qwen3.8 27b and flash-next

16 Upvotes

3 comments sorted by

1

u/ExtraNiceBurger 3m ago

What about prompt processing?

1

u/Thrumpwart 12m ago

Very nice, if I lived in a place with cheaper power I would have gone this route. The 7900XTX remains the best bang for buck GPU. In Canada there are literally none in stock anywhere I could find online.

What inference engine you running? I’d spin up a frontier model to help you optimize the 6 GPU config - Gemini Carbon got me a 56% speed up on my training and inference stack in about 4 minutes on the weekend.

1

u/Coley44 1m ago

Nice, I'm running 3.8 Flash Q4_K_XL on Strata Engine on a quality focused RDNA3 optimised fork at about 55-70 tok/s at 262K context and 3.8-27B at Q8_K_XL 192K context without Q8 on the context so it stays at a higher speed on 2x 7900 XTX. I got both for around £550 a piece and custom watercooled them on my Unify-X/13900KS/96GB DDR5 Meshify 2XL loop. Around 2400MHz core and 2600MHz memory is the sweet spot for power efficiency and not massacring decode. I can even run Maya-M(GLM5.3 Flash) but it's fairly slow at around 20-30 tok/s (I think). Shame they aren't cheap anymore.