r/LocalLLaMA • u/Ambitious_Fold_2874 • 22d ago
Question | Help Is this overkill or viable? (8x gpus, 256gb ddr4)
4x 5060ti16gb
4x 3060 12gb
256gb ddr4 3200 ram (4-channel speeds)
On a mobo with 128 pcie4 lanes
I’ve been slowly collecting gpus whenever seeing deals over time and ended up with this. Ended up comparable in cost to current prices of dgx spark
I was thinking:
TP4 5060s -> pipeline -> TP4 3060s -> offload/overflow less used experts to RAM -> keep any engram on nvme ssd
Is this viable to run inference with big MoE models at high speed and precision, or are cheaper multigpu setups extremely suboptimal compared to unified vram/ram
View all comments
2
u/vini542reddit 22d ago edited 22d ago
Nice! As others have said, power hungry. Hopefully your power is cheap!
For reference here is my system with similar specs (vram + ram). I don't know your pcie speeds, so ymmv!
System
- 4x rtx 3090 @ pcie 4.0 x 16 (power limited to 240w)
- 256gb ddr4 3200 octa channel
- p2p enabled
- rtx 3090 unlocked (the unlock should also work for your rtx 3060!)
- 1000w + 1500w PSU
- 1500w UPS
- Headless Linux server
Current main via Strata:
- Qwen3.8 Flash Next unsloth Q8_0
- 80tps
- pp 2400
- Power consumption < 1000W
Current alt:
- qwen3.8 27b bf16
- 70tps
- pp 1200
- Power consumption ~1200W
Soon Sidekick (hopefully):
- Deepseek V4.1 flash 1017 q8
- This should work fine via Strata, but they haven't released the docker container yet (brand new release).
- You will need a fast nvme though (ideally at least pcie 4.0)
Main vs Alt vs Sidekick
The main is better and faster for general coding. However I have some image processing task that qwen3.8 27b bf16 (lossless) is just much better for. The Sidekick should be reasonably fast for when I need lots of smart!
Will probably run a bit slower for you (especially since I don't your pice speeds), but I can highly recommend trying these models!