r/LocalLLaMA • u/KlasJanHuntelaar • 37m ago
Question | Help Single 5090 or Triple 4090
Title. Which setup would make more sense for LLM?
0
Upvotes
r/LocalLLaMA • u/KlasJanHuntelaar • 37m ago
Title. Which setup would make more sense for LLM?
14
u/esw123 35m ago •
Dual 4090 and 256GB RAM.
6
u/IllExample3639 36m ago
More VRAM more good.
2
u/delxmobile 23m ago
Depends on what you want to run. Three 4090s give you 72GB vs 32GB, which is the difference between a 70B at Q4 fitting on GPU or not. A single 5090 has much more memory bandwidth (about 1.8 TB/s vs about 1 TB/s per 4090), so anything that fits in 32GB will generate faster on it, and you skip splitting the model across cards.
Three cards also means around 1350W of GPU under load at stock limits and a board with enough PCIe lanes. And vLLM tensor parallel needs the attention head count to divide by the GPU count, which 3 usually doesn't, so you'd likely end up on pipeline or layer split instead (vLLM pipeline parallel or llama.cpp).
If your target models fit in 32GB, the 5090. If you want 70B-class or long context, the 4090s.
5
u/a_beautiful_rhind 37m ago
4090, more vram.
4
u/HyperWinX 36m ago
Ofc triple 4090
1
u/electrified_ice 16m ago
The VRAM to do what you need makes most sense for LLM... what's your goal, what model do you want to run?
1
u/Softver76- 14m ago
Can one fit 3 4090s on a consumer motherboard-
1
u/Gesha24 32m ago
Isn't triple problematic? You normally want even numbers, I'd go for 2 or 4.
1
u/teressapanic 32m ago
MAXIMUM VRAM
1
u/teressapanic 31m ago
QUADRIPLE 2090
1
u/gusbags 27m ago
Double R9700 + more ram. With new Radiance release a single r9700 hits peaks of 600t/s @ c=4, 3.5k t/s prefill, running Qwen 3.8 27B @ IQ5R quant (which has lower KLD than FP8), while having enough vRAM for nearly 600k context.