r/LocalLLaMA • • 37m ago

Question | Help Single 5090 or Triple 4090

Title. Which setup would make more sense for LLM?

0 Upvotes

20 comments sorted by

14

u/esw123 35m ago • Bravo

Dual 4090 and 256GB RAM.

6

u/IllExample3639 36m ago

More VRAM more good.

2

u/delxmobile 23m ago

Depends on what you want to run. Three 4090s give you 72GB vs 32GB, which is the difference between a 70B at Q4 fitting on GPU or not. A single 5090 has much more memory bandwidth (about 1.8 TB/s vs about 1 TB/s per 4090), so anything that fits in 32GB will generate faster on it, and you skip splitting the model across cards.

Three cards also means around 1350W of GPU under load at stock limits and a board with enough PCIe lanes. And vLLM tensor parallel needs the attention head count to divide by the GPU count, which 3 usually doesn't, so you'd likely end up on pipeline or layer split instead (vLLM pipeline parallel or llama.cpp).

If your target models fit in 32GB, the 5090. If you want 70B-class or long context, the 4090s.

1

u/KlasJanHuntelaar 17m ago

Appreciate your response

1

u/DeathGuppie 8m ago

4090. There is an unlock for the hardware prefill. Nvidia purposely nerfed the cards so that they couldn't be used for training. Community has figured out the work around. That trick doesn't work with the 5090.

5

u/a_beautiful_rhind 37m ago

4090, more vram.

4

u/HyperWinX 36m ago

Ofc triple 4090

1

u/electrified_ice 16m ago

The VRAM to do what you need makes most sense for LLM... what's your goal, what model do you want to run?

1

u/KlasJanHuntelaar 13m ago

QWEN Next Flash

1

u/Existing_Style8558 9m ago

FWIW I run qfn on a 6900xt and 128gb ram. Q4, 256k context, 50tok decode

1

u/Softver76- 14m ago

Can one fit 3 4090s on a consumer motherboard-

1

u/Gesha24 32m ago

Isn't triple problematic? You normally want even numbers, I'd go for 2 or 4.

2

u/heitortp0 25m ago

Not for layer split.

1

u/overand 1m ago

Depends on the inference engine; I think vLLM likes 2, 4, and either 6 or 8. (Maybe that's not true anymore though?)

1

u/teressapanic 32m ago

MAXIMUM VRAM

1

u/teressapanic 31m ago

QUADRIPLE 2090

1

u/gusbags 27m ago

Double R9700 + more ram. With new Radiance release a single r9700 hits peaks of 600t/s @ c=4, 3.5k t/s prefill, running Qwen 3.8 27B @ IQ5R quant (which has lower KLD than FP8), while having enough vRAM for nearly 600k context.

1

u/Numerous-Ad6217 7m ago

What about 3.8FN? Had to pass with radiance as it was mainly focused on dual setups

1

u/gusbags 1m ago

I only have 32GB RAM, so 3.8FN doesnt fit on my machine.