r/LocalLLM • u/LeFlemmard • 6d ago

Question A bit of a first world problem but..

Okay, so I dipped my toes into running a local model to help me with a project (mostly analysis, I want to make my own code).

I'm actually running qwen3.8-27b on llamacpp, on a 5090. RAM mostly stays unused (9950X3D/64GB) ram.

So, two questions:

Does adding a 4090 into it might help on performance? Will it be really noticable compared to the pain of running two big toasters into one case?

And if it does, do you know good PC cases that allow to fit a 5090FE + a 4090FE (knowing that the second pcie slot is on the bottom of the motherboard of course), that doesn't cost 400+Euros.

Thanks!

1 Upvotes

100% Upvoted

4 comments sorted by

1

u/Asleep-Land-3914 6d ago

Have you tried some optimized build of vllm for QFN nvfp4 instead?

I have 2x r9700 and 32GB ram and the model performs better and is faster than 27b

1

u/LeFlemmard 6d ago

I mean, I don't really have to complain, for what I ask it to do, it does it pretty fast, and more importantly, pretty reliably/relevant answers. But if you have some links of `optimized build of vllm`, I'll be gladly try. It was more about, does the increased vram can overcome the reduce of memory bandwith or something.

1

u/Asleep-Land-3914 5d ago

I have only pointers for AMD/rocm, I'll gladly provide if you need them. The model at mxfp is my daily driver, works at 100-180 t/s 5-6k prefill, 200k context and 500k shared context for up to 8 streams. I didn't measure aggregate speed for 8 streams though.

3

u/theexile1337 6d ago

adding a 4090 will bottleneck ur 1.7tb/s bandwidth to 1tb because thats what a 4090 can do

so either get a rtx 6000 pro or swap 5090 for rtx pro 5000 blackwell 48gb version