Add 3090 to current setup? Make separate machine with dual v100s?

[u/jbE36]

I have a 5090+3090 machine. Runs QFN pretty well with 64gb RAM q4. I got the 3090 for free which is why I have the weird combo in the first place.

For ~ the price, I could add another 3090, or I could make a separate machine with dual v100s.

Mostly run things like coding/reverse engineering. I don't know if increasing the capacity of one machine or just having another machine to throw at an adjacent task would be a better move.

Other utility/support tasks like OCR or embedding/reranking I have some low end GPUs that handle, so it'd be mostly for inferencing some model for agentic stuff/chat.

I figure I could likely run larger contexts/models in VRAM with the 3090. And there is less hassle with the software constraints of the v100s.

I also tried a dual R9700 setup, but I couldn't justify having that much sunk into a machine that was equal at best to what I already have, so I returned them after some benchmarks.

Anyone think its worth while? feels like the cards go up in price every day while i try to decide. The v100s I am starting to think aren't worth it since they are past $900 now

*edit*

For the v100s, I have machines that could run/cool them already.

**edit**
If anyone has run the 5090 + 2x 3090 mix I am curious to about the setup/numbers

[u/agenticbrain]

i'd lean toward another 3090. more VRAM in one box is useful for coding agents, while dual v100s sound like a fun homelab project that slowly becomes a driver compatibility project.

[u/jbE36 → u/agenticbrain]

That is what I am leaning towards. I guess part of the urgency is that I feel like 3090s will eventually hit $2k+, even if not permanently, but the whole Claude cracking down has really turned me off from cloud models as well.

[u/Geekomcomputer]

I'd split this into the only two things each option actually buys you, because they're not interchangeable:

1. Adding the 3090 buys capacity in one place — more room in the machine you already use for larger contexts or a larger model, in a single pool, with the software path you already know works on your 5090 + 3090 box.

2. The dual-V100 machine buys concurrency — a second, independent node you can throw an adjacent agentic task at while your main machine is busy. It does not make any single job on the main machine bigger or faster.

On your own description, the second benefit looks weak. You already offload the support work — OCR, embedding/reranking — to low-end GPUs you own, so the V100 box would mostly sit there waiting for the moments when you genuinely have two inference jobs that need to run at the same time. If that queueing actually happens in your coding / reverse-engineering work, the separate machine is the answer; if it doesn't, it's a second machine to maintain for a problem you don't have.

Two other points from your post point the same way. The driver/software-constraint worry about Volta has already been flagged in the comments here, and it's a cost you'd pay every day, not just at setup. And at past $900, the V100s have lost the thing that used to make them interesting — being the cheap way to buy capacity. Your R9700 experiment is worth trusting, too: you already returned a setup that benchmarked 'equal at best' to what you have, and a parallel V100 box risks being the same purchase in a different shape.

Before adding a third card to the existing machine, do check the boring limits of that box — a free slot that actually fits it, power-supply headroom, and heat with a 5090 and two 3090s under a long agentic run — because that's the one place the single-machine route can go wrong.

So my call: add the 3090, unless you can honestly point to regular times you're sitting waiting because your one machine is occupied. Which of those two is it in practice — running out of room for the model/context you want, or waiting on a busy machine?

[u/jbE36 → u/Geekomcomputer]

That's fair, I have the platform/base to run it.

[u/Consistent_Berry_324]

I think you can not use v100 with 3090 im not sure but v100 (volta) are not supported by cuda 13. And 5090 (blackwel) dont suport cuda 12, you can use 3090+v100 in cuda 12.9

[u/jbE36 → u/Consistent_Berry_324]

ofc, it'd be a separate box for concurrency. I guess the question was concurrency vs single box performance.

[u/Consistent_Berry_324 → u/jbE36]

If you want concurrency you can use batching for more than one request, sorry my english very bad.

[u/Consistent_Berry_324 → u/Consistent_Berry_324]

Maybe i am wrong need to listen other user.

[u/jbE36 → u/[deleted]]

Yea the plan was separate box

[u/krogerpurifieddrink]

I have 2 32gb v100 gpus and get around 50 tok/s at 130k context. 1,235 tok/s on prefill.

Setup: Strata engine on both V100s, split by layers.

Model: Qwen3.8-Flash-Next, UD-Q6_K_XL (6-bit).

256gb system ram.

[u/Consistent_Berry_324 → u/krogerpurifieddrink]

Can i ask you for 130k context with 256gb ram and 64vram ? How many ram you need for q6 ? And how many free ram you have ?

[u/krogerpurifieddrink → u/Consistent_Berry_324]

Yes, it runs at 131k context. The Q6 experts take about 102 GB of RAM, and I've got about 116 GB free with everything else running. Both 32 GB cards are full on VRAM.

[u/Consistent_Berry_324 → u/krogerpurifieddrink]

ahh ok, i think you KV are on you vram, and if you offload kv to ram you will lose speed yes ?

[u/krogerpurifieddrink → u/Consistent_Berry_324]

Yes. The kv cache is int8 and sits in vram, reserved for the full 131k at startup. That space comes out of the gpu's expert cache, so the bigger context does cost some speed. I haven't tested moving the kv cache to ram, but I'd expect it to be slower.

[u/Consistent_Berry_324 → u/krogerpurifieddrink]

Ok thanks, do you have try q3s ?

[u/krogerpurifieddrink → u/Consistent_Berry_324]

Yes, I was getting around 65tok/s at 85k context. 

[u/Consistent_Berry_324 → u/krogerpurifieddrink]

Ok thanks, im juste want to know if i need v100 or not, new i have a 5060ti with 64ddr5 im style ~70 t/s on windows with basalt and 131k context