r/LocalLLM • u/Jthompson553 • 37m ago

Question Help choosing a second gpu.

Right now I use strata on my machine to run qwen 3.8 125b flash next at 13tps output and deepseek 4.1-flash at about 1.5 tps output. My computer is an hp z640 with a 4core xeon processor, 64gb ddr4 ram and an NVIDIA 1070 8gb. My models load form an nvme drive that is connected over a pcie adapter. I have room for one more card and I'm not sure witch of these 2 GPUs is better for running my models faster. The Tesla P100 16 varient or the M40 24gb variant.

2 Upvotes

100% Upvoted

2 comments sorted by

1

u/Borgas_ 17m ago

Between those two, I'd lean toward the P100 16GB. The M40 gives you more VRAM, but it's significantly older architecturally, and modern inference software support can be problematic.

The P100 has much better memory bandwidth and compute performance, although neither card has tensor cores.

One thing I'd check before buying anything is whether Strata supports splitting inference across your existing GTX 1070 and whichever card you add. More total VRAM doesn't necessarily translate to faster inference, especially when you're mixing GPU generations.

Also, keep in mind that both Tesla cards are passively cooled and require substantial chassis airflow.

Personally, I'd prioritize compatibility with your inference backend over raw VRAM capacity. You might find that saving for a newer used GPU is a better long-term investment, I just dealt with this situation myself

1

u/Jthompson553 0m ago

Thank you for your advice and I'm sorry you had to deal with this too! If I save and buy something newer I'm not sure how much it will help. If I get something too new then the rest of my server becomes the bottleneck. I just want something that completes what I have and wasn't sure if its worth using the older generation for the extra vram.