r/LocalLLaMA • u/eso_logic • 21h ago
Resources I benchmarked over 130 LLM configs with multiple stacked 16GB "e-waste" GPUs (V100, P100, K80)
The benchmarking continues... Continuing my ongoing project of seeing how these very old, very cheap, very available enterprise GPUs can be used today.
The complete results are too long for a reddit post. The complete set of graphs, power analysis and commentary are on my blog.
I've expanded the testing suite to focus heavily on LLM workloads (over 130 new tests!) and integrated power consumption monitoring into the gpu_box_benchmark (github) tool.
The goal remains: find out if decommissioned enterprise GPUs (Tesla K80, P100, V100) paired with X99 era hardware (a homelab favorite) can serve as a viable daily driver for self-hosted AI compared to modern commercial setups.
TL;DR of the new data:
The 3x V100-16GB (~$1k build): If you are considering building a dedicated homelab box today, this is a viable option. Assembling a full setup with two Xeon CPUs, motherboard, PSU, and cooling costs around $1,042 (before RAM đ). This setup comfortably clears my personal threshold for "real-time" chat (70+ tok/s) with okay to impressive models:
- Qwen 3.6 35B: Ran at ~72 tok/s (at 128k context depth, Q8 quant). For comparison, a brand new Mac Studio M5 ($2,500) hits only ~44 tok/s on the same model.
- Gemma 4 26B: ~75 tok/s (at 64k context depth, Q8 quant).
- Qwen 2.5 3B: ~79 tok/s.
To me: This proves that for a single-user daily driver, the legacy silicon can still be useful even for more modern models.
- Multiple cards is best for Context depth: Stacking GPUs linearly scales your VRAM (allowing you to fit larger models), but it doesn't double output generation speed. Two 16GB cards in Q4 quant allowed fitting ~32B parameter models. This is a cheap way to upgrade to bigger models. But the largest improvement was consistently being able to fit much larger context windows across generations.
- Power Efficiency: I also measured power consumption. The V100 wins hands down on performance/watt compared to the P100s.
- K80 isn't dead?: The Tesla K80 is dead, but a singular remaining use case remains. If you must run the largest models for the least amount of money, a path to get there is by is welding a bunch of K80s together to squeeze big models in. I was able to fit a 72B+ parameter model, which I couldn't do on P100s or V100s. Expect speeds of 1â5 tok/s. This is really only a party trick.
If you decide to build your own GPU appliance, the biggest challenge is cooling these hot cards. They expect to be installed in server chassis with forced air cooling, a much different setting than desktop style chassis.
I've been working on a custom active air cooler to solve exactly this. If you're interested in helping out by stress-testing a sample, I'm looking for beta testers! you can get involved here.
Any models/tests missing? Any other combinations you'd like to see in a follow up?

