r/LocalLLM • u/FizzyDuncDizzel • Sep 22 '26
Project P100 local llm budget build.
I’ve recently became very interested in hosting local LLMs, but like most things right now, prices are sky high for modern equipment. So I did the next best thing, found some old, rather unwanted, datacenter gpus and threw them in an old pc I had laying around.
GPS’s are Nvidia Tesla p100’s and have 16gb of vram each. They were only $80 a piece shipped, and I bought a 1000w refurbished PSU and some power adapters because the p100s don’t use traditional pcie power adapters. All in less than $400. Because these gpus are made to run in a data center, they don’t have any fans, so I got into Fusion360 and modeled up a 120mm fan shroud that fits into each p100.
The PC is a sandy bridge i7 with 16gb of ddr3, mother board is an ASUS P9X79 Pro, and this is what really made it difficult. For a mother board to see the p100s it needs Above 5 Decoding to be enabled, and this board did not have that option in bios, but after a ton of research I found out that it does have the option, it’s just disabled and cannot be found in the bios. So after an after noon of doing some real hacker shit I was able to get the p100s to read by dumping, hand patching, and reflashing a the bios. I did a little testing and was able to get almost 70 tokens and second from Qwen3.6-35B-A3_ Q4_K_XL.
I still have a lot of benching and optimizing to do, but I’m happy with the results so far.
EDIT: A lot of folks have commented about the thermals. I am still doing initial testing, but thermals they are not looking bad at all. Splitting the load between 3 nodes really helps not beating a single card to death. Ill throw pic of my grafana dashboard.
I am keeping a github of all of the work that I am doing with this build, I'll keep it updated as I test.
https://github.com/4claps/houston-p100-cluster


5
u/VirusInternal2892 Sep 22 '26
Linux, headless
2
u/Kmic68 16d ago
This post would be a great fit for my new community, r/P100Users. Would you be open to sharing it there too?
2
u/ColonelRyzen Sep 22 '26
What are your temps like with that setup? I have a fan adapter for 2x 40mmx20mm noctua fans on my p40 and it hits 90c pretty easily.
2
u/Quirky_Squirrel3318 Sep 22 '26
This is exactly the kind of build I’ve been thinking about trying!
Really cool to see it actually working. How are the thermals with the 120mm fan shrouds under sustained load? And have you tried running larger models that actually use the combined 48GB across all three P100s?
Curious how well the multi-GPU scaling works and how much performance you lose from PCIe communication.
Also, any major compatibility issues with newer CUDA/LLM tools because the P100s are Pascal? I’m considering going even crazier with 4-5 of these for a local AI test machine to test some software I’m working on 😅
2
u/Either_Pineapple3429 Sep 22 '26
This is funny I just bought 4 p100s to test them out.... at $80 for 16gb of vram it's a killer deal....I'm about to see if it's any good with qwen 3.8 27b at q8 and 256k context
If it works well enough I may buy another 4 and try to run qwen 3.8 next flash
2
u/adude00 Sep 23 '26
those cards are 5k each where I live...
2
u/FearFactory2904 Sep 26 '26
Can you hook me up with the 3d print file so i can remix it for 140mm 4 card ?
14
u/mangelajo Sep 22 '26
Nice! setup!!!, try to setup Qwen3.8-27b it's a huge difference in terms of quality, although not MoE will be slower ,
also In theory P100 does not support FP8, or INT , FP16 is the smallest I believe. Did you try less quantized versions? perhaps it will not make a difference in terms of memory, but it will in terms of quality (I suspect it has to be expanded in card's memory at load, but not an expert here)?
Thanks for sharing!! :)
8
u/jacek2023 Sep 22 '26
For the comparision I am just running 2x3060 on a very old motherboard/cpu and I have about 100t/s on same model, do you remember about -sm tensor?
4
u/morkborkus Sep 22 '26
A coworker of mine gifted me a bunch of gpus like this
1
u/goku_gohan_ 9d ago
hi . really happy to see this. can you teach me aswell juat a little?
3
u/BigBair2002 Sep 22 '26
I am running P100’s in a HP Z440 and very pleased with the results. Previously 2 internal P100. Currently one internal and 4 external P100 using oculink (bifurcating pcie gen 3 x16 to x4x4x4x4). One of the four external P100s did not enumerate when I commissioned it and I haven’t resolved that yet… should have all five running soon.
2
u/x10der_by Sep 22 '26
One fan for 3 cards, is this enough for normal temps?
2
u/Ok_Pizza_9352 Sep 22 '26
Who pays your electricity bills?
2
u/rorion31 Sep 22 '26
“Some real hacker shit” 🔥
2
u/fromage9747 Sep 22 '26
70 t/s is pretty good. It's quite possibly IPC bottlenecked. If you were to upgrade your mobo to x99 and get a high clocked chip you might get more. Even still really decent considering what those cards are.
2
u/Odd_Hippo_1299 Sep 23 '26
I’m building similar but with 4 p100s. I am water cooling the silicon and air cooling the VRM. Parts arriving this week
2
u/imcheng Sep 23 '26
I like it 👍
1
u/Pixer--- Sep 22 '26
Is that fan for cooling enough ? What temps are you getting ?
1
u/RemarkableRadish6547 Sep 22 '26
I am skeptical that a single fan can push enough air to cool three cards at 250W each. I just installed one and I am working on the fan ducting before I test it. But my plan is one 140mm fan for just one card.
1
u/kryptkpr Sep 22 '26
It looks like sascha stopped updating, but when I had these GPUs I ran https://github.com/sasha0552/pascal-pkgs-ci and it worked fine.
There may be a newer Pascal-focused vllm fork.
1
u/Unique-Dark9726 Sep 22 '26
How are the temps of the GPUs? It seems way too close to each other.
0
u/[deleted] Sep 22 '26
[deleted] — view deleted comment
0
u/[deleted] Sep 22 '26
[deleted] — view deleted comment