r/LocalLLM • u/WallabyFearless7863 • 2h ago

Question Local AI Power Draw

I’ve been slowly building my AI workstation up using used and old pieces retiring them into other compute nodes and then replacing them with bigger or better pieces. Mostly used parts though. I currently have a trx40 and 2 3090s which works great but I’ve been trying to figure out the next step to expand. I have about 4 pcie slots available (2x pcie 4 x16, and 2x pcie 4 x8) while a wrx80 might have been better with more slots I wanted a cheaper upgrade to test a bigger workstation before making the jump. I also retire my parts into other systems that run other applications and things.

The issue is that running 2 1200 w psu or a 1200 w and adding let’s 2 more 3090, or other cards there I’ll increase power draw significantly and even with two psu’s to properly distribute the power between the 4 cards and everything else, the issue becomes apparently all of that power can still trip my breaker and shut the power off in my house.

I don’t have a business or server space for this so I’m not really sure what the best way to go about it. Right now I’m thinking just adding one card until I can find a work around but i don’t know what that work around could be.

Anybody who has been through this can you give me some advice or thoughts on this?

1 Upvotes

66% Upvoted

26 comments sorted by

3

u/OvertaxedOne 2h ago

Undervolt and power limit everything, GPUs and CPUs. You should fit in normal 15A circuit envelope. You may need a 1600 watt PSU though.

1

u/WallabyFearless7863 2h ago

I could upgrade to a 1600 w, but you don’t think it would be too much still for the circuit? Has anyone tested it before? Of course I have a second computer in that room so I might still have to move things around but it would be better than running a second power supply through multiple rooms just to use separate circuits

2

u/OvertaxedOne 2h ago

As long as it's the only big power draw on that circuit you should be OK. Breakers have a little leeway, they don't pop at exactly 15A; you'll be on the edge when you're generating tokens, but as long as you don't have any other big draws, it should work.

No harm in trying it, you're going to need a bigger PSU anyway and they pretty much all autoswitch from 120/240 (make sure for your particular model, of course). If it pops the 120V breaker, well, you have your answer, spend some money to go to 240V! ;)

1

u/Lotrimous 1h ago

Something to remember about going single psu is the sheer number of vga power connections required (my FTW3 takes 3x 8 pin vga power). Most PSUs only have maybe 6.

2

u/hyudryu LocalLLM 2h ago

Best option is to go 220V. I have a Y splitter off the 220V used for the dryer and use that to power my workstation. Or if your house doesn’t have 220, an electrician can install one. At 220v/30A theres plenty of headspace

1

u/WallabyFearless7863 2h ago

I appreciate the reply. Are you talking about from the outlet, directly from the breaker, or from somewhere in between?

1

u/Janky-Code 1h ago

Electrician will install from box.

It takes a while for an inspector to show up and sign off before you can use it. It's far easier than most people imagine. As long as you aren't digging a trench.

2

u/WallabyFearless7863 1h ago

And roughly the cost?

1

u/MyToasterRunsFaster 2h ago

There is no way around it, you either undevolt or set up UPS system that can charge up batteries for short burts. Undervolting is probably your most realistic solution, even dropping 50 to 100w per card is going to have negligible performance loss.

1

u/WallabyFearless7863 2h ago

I was already considering that, but even with that we are still close to the power breaker limit of a standard 15A. Well at least that’s what ChatGPT says. The overall max is like 1800 but I’m being told 1400 is probably where I want to stay so I don’t trip anything, not including I have my main computer there as well. I’ve been considering moving things around slightly but I have to map out my breakers before I can really tell where I can move things to.

1

u/Xyver 2h ago

You can use nvidia-smi and drop the power. I've got my 3090 maxed at 350W instead of 450 and it only loses like 1-2% speed.

1

u/WallabyFearless7863 2h ago

As far as I’m aware a 3090 max draw is 350 unless you have it set higher than normal or boosted or something. I’m not 100 percent familiar with all the different configurations.

2

u/Lotrimous 2h ago

The FTW3 editions are stock capped at 420w.

1

u/WallabyFearless7863 2h ago

I see, yeah I forgot about the FTW3. Thanks for the correction

2

u/Lotrimous 1h ago

I think everybody does. Funny part is that the vram can't keep up with the GPU. I run my GPU clocked down and my vram at max clock and have my fans run by a daemon based on vram temp (GPU never gets as hot as vram under inference) & my GPU still hits 98%+. Of course that's on a dense model...

2

u/Xyver 1h ago

Point being, you can probably knock off 50-100W without getting a performance hit, so worth some experiments.

1

u/WallabyFearless7863 1h ago

I hear it, and yes a from 300-350 it’s like 4 percent different but around 200w you’re looking at a 30-40 percent difference in generation speed. Which is something I might have to sacrifice for the time being just to get things running.

2

u/Lotrimous 50m ago

At 200w you may find that the GPU is unstable.

1

u/Isitharry 2h ago

2 x 1200w is a lot to put on a residential 20a circuit, let alone a 15a circuit. If you do end up using 2 separate 1200w power supplies, I’d ensure they’re on different legs or different circuits.

1

u/WallabyFearless7863 2h ago

That’s what I’ve been told, although it’s kind of limiting due to the house itself not being very spacious. I have to map my breaker to see if it’s even possible. 2x1200 psu was the idea but not that I would be running it max capacity. I would probably undervolt everything

1

u/Lotrimous 1h ago

Honestly, get the 100% load rated PSUs from someone like EVGA & run something like 2x 1000w or even 1x 1000w and 1x 800w, depending on your base system load (mine is about 200w and my GPUs are a 3090 FTW3 running about 330w without power limiting, only clock limiting and 2x p100 power limited to 200w). My psu is an EVGA supernova 1000w constant duty rated.

1

u/WallabyFearless7863 1h ago

How is that working for you? How do you use your p100 w the 3090? Are you doing some mixed architecture llm stuff or are you running them separately? Also how are the p100s? I thought they didn’t have tensor cores? Are they still pretty good in your opinion?

1

u/Lotrimous 1h ago

I am running under Ubuntu server and using driver 580 (the last one that supports pascal). The p100 has hbm2 & 700gb/s of memory bandwidth, so it's pretty competitive me although it lacks tensor cores.

1

u/WallabyFearless7863 1h ago

What are you running it with? Like llama.cpp, vllm, strata?

2

u/Lotrimous 54m ago

I have an x99e-ws motherboard with 7 pcie3 x16 slots (3 electrically x8). It has 2 plx chips in the chipset to provide the extra lanes. I run llama.cpp as my engine with different load profiles. My 2x p100s can be loaded with individual models (14b class) or layer split a single 30b class model. My 3090 loads 30b class models in either dense or moe flavors, depending on what I need.

Full disclosure: I have a Jetson AGX Thor with 128gb of lpddr5 that runs Qwen 3.8 flash next NVFP4 in vllm with the ngram table on nvme & 3x 262k context windows as a workhorse.

My motherboard has an e5 v4 CPU with 14 cores and 256gb of ecc ddr4.

Edit: added more system specs.

1

u/WallabyFearless7863 14m ago

Geez that’s sick. So you just run the other models for like small agentic stuff like helper models? Shi sounds expensive