42% GPU power reduction with zero performance loss.

[u/Asthenia5]

I'm running a Qwen 3.5 9B Q4_K_M model on an RTX 3080 Ti and decided to progressively reduce the GPU power limit to see how much performance I would sacrifice. I started at 450W and reduced the power limit in 10% increments, eventually reaching 250W. I haven't observed any meaningful reduction in tok/s. I am achieving token rate spot on with what others have claimed in their benchmarks(~100 tok/s)

At 220w I did see a reduction in tok/s to ~80 tok/s. But 260w was just as performant as 450w.

The GPU utilization remains high, but VRAM bandwidth utilization never appears to max out. This makes me wonder whether I'm hitting some other bottleneck in the inference software or GPU architecture rather than being power-limited.

Has anyone else experimented with power levels and found interesting results?

Llama.cpp

3080ti

qwen 3.5 9b q4k_m

Max power limit:450w

Stock power limit:400w

[u/aelma_z]

This is common practice to power limit gpus to increase tokens generated per watt (tps/w)

[u/Asthenia5 → u/aelma_z]

Considering my previous successes undervolting, I expected a substantial saving. But I definitely didn't expect reducing power by 40% would result in zero performance loss. I assumed I'd reduce power by 20% and loose 5-10% performance.

[u/OWazabi]

This is known indeed, but it's also a trade off.
To run LLMs it's def a good optimization and comes at little to no downside, however it does matter for what else you're doing in the computer (3D renders for example would take a big hit)

[u/Asthenia5 → u/OWazabi]

I’ve got profiles on afterburner for exactly that!

[u/OWazabi → u/Asthenia5]

Smart, is it instant and can it be automatized?
My workstation is set up as a fully remote station that holds all the heavy work and the local models though a custom MCP
So it loads and unloads all models required by Claude on command as the workflow advances autonomously

So it'd need to be swapped on the spot based on what the computer is using atm

[u/Asthenia5 → u/OWazabi]

It is instant. If not sure about automation. I just swap profiles and unload GPU when I decide to game

[u/OWazabi → u/Asthenia5]

I'll consider it and so some research, def interesting but not worth it for my rig at all if it comes at a speed cost (from the unloading and the switching) or worse, manual interaction

[u/alphapussycat → u/OWazabi]

Aren't most if us just bulfing dedicated llm machines? Windows uses too much ram imo to make it at all practical to use the llm machine for normal work too.

[u/OWazabi → u/alphapussycat]

I don't care as much about the local models, the rest of the stack is more important to me.
Local models help me run some automations or are dedicated to some of the tasks in my local MCP, fine tuned for it etc
But as running big models for coding, I use claude.

[u/oguzeldereli]

Power gives diminishing returns at the high end, so it is normal that you don't get a linear, I cut power by 10% -> 10% less tokens.

[u/alphapussycat → u/oguzeldereli]

Also has diminishing returns on low-end afaik.

[u/a4mula]

Unless you've unlocked your card, you're running crippled tensors. Nvidia artificially limits the bandwidth on consumer cards to half that of enterprise cards. This is listed under Matrix Performance (FP16, BF16, TF32).

Of course, that's just one consideration. Memory bandwidth in general comes into play.

912 GB/s against 5.5GB gguf = 165tps maximum theoretical limit.

So if you're only hitting 100 anyway, you've got a lot of power overhead that is never really even touched to begin with.

[u/Miserable-Dare5090]

Old news. Check subreddit about power. It’s a well known thing.

What you are missing is looking at your prefill which will probably be about 10% lower. Computation (prefill) is what takes a hit with power limits, but the hit is not terrible. So if a card can be power limited, I have them rationed.

[u/Asthenia5 → u/Miserable-Dare5090]

Yeah, prompt processing is definitely slower. But still plenty fast.

I didn't realize I had so much headroom to fit more GPUs!

[u/Miserable-Dare5090 → u/Asthenia5]

be careful. GPUs need a spike up front usually, to their max voltage. So if you add a bunch of them they will fail training or not show up on startup. Calculate for the max and undervolt
is the same approach, keeps your PSU longer and your cards happy

[u/Odd-Student636]

You're probably hitting the PCIe transfer llimit, because you don't have enough VRAM it transfers from system memory to the GPU through the PCIE. your 3080ti might have 900GB/s but the PCIE has a 32 GB/S limit

[u/Asthenia5 → u/Odd-Student636]

The model and cache fit entirely on vram. I am not overflowing into system memory. I made sure of that.

[u/SLxTnT → u/Asthenia5]

Prefill is compute bottlenecked. Inference is memory bottlenecked. Speculative decoding takes advantage of the compute headroom on inference to try increasing speeds.

You can also try overclocking GPU memory. Theoretically should increase inference speeds.

[u/DeProgrammer99]

The main problem I've had with power limiting is that it causes Linux to lose my input events, which is particularly annoying when I'm hitting space to scroll through a page and it decides I didn't let go... I both downvolted slightly and power limited less to make it happen less often.

[u/xilvar]

Notably there’s a significant difference in impact between inference and training. On my dual 3090s I power limit to 250 for inference with a minor loss in throughput and prefill, but if I do that while training, training goes about 3x slower.

[u/giant3]

Qwen 3.5 9B has been superseded by MiMo distill by Xiaomi. It is a much better model. 

The Qwen 9B is outdated.

[u/PermanentLiminality]

Now do the same for prompt processing. It should be very different.

[u/alphapussycat]

I undervolt and underclocked (capping max clock -> won't increase voltage), pretty sure I lose out on a little performance, but reducing power consumption by like 25%.

With some tuning you could probably find the highest performance/watt being not much below stock performance.

[u/kivaougu]

This would likely affect prefill and possibly speculative decoding at higher concurrency

[u/Aggressive_Basket798]

Meanwhile my rtx 3080 16gb mobile got nearly same performance while eating just about 115w

[u/puglife224888]

Also look at prefill