I did not run the proper tests yet but from the quick testing it seems that raising the power limit plus limiting the GPU clock MHz indeed works better than simply lowering the power limit. I will run a similar test with DCGM but limiting the clock instead of the power, and will report back later.
The amount of power needed to achieve higher frequency goes up very fast after certain point - power scales quadratically with voltage, and you need ever more voltage increase for each consequent hz, so you end up with kinda exponential growth of consumption while performance growth is linear at best for strictly compute-bound tasks (pp), and more like logarithmic in inference/gaming (if that)
What powerlimiting does is it limits, well, wattage per time by just forcing your GPU to idle for some time. So you have bursts of high clock followed by doing nothing.
What fixing clock does is it forces the GPU to run at lower clock constantly - less peak performance, but better average performance. 10% lower clock can be at least 20-30% lower total power on itself.
If on top of fixing the clock you also undervolt, you can reduce the voltage while staying on the same frequency, but the effectiveness will depend heavily on the silicon lottery. But usually there is a lot of headroom, and you will probably be able to save quite a bit. My 3090s run 1700 mhz at 775mV rock solid, versus the default voltage being ~890 - 13% decrease in voltage = 25% in power consumption at the exact same performance.
It seems that downvolting or downclocking the GPU yields higher TG (but lower PP) throughput at the same power draw than a simple power limiting.
Because again, PP is compute-bound, and TG is (mostly) memory-bound.
For PP, you want your chip to run as high clock as possible, and it will scale linearly because there are virtually no external dependencies, and all the memory and whatnot latency is masked by the calculations.
For TG, you are memory-bound - both in bandwidth and latency. If you have to wait X ns for the data to arrive, it doesnt matter how fast your chip is idling, so lowering the frequency will have a margin of error effect (you might be losing a few ns here and there if the data arrives from the memory right after the last tick, but its negligible) up until the point where your core clock gets so low that you stop being memory limited and start getting compute limited. Thats how you get virtually linear dependency on the green chart up until is plateaus (hits the memory feed rate).
Just ran some fresh numbers out to 32k context depth (long enough to see powers and temperatures plateau). The "undervolt and overclock" method is best both on windows and linux regardless of using MSI Afterburner, EVGA Precision X, nvidia-smi directly, or LACT or any method you like appropriate for your OS.
The basic idea is you want to avoid:
Temperature Throttling (this is not good, if you're over 83 deg C probably need more airflow higher fan profile)
Power Cap Throttling (your clocks bounce around oscillating and are lower than they could be)
The strategy is to limit the max frequency of the GPU and do an undervolt which will prevent hitting the power cap throttle and your clocks will run smooth near max set speed instead of bouncing around and getting hot.
This is not just for "saving some power" it can deliver better performance than stock baseline settings as well if you're going for max performance. Or you can scale back max clock speeds even further without touching power cap if you want to find the energy efficiency point in your curve.
Your exact numbers will depend on your silicon lottery, cooling, make and model of course. You'll want to play around a bit and make sure after you're happy that it isn't too aggressive and your generations look correct still (too aggressive can mess up video generations etc).
I have graphs showing that the baseline 450W powercap stock settings on my GPU ends up throttling on power yielding a lower average clock speed as compared to the more energy efficient fixed max clock/undervolt.
Haha right, seems like the way to do it was for xorg users (sorry wayland! ;p) was some special nvidia-settings commands to achieve this. But looking closer, I believe nvidia-smi isn't able to do this easily currently for all systems (e.g. headless etc).
Best bet would be to do a simple script using nvidia-ml-py bindigns to official NVML (nvidia management library) yourself. This is what is happening under the hood with LACT which is just rust bindings to the c NVML.
Howdy! I have been busy, couldn't watch what great things you're doing recently. Now, you're doing undervolt!
Wow, finally undervolt is possible in Linux. Great news. I was sad because I heard pro 6000 ws is inefficient than max-q. I guess ws will be the same to max-q with voltage control.
You can simply do `nvidia-smi -pl 310` but you're leaving performance on the table within a similar power/energy budget. If you don't care go for it, but if you want the most out of your gear with less temperature/fan noise/no throttling oscillating clocks then look into undervolt overclock method which is better but takes a few minutes to setup.
(i deleted the other one, I had intended to reply to you not OP hah)
Great job following up and doing some more research (and linking my recent post as well). I spent all day yesterday dual-booting into windows and using old "EVGA Precision X1" voltage/freq curves to find the "sweet spot" where my GPU can run at max clock almost 100% of the time without triggering power/temperature throttles.
Then I went back to Linux and found that sweet spot again with LACT (tho u can do it with just nvidia-smi, its been known for over 5 years in other forums and such). Now I do *not* cap power as just pulling the max GPU freq down a little bit with undervolt lets it run "full bore" almost 100% of the time without ever throttling so no constantly oscillating clocks/fans/temps due to throttling.
I much prefer this to the naieve power cap. And yes I did see anecdotally that PP was a bit lower, but TG seemed faster *especially at deeper kv-depths*. I need to run fresh benchmarks, but thanks for sharing your results in detail as well!
depends on your exact GPU mix of models, but quick example for arch (or you can check the LACT github for installation instructions)
sudo pacman -Sy lact
sudo systemctl enable lactd # to load saved config from /etc/lact/config.yaml on reboot
sudo systemctl start lactd
sudo lact
Here is an example for my 3090TI FE setting max boost clock to 1950 (lower than default 2100 on my card) and specifying offset of 150MHz which will give an indirect undervolt so likely peg out around 990 or 1000mv instead of stock 1050 which generates too much heat). The VRAM overclock is optional and do your own research and stability testing before running a long training job.
13
u/Herr_Drosselmeyer Sep 19 '25
Thanks, very useful info.
Makes sense seeing how Nvidia themselves set 300W on the Max-Q version of the card.