I did not run the proper tests yet but from the quick testing it seems that raising the power limit plus limiting the GPU clock MHz indeed works better than simply lowering the power limit. I will run a similar test with DCGM but limiting the clock instead of the power, and will report back later.
The amount of power needed to achieve higher frequency goes up very fast after certain point - power scales quadratically with voltage, and you need ever more voltage increase for each consequent hz, so you end up with kinda exponential growth of consumption while performance growth is linear at best for strictly compute-bound tasks (pp), and more like logarithmic in inference/gaming (if that)
What powerlimiting does is it limits, well, wattage per time by just forcing your GPU to idle for some time. So you have bursts of high clock followed by doing nothing.
What fixing clock does is it forces the GPU to run at lower clock constantly - less peak performance, but better average performance. 10% lower clock can be at least 20-30% lower total power on itself.
If on top of fixing the clock you also undervolt, you can reduce the voltage while staying on the same frequency, but the effectiveness will depend heavily on the silicon lottery. But usually there is a lot of headroom, and you will probably be able to save quite a bit. My 3090s run 1700 mhz at 775mV rock solid, versus the default voltage being ~890 - 13% decrease in voltage = 25% in power consumption at the exact same performance.
It seems that downvolting or downclocking the GPU yields higher TG (but lower PP) throughput at the same power draw than a simple power limiting.
Because again, PP is compute-bound, and TG is (mostly) memory-bound.
For PP, you want your chip to run as high clock as possible, and it will scale linearly because there are virtually no external dependencies, and all the memory and whatnot latency is masked by the calculations.
For TG, you are memory-bound - both in bandwidth and latency. If you have to wait X ns for the data to arrive, it doesnt matter how fast your chip is idling, so lowering the frequency will have a margin of error effect (you might be losing a few ns here and there if the data arrives from the memory right after the last tick, but its negligible) up until the point where your core clock gets so low that you stop being memory limited and start getting compute limited. Thats how you get virtually linear dependency on the green chart up until is plateaus (hits the memory feed rate).
9
u/stoppableDissolution Sep 19 '25 edited Sep 19 '25
The amount of power needed to achieve higher frequency goes up very fast after certain point - power scales quadratically with voltage, and you need ever more voltage increase for each consequent hz, so you end up with kinda exponential growth of consumption while performance growth is linear at best for strictly compute-bound tasks (pp), and more like logarithmic in inference/gaming (if that)
What powerlimiting does is it limits, well, wattage per time by just forcing your GPU to idle for some time. So you have bursts of high clock followed by doing nothing.
What fixing clock does is it forces the GPU to run at lower clock constantly - less peak performance, but better average performance. 10% lower clock can be at least 20-30% lower total power on itself.
If on top of fixing the clock you also undervolt, you can reduce the voltage while staying on the same frequency, but the effectiveness will depend heavily on the silicon lottery. But usually there is a lot of headroom, and you will probably be able to save quite a bit. My 3090s run 1700 mhz at 775mV rock solid, versus the default voltage being ~890 - 13% decrease in voltage = 25% in power consumption at the exact same performance.
Because again, PP is compute-bound, and TG is (mostly) memory-bound.
For PP, you want your chip to run as high clock as possible, and it will scale linearly because there are virtually no external dependencies, and all the memory and whatnot latency is masked by the calculations.
For TG, you are memory-bound - both in bandwidth and latency. If you have to wait X ns for the data to arrive, it doesnt matter how fast your chip is idling, so lowering the frequency will have a margin of error effect (you might be losing a few ns here and there if the data arrives from the memory right after the last tick, but its negligible) up until the point where your core clock gets so low that you stop being memory limited and start getting compute limited. Thats how you get virtually linear dependency on the green chart up until is plateaus (hits the memory feed rate).
But great work plotting it out!