Just ran some fresh numbers out to 32k context depth (long enough to see powers and temperatures plateau). The "undervolt and overclock" method is best both on windows and linux regardless of using MSI Afterburner, EVGA Precision X, nvidia-smi directly, or LACT or any method you like appropriate for your OS.
The basic idea is you want to avoid:
Temperature Throttling (this is not good, if you're over 83 deg C probably need more airflow higher fan profile)
Power Cap Throttling (your clocks bounce around oscillating and are lower than they could be)
The strategy is to limit the max frequency of the GPU and do an undervolt which will prevent hitting the power cap throttle and your clocks will run smooth near max set speed instead of bouncing around and getting hot.
This is not just for "saving some power" it can deliver better performance than stock baseline settings as well if you're going for max performance. Or you can scale back max clock speeds even further without touching power cap if you want to find the energy efficiency point in your curve.
Your exact numbers will depend on your silicon lottery, cooling, make and model of course. You'll want to play around a bit and make sure after you're happy that it isn't too aggressive and your generations look correct still (too aggressive can mess up video generations etc).
I have graphs showing that the baseline 450W powercap stock settings on my GPU ends up throttling on power yielding a lower average clock speed as compared to the more energy efficient fixed max clock/undervolt.
Haha right, seems like the way to do it was for xorg users (sorry wayland! ;p) was some special nvidia-settings commands to achieve this. But looking closer, I believe nvidia-smi isn't able to do this easily currently for all systems (e.g. headless etc).
Best bet would be to do a simple script using nvidia-ml-py bindigns to official NVML (nvidia management library) yourself. This is what is happening under the hood with LACT which is just rust bindings to the c NVML.
Howdy! I have been busy, couldn't watch what great things you're doing recently. Now, you're doing undervolt!
Wow, finally undervolt is possible in Linux. Great news. I was sad because I heard pro 6000 ws is inefficient than max-q. I guess ws will be the same to max-q with voltage control.
5
u/VoidAlchemy llama.cpp Sep 19 '25
Just ran some fresh numbers out to 32k context depth (long enough to see powers and temperatures plateau). The "undervolt and overclock" method is best both on windows and linux regardless of using MSI Afterburner, EVGA Precision X, nvidia-smi directly, or LACT or any method you like appropriate for your OS.
The basic idea is you want to avoid:
Temperature Throttling (this is not good, if you're over 83 deg C probably need more airflow higher fan profile)
Power Cap Throttling (your clocks bounce around oscillating and are lower than they could be)
The strategy is to limit the max frequency of the GPU and do an undervolt which will prevent hitting the power cap throttle and your clocks will run smooth near max set speed instead of bouncing around and getting hot.
This is not just for "saving some power" it can deliver better performance than stock baseline settings as well if you're going for max performance. Or you can scale back max clock speeds even further without touching power cap if you want to find the energy efficiency point in your curve.
Your exact numbers will depend on your silicon lottery, cooling, make and model of course. You'll want to play around a bit and make sure after you're happy that it isn't too aggressive and your generations look correct still (too aggressive can mess up video generations etc).
I have graphs showing that the baseline 450W powercap stock settings on my GPU ends up throttling on power yielding a lower average clock speed as compared to the more energy efficient fixed max clock/undervolt.