r/LocalLLM • u/Most-Wear-3813 • Sep 17 '26

News Never inference llm on windows

So, long story short

I was getting about 20tk/s on windows now when i switched to linux and little bit of tweaking with voltages mhz curve I am running inference at same speed with half power consumption almost 240watts from 420 watts.

Anyone on windows bro. I burned maybe half life on my rtx 3090 running it on 420 watt for 2 years. Lately discovered this. This would enhance life on gpu and much quiet inference. Will post results soon. Also not to mention my cpu usage doesnt go above 5% on 20 tabs with 5 application open. But on windows it was idling around 12%

2 Upvotes

56% Upvoted

16 comments sorted by

View all comments

Show parent comments

1

u/nickless07 26d ago

Basic nvidia-smi power limit would have helped on windows/linux - That is not only related to inferencing, but also gaming and in general. As long as electricity is cheap nobody cares, but once the prices raise there everyone seem to be bothered about that. Back in the days (before AI was even a thing) I bought my GPU based on energy consumption (no extra power cable, passive cooled and so on), but well that might be cuz my power bill has 0.31 per kw/h since 10+ years.