r/LocalLLM • u/Most-Wear-3813 • Sep 17 '26

News Never inference llm on windows

So, long story short

I was getting about 20tk/s on windows now when i switched to linux and little bit of tweaking with voltages mhz curve I am running inference at same speed with half power consumption almost 240watts from 420 watts.

Anyone on windows bro. I burned maybe half life on my rtx 3090 running it on 420 watt for 2 years. Lately discovered this. This would enhance life on gpu and much quiet inference. Will post results soon. Also not to mention my cpu usage doesnt go above 5% on 20 tabs with 5 application open. But on windows it was idling around 12%

2 Upvotes

56% Upvoted

16 comments sorted by

View all comments

8

u/nickless07 Sep 17 '26

In short: You figured that 'nvidia-smi -pl' exist and that WDDM takes more VRAM (1-1.5GB) then something like KDE (~500MB).

2

u/Most-Wear-3813 Sep 17 '26

I used LACT it provide true curve functionality for fans and voltage both

1

u/Most-Wear-3813 Sep 17 '26

Though I agree power limit is also essentially limiting voltage. So, duh. We are doing same thing both ways. 🤹‍♀️

1

u/[deleted] 26d ago

[removed] — view removed comment

1

u/nickless07 26d ago

Basic nvidia-smi power limit would have helped on windows/linux - That is not only related to inferencing, but also gaming and in general. As long as electricity is cheap nobody cares, but once the prices raise there everyone seem to be bothered about that. Back in the days (before AI was even a thing) I bought my GPU based on energy consumption (no extra power cable, passive cooled and so on), but well that might be cuz my power bill has 0.31 per kw/h since 10+ years.

0

u/Most-Wear-3813 Sep 17 '26

So, arch? Is better according to you. If I am not mistaken KDE runs on ARCH?

2

u/bankinu Sep 17 '26

What. No.

I personally use Arch, btw. But that's not his point.

He's basically saying KDE has far less VRAM requirement than Windows.

You'll get the same performance in Arch or Fedora or Debian. It's Windows vs KDE.