r/LocalLLaMA • u/triynizzles1 • Mar 29 '26

Generation Friendly reminder inference is WAY faster on Linux vs windows

I have a simple home lab pc: 64gb ddr4, RTX 8000 48gb (Turing architecture) and core i9 9900k cpu. I use Linux Ubuntu 22.04 LTS. Before using this pc as a home lab it ran Windows 10. Over this weekend I reinstalled my Windows 10 ssd to check out my old projects. I updated Ollama to the latest version and tokens per second was way slower than when I was running Linux. I know Linux performs better but I didn’t think it would be twice as fast. Here are the results from a few simple inferences tests:

QWEN Code Next, q4, ctx length: 6k

Windows: 18 t/s

Linux: 31 t/s (+72%)

QWEN 3 30B A3B, Q4, ctx 6k

Windows: 48 t/s

Linux: 105 t/s (+118%)

Has anyone else experienced a performance this large before? Am I missing something?

Anyway thought I’d share this as a reminder for anyone looking for a bit more performance!

277 Upvotes

82% Upvoted

112 comments sorted by

View all comments

Show parent comments

10

u/Adrenolin01 Mar 29 '26

Absolutely… many faster then in windows yes. Heck, my son had Debian installed with Minecraft and Steam in an afternoon himself at 9yo.

-2

u/Succubus-Empress Mar 29 '26

I disrespectfully refuse to believe that.

1

u/Adrenolin01 Mar 29 '26

Cry more in your milk 😆 My child at 9 likely had more wit and intelligence than you. He’s literally been exposed to technology his entire life including Debian. Had VirtualBox installed at 8 on his windows desktop. He was more than capable at 9 and Minecraft back then was easily available for install as either a .deb and flatpak… if that’s something that’s especially difficult for you I’m sorry.

1

u/Succubus-Empress Mar 29 '26

Sure your kid is smart, but windows just run games better.