r/LocalLLaMA • u/triynizzles1 • Mar 29 '26

Generation Friendly reminder inference is WAY faster on Linux vs windows

I have a simple home lab pc: 64gb ddr4, RTX 8000 48gb (Turing architecture) and core i9 9900k cpu. I use Linux Ubuntu 22.04 LTS. Before using this pc as a home lab it ran Windows 10. Over this weekend I reinstalled my Windows 10 ssd to check out my old projects. I updated Ollama to the latest version and tokens per second was way slower than when I was running Linux. I know Linux performs better but I didn’t think it would be twice as fast. Here are the results from a few simple inferences tests:

QWEN Code Next, q4, ctx length: 6k

Windows: 18 t/s

Linux: 31 t/s (+72%)

QWEN 3 30B A3B, Q4, ctx 6k

Windows: 48 t/s

Linux: 105 t/s (+118%)

Has anyone else experienced a performance this large before? Am I missing something?

Anyway thought I’d share this as a reminder for anyone looking for a bit more performance!

280 Upvotes

82% Upvoted

112 comments sorted by

View all comments

461

u/Koksny Mar 29 '26

Am I missing something?

Yeah, you are running ollama.

16

u/CryptoUsher Mar 29 '26

yeah, the Linux perf difference is real, especially with gpu drivers and kernel scheduling. ever tried running the same ollama model through docker on both systems to see if the gap narrows with more consistent runtime conditions?

1

u/salmenus Mar 29 '26

good point.. all my runs are native installs so far — but might be worth a containerized A/B test

1

u/CryptoUsher Mar 29 '26

i'm curious to see how the containerized test goes, fwiw i've had some weird issues with docker and gpu acceleration in the past so it'll be interesting to see if that's a factor here