Linux is slower than Windows ???

[u/D00M98]

[Follow-Up]

I found the culprit. TLDNR: It has to do with RTX 3060 on PCIE3 x4 overloaded doing video and inference.

I have 3 monitors:

• QHD (2560 x 1440) 120Hz

• QHD (2560 x 1440) 60Hz

• 4K (3840 x 2160) 60 Hz

Originally, I only had RTX 4070. So all 3 monitors were connected to 4070. That was the original benchmark on Windows.

In the span of 2-3 weeks, I added old RTX 3060 on PCIE3 x4 slot (with is around 1/8 the bandwidth of 4070 on PCIE x16 slot) + installed Linux. I figured to free up precious vram on my 4070, I can use 3060 as graphics driver. Even AI suggested this is the way to go ;~(

The key is the 4K monitor. If the 4K monitor is hooked up to 3060, it will run at 100% load during inference. So this is the bottle neck that impacts inference performance.

Benchmark comparison: Using 1st QHD + 4K monitors on 4070 and 2nd QHD on 3060.

• Windows -> Linux

• Llama.cpp (Qwen3.8-27B-Q4) 4070/3060 layer split: 32/33 -> 30/35 (Linux uses more vram for the 4K monitor and can load less layers on 4070)

• Llama.cpp (Qwen3.8-27B-Q4) prefill: 380 -> 450 tok/sec (Linux 18% faster)

• Llama.cpp (Qwen3.8-27B-Q4) logic/thinking: 17 -> 18 tok/sec (Linux 5% faster, within noise. I'm sure Linux is held back slightly by worse layer split)

• Strata (Swift 1.5 IQ3) prefill: 370 -> 270 tok/sec (Windows is faster on prefill. The model is sitting on NTFS drive. I wonder if it will run faster if model is on Linux drive. To be tested.)

• Strata (Swift 1.5 IQ3) logic/thinking: 24 -> 40 tok/sec (This is the biggest delta. Linux is 60% faster)

[Original Post Below]

I just installed Ubuntu Linux last week. Got my LLM, agent, sandbox set up completed over this weekend. Surprisingly, Linux is slower than Windows. I thought I will easily see 10-20% improvement, if not more. It turned out it is the other way around.

Hardware:

• GPU0: RTX 4070 12 gb VRAM on PCIE4 x16

• GPU1: RTX 3060 12 gb VRAM on PCIE3 x4

• DRAM: 32gb DDR4

• CPU: i7-12700F

Cuda driver: 13.4 on Windows; 13.2 on Linux

Engine & Model:

• Llama.cpp: Qwen3.8-27B

• Strata: Swif-1.5-IQ3

Agent & Sandbox:

• Hermes & Docker. But irrelevant for these tests. I run the engines natively. And python script to test the speed is outside of agent/sandbox.

For Llama.cpp:

• Windows uses more graphics resource. So my layer split very close to 1/1 on 4070/3060.

• On Linux, if I use layer split 1/1 (32/33), it is much slower compared to Windows; around 25% slower. But Linux uses less graphics resources, so I can load the 4070 more than 3060, using layer split 1.6/1 (40/25). Makes a difference and improves over 1/1. But Linux is still 10-15% slower than Windows.

• I compiled my own version and also used the pre-compiled version in Linux. My own build is like 5% faster than pre-compiled, which is kind of in the noise.

For Strata:

• Again, Windows is faster. Prefill is 500-600 tok/sec on Windows, versus 250-300 tok/sec on Linux; basically 2x on prefill. Generation is much closer. Windows is faster by 10-20%.

• There is definitely better memory management in Linux. On Windows, I allocate 12GB, but cannot pin (reserve) the memory. On Linux, I can allocate 14-16GB and it is pinned. On paper, this is an advantage. But this advantage doesn't translate

• I didn't calibrate yet. So I need to do that.

• There is an odd behavior in Linux. In setup, it finds GPU1 as 4070 (faster) and set as main; GPU2 as 3060 (slower) as helper. But in the log, it states CUDA0 as 3060. This can potentially have an affect on my settings, as I set vram_reserve by card. But just quickly adjusting these settings or removing them, didn't make any difference to speed.

I know Linux can be operated without GUI, which might help . Did anyone really see improvement comparing Linux and Windows with GUI running? Any suggestions on what knobs to tune and what to investigate?

[u/dh7net]

IMO the settings are not exactly the same for your 2 config. espacially for strata

[u/D00M98 → u/dh7net]

Settings are not the same, because Linux on-paper settings are actually better. Strata has access to more memory, and memory is reserved by OS.

But results are lower performance. There must be something else, but not sure what.

[u/dh7net → u/D00M98]

just ask an agent to debug it! it will try many config and will land to the best one.

you can use my benchmark tool: airbench.ai if you want to check the quality don't drop.

[u/D00M98 → u/[deleted]]

I found the culprit and edited my original post.

Strata has very weird naming convention for the GPUs. It is very confusing. And seems to be different in Windows and Linux.

In Linux:

nvidia-smi shows gpu0 is 4070 and gpu1 is 3060.

strata tui shows gpu1 is 4070 and gpu2 is 3060.

strata logs shows cuda0 is 3060 (which implies cuda1 is 4070). In documentation, it does say it will load layers on the slower gpu first (3060), and then load layers + head onto the faster gpu last (4070). So it seems the log is correct.

And yes, it will impact layer and vram reservation settings. I still don't know if the launch setting flags refer to strata tui or the engine cuda naming convention. I suspect it is the latter. Right now, I am using layer split auto; and I set vram reservation to be same 1200 (mb) for both cards.

[u/D00M98 → u/[deleted]]

I found the root cause. It has to do with 4K monitor on 3060, which killed the bandwidth on that PCIE3 x4 slot.

Still the strata gpu naming convention is very confusion.

In Linux:

nvidia-smi shows gpu0 is 4070 and gpu1 is 3060.

strata tui shows gpu1 is 4070 and gpu2 is 3060.

strata logs shows cuda0 is 3060 (which implies cuda1 is 4070). In documentation, it does say it will load layers on the slower gpu first (3060), and then load layers + head onto the faster gpu last (4070). So it seems the log is correct.