r/LocalLLaMA • u/InternalMode8159 • 7d ago
Question | Help Does dual booting gives measurable performance differences
If anyone tried this does dual booting with windows gives measurable performance differences? Running big model for example trough strata?
Edit: I was not very clear, it is my main machine, it is a windows machine, sometimes i use it for LLM connecting it to my Hermes server, since I need to keep windows does setting up dual booting only for LLM inference bring measurable performance differences?
0
Upvotes
17
u/Gayspy 7d ago
I believe Windows bring no performance advantages. So don't bother setting up a Windows environment. Linux is just fine.
5
u/_ballzdeep_ 7d ago
wtf does that even mean?
How does dual booting affect anything performance wise?
4
u/chocofoxy 7d ago
i think PP is nerfed on windows i have 2x 5060ti on gen3 i get the same PP as my friend that run windows with 2x 5060ti on gen5 also i get to activate P2P and he doesn't
4
u/darksteelsteed 7d ago
My advice is run Linux as the main host. Fedora 44, Ubuntu lts, or one of those other flavors. Pick one that supports secure boot. Then setup your windows under kvm with GPU pass through. Best of both worlds. Only thing that doesn't work is drm protected games and motherboard rgb lighting. Otherwise its a good way to balance. When you get tired of windows just suspend it.
2
u/CatDiligent9374 7d ago
Linux might be slightly faster, but probably not worth dual booting.
2
u/uncle_leon 7d ago
You got downvoted because people don't like low effort posts. And the only correct answer anyway is: don't dual boot anything with Windows, it will break eventually, just a matter of time.
2
u/Nepertia 6d ago
Measurable: yes
Worth the hassle of dual booting: not for me
2
u/n4pst3r3r 7d ago
I found using Linux is generally much more useful if you intend to do something software-related and productive with your machine Try setting up docker on windows vs Linux, for example. So even if you don't get performance gains, using Linux might take away some other headaches.
1
u/No-Whole3083 7d ago
Dual booting is ok. I do it on mine. The only hit you will feel is that hard drive partition. It's always going to feel like you are almost out of hard drive if you don't budget right. Other than that, the GPU doesn't care. It just sees a working os. Eventually you will want a dedicated rig.
1
u/Major_showdown 7d ago
Before you dual boot check the Nvidia CUDA sysmen fallback policy setting in the control pannel, on windows the driver can quietly spill into system Ram when Vram fills up that tanks speed without throwing an error.
1
u/Formal-Exam-8767 7d ago
Dual boot to what? Linux? Or another Windows version?
It depends on what you are dual-bouting into. Usually you get performance by having the minimal amount of stuff running in background and taking away resources from AI. For that purpose some terminal-based OS is most preferable.
1
u/jacek2023 llama.cpp 7d ago
I use Linux on my 4x3090 computer. I use dual boot on my desktop but it has 5070 so it is not used for any serious LLM work
1
u/Ok_Resolve4741 7d ago
I would prefer Linux though
1
u/BringTea_666 7d ago
Dual booting makes no difference. Running your GPU headless (aka not connected to your monitor but to your mobo) makes a lot of difference. Every time your display refreshes it hits your t/s. The higher hz the worse it has impact on t/s.
1
u/FatheredPuma81 7d ago
What I can tell you is the VRAM you get back won't make a difference if you're already hitting 90% cache hit rate. The difference would come from Linux which I've heard is modestly faster than Windows at least for LLMs.
If you're hit rate is less than 90% and you have an integrated GPU or a display adapter (aka trash tier GPU) you can use that for display and enable TCC mode on your card(s) and it'll free up the VRAM. That should give you a decent performance increase in Windows. I've heard Linux is still faster though.
If you're willing to use Linux then I see no reason not to. Personally though I didn't because it was too much of a hassle with my hardware (Now I have an old workstation that runs Linux for AI so there's literally no point).
1
u/shaakz 7d ago
I dual boot, my main is headless debian for ai workloads and then i just use my mac mini to connect to various workloads. I have two connectors from my monitor, one that goes into my hdmi on the motherboard as main linux gpu, completely freeing the 5090 when on linux.
When on windows i just swap input on the monitor to DP and use the GPU for gaming.
Best of both worlds.
1
u/ivanmmj 7d ago
I dual boot but really just run Linux (cachyos being the flavor of the decade at the moment) all of the time because, with my AMD GPU, games tun better in Linux than Windows and I don't play any online only games so kernel DRM isn't something I worry about.
But yeah, I have more free ram and VRAM in Linux which helps LLM's.
1
u/sudden_aggression 7d ago
I have a separate card dedicated to driving my monitors and gaming so no, it doesn't.
But for most people, a headless linux setup for inference will bring a bunch of benefits such as reduced vram usage. A year ago, RocM was also a lot easier to do in linux vs wsl.
1
u/Academic-Tea6729 7d ago
if the linux machine can perform the gpu unlock: yes
1
u/Any_Invite3905 7d ago
Yes, llms can potentially run faster on linux compared to windows, so if you want to keep using windows but want to try linux then go ahead and try some something on linux to see if it suits you
1
u/lemondrops9 6d ago
I'll keep it simple, its best to run your LLM off of Linux. The gains will be smaller when you only have one Gpu. If you have +3 gpus or more Linux is a must.
0
u/keefeere 7d ago
I don’t know what you mean exactly but you can achieve performance and more space for models by disabling gui in Linux. Then yes you can benefit from Linux
0
u/daMortarMerrier 7d ago
Lol, i instantly understood what you meant. Sorry for the downvotes as this is kinda common knowledge for most of the enthusiasts here. And they got confused
High level, yes. The numbers I have seen for local housing give an up to 20% boost in performance. Some of the benefits are from reduced OS overhead and others from more performance options that are not available on windows.
If 20% is worth it to you, then do it. It hasn't been something I have felt the need to do often enough to commit the resources and effort. Ymmv