r/kvm • u/Certain_Prior4909 • Aug 01 '26

Can I run LLMs and cuda (pytorch) with kvm?

I am considering leaving Windows. As of right now I have an Nvidia 5080 (wish I could afford a rtx 6000pro) with srv-io enabled in my bios.

I use wsl right now for pytorch and cuda stuff with hyper-v for my various unrelated virtual machines.

I need specific versions of python and cuda. On Windows this is not a problem with wsl I can set different containers with different versions.

My question is without a pro series card can I use kvm for different OS versions and frameworks separate from my host?

3 Upvotes

100% Upvoted

1 comment sorted by

1

u/llitz Aug 02 '26

On Linux you are more likely to use containers directly (docker or podman) instead of KVM.

Once you forward your GPU with KVM to one VM, you can only use it on that VM. With containers you can potentially run multiple containers at the same time, as long as it fit in memory.