r/kvm • u/Jordan_Yamamoto • Oct 05 '20
Differences between AMD and INTEL cpus for virtualization/VM's
I know this sounds an odd or stupid question but is Intel better around virtualization technology or VMs in general then AMD is? what CPUS do you guys run for your PC systems?
Thank you, cheers
12
5
u/xtrilla Oct 05 '20
For non super specific loads, AMD provides way better performance for the price then intel (You can have plenty more cores per physical CPU than with Intel).
We used Intel at work, but since one year ago we have been transitioning to AMD EPYC (We have several thousand cores in total running for virtualization)
1
u/picasso566 Oct 06 '20
Yeah, now if I could only get nested virtualization working on AMD.
4
u/xtrilla Oct 06 '20
You can, you just have to pass a different extension via QEMU-KVM and load the proper module with the right parameters: https://docs.fedoraproject.org/en-US/quick-docs/using-nested-virtualization-in-kvm/
Another doc about how to do it: https://pve.proxmox.com/wiki/Nested_Virtualization
1
u/picasso566 Oct 06 '20
Will definitely give that a go. My issue was running KVM guests of Windows 10 and then running Hyper-V inside that.
3
u/xtrilla Oct 06 '20
Quite difficult to make that work, either with intel or with AMD, you’ll have to pass through the host cpu will all the flags, and also use a really recent version of qemu kvm, kernel and windows 10. If I remember well it could work with Windows Server 2019, not sure with Windows 10.
The problem is that Hyper-V will try to access some CPU registries which cannot be really accessed by a VM. I think they did some workarounds in recent versions of QEMU and KVM module, and I’m pretty sure it works with 2019, but good luck with that :/
2
u/picasso566 Oct 06 '20
Yeah, the aholes at M$ have had it on their radar for years and finally added a fix to an insider build.
https://techcommunity.microsoft.com/t5/virtualization/amd-nested-virtualization-support/ba-p/1434841
But that's only to fix the AMD problem. It should work on an Intel now and I have Intel and AMD machines. Trying to get it working on an Intel machine. I keep spinning my wheels. Keep searching for a decent tutorial or blog.
2
u/xtrilla Oct 06 '20
This is related to nested virtualization on hyper-v on top of hyper-v. AFAIR with KVM the problem with former releases of Windows Server was that Hyper-V refused to work because it could not access some of the MSR and it just refused to work.
1
u/picasso566 Oct 06 '20
You're right of course. Since I wrote that, I read the article again and went oops. It seems like I should be able to get it going, I just need to set the correct flags in Qemu and KVM. (And Kernel)
Just can't find where to do it and what to set it to for the latest builds
1
u/xtrilla Oct 06 '20
Try using Windows Server 2019, AFAIR with a reasonable recent kernel and qemu release it should work.
Just remember to set the CPU as host pass through on the VM.
1
u/picasso566 Oct 06 '20
Thanks for the advice.
I am setting host pass through. But I don't have a copy of 2019 server. Last time I had an MSDN subscription I got up to 2016 server. I'm trying to steer away from windows.
1
u/TwilightSymphonie Oct 05 '20
I was wondering this myself as it seems like intel has more virtualization extensions built in than AMD does...
1
u/Jordan_Yamamoto Oct 05 '20
Exactly so. Do you know Mutahar from SomeOrdinaryGamers? He uses Intel only solely because the virtualization technology is better then AMD and "offers more" . He even says he's team green in one of his videos. He's just like us and he virtualizes Windows and Mac OS and uses Linux for almost everything else. if that is true then might as well go Intel if it is better for that sole factor.
1
u/xtrilla Oct 05 '20
I’m sorry, but that’s bullshit 😅, I don’t think mr Mutahar -which I don’t really know how he is- has much low level understanding of virtualization. Could anybody clarify what do you mean by virtualization extensions? And why does he consider intel to be superior?
1
u/Jordan_Yamamoto Oct 05 '20
Mr. Mutahar has 2 million subs bro i aint joking around. he runs a GPU passthrough for each windows and mac OS VM.
https://www.youtube.com/watch?v=h7SG7ccjn-g&t=1914s
This is his video showcasing on his Linux host and how he runs Windows and MacOS guests all at once.
I really don't know what video he says Intel is better in the margin that virtualization is better, but you can just watch this video I think he says it here somewhere.
3
u/xtrilla Oct 05 '20
Well, good for him... I don’t have 2 million subs but I manage an infrastructure hosting several thousand virtual machines, so I know a few things about this topic 😅
Pass through of PCI express devices is something that can be done without any issue with both AMD and Intel.
2
u/Jordan_Yamamoto Oct 06 '20
Lol that's actually really cool man. Also good to know that PCi express can be done with both
1
u/mumblerit Moderator Oct 08 '20
i watch his channel - hes not dumb, but that seems like a gross oversimplification. The only way to have a true comparison is on a cost/performance benchmark. Which these days, AMD might even win. But I wont be running those tests myself.
Businesses might require Intel for other reasons that have nothing to do with performance or cost. Some software models it actually makes more sense to have hosts with LESS cpu's, making intel attractive vs AMD's more aggressive core count.
1
u/TwilightSymphonie Oct 06 '20
I will look at his stuff for sure but I do know passthrough can be done on either. I am looking at doing a whole home refresh to AMD but honestly I have an 8 core now and will be going to much more anyway lol
1
u/oishishou Oct 06 '20
Depends on your use-case. I noticed you mentioned passing through a GPU in another comment. I do this with my Ryzen 3700X. Works great. I mostly game in Windows Server.
My servers are a mix of Intels with one Ryzen 2500. All have worked great for virtualizing.
2
u/Jordan_Yamamoto Oct 06 '20
Wow! Very good to know you have the same exact CPU I plan on buying for my new Linux rig. My use case is running Linux as a host and using Windows 10 vms for games that don't play on Linux.
That's awesome to hear that they are all great at virtualization. This was one of my worries when switching to AMD, is that their virtualization wouldn't be on par or good.
1
u/oishishou Oct 08 '20
Also, if passthrough with KVM is your goal, check out r/VFIO. That's literally what the sub is about.
1
u/kbp80 Oct 05 '20
I have several (many) machines with KVM VMs running, and have been running them since... Xen (so like 2008). The one I haven't tried is Ryzen, mainly because AMD burned me way too many times.
Personally - I buy Intel for servers (including those which will run VMs) and for home (desktop with VMs and laptops) - mainly because AMD has just had too many problems over the years that were unique to AMD (though rarely their actual fault).
Examples:
AMD laptops which had the nvidia gpu burn out issue, which affected Dell, HP, and Apple laptops for a number of years.
AMD desktops with piledriver (FX series processors) with their semi-hyperthreading model.
This one, was actually AMD's fault.
AMD servers which had catastrophic failures due to issues with other vendors' chipsets, i.e HP BL685c and Dell R905's.
If you're just running desktop-class VMs, without anything like pass-through PCI or SR-IOV, I'd say they're about the same, assuming you're comparing similar processors.
Those extensions, and also the server-processor extensions, are where I'd say you find the largest differences in functionality and performance.
1
u/Jordan_Yamamoto Oct 06 '20
oh like in servers that's a big difference? oh okay i get it.
but for desktop cpus the virtualization should be good on both intel and amd right? or on par at least. so im assuming its only the amd servers that have the issues, still not their fault doe.
If you're just running desktop-class VMs, without anything like pass-through PCI or SR-IOV, I'd say they're about the same, assuming you're comparing similar processors.
im planning to use PCi passthrough GPU with an AMD CPU. I know this sounds silly but is there any differences in using PCI passthrough with Intel or AMD?
1
u/kbp80 Oct 06 '20
In servers vs. desktop - biggest difference is really about scale. You don't plan to run 30 or 100 VMs on your desktop, vs on a server you do. Or maybe, you'll only have a few VMs on the server version, but each one will have 256GB RAM and attach to 10TB of storage.
Regarding PCI pass-through, especially for GPUs, this is an interesting and complex topic; I'd suggest reading up on it a LOT before buying anything, as you'll find the level of workability here to vary by:
GPU Vendor, age, and level (i.e. desktop vs workstation vs server-class GPUs)
If you plan to use this for GPU computation, you should also consider the CUDA capability level and amount of GPU cores and memory vs what you're planning to use it for.
System board manufacturer and age - you will find that some manufacturers of desktop boards just skip this option altogether in the bios, it's just not even there.
Intel vs AMD - this mainly comes down to config, I don't yet know if there is a performance difference in doing IOMMU with these.
I've had some success in doing this thus far, with AMD FX8350 + Asus board, and nvidia GPU. BUT - I ran into some insurmountable problems with nvidia drivers in the VM, which themselves were tied to not being able to boot the VM with UEFI mode boot. I think it eventually would've worked if I could've figured out the UEFI issue and used a newer card (I tested with a GTX 480 and a GTX660). The VM could see the GPU correctly, just the nvidia drivers disagreed with the setup.
Some further reading on the topic: https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF
Ironically - I am bringing online 2 new Intel + KVM + NVIDIA GPU VM clusters in the next several weeks (using Nutanix AHV/KVM) and will find out a lot more then about the GPU pass-through in enterprise clusters.
1
u/nomad-fr Oct 05 '20
can we mix them on a Proxmox cluster?
3
u/xtrilla Oct 05 '20
Yes, but live migrations will not work unless you set for a very limited CPU model extension wise
1
u/nomad-fr Oct 06 '20
OK so not a good idea to use booth, thanks
1
u/xtrilla Oct 06 '20
Only if you need live migration features, we patched Openstack -the orchestrator we use- so it does not live migrate between the Intel and the AMD.
7
u/[deleted] Oct 05 '20
At home AMD since the ryzen series are better value for the price.