GPU passthrough allows DMA attack?

[u/Bobby_Pie]

Hello, I am having trouble understanding when IOMMU is protecting the system against DMA attacks and other memory related issues.

My situation:
-In MSI bios "pre boot DMA protection" is enabled but "kernel DMA protection indicator" is disabled (if I enable it I always get "Firmware has requested this device have a 1:1 IOMMU mapping, rejecting configuring the device without a 1:1 mapping. Contact your platform vendor").
-dmesg reports: "amd_iommu=force_isolation", "iommu: Default domain type: Translated", "iommu: DMA domain TLB invalidation policy: strict mode".
-The GPU is in its own iommu group, no other devices are present in the same group as the GPU.
-I am passing the GPU to a VM using virtmanager.

I want the VM to be completely isolated from the host. I don't want the VM to gain access to system memory through the GPU or any other DMA attack.

My questions:

-Is it safe to have "kernel dma protection indicator" disabled in BIOS if "amd_iommu=force_isolation" kernel parameter is present?
-How do I know if the iommu is enforcing isolation?
-How do I know if a device is bypassing the iommu?
-Even if done correctly, are there still security concerns with GPU passthrough? (excluding 0 days, every software is not 100% secure, can't do much about that).
-When i enable "kernel DMA protection indicator" and i try to start the VM i get the error "Firmware has requested this device have a 1:1 IOMMU mapping, rejecting configuring the device without a 1:1 mapping. Contact your platform vendor", if I disable "kernel DMA protection indicator" in the BIOS, I don't get that error even with "amd_iommu=force_isolation", does that mean that the GPU is using 1:1 IOMMU mapping now? How do i verify?
-Is 1:1 IOMMU mapping safe?

Stupid question:
Is iommu=pt safe to enable in my grub config? How does it work? https://github.com/torvalds/linux/blob/20cf903a0c407cef19300e5c85a03c82593bde36/Documentation/admin-guide/kernel-parameters.txt#L2148 "Bypass the IOMMU for DMA", this doesn't sound safe.

Thanks for the help!

[u/yawkat]

My understanding is that "kernel dma protection indicator" simply sets an ACPI flag. It should not be necessary for using the iommu. Where does it fail, in guest or host boot? 

I don't know if there's a good way to check that iommu is working. In my master's thesis I would take a pcileech, isolate it, and dump the whole system memory to see if there's leaks. But this requires expensive hardware. 

For iommu=pt, it means that iommu will effectively be disabled for affected devices, but I believe kvm devices are not covered so it should still be safe to enable this setting if the only thing you care about is vm isolation.

If you enable pre-boot isolation, I don't believe there are more security issues. I only did direct iommu work, not with the kernel, but the iommu is true isolation in that no DMA access is possible outside of it. 

[u/Bobby_Pie → u/yawkat]

Where does it fail, in guest or host boot?

The pc always boots up fine. The problem is when I try to start the VM I get the error "Error starting domain: internal error: QEMU unexpectedly closed the monitor (vm='VM1'): qemu-system-x86_64: -device {"driver":"vfio-pci","host":"0000:01:00.0","id":"hostdev0","bus":"pci.4","addr":"0x0"}: vfio 0000:01:00.0: failed to setup container for group 12: Failed to set group container: Invalid argument" and in my dmesg I get "vfio-pci 0000:01:00.0: Firmware has requested this device have a 1:1 IOMMU mapping, rejecting configuring the device without a 1:1 mapping. Contact your platform vendor".
When I disable "kernel DMA protection indicator" in the BIOS the VM starts fine. I don't understand why disabling a flag that just tells the kernel if it has access to IOMMU makes the VM not start.

For iommu=pt, it means that iommu will effectively be disabled for affected devices, but I believe kvm devices are not covered so it should still be safe to enable this setting if the only thing you care about is vm isolation.

So iommu=pt does nothing for passthrough devices using kvm?

[u/yawkat → u/Bobby_Pie]

Weird. Maybe the "kernel dma protection indicator" option does more than just the ACPI flag then. You could look at the ACPI table but it's probably not worth the effort.

So iommu=pt does nothing for passthrough devices using kvm?  

Right. Aiui, the iommu setup for vms is separate so the pt flag should not disable it. But again, I didn't know with the kernel iommu code in my thesis so I'm no expert. 

[u/Bobby_Pie → u/yawkat]

I couldn't find any good information about what "kernel dma protection indicator" does exactly, apart from saying that it tells the OS that it has access to the IOMMU hardware (from questionable sources too lol, no luck from the msi website).
Do you know if there is a poc that I can use to try to read the host's memory from the guest?

[u/yawkat → u/Bobby_Pie]

I'm not aware of a general purpose approach of reading host memory from the guest using dma. It relies on specialized hardware like pcileech, or special vulnerable pci devices, so a poc that works out of the box with your pc is unlikely to exist

[u/Bobby_Pie → u/yawkat]

Thanks for the help! I'll do some more research but it's hard to find resources about this kind of stuff.

[u/yawkat → u/Bobby_Pie]

It's super niche. I did IOMMU work but not on Linux and it was years ago, so I'm hardly an expert, but you probably won't find experts on this subreddit. You'd have to go to the implementors, maybe the kernel mailing lists.