Hi, Problem: Pass through of a AMD Vega 56 or 64 on a AMD Threadripper 1950X with X399 Motherboard , Host OS: Linux Kernel 5.1.5, did not work.
Various problems were observed: 1) PCI BAR resizing - The GPU tries to resize the BAR from 256M to 8G. The resize fails, but the Guest does not know this, and continues on as if it was resized, resulting in a VM_L2_PROTECTION_FAULT_STATUS in the guest. Symptoms of (1): Able to passthru most PCI devices except the GPU. Solution: This problem has been worked-around with (See other email by Alex W): [PATCH] vfio/pci: Hide Resizable BAR capability The reason the GPU was problematic, is because it was the only device that tried to resize the BAR. Fortunately, the AMD GPU driver can work with a 256M or 8G BAR. The real fix will be the implementation of support for PCI BAR Resizing in QEMU and VFIO. 2) The QEMU fails to start with GPU passthrough, in that it does not reach the Seabios messages and sits at 100% CPU. I verified that the AMD GPU card's option ROM contained both Legacy BIOS and EFI images in it. Removing the GPU PCI device in virt-manager allows the VM to boot, but adding the GPU PCI device causes it to fail to boot. Symptoms: VM won't start, left with blank screen. Solution: Using Q35 and OVMF EFI works. Using 440 and Seabios does not work. My guess is there is a bug in Seabios preventing it from working. 3) Use case: Have 2 AMD Vega CPUs and wish only one to be passthru to guest and the other to be used by the Host. Problem: Currently you can passthru both, or neither. Solution: I have written a patch for VFIO to handle this, but it is still work in progress before it will be accepted into kernel mainstream. Kind Regards James
