** Description changed: + [ Impact ] + + * This is about nested virtualization on PowerPC + + * An L2 fails to boot inside an L1 when booted with a memory more than + 128G and a VFIO device passthrough. + + * The issue is seen with both, when guest booted with virsh and as well + as with a direct qemu cmdline. + + * The issue arises from a conditional check for `levels > 1` in + `spapr_tce_create_table()` within L1 KVM. This check is meant to prevent + multi-level TCEs, which are not supported by the PowerVM hypervisor. As + a result, when QEMU makes a `VFIO_IOMMU_SPAPR_TCE_CREATE` ioctl call + with `levels > 1`, it triggers the conditional check and returns + `EINVAL`, causing the guest to crash. + + * This is addressed by cherry-picking 2 upstream commits, checking the + supported DDW "levels" returned by the VFIO_IOMMU_SPAPR_TCE_GET_INFO + ioctl before attempting the TCE create ioctl in KVM. + + [ Test Plan ] + + * We need a PowerPC machine with > 128 GB memory. Therefore, IBM will + help with testing this. + + * Boot a nested VM (through virsh or qemu directly) using a VFIO device + passthrough, on such a big machine. + + * Confirm the L2 guest inside an L1 hypervisor can boot successfully + + * check dmesg and confirm the following error is not visible: + ``` + qemu-system-ppc64: Failed to create a window, ret = -1 (Invalid argument) + qemu-system-ppc64: Failed to create SPAPR window: Invalid argument + qemu: hardware error: vfio: DMA mapping failed, unable to continue + ``` + + [ Where problems could occur ] + + * This changes things in vfio/spapr, so is mostly isolated to PowerPC + functionality. + + * Should something go wrong, we could end up with broken booting of + virtualization guests on PowerPC. + + [ Other Info ] + + * Upstream cherry-picks (included in Questing, but not yet part of an upstream release): + - https://gitlab.com/qemu-project/qemu/-/commit/d5e8e6195bcd ("vfio/spapr: Fix L2 crash with PCI device passthrough and memory > 128G") + - https://gitlab.com/qemu-project/qemu/-/commit/6a7abe1c96bf ("vfio/spapr: Enhance error handling in vfio_spapr_create_window()") + + + ---- original bug report ---- + + == Comment: #0 - SEETEENA THOUFEEK <[email protected]> - 2025-04-15 09:03:28 == +++ This bug was initially created as a clone of Bug #212409 +++ ---Problem Description--- ---Problem Description--- An L2 fails to boot inside an L1 when booted with a memory more than 128G and a VFIO device passthrough. The issue is seen with both, when guest booted with virsh and as well as with a direct qemu cmdline. Error: ------ [ 4.708646] tg3 0001:00:01.0: enabling device (0140 -> 0142) [ 4.734333] tg3 0001:00:01.0: ibm,query-pe-dma-windows(2026) 800 8000000 20000001 returned 0, lb=2000000000 ps=100 wn=1 qemu-system-ppc64: Failed to create a window, ret = -1 (Invalid argument) qemu-system-ppc64: Failed to create SPAPR window: Invalid argument qemu: hardware error: vfio: DMA mapping failed, unable to continue - - Contact Information = [email protected] - + + Contact Information = [email protected] + ---uname output--- na - - Machine Type = na - + + Machine Type = na + ---Debugger--- A debugger is not configured - The patches are now in vfio-next branch in Cedric's tree: - d7ee6466d4ab ("vfio/spapr: Fix L2 crash with PCI device passthrough and memory > 128G") - ce07cf682e54 ("vfio/spapr: Enhance error handling in vfio_spapr_create_window()") + d7ee6466d4ab ("vfio/spapr: Fix L2 crash with PCI device passthrough and memory > 128G") + ce07cf682e54 ("vfio/spapr: Enhance error handling in vfio_spapr_create_window()") Links to commits: - https://github.com/legoater/qemu/commit/ce07cf682e54eba1360e1546c248f1485016bb79 - https://github.com/legoater/qemu/commit/d7ee6466d4ab2515f57f41e1ee6015704d7938d3 + https://github.com/legoater/qemu/commit/ce07cf682e54eba1360e1546c248f1485016bb79 + https://github.com/legoater/qemu/commit/d7ee6466d4ab2515f57f41e1ee6015704d7938d3 ~Amit - - Machine Type = na - + + Machine Type = na + ---Debugger--- A debugger is not configured - - Contact Information = [email protected] - + + Contact Information = [email protected] + ---uname output--- na
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2107396 Title: [KOP]:: L2 fails to boot with vfio passed through device and memory > 128G To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/2107396/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
