** Description changed: [ Impact ] - * This is about nested virtualization on PowerPC + * This is about nested virtualization on PowerPC - * An L2 fails to boot inside an L1 when booted with a memory more than + * An L2 fails to boot inside an L1 when booted with a memory more than 128G and a VFIO device passthrough. - * The issue is seen with both, when guest booted with virsh and as well + * The issue is seen with both, when guest booted with virsh and as well as with a direct qemu cmdline. - * The issue arises from a conditional check for `levels > 1` in + * The issue arises from a conditional check for `levels > 1` in `spapr_tce_create_table()` within L1 KVM. This check is meant to prevent multi-level TCEs, which are not supported by the PowerVM hypervisor. As a result, when QEMU makes a `VFIO_IOMMU_SPAPR_TCE_CREATE` ioctl call with `levels > 1`, it triggers the conditional check and returns `EINVAL`, causing the guest to crash. - * This is addressed by cherry-picking 2 upstream commits, checking the + * This is addressed by cherry-picking 2 upstream commits, checking the supported DDW "levels" returned by the VFIO_IOMMU_SPAPR_TCE_GET_INFO ioctl before attempting the TCE create ioctl in KVM. [ Test Plan ] - * We need a PowerPC machine with > 128 GB memory. Therefore, IBM will - help with testing this. + * We need a PowerPC machine with > 128 GB memory. Therefore, + IBM will need to help with testing this. - * Boot a nested VM (through virsh or qemu directly) using a VFIO device + * Boot a nested VM (through virsh or qemu directly) using a VFIO device passthrough, on such a big machine. - * Confirm the L2 guest inside an L1 hypervisor can boot successfully + * Confirm the L2 guest inside an L1 hypervisor can boot successfully - * check dmesg and confirm the following error is not visible: + * check dmesg and confirm the following error is not visible: ``` qemu-system-ppc64: Failed to create a window, ret = -1 (Invalid argument) qemu-system-ppc64: Failed to create SPAPR window: Invalid argument qemu: hardware error: vfio: DMA mapping failed, unable to continue ``` [ Where problems could occur ] - * This changes things in vfio/spapr, so is mostly isolated to PowerPC + * This changes things in vfio/spapr, so is mostly isolated to ppc64 functionality. - * Should something go wrong, we could end up with broken booting of - virtualization guests on PowerPC. + * Should something go wrong, we could end up with broken booting of + virtualization guests on ppc64. [ Other Info ] - * Upstream cherry-picks (included in Questing, but not yet part of an upstream release): - - https://gitlab.com/qemu-project/qemu/-/commit/d5e8e6195bcd ("vfio/spapr: Fix L2 crash with PCI device passthrough and memory > 128G") - - https://gitlab.com/qemu-project/qemu/-/commit/6a7abe1c96bf ("vfio/spapr: Enhance error handling in vfio_spapr_create_window()") - + * Upstream cherry-picks (included in Questing, but not yet part of an upstream release): + - https://gitlab.com/qemu-project/qemu/-/commit/d5e8e6195bcd ("vfio/spapr: Fix L2 crash with PCI device passthrough and memory > 128G") + - https://gitlab.com/qemu-project/qemu/-/commit/6a7abe1c96bf ("vfio/spapr: Enhance error handling in vfio_spapr_create_window()") ---- original bug report ---- - == Comment: #0 - SEETEENA THOUFEEK <[email protected]> - 2025-04-15 09:03:28 == +++ This bug was initially created as a clone of Bug #212409 +++ ---Problem Description--- ---Problem Description--- An L2 fails to boot inside an L1 when booted with a memory more than 128G and a VFIO device passthrough. The issue is seen with both, when guest booted with virsh and as well as with a direct qemu cmdline. Error: ------ [ 4.708646] tg3 0001:00:01.0: enabling device (0140 -> 0142) [ 4.734333] tg3 0001:00:01.0: ibm,query-pe-dma-windows(2026) 800 8000000 20000001 returned 0, lb=2000000000 ps=100 wn=1 qemu-system-ppc64: Failed to create a window, ret = -1 (Invalid argument) qemu-system-ppc64: Failed to create SPAPR window: Invalid argument qemu: hardware error: vfio: DMA mapping failed, unable to continue Contact Information = [email protected] ---uname output--- na Machine Type = na ---Debugger--- A debugger is not configured The patches are now in vfio-next branch in Cedric's tree: d7ee6466d4ab ("vfio/spapr: Fix L2 crash with PCI device passthrough and memory > 128G") ce07cf682e54 ("vfio/spapr: Enhance error handling in vfio_spapr_create_window()") Links to commits: https://github.com/legoater/qemu/commit/ce07cf682e54eba1360e1546c248f1485016bb79 https://github.com/legoater/qemu/commit/d7ee6466d4ab2515f57f41e1ee6015704d7938d3 ~Amit Machine Type = na ---Debugger--- A debugger is not configured Contact Information = [email protected] ---uname output--- na
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2107396 Title: [KOP]:: L2 fails to boot with vfio passed through device and memory > 128G To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/2107396/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
