Public bug reported:

host Ubuntu 22.04 + Kernel 6.8 HWE
OpenStack Epoxy (openstack-helm-based), 
AMD EPYC 9754 128-Core Processor

Qemu runs in a ubuntu-noble-based docker container.
Package: qemu-system-x86
Version: 1:8.2.2+ds-0ubuntu1.9

Previous Qemu version was 1:6.2+dfsg-2ubuntu6.26 (runs in ubuntu-jammy-
based container).

After upgrading the cloud to OpenStack Epoxy and thus qemu 6.2 to qemu 8.2 as 
pointed above,
we are seeing boot failures for new VMs using our largest flavor (~900GB RAM, 
1GB hugepages, 112 vCPU).

During early boot the guest reports virtio BAR reservation failures such
as:

virtio-pci ... BAR ... can't reserve
virtio-pci: probe failed with error -16

When this occurs the virtio-scsi controller never initializes, so the
root disk is not detected and the VM drops to initramfs with a missing
root UUID.

Smaller flavors appear unaffected.

Comparing working vs failing instances shows the same device layout
(virtio-scsi disk, virtio NIC, etc.), but the PCI MMIO window placement
differs:

Working host (older VM):
64-bit MMIO window at 0xe040000000  (above guest RAM end)

Failing host (new VM):
64-bit MMIO window at 0xc040000000  (inside guest RAM range)

When the MMIO window overlaps guest RAM, virtio BAR allocation fails
with -EBUSY.

Machine types

Working VM:
pc-i440fx-6.2

New VM:
pc-i440fx-8.2

Both are still i440fx (BIOS) machines. The version change alone doesn’t
appear to explain the issue but likely changes QEMU defaults (PCI hole
sizing / BAR placement).

Attaching XML for the failing VM as generated by OpenStack Nova (epoxy
release) for libvirt 10.0 (from ubuntu 24.04)

Rebuilding VM with q35-* machine type seems to solve the issue.

We suspect this might be related to the following patch/fix "hw/i386/pc:
Avoid overlap between CXL window and PCI 64" which is present in qemu
10.2.0 https://gitlab.com/qemu-
project/qemu/-/commit/d1193481dee63442fc41e47ca6ebc4cd34f1f69c

Please consider porting that patch to qemu 8.2 as in Ubuntu 24.04.

** Affects: qemu (Ubuntu)
     Importance: Undecided
         Status: Confirmed

** Attachment added: "libvirt 10.0 XML for large mem VM that fails on qemu 8.2"
   
https://bugs.launchpad.net/bugs/2146021/+attachment/5955323/+files/large-vm-failing.xml

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2146021

Title:
  Qemu 8.2 fails to boot VM with very large memory

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/2146021/+subscriptions


-- 
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to