Theory given what we know so far: - only fails if LVL1 is at 4.4 - not failing if LVL1 is at 3.13 - 4.4 might have more CPU features - qemu 2.0 when using host-model is passing ALL features - qemu 2.5 works, but we now know it filters some flags that 2.0 doesn't => one of these extra flags disturbs the guests bug detection
Check extra flags in LVL1 between 3.13 and 4.4 3.13 -> 4.4 has in addition (Host): > clflushopt > kaiser > mpx > tsc_known_freq > xgetbv1 > xsavec < eagerfpu Comparing LVL2 between case 07 and 10 < arch_capabilities > arat So interestingly, none of the flags that are added on 4.4 on LVL1 show up in the guest. But one more that also seems interesting is showing up "arch_capabilities". I haven't found a good way to control arch_capabilities yet. It is part of the Spectre backports actually like [1] - I haven't seen it like that in the code that you added to qemu 2.0 but it is at least related. So the LVL1 4.4 has some empty flags/features that the older qemu 2.0 does not filter and hence the guest gets an broken MSR for MSR_IA32_ARCH_CAPABILITIES. That is what breaks the guests. Given that: - nested (especially in these much older versions of KVM/Qemu) is not very well supported - this issue seems to depend on other security fixes (in the 4.4 kernel) - qemu 2.0 is out in ESM, and this is not a fix required for that I'd call it confirmed but prio wishlist and probably, unless convinced won't work on it for now. I hope the analysis helps if e.g. the security Team wants to take a look at all MSR_IA32_ARCH_CAPABILITIES related changes. One could e-g- actually read CPUID_7_0_EDX_ARCH_CAPABILITIES in the LVL2 guest that is broken. I'm rather sure it has malformed or incomplete content. [1]: https://lwn.net/Articles/746119/ ** Changed in: qemu (Ubuntu) Importance: Undecided => Low -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1829555 Title: nested virtualization w/first level trusty guests has odd MDS behavior Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: When nested kvm virtualization is used (with host-passthrough), if the first level guest is a trusty vm, odd behavior is seen in the second level guest: host os: disco/5.0.0-15.16-generic/qemu 1:3.1+dfsg-2ubuntu3.1 contents of /sys/devices/system/cpu/vulnerabilities/mds: Mitigation: Clear CPU buffers; SMT vulnerable 1st level vm: trusty/4.4.0-148.174~14.04.1-generic/qemu 2.0.0+dfsg-2ubuntu1.46 contents of /sys/devices/system/cpu/vulnerabilities/mds: Mitigation: Clear CPU buffers; SMT Host state unknown 2nd level vm: bionic/4.15.0-50.54-generic contents of /sys/devices/system/cpu/vulnerabilities/mds: Not affected This behavior is not seen when the first level guest is a xenial or bionic vm (same bare metal hardware): 1st level vm: bionic/4.15.0-50.54-generic/qemu 1:2.11+dfsg-1ubuntu7.13 contents of /sys/devices/system/cpu/vulnerabilities/mds: Mitigation: Clear CPU buffers; SMT Host state unknown 2nd level vm: bionic/4.15.0-50.54-generic contents of /sys/devices/system/cpu/vulnerabilities/mds: Mitigation: Clear CPU buffers; SMT Host state unknown and: 1st level vm: xenial/4.4.0-148.174-generic/qemu 1:2.5+dfsg-5ubuntu10.39 contents of /sys/devices/system/cpu/vulnerabilities/mds: Mitigation: Clear CPU buffers; SMT Host state unknown 2nd level vm: bionic/4.15.0-50.54-generic contents of /sys/devices/system/cpu/vulnerabilities/mds: Mitigation: Clear CPU buffers; SMT Host state unknown It's not clear whether this is an issue with linux/kvm or qemu in trusty. --- ApportVersion: 2.14.1-0ubuntu3.29 Architecture: amd64 AudioDevicesInUse: USER PID ACCESS COMMAND /dev/snd/controlC0: ubuntu 2239 F.... pulseaudio DistroRelease: Ubuntu 14.04 HibernationDevice: RESUME=UUID=4fa9460d-7ed4-49db-8e22-86a5107d0062 InstallationDate: Installed on 2019-02-14 (92 days ago) InstallationMedia: Ubuntu 14.04.5 LTS "Trusty Tahr" - Release amd64 (20160803) Lsusb: Bus 001 Device 002: ID 0627:0001 Adomax Technology Co., Ltd Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub MachineType: QEMU Standard PC (i440FX + PIIX, 1996) Package: qemu 2.0.0+dfsg-2ubuntu1.46 PackageArchitecture: amd64 ProcEnviron: TERM=screen PATH=(custom, no user) XDG_RUNTIME_DIR=<set> LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 qxldrmfb ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-148-generic root=UUID=9a35107e-83fa-4010-81e1-235a4ea14fe6 ro quiet splash vt.handoff=7 ProcVersionSignature: User Name 4.4.0-148.174~14.04.1-generic 4.4.177 RelatedPackageVersions: linux-restricted-modules-4.4.0-148-generic N/A linux-backports-modules-4.4.0-148-generic N/A linux-firmware 1.127.24 RfKill: Tags: trusty trusty Uname: Linux 4.4.0-148-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: adm cdrom dip libvirtd lpadmin plugdev sambashare sudo _MarkForUpload: True dmi.bios.date: 04/01/2014 dmi.bios.vendor: SeaBIOS dmi.bios.version: 1.12.0-1 dmi.chassis.type: 1 dmi.chassis.vendor: QEMU dmi.chassis.version: pc-i440fx-bionic dmi.modalias: dmi:bvnSeaBIOS:bvr1.12.0-1:bd04/01/2014:svnQEMU:pnStandardPC(i440FX+PIIX,1996):pvrpc-i440fx-bionic:cvnQEMU:ct1:cvrpc-i440fx-bionic: dmi.product.name: Standard PC (i440FX + PIIX, 1996) dmi.product.version: pc-i440fx-bionic dmi.sys.vendor: QEMU To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1829555/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
