Hi Steve,
while IBM wanted to do the testing I can agree that this was too short in the 
Template for the SRU Team. The description wasn't far away thou, as I said in 
the template "IBM outlined ...", so this is from the comments that already were 
in the bug and description.

I made sure by testing against the PPA that this works as expected.

(one way to) reproduce:
1. apt install libguestfs-tools python-libguestfs linux-image-generic qemu
2. sudo libguestfs-test-tool
This will by default break on e.g.:
  qemu-system-s390x: -device virtio-rng-pci,rng=rng0: MSI-X support is 
mandatory in the S390 architecture
  libguestfs: error: appliance closed the connection unexpectedly, see earlier 
error messages
3. (upgrade from ppa or proposed depending on what you test)
4. sudo libguestfs-test-tool
[...]
===== TEST FINISHED OK =====

I hope this better description will suffice, otherwise let me know.
I'll add it to the SRU template and set back to in progress to wait for 
re-review.

** Changed in: libguestfs (Ubuntu Bionic)
       Status: Incomplete => In Progress

** Description changed:

  [Impact]
  
-  * the way libguestfs launches libvirt is not working on s390x
- 
-  * backport fixes to work on that platform as well
- 
-  * The TL;DR of the changes is to add arch specific options to the calls
+  * the way libguestfs launches libvirt is not working on s390x
+ 
+  * backport fixes to work on that platform as well
+ 
+  * The TL;DR of the changes is to add arch specific options to the calls
  being made
  
  [Test Case]
  
-  * TL;DR: install and use libguestfs-tools on s390x
-    IBM outlined a test using the libvrtguestfs-test-tool, but that is 
-    just one of too many options to try - find more details in the comments 
below.
-    Note: IBM will assist in verifying those and already has on the PPA
+  * Note: IBM will assist in verifying those and already has on the PPA
+  * To run yourself from the original description you can use 
libguestfs-test-tool:
+   1. apt install libguestfs-tools python-libguestfs linux-image-generic qemu
+   2. sudo libguestfs-test-tool
+   This will by default break on e.g.:
+     qemu-system-s390x: -device virtio-rng-pci,rng=rng0: MSI-X support is 
mandatory in the S390 architecture
+     libguestfs: error: appliance closed the connection unexpectedly, see 
earlier error messages
+   3. (upgrade from ppa or proposed depending on what you test)
+   4. sudo libguestfs-test-tool
+     [...]
+     ===== TEST FINISHED OK =====
  
  [Regression Potential]
  
-  * This modifies how arguments are created, but it does so only for s390x so 
the regression should be limited to that architecture.
-    The issue would be that it doesn't start properly, but since that is the 
current state there is no real regression - it would just replacve one way of 
failing for another - and since the tests by IBM on the PPA were good we are 
convinced that even that regression-risk would not trigger.
- 
-  * The FTBFS is a build time only change and should not trigger
+  * This modifies how arguments are created, but it does so only for s390x so 
the regression should be limited to that architecture.
+    The issue would be that it doesn't start properly, but since that is the 
current state there is no real regression - it would just replacve one way of 
failing for another - and since the tests by IBM on the PPA were good we are 
convinced that even that regression-risk would not trigger.
+ 
+  * The FTBFS is a build time only change and should not trigger
  regressions later on.
  
  [Other Info]
-  
-  * Along the fix requested this fixes an FTBFS in Bionic due to the move to a 
new default JDK version.
+ 
+  * Along the fix requested this fixes an FTBFS in Bionic due to the move
+ to a new default JDK version.
  
  ---
  
  The libguestfs versions < 1.37.22 don't work on s390x, as they don't
  contain the patches which include the s390x specifics to instruct qemu-kvm
  correctly. Ubuntu 16.04 contains an older version than that, which makes
  libguestfs not usable on s390x.
  
  Steps to reproduce
  ==================
  A chronological list of steps which will bring off the issue I noticed:
  
      mz@s390xhost$ docker run -it --privileged --rm ubuntu:xenial bash
      root@5dde0aef5d2e:/#
      root@5dde0aef5d2e:/#
      root@5dde0aef5d2e:/# apt update
      root@5dde0aef5d2e:/# DEBIAN_FRONTEND=noninteractive apt install -y \
      libguestfs-tools \
      python-libguestfs \
      linux-image-generic \
      qemu
      root@5dde0aef5d2e:/#
      root@5dde0aef5d2e:/#
      root@5dde0aef5d2e:/# export LIBGUESTFS_DEBUG=1
      root@5dde0aef5d2e:/# export LIBGUESTFS_TRACE=1
      root@5dde0aef5d2e:/# libguestfs-test-tool
  
  Expected result
  ===============
  The libguestfs-test-tool passes and I can use it on s390x.
  
  To check if qemu on s390x works independently from libguestfs, I use this
  command:
  
      root@5dde0aef5d2e:/# qemu-system-s390x \
      -enable-kvm \
      -nographic \
      -kernel /boot/vmlinuz-4.4.0-119-generic \
      -initrd /boot/initrd.img-4.4.0-119-generic \
      -m 1G \
      -M s390-ccw-virtio
  
      [...]  # boot
  
      BusyBox v1.22.1 (Ubuntu 1:1.22.0-15ubuntu1) built-in shell (ash)
      Enter 'help' for a list of built-in commands.
  
      (initramfs)
  
  Actual result
  =============
  
  libguestfs instructs qemu to use pci devices and serial consoles
  which doesn't work on s390x, e.g.:
  
      root@5dde0aef5d2e:/# libguestfs-test-tool
           ************************************************************
           *                    IMPORTANT NOTICE
           *
           * When reporting bugs, include the COMPLETE, UNEDITED
           * output below in your bug report.
           *
           ************************************************************
      libguestfs: trace: set_verbose true
      libguestfs: trace: set_verbose = 0
      libguestfs: trace: set_verbose true
      libguestfs: trace: set_verbose = 0
      LIBGUESTFS_DEBUG=1
      LIBGUESTFS_TRACE=1
      PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
      SELinux: sh: 1: getenforce: not found
      libguestfs: trace: add_drive_scratch 104857600
      libguestfs: trace: get_tmpdir
      libguestfs: trace: get_tmpdir = "/tmp"
      libguestfs: trace: disk_create "/tmp/libguestfs82Gj1d/scratch.1" "raw" 
104857600
      libguestfs: trace: disk_create = 0
      libguestfs: trace: add_drive "/tmp/libguestfs82Gj1d/scratch.1" 
"format:raw" "cachemode:unsafe"
      libguestfs: trace: add_drive = 0
      libguestfs: trace: add_drive_scratch = 0
      libguestfs: trace: get_append
      libguestfs: trace: get_append = "NULL"
      guestfs_get_append: (null)
      libguestfs: trace: get_autosync
      libguestfs: trace: get_autosync = 1
      guestfs_get_autosync: 1
      libguestfs: trace: get_backend
      libguestfs: trace: get_backend = "direct"
      guestfs_get_backend: direct
      libguestfs: trace: get_backend_settings
      libguestfs: trace: get_backend_settings = []
      guestfs_get_backend_settings: []
      libguestfs: trace: get_cachedir
      libguestfs: trace: get_cachedir = "/var/tmp"
      guestfs_get_cachedir: /var/tmp
      libguestfs: trace: get_direct
      libguestfs: trace: get_direct = 0
      guestfs_get_direct: 0
      libguestfs: trace: get_hv
      libguestfs: trace: get_hv = "/usr/bin/qemu-system-s390x"
      guestfs_get_hv: /usr/bin/qemu-system-s390x
      libguestfs: trace: get_memsize
      libguestfs: trace: get_memsize = 500
      guestfs_get_memsize: 500
      libguestfs: trace: get_network
      libguestfs: trace: get_network = 0
      guestfs_get_network: 0
      libguestfs: trace: get_path
      libguestfs: trace: get_path = "/usr/lib/s390x-linux-gnu/guestfs"
      guestfs_get_path: /usr/lib/s390x-linux-gnu/guestfs
      libguestfs: trace: get_pgroup
      libguestfs: trace: get_pgroup = 0
      guestfs_get_pgroup: 0
      libguestfs: trace: get_program
      libguestfs: trace: get_program = "libguestfs-test-tool"
      guestfs_get_program: libguestfs-test-tool
      libguestfs: trace: get_recovery_proc
      libguestfs: trace: get_recovery_proc = 1
      guestfs_get_recovery_proc: 1
      libguestfs: trace: get_selinux
      libguestfs: trace: get_selinux = 0
      guestfs_get_selinux: 0
      libguestfs: trace: get_smp
      libguestfs: trace: get_smp = 1
      guestfs_get_smp: 1
      libguestfs: trace: get_tmpdir
      libguestfs: trace: get_tmpdir = "/tmp"
      guestfs_get_tmpdir: /tmp
      libguestfs: trace: get_trace
      libguestfs: trace: get_trace = 1
      guestfs_get_trace: 1
      libguestfs: trace: get_verbose
      libguestfs: trace: get_verbose = 1
      guestfs_get_verbose: 1
      host_cpu: s390x
      Launching appliance, timeout set to 600 seconds.
      libguestfs: trace: launch
      libguestfs: trace: version
      libguestfs: trace: version = <struct guestfs_version *>
      libguestfs: trace: get_backend
      libguestfs: trace: get_backend = "direct"
      libguestfs: launch: program=libguestfs-test-tool
      libguestfs: launch: version=1.32.2
      libguestfs: launch: backend registered: unix
      libguestfs: launch: backend registered: uml
      libguestfs: launch: backend registered: libvirt
      libguestfs: launch: backend registered: direct
      libguestfs: launch: backend=direct
      libguestfs: launch: tmpdir=/tmp/libguestfs82Gj1d
      libguestfs: launch: umask=0022
      libguestfs: launch: euid=0
      libguestfs: trace: get_backend_setting "force_tcg"
      libguestfs: trace: get_backend_setting = NULL (error)
      libguestfs: trace: get_cachedir
      libguestfs: trace: get_cachedir = "/var/tmp"
      libguestfs: [00000ms] begin building supermin appliance
      libguestfs: [00000ms] run supermin
      libguestfs: command: run: /usr/bin/supermin
      libguestfs: command: run: \ --build
      libguestfs: command: run: \ --verbose
      libguestfs: command: run: \ --if-newer
      libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock
      libguestfs: command: run: \ --copy-kernel
      libguestfs: command: run: \ -f ext2
      libguestfs: command: run: \ --host-cpu s390x
      libguestfs: command: run: \ /usr/lib/s390x-linux-gnu/guestfs/supermin.d
      libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d
      supermin: version: 5.1.14
      supermin: package handler: debian/dpkg
      supermin: acquiring lock on /var/tmp/.guestfs-0/lock
      supermin: build: /usr/lib/s390x-linux-gnu/guestfs/supermin.d
      supermin: reading the supermin appliance
      supermin: build: visiting 
/usr/lib/s390x-linux-gnu/guestfs/supermin.d/base.tar.gz type gzip base image 
(tar)
      supermin: build: visiting 
/usr/lib/s390x-linux-gnu/guestfs/supermin.d/daemon.tar.gz type gzip base image 
(tar)
      supermin: build: visiting 
/usr/lib/s390x-linux-gnu/guestfs/supermin.d/excludefiles type uncompressed 
excludefiles
      supermin: build: visiting 
/usr/lib/s390x-linux-gnu/guestfs/supermin.d/hostfiles type uncompressed 
hostfiles
      supermin: build: visiting 
/usr/lib/s390x-linux-gnu/guestfs/supermin.d/init.tar.gz type gzip base image 
(tar)
      supermin: build: visiting 
/usr/lib/s390x-linux-gnu/guestfs/supermin.d/packages type uncompressed packages
      supermin: build: visiting 
/usr/lib/s390x-linux-gnu/guestfs/supermin.d/packages-hfsplus type uncompressed 
packages
      supermin: build: visiting 
/usr/lib/s390x-linux-gnu/guestfs/supermin.d/packages-reiserfs type uncompressed 
packages
      supermin: build: visiting 
/usr/lib/s390x-linux-gnu/guestfs/supermin.d/packages-xfs type uncompressed 
packages
      supermin: build: visiting 
/usr/lib/s390x-linux-gnu/guestfs/supermin.d/udev-rules.tar.gz type gzip base 
image (tar)
      supermin: mapping package names to installed packages
      supermin: resolving full list of package dependencies
      supermin: build: 194 packages, including dependencies
      supermin: build: 6751 files
      supermin: build: 3694 files, after matching excludefiles
      supermin: build: 3696 files, after adding hostfiles
      supermin: build: 3696 files, after removing unreadable files
      supermin: build: 3700 files, after munging
      supermin: kernel: picked kernel vmlinuz-4.4.0-119-generic
      supermin: kernel: picked modules path /lib/modules/4.4.0-119-generic
      supermin: kernel: kernel_version 4.4.0-119-generic
      supermin: kernel: modules /lib/modules/4.4.0-119-generic
      supermin: ext2: creating empty ext2 filesystem 
'/var/tmp/.guestfs-0/appliance.d.s5hkw6ja/root'
      supermin: ext2: populating from base image
      supermin: ext2: copying files from host filesystem
      supermin: ext2: copying kernel modules
      supermin: ext2: creating minimal initrd 
'/var/tmp/.guestfs-0/appliance.d.s5hkw6ja/initrd'
      supermin: ext2: wrote 9 modules to minimal initrd
      supermin: renaming /var/tmp/.guestfs-0/appliance.d.s5hkw6ja to 
/var/tmp/.guestfs-0/appliance.d
      libguestfs: [01090ms] finished building supermin appliance
      libguestfs: [01090ms] begin testing qemu features
      libguestfs: command: run: /usr/bin/qemu-system-s390x
      libguestfs: command: run: \ -display none
      libguestfs: command: run: \ -help
      libguestfs: command: run: /usr/bin/qemu-system-s390x
      libguestfs: command: run: \ -display none
      libguestfs: command: run: \ -version
      libguestfs: qemu version 2.5
      libguestfs: command: run: /usr/bin/qemu-system-s390x
      libguestfs: command: run: \ -display none
      libguestfs: command: run: \ -machine accel=kvm:tcg
      libguestfs: command: run: \ -device ?
      libguestfs: [01166ms] finished testing qemu features
      libguestfs: trace: get_backend_setting "gdb"
      libguestfs: trace: get_backend_setting = NULL (error)
      [01172ms] /usr/bin/qemu-system-s390x \
          -global virtio-blk-pci.scsi=off \
          -nodefconfig \
          -enable-fips \
          -nodefaults \
          -display none \
          -machine accel=kvm:tcg \
          -cpu host \
          -m 500 \
          -no-reboot \
          -rtc driftfix=slew \
          -global kvm-pit.lost_tick_policy=discard \
          -kernel /var/tmp/.guestfs-0/appliance.d/kernel \
          -initrd /var/tmp/.guestfs-0/appliance.d/initrd \
          -device virtio-scsi-pci,id=scsi \
          -drive 
file=/tmp/libguestfs82Gj1d/scratch.1,cache=unsafe,format=raw,id=hd0,if=none \
          -device scsi-hd,drive=hd0 \
          -drive 
file=/var/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none
 \
          -device scsi-hd,drive=appliance \
          -device virtio-serial-pci \
          -serial stdio \
          -chardev socket,path=/tmp/libguestfs82Gj1d/guestfsd.sock,id=channel0 \
          -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 
\
          -append 'panic=1 console=ttyS0 udevtimeout=6000 
udev.event-timeout=6000 no_timer_check acpi=off printk.time=1 
cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm'
      qemu-system-s390x: Warning: global kvm-pit.lost_tick_policy has invalid 
class name
      Alarm clock
  
  Environment
  ===========
  
  This affects ubuntu 16.04 (see below)
  
  host
  ----
  
      $ lsb_release -a
      LSB Version:      
core-9.20160110ubuntu0.2-noarch:core-9.20160110ubuntu0.2-s390x:printing-9.20160110ubuntu0.2-noarch:printing-9.20160110ubuntu0.2-s390x:security-9.20160110ubuntu0.2-noarch:security-9.20160110ubuntu0.2-s390x
      Distributor ID:   Ubuntu
      Description:      Ubuntu 16.04.4 LTS
      Release:  16.04
      Codename: xenial
  
  container
  ---------
  
      root@5dde0aef5d2e:/# lsb_release -a
      No LSB modules are available.
      Distributor ID:   Ubuntu
      Description:      Ubuntu 16.04.3 LTS
      Release:  16.04
      Codename: xenial
  
      root@5dde0aef5d2e:/# apt-cache policy libguestfs-tools
      libguestfs-tools:
        Installed: 1:1.32.2-4ubuntu2
        Candidate: 1:1.32.2-4ubuntu2
        Version table:
       *** 1:1.32.2-4ubuntu2 500
              500 http://ports.ubuntu.com/ubuntu-ports xenial/universe s390x 
Packages
              100 /var/lib/dpkg/status
  
  Additional information
  ======================
  
  Please be aware that there this is a known libguestfs upstream bug [1]
  which is already fixed and backported by RedHat [2][3] and was the
  trigger for a patch series on the libguestfs mailing list [4]. As an
  example, the latest known needed patch is [5]. As a smoke test, in case
  the libguestfs-test-tool result was a false positive, I used the python
  binding code in Appendix A below, which is based on [6].
  
  FWIW, I checked Ubuntu 18.04 (beta), and it doesn't work there too.
  The packaged version of libguestfs is v1.36 in 18.04 bionic beaver,
  which is too old (and/or doesn't contain the needed backports).
  
  References
  ==========
  [1] Bug 1376547 - qemu-system-s390x: -device isa-serial,chardev=charserial0,
      id=serial0: 'isa-serial' is not a valid device model name
      https://bugzilla.redhat.com/show_bug.cgi?id=1376547
  [2] [Private] Bug 1479526 - [RFE] libguestfs for z Systems
      https://bugzilla.redhat.com/show_bug.cgi?id=1479526
  [3] The RHEL7.5 branch of libguestfs with backports to v1.36
      https://github.com/libguestfs/libguestfs/tree/rhel-7.5
  [4] The patch series of libguestfs on s390x:
      https://www.redhat.com/archives/libguestfs/2017-May/msg00066.html
  [5] Fix of an x86 specific item to enable s390x in v1.37.22
      
https://github.com/libguestfs/libguestfs/commit/5b60dd4eff02f48d344bcdad0d4bad4676ca9168
  [6] https://rwmj.wordpress.com/2010/02/01/libguestfs-python-example/
  
  Appendix A
  ==========
  
  Smoke test, based on the example from the author of libguestfs [6]:
  
      root@26248e573a87:/# cat smoke.py
      #!/usr/bin/python
  
      # To see the Python docs, do: help (guestfs)
      # To see the general docs http://libguestfs.org/guestfs.3.html
      # For FAQ see http://libguestfs.org/FAQ.html
      import os
      import guestfs
      g = guestfs.GuestFS()
      f = open("/tmp/test.img", "w")
      f.truncate(500 * 1024 * 1024)
      f.close()
      g.add_drive("/tmp/test.img")
      g.launch()
      g.part_disk("/dev/sda", "mbr")
      g.mkfs("ext2", "/dev/sda1")
      g.mount("/dev/sda1", "/")
      f = open("/tmp/ks.cfg", "w")
      f.write("# this would be the kickstart file\n")
      f.write("#\n")
      f.write ("# end of file\n")
      f.close()
      g.upload("/tmp/ks.cfg", "/ks.cfg")
      g.sync()
      g.umount_all()
  
  == Comment: #1 - Markus Zoeller <mzoel...@de.ibm.com> - 2018-04-20 06:51:10 ==
  With the Debian Sid repository I could install libguestfs v1.38 in an
  Ubuntu 18.04 (bionic beaver) container, all on s390x:
  
      [mz@s390xhost]$ docker run -it --privileged --rm s390x/ubuntu:18.04 bash
      $ apt update
      $ apt install -y vim
      $ apt-cache policy libguestfs-tools
      $ apt install -y software-properties-common \
      debian-archive-keyring \
      dirmngr
      $ apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 
8B48AD6246925553
      $ apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 
7638D0442B90D010
      $ add-apt-repository "deb http://ftp.de.debian.org/debian sid main"
  
      $ cat << EOF > /etc/apt/preferences.d/libguestfs.pref
      # Note: 2 blank lines are required between entries
      Package: *
      Pin: release a=bionic
      Pin-Priority: 500
  
      Package: *
      Pin: origin "ftp.de.debian.org"
      Pin-Priority: 300
  
      # Pattern includes 'python-guestfs' and 'libguestfs-tools' and similarily 
named dependencies
      Package: *guestfs*
      Pin: origin "ftp.de.debian.org"
      Pin-Priority: 700
      EOF
      $ cat /etc/apt/preferences.d/libguestfs.pref
  
      $ apt update
      $ apt-cache policy libguestfs-tools
  
      $ DEBIAN_FRONTEND=noninteractive apt install -y \
          libguestfs-tools \
          python-libguestfs \
          linux-image-generic \
          qemu-system \
          qemu-kvm
  
      $ apt-cache policy libguestfs-tools
      $ export LIBGUESTFS_DEBUG=1
      $ export LIBGUESTFS_TRACE=1
      $ libguestfs-test-tool
      [...]
      ===== TEST FINISHED OK =====
  
  This verifies that libguestfs v1.38 has the needed code to work on
  s390x.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1766534

Title:
  libguestfs cannot start the qemu process on s390x

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/1766534/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to