After some setup and preparation I was bisecting through v5.2.0..v6.0.0
to see which change exactly added this new behavior.

Summary:
 - build reduced qemu
 - build, configure and log-check errors will be skips
 - run the extracted systemd test
 - redirect serial to file
 - check log if the service failed
 - unlikely to be related, but to be sure I as using the same slof.bin on all 
runs
 - verified with self-built v5.2.0 (food) and v6.0.0 (bad)

The full log is here:
$ git bisect log
git bisect start
# new: [609d7596524ab204ccd71ef42c9eee4c7c338ea4] Update version for v6.0.0 
release
git bisect new 609d7596524ab204ccd71ef42c9eee4c7c338ea4
# old: [553032db17440f8de011390e5a1cfddd13751b0b] Update version for v5.2.0 
release
git bisect old 553032db17440f8de011390e5a1cfddd13751b0b
# new: [57206696058e3a8b1ee6ec8a129722baa6e7ec9f] hw/block/nvme: merge 
implicitly/explicitly opened processing masks
git bisect new 57206696058e3a8b1ee6ec8a129722baa6e7ec9f
# new: [4663b72a48fd540cbe16053b01d6839a95656440] intc/arm_gic: Fix 
gic_irq_signaling_enabled() for vCPUs
git bisect new 4663b72a48fd540cbe16053b01d6839a95656440
# new: [55810e90cc9a5ca18289c25aa5c1e0e2dc77eadb] ppc/spapr: cleanup -machine 
pseries,nvdimm=X handling
git bisect new 55810e90cc9a5ca18289c25aa5c1e0e2dc77eadb
# old: [cd08b80952657ab9f9ca5b8f374982d9c3159f4a] Replace HTTP links with HTTPS 
ones: documentation
git bisect old cd08b80952657ab9f9ca5b8f374982d9c3159f4a
# old: [5f3013654e879bb4b22876617fdb235aa22568d3] target/mips: Remove unused 
headers from op_helper.c
git bisect old 5f3013654e879bb4b22876617fdb235aa22568d3
# new: [ab32b78cd1b3b31950c4332f0fa8b192295d77fd] audio: Simplify audio_bug() 
removing old code
git bisect new ab32b78cd1b3b31950c4332f0fa8b192295d77fd
# new: [711dfb2423a85fba91884b5a90a8c0712c152ce2] hw/ppc: Do not re-read the 
clock on pre_save if doing savevm
git bisect new 711dfb2423a85fba91884b5a90a8c0712c152ce2
# old: [cf77e27f1e16d8eca761e5c966ea80291254a32c] target/ppc: replaced the TODO 
with LOG_UNIMP and add break for silence warnings
git bisect old cf77e27f1e16d8eca761e5c966ea80291254a32c
# new: [728aa6f6ffd8cf21d9485eff7ac1926b00155ed2] hw/ppc/spapr_tpm_proxy: Fix 
hexadecimal format string specifier
git bisect new 728aa6f6ffd8cf21d9485eff7ac1926b00155ed2
# old: [132954a8307635c08c8e3c4d63d4af9fa865a459] ppc/translate: Turn the 
helper macros into functions
git bisect old 132954a8307635c08c8e3c4d63d4af9fa865a459
# old: [91699dbf30a94dea2575ae193412c364c7f3a5fd] ppc/translate: Raise 
exceptions after setting the cc
git bisect old 91699dbf30a94dea2575ae193412c364c7f3a5fd
# new: [bcb0b7b1a1c05707304f80ca6f523d557816f85c] ppc/translate: Rewrite 
gen_lxvdsx to use gvec primitives
git bisect new bcb0b7b1a1c05707304f80ca6f523d557816f85c
# first new commit: [bcb0b7b1a1c05707304f80ca6f523d557816f85c] ppc/translate: 
Rewrite gen_lxvdsx to use gvec primitives


** Bug watch added: gitlab.com/qemu-project/qemu/-/issues #212
   https://gitlab.com/qemu-project/qemu/-/issues/212

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1935617

Title:
  systemd autopkgtest broken on ppc64el with qemu 6.0

Status in The Ubuntu-power-systems project:
  New
Status in qemu package in Ubuntu:
  New
Status in systemd package in Ubuntu:
  New

Bug description:
  I'm not sure yet if this is flaky or a real issue, but I'm filing it
  to avoid multiple people analyzing the same.

  The Qemu 6.0 upload 
https://launchpad.net/ubuntu/+source/qemu/1:6.0+dfsg-1~ubuntu2 triggers a test 
failure like
  
https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/ppc64el/s/systemd/20210708_223311_e3bbb@/log.gz

  I have tested the new qemu on ppc64 and it worked fine for device emulation 
and migration cases.
  But this is suspicious.
  Of the last tests exactly and only those with the new qemu failed.

  
  impish
    ppc64el
      tests-in-lxd                   (F  5% f  0% S  0% B  0% => P 95%/) 
F.........F.............................
      systemd-fsckd                  (F  0% f  0% S 100% B  0% => P  0%/) 
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
      upstream-1                     (F 15% f  0% S  0% B  0% => P 85%/) 
F..FFFF....F............................
      upstream-2                     (F 12% f  0% S  0% B  0% => P 87%/) 
F..FFFF.................................

  
  For an insight in flakyness/reproducibility I've retriggered the missing qemu 
and the non-qemu cases a few times. If those reproduce all-bad vs all-good 
again this would further indicate a real issue.

  Unfortunately the ppc maas seems down right now and canonistack also
  isn't too nice this week - overall that inhibits the testing a bit :-/

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1935617/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to     : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help   : https://help.launchpad.net/ListHelp

Reply via email to