I spend a bit more time bisecting the various core18 snaps to see at what revision the failure started to become reproducible. Interestingly I found that it started with r2208 AFAICT. It's all a bit annoying because it's a race so I don't trust it 100% but the previous r2206 got 2 good runs in a row but the r2208 failed also two times in a row. The only change there is cloud-init https://people.canonical.com/~mvo/core18-changes/html/edge/'20210928'r2206_'20210930'r2208.html which might explain why we see the failure in our integration tests in GCE but not inside QEMU. Unfortuantely the cloud-init diff is pretty big.
For reference, here is what I ran: """ $ git status On branch tests-use-core18-from-gce Your branch is up to date with 'sergiocazzolato/tests-use-core18-from-gce'. $ git describe 2.53.1-460-gd171d434ed diff --git a/overlord/devicestate/devicestate.go b/overlord/devicestate/devicestate.go index d40b32bcbd..77ec3fc0a8 100644 --- a/overlord/devicestate/devicestate.go +++ b/overlord/devicestate/devicestate.go @@ -117,6 +117,8 @@ func canAutoRefresh(st *state.State) (bool, error) { if !seeded { return false, nil } + // HACK + return false, nil // Try to ensure we have an accurate time before doing any // refreshy stuff. Note that this call will not block. diff --git a/spread.yaml b/spread.yaml index 0aef5dea7c..68cbdc09b7 100644 --- a/spread.yaml +++ b/spread.yaml @@ -37,7 +37,7 @@ environment: MANAGED_DEVICE: "false" # a global setting for LXD channel to use in the tests LXD_SNAP_CHANNEL: "latest/edge" - UBUNTU_IMAGE_SNAP_CHANNEL: "latest/candidate" + UBUNTU_IMAGE_SNAP_CHANNEL: "beta/1.11" CORE_CHANNEL: '$(HOST: echo "${SPREAD_CORE_CHANNEL:-edge}")' BASE_CHANNEL: '$(HOST: echo "${SPREAD_BASE_CHANNEL:-edge}")' KERNEL_CHANNEL: '$(HOST: echo "${SPREAD_KERNEL_CHANNEL:-edge}")' diff --git a/tests/lib/prepare.sh b/tests/lib/prepare.sh index e6b984c4d0..14b9608fb9 100755 --- a/tests/lib/prepare.sh +++ b/tests/lib/prepare.sh @@ -973,7 +973,26 @@ EOF fi if os.query is-core18; then - curl -s -o core18.snap https://storage.googleapis.com/snapd-spread-tests/snaps/core18_20211102_amd64.snap + # GOOD + #curl --insecure -o core18.snap https://people.canonical.com/~mvo/tmp/CSO04Jhav2yK0uz97cr0ipQRyqg0qQL6_2081.snap + # GOOD + #curl --insecure -o core18.snap https://people.canonical.com/~mvo/tmp/CSO04Jhav2yK0uz97cr0ipQRyqg0qQL6_2128.snap + # BAD + #curl --insecure -o core18.snap https://people.canonical.com/~mvo/tmp/CSO04Jhav2yK0uz97cr0ipQRyqg0qQL6_2239.snap + # GOOD + #curl --insecure -o core18.snap https://people.canonical.com/~mvo/tmp/CSO04Jhav2yK0uz97cr0ipQRyqg0qQL6_2141.snap + # GOOD (3 good runs in a row) + #curl --insecure -o core18.snap https://people.canonical.com/~mvo/tmp/CSO04Jhav2yK0uz97cr0ipQRyqg0qQL6_2191.snap + # BAD + #curl --insecure -o core18.snap https://people.canonical.com/~mvo/tmp/CSO04Jhav2yK0uz97cr0ipQRyqg0qQL6_2213.snap + # BAD + #curl --insecure -o core18.snap https://people.canonical.com/~mvo/tmp/CSO04Jhav2yK0uz97cr0ipQRyqg0qQL6_2228.snap + # BAD + #curl --insecure -o core18.snap https://people.canonical.com/~mvo/tmp/CSO04Jhav2yK0uz97cr0ipQRyqg0qQL6_2219.snap + # BAD + curl --insecure -o core18.snap https://people.canonical.com/~mvo/tmp/CSO04Jhav2yK0uz97cr0ipQRyqg0qQL6_2208.snap + # GOOD (2 good run in a row) + #curl --insecure -o core18.snap https://people.canonical.com/~mvo/tmp/CSO04Jhav2yK0uz97cr0ipQRyqg0qQL6_2206.snap EXTRA_FUNDAMENTAL="$EXTRA_FUNDAMENTAL --snap $PWD/core18.snap" fi $ $GOPATH/bin/spread -repeat 2 google:ubuntu-core-18-64:tests/main/services-disabled-kept-happy ... """ Each run takes ~12min so it's a bit cumbersome. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1949089 Title: systemd randomly fails to activate mount units in Ubuntu Core 18 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1949089/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs