Source: shasta Version: 0.11.1-1 Severity: important User: debian...@lists.debian.org Usertags: regression
Dear maintainer(s),Your package has an autopkgtest, great. However, it recently started failing on arm64 in unstable, testing *and* stable. Until mid September 2023, most runs passed (there are some historical rare failures where we don't have logs anymore), but somewhere between 2023-09-17 and 2023-10-01 the tests started failing (although, in testing we had some passes on 2023-09-28 and 2023-10-04, interestingly these runs too *much* longer than normal (factor ~20)). I first suspected that this regression aligned with my fix for bug 1050256 where I switched the kernel of our hosts to backports, but there are multiple good runs with that kernel. Today I learned that "Killed" in the log is often coming from the kernel when a program is allocating too much memory and the kernel kills the process. We have some striking logs of failures [1] where the shasta test actually bails out itself for lack of memory.
So, there's a couple of things (weirdness and options):1) why does it now suddenly start to (nearly always) fail across the board on arm64 (in Debian, Ubuntu still seems fine), without changes to the infrastructure that I know of?
2) do you also believe this is related to memory consumption?3) If 2 == yes, what are the memory requirements for the test? The test *could* test for that before it starts and bail out (restriction: skippable with exit code 77 [2]) if the amount of memory available is too small. 4) just stop testing on arm64 (but again, in Ubuntu the test is still running fine) 5) the recent glibc security update also came to my mind, but that's not *yet* installed on our hosts, nor is it installed in the stable testbed.
The release team has announced [3] that failing autopkgtest on amd64 and arm64 are considered RC in testing. However, due to nature of the symptoms, I've filed this as important for now.
Paul[1] https://ci.debian.net/data/autopkgtest/testing/arm64/s/shasta/38544284/log.gz 48s 2023-Oct-04 12:01:05.603191 Memory allocation failure during mremap call for MemoryMapped::Vector.
48s This assembly requires more memory than available. 48s Rerun on a larger machine.[2] https://salsa.debian.org/ci-team/autopkgtest/-/blob/master/doc/README.package-tests.rst
[3] https://lists.debian.org/debian-devel-announce/2019/07/msg00002.html
OpenPGP_signature.asc
Description: OpenPGP digital signature
