On Mon, Nov 18, 2024 at 04:34:37PM +0100, Christian Heusel wrote: > On 24/11/17 10:08PM, David Kalnischkies wrote: > > Control: retitle -1 apt: FTBFS on Arch Linux with libapt test failures > > Control: severity -1 wishlist > > > > Hi Christian Heusel, > > Hello David, > > > as said on IRC, we are happy to receive and help with bug reports > > here in the BTS, but I had to dial back on the severity quite a bit as > > this is not applying to Debian and hence not release-critical to Debian > > (apt is not in any danger of being removed from testing or a future > > release for obvious reasons, but it could still confuse unsuspecting > > users and automatic processes working with and against bugreports). > > > > No worries, severity has hardly any practical meaning for us so your > > bug report isn't handled and dealt with slower or faster regardless > > of it being tagged serious or wishlist. > > > > sure, that is not an issue for me! I think I just selected this severity > as it's descriptions sounded the most fitting when using reportbug. > (Fun fact, in the Arch bugtracker the severity field means almost > nothing aswell :D) > > > > > But enough of the pretext, lets start with the on topic: > > > > Am Sun, Nov 17, 2024 at 08:15:21PM +0100, schrieb Christian Heusel: > > > * What was the outcome of this action? > > > Building the package yields test failures for 2.9.11 > > > > Did you try with earlier versions or is 2.9.11 the first you try? > > Yes I tried earlier versions and they all worked fine. The last version > that worked was the previous release 2.9.10 but I package it for [a > while now] (since 2.9.2). > > In the meantime we had another release (2.9.12) which fixed the > URITest.AutoProxyTest issue, so let's just ignore that for now. The > crash in the other tests persists though! > > I also took the time and bisected between 2.9.10 and 2.9.11 which commit > introduced the failure and I landed on the following: > > commit b430a53e45b6d676142af7da78564de3bd97ef5f (HEAD) > Author: наб <nabijaczlew...@nabijaczleweli.xyz> > Date: Tue Nov 12 19:50:56 2024 +0100 > > methods/http.cc: APT::StringView -> std::string_view >
There seems to be an issue with your bisecting, as that change introduced no difference unless Acquire::http::User-Agent-Non-Interactive is set, where it sends a "non-interactive" flag in the user agent. > > > > The tests in question themselves haven't changed in months. > > The code they test might have, but to figure that out it would be > > very beneficial to know if that was "always" the case or a recent > > change. > > > > > * What outcome did you expect instead? > > > The application builds without test failures > > > > Can you perhaps provide some instructions so that we with Debian as > > a starting base could reach a point where we can reproduce this failure > > which preferably doesn't involve "install Arch on a spare machine" or > > (at least for me, others might have other opinions) the word "Docker". > > Docker or distrobox is of course the easy route, getting a working Arch > Linux chroot is not too hard aswell and is sufficient to repro this bug > (although we need to do a ton of trickery that wouldn't be neccessary on > a regular Arch system since things aren't set up): > > # apt install arch-install-scripts pacman-package-manager > archlinux-keyring > # curl > https://gitlab.archlinux.org/archlinux/packaging/packages/pacman/-/raw/main/pacman.conf\?ref_type\=heads > > /etc/pacman.conf > # mkdir -pv /etc/pacman.d/ > # echo "Server = https://geo.mirror.pkgbuild.com/\$repo/os/\$arch"; > > /etc/pacman.d/mirrorlist > # pacman-key --init && pacman-key --populate > # cd /path/to/chroots > # mkdir -pv archlinux-chroot > # pacstrap -KP archlinux-chroot base base-devel git devtools nvchecker > less gdb > # arch-chroot archlinux-chroot > > ># useradd -m -G wheel username > ># echo "%wheel ALL=(ALL:ALL) NOPASSWD: ALL" >> /etc/sudoers > ># sed -i "s/CheckSpace/#CheckSpace/" /etc/pacman.conf > ># sudo -u username -i > ># pkgctl repo clone --protocol=https apt > ># cd apt > ># pkgctl version upgrade > ># gpg --import < keys/pgp/* > ># export MAKEFLAGS="-j$(nproc)" > ># makepkg --syncdeps > ># cat /home/username/apt/src/build/Testing/Temporary/LastTest.log > ># gdb src/build/test/libapt/libapt_test You can't run the unit tests that way, this doesn't work, you need to run them using `make test` (or `ninja test` is what we'd use, we do not test the Makefile generator). Well you could run it manually but it needs to be with the source directory test/libapt as the working directory. That explains all your failures with the AutoProxyTest, fwiw. > > With "native" Arch tooling this would have been just a few commands (we > do all our builds in clean chroots), but sadly our devtools arent't > available in debian. > > > As in, a Debian chroot can be relatively easy bootstrapped even on > > foreign Linux distros with (c)debootstrap. I suppose something like this > > exists for Arch, too? And how can I check out your work so far and build > > that to see the error(s) myself? > > > > > > > [ FAILED ] URITest.AutoProxyTest (0 ms) > > > > That test has 4 checks and they all fail… they don't do much through, > > they just call a script, so that screams setup problem like that the > > script isn't executable or you are building on tmpfs with noexec or > > something like that. > > As I have said previously, let's just ignore this then! > > > > [ RUN ] TreeParserTest.ParseInvalid > > > /usr/include/c++/14.2.1/string_view:256: constexpr const > > > std::basic_string_view<_CharT, _Traits>::value_type& > > > std::basic_string_view<_CharT, _Traits>::operator[](size_type) const > > > [with _CharT = char; _Traits = std::char_traits<char>; const_reference = > > > const char&; size_type = long unsigned int]: Assertion '__pos < > > > this->_M_len' failed. > > > Aborted (core dumped) > > > > While the libapt code this is crashing in should be rewritten to not > > crash… (which I did once, but it was kinda ugly, so we discarded it, but > > now that a bunch of std::string_view recently landed perhaps…) anyway, > > this is a test that is trying to trigger an exception. Do you perhaps > > build with exceptions disabled? > > No, our buildflags are the following[1]: > > CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions \ > -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security \ > -fstack-clash-protection -fcf-protection \ > -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer" > CXXFLAGS="$CFLAGS -Wp,-D_GLIBCXX_ASSERTIONS" > > But I guess the added -D_GLIBCXX_ASSERTIONS is relevant to trigger the > bug..? So somehow (just judging from the backtrace) there seems to be a > bounds issue in APT::Internal::PatternTreeParser::parseShortPattern with > the check that might now be caught since the switch to std::string_view..? This should be it. > > > Or was GoogleTest perhaps built without them? Which type of GoogleTest > > is that anyhow, the pre-built or the one that is build with apt. > > It is using the system one: > > $ ldd src/build/test/libapt/libapt_test | grep gtest > libgtest.so.1.15.2 => /usr/lib/libgtest.so.1.15.2 (0x00007f684c157000) > > Also I think the test suite is not responsible given that one version of > the upstream code works and another one does not. I think there's some issue with the system one; I was building apt in a docker.io/archlinux container earlier and that found no tests at all for some reason. > > > > Our buildsystem supports both in theory, but > > GoogleTest upstream tends to advice against building the goods and the > > tests differently – in Debian that doesn't usually happen, but in a more > > source-orientated (AUR stuff is built on user-systems, right?) this > > might be more of a factor, maybe? > > > > At least I assume that both "failures" are a red herring and the actual > > problem causing them is deeper and more related to how Arch is setup. > > Hence my interest in chroot instructions… if you can't reproduce it > > there, we have a first good lead. > > I hope you can work with the above instructions, they are roughly based > on [this][2] wiki article. > > > Best regards > > > > David Kalnischkies > > Thanks for being awesome, > Chris > > [0]: > https://gitlab.archlinux.org/archlinux/packaging/packages/apt/-/commits/main/?ref_type=HEADS > [1]: > https://gitlab.archlinux.org/archlinux/devtools/-/blob/master/config/makepkg/x86_64.conf?ref_type=heads#L43-57 > [2]: > https://wiki.archlinux.org/title/Install_Arch_Linux_from_existing_Linux#Using_a_chroot_environment -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en
signature.asc
Description: PGP signature
