Re: debvm for autopkgtests with multiple host?
Hi,
Quoting Helmut Grohne (2023-09-23 20:18:21)
> Quoting Ian Jackson (2023-09-23 12:19:27)
> > To summarise that discussion: at that time the best available solution that
> > worked in ci.d.n seemed to be to write an ad-hoc script to run the tests in
> > qemu; three packes had done that, each separately, with complex scripts
> > with many moving parts.
the three packages are probably sbuild, dropbear and cryptsetup?
> > I saw debvm, and wondered if it was suitable for this purpose. But, then I
> > looked at its debian/test/control and I see that the tests are marked as
> > flaky.[2] So maybe it isn't reliable enough :-/.
>
> The reliability of tests is ok. The reason for marking them flaky is that
> they currently test the "wrong" packages. ci.d.n sets up chroots in a
> delicate way to combine particular packages to see which combinations cause
> breakage. Then debvm just creates an unstable system and tests that. In
> effect, it currently tests unstable (inside those virtual machines) rather
> than what it is supposed to be testing.
>
> Johannes solved this problem on the mmdebstrap side and mmdebstrap's
> tests no longer are flaky in this way. Therefore this should be solvable
> on the debvm side. I just haven't gotten do figuring out the right runes
> thus far. Roughly speaking, the hosts' apt configuration, pinning and
> sources.lists should be used inside the created virtual machine.
right now the mmdebstrap autopkgtest only mimics sources and pinning of the
outside system. I had not considered that apt configuration on salsaci or debci
was set to something that influenced the tests. Is the apt configuration on
those systems set to something that is not the default and should be considered
as well?
There is really not much magic. The core of it is to pass this to your
mmdebstrap or debvm-create invocation:
--setup-hook='for f in /etc/apt/sources.list /etc/apt/sources.list.d/*
/etc/apt/preferences.d/*;
do [ -e "$f" ] && { echo; sed "s| file://| copy://|" "$f"; }
| tee "$1/$f" >&2; done'
--hook-dir=/usr/share/mmdebstrap/hooks/file-mirror-automount
The first will hook will make sure that the chroot receives the sources and
pinning values of the outside system. The second will do some bind-mount magic
which allows the chrooted processes to access even file:// repositories from
outside the chroot. Full script here:
https://sources.debian.org/src/sbuild/0.85.3/debian/tests/unshare-qemuwrapper/
> There is another practical problem. None of the autopkgtest nodes support
> kvm. Emulation will always use tcg. For one thing, tcg is slow. It can be so
> slow on some architectures that RCU becomes unhappy as its grace periods
> become too long. For another, tcg is buggy. It has emulation bugs even on
> release architectures that make some expected functionality fail. For
> instance, gdb reliably segfaults when run in s390x tcg emulation.
Architectures that are not amd64 or arm64 are a common source of problems. The
current sbuild autopkgtest hits another issue when running qemu on s390x:
514s ^M[ 28.399829] illegal operation: 0001 ilc:1 [#1] SMP
514s ^M[ 28.400490] Modules linked in: chacha_s390(+) libchacha virtio_pci
virtio_pci_legacy_dev virtio_pci_modern_dev virtio_blk
514s ^M[ 28.402977] CPU: 4 PID: 163 Comm: cryptomgr_test Not tainted
6.5.0-1-s390x #1 Debian 6.5.3-1
514s ^M[ 28.403150] Hardware name: QEMU 8561 QEMU (KVM/Linux)
514s ^M[ 28.403285] Krnl PSW : 0704c0018000 0042 (0x42)
514s ^M[ 28.403964]R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:0
PM:0 RI:0 EA:3
[...]
514s ^M[ 28.408739] Last Breaking-Event-Address:
514s ^M[ 28.408758] [<03ff800fb084>] chacha_crypt_generic+0x54/0xfd0
[libchacha]
514s ^M[ 28.409610] ---[ end trace ]---
https://ci.debian.net/data/autopkgtest/testing/s390x/s/sbuild/38123402/log.gz
In addition to debvm, these tests should maybe directly depend on qemu so that
they get run on new qemu uploads that trigger these kinds of regressions.
Assuming this is really a qemu problem and not something a new kernel version
introduced...
Thanks!
cheers, josch
signature.asc
Description: signature
Bug#1052536: ITP: cowsql -- Embeddable, replicated and fault tolerant SQL engine
Package: wnpp Owner: Free Ekanayaka Severity: wishlist * Package name: cowsql Version : 0.1 Upstream Author : Free Ekanayaka * URL or Web page : https://github.com/cowsql/cowsql * License : LGPLv3 with Static Linking Exception Description : Embeddable, replicated and fault tolerant SQL engine cowsql is a C library that implements an embeddable and replicated SQL database engine with high availability and automatic failover. cowsql extends SQLite with a network protocol that can connect together various instances of your application and have them act as a highly-available cluster, with no dependency on external databases. The name "cowsql" loosely refers to the "pets vs. cattle" concept, since it's generaly fine to delete or rebuild a particular node of an application that uses cowsql for data storage.
Re: debvm for autopkgtests with multiple host?
Hi, On 24-09-2023 10:27, Johannes Schauer Marin Rodrigues wrote: Is the apt configuration on those systems set to something that is not the default and should be considered as well? How the unstable to testing migration runs work is that they have a testing testbed (with apt pinning making testing the APT::Default-Release without using that configuration option for $REASONS) with unstable added as an available suite. autopkgtest will create apt pinning for only those packages that were requested on the interface (by britney2, our migration software) to be added to testing. That way, we try to see what happens in testing if we would migrate the candidate source package to testing without all the rest of unstable. Be aware, there's also an ugly fall back in autopkgtest where it will remove all the pinning if it can't install all test dependencies with the pinning set-upped as described above, effectively allowing all packages from unstable to be installed. However, for the current use-case that probably happens *before* debvm/mmdebstrap runs, so that detail should not matter. Paul OpenPGP_signature.asc Description: OpenPGP digital signature
Kudos to everyone who has worked on the Debian build & packaging infrastructure
Hi all, I set out this weekend to get up to speed on maintaining and packaging Debian packages and spent many hours reading documentation and playing around with the tooling surrounding building packages. This prompted me to drop in here to say just how darned impressive everything is. The documentation is both helpful and extensive, and the tooling is just amazing. I could give example after example of the things that impressed me, but I'll spare your inboxes and mention just a few: 1) Debmake's ability to produce a mostly working control information file set from any random source tarball seems nearly magical. 2) It's amazing how easy it is to set up a chroot'd build environment and use it to test package builds, which is incredibly essential if for no other reason than because it makes it trivially easy to figure out what dependencies you need to list in your control file. 3) Lintian's ability to find and flag most of the issues that detract from package quality and may prevent it from being accepted is both impressive and useful. I've been maintaining free software for 30 years so I've got a lot of experience with a lot of different tools, and I've rarely encountered anything that is as comprehensive and well-documented as all this stuff is. To be sure, there's a huge amount to learn! But to me, at least, it doesn't seem like that's because there's a lot of cruft. On the contrary, the stuff that's there seems absolutely necessary to maintain high quality in Debian. Thank you so much to everyone who has worked on this. Regards, Jonathan Kamens
Re: debvm for autopkgtests with multiple host?
Hi Johannes,
On Sun, Sep 24, 2023 at 10:27:37AM +0200, Johannes Schauer Marin Rodrigues
wrote:
> There is really not much magic. The core of it is to pass this to your
> mmdebstrap or debvm-create invocation:
>
> --setup-hook='for f in /etc/apt/sources.list /etc/apt/sources.list.d/*
> /etc/apt/preferences.d/*;
> do [ -e "$f" ] && { echo; sed "s| file://| copy://|" "$f";
> } | tee "$1/$f" >&2; done'
> --hook-dir=/usr/share/mmdebstrap/hooks/file-mirror-automount
This sounds simple, but reality is a little more elaborate.
For one thing, there also is
/usr/share/mmdebstrap/hooks/copy-host-apt-sources-and-preferences. This
hook directory is similar but subtly different from the above setup
hook:
* It does not perform the translation of file:// uris into copy://uris.
* It is more accurate in terms of following non-standard locations for
the various configuration items.
* Neither of these clear the sources.list created by mmdebstrap by
default.
* The latter one verifies that you have the same package versions
inside and outside.
Did I accurately represent the differences? Which one would you prefer
in which situation?
Adding any of this to debvm-create will not just work, because
debvm-create also adds the maybe-merged-usr hook and any pass-through
arguments you add come later. Therefore the maybe-merged-usr hook would
come before this hook and it fails if you pass an empty sources.list,
which would be most useful. As a workaround, you may add
--skip=usrmerge -- --hook-dir=/usr/share/mmdebstrap/hooks/maybe-merged-usr
to the debvm-create invocation to reorder the hooks to actually work.
What seems to work is this:
debvm-create
--skip=usrmerge
...
--
--hook-dir=/usr/share/mmdebstrap/hooks/file-mirror-automount
--hook-dir=/usr/share/mmdebstrap/hooks/copy-host-apt-sources-and-preferences
--hook-dir=/usr/share/mmdebstrap/hooks/maybe-merged-usr
""
That final empty string supplies the apt sources. Does this sound about
right? If yes, I'd like to add this as a non-flaky autopkgtest to debvm.
Helmut
Re: Questionable Package Present in Debian: fortune-mod
> Without an ftpteam hat on, but my point of view -- I believe the team > would absolutely reject a package only based on its name (see: > #914179). Not very consistently though: $ apt search penis | grep penis | wc -l 2 And that's without considering that every language has hundreds of synonyms :) -- Salvo Tomaselli "Io non mi sento obbligato a credere che lo stesso Dio che ci ha dotato di senso, ragione ed intelletto intendesse che noi ne facessimo a meno." -- Galileo Galilei http://ltworf.github.io/ltworf/ signature.asc Description: This is a digitally signed message part.
Re: lpr/lpd
Thanks to all of you for your insights and experiences. So I guess those lpr/lpd packages should stay within Debian and should be maintained by the debian-printing team. Thorsten
Re: lintian.debian.org off ?
On Sat, Sep 23, 2023 at 11:02:08AM +0800, Paul Wise wrote: > On Fri, 2023-09-22 at 09:27 +0200, Jérémy Lal wrote: > > > Host lintian.debian.org not found: 3(NXDOMAIN) > > > > is this expected ? > > Yes, it is replaced by the UDD interface: > > https://wiki.debian.org/Services/lintian.debian.org > https://udd.debian.org/lintian/ > > There is no web based location for the description of each tag yet. I don't know if it is just me, but even udd gives me a 500 when I try to check lintian output for any (existing) package. For example: https://udd.debian.org/lintian/?packages=nim Best, Nilesh signature.asc Description: PGP signature
Re: lintian.debian.org off ?
> > > Host lintian.debian.org not found: 3(NXDOMAIN) > > > > > > is this expected ? > > > > Yes, it is replaced by the UDD interface: > > > > https://wiki.debian.org/Services/lintian.debian.org > > https://udd.debian.org/lintian/ Could we please have a HTTP 301 redirect from lintian.debian.org to https://udd.debian.org/lintian/ so people will automatically find the new location? And perhaps a separate redirect from URL https://lintian.debian.org/manual to the new location, wherever the guide now is to be read. I used the old website frequently, and also shared links to the Lintian error explanations there frequently. Currently all links from all HTML reports from Lintian are broken, see https://mariadb-team.pages.debian.net/-/galera-4/-/jobs/4709137/artifacts/debian/output/lintian.html for example. Maybe they should now be updated to be links to UDD instead? Thanks for everyone maintaining our infra and working towards resolving this. > > There is no web based location for the description of each tag yet. > > I don't know if it is just me, but even udd gives me a 500 > when I try to check lintian output for any (existing) package. > > For example: https://udd.debian.org/lintian/?packages=nim I also just get error 500 when trying to look up LIntian errors for my own packages..
Salsa-CI featured on GitLab.com blog
Hi all! I just wanted to share that the story about Salsa-CI was featured a couple days ago at https://about.gitlab.com/blog/2023/09/19/debian-customizes-ci-tooling-with-gitlab/ Personally I think Salsa-CI is extremely useful and pleasant to use and understand, and it surely helped make Debian 12 a very solid and high quality release. Nice to see this getting more visibility! If you want to help boost the visibility, please vote on HackerNews: https://news.ycombinator.com/item?id=37635937 Thanks to Alexander and Joerg for maintaining Salsa itself, and to Santiago, Iñaki and Agustin for being the main drivers of https://salsa.debian.org/salsa-ci-team/pipeline since 2018. - Otto
Bug#1052560: ITP: zsh-antidote -- ZSH plugin manager
Package: wnpp Severity: wishlist Owner: Michael Fladischer X-Debbugs-Cc: debian-devel@lists.debian.org -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 * Package name: zsh-antidote Version : 1.9.2 Upstream Contact: Matt McElheny * URL : https://github.com/mattmc3/antidote * License : Expat Programming Lang: Shell Description : ZSH plugin manager Antidote is a feature-complete Zsh implementation of the legacy Antibody plugin manager, which in turn was derived from Antigen. Antidote not only aims to provide continuity for those legacy plugin managers, but also to delight new users with high-performance, easy-to-use Zsh plugin management. -BEGIN PGP SIGNATURE- iQFPBAEBCgA5FiEEqVSlRXW87UkkCnJc/9PIi5l90WoFAmUQhvUbHGZsYWRpc2No ZXJtaWNoYWVsQGZsYWRpLmF0AAoJEP/TyIuZfdFqMjAH+wZrZwc18FWvmN4LTrgH f81xMrc8IgxkxHq8V6QV+s/eXWkEx/PKqZ7bTTa4+Nl6cCDF4rdJ7LUH1tpw8biE znLKt3PJyU3lOelqMib3dL/G46Rcn2Lc3jpw3jZ3VkD5b4AX7Ia4OplpwRuNJw6x ovBmIti9/nY9mPJYNHRbhLsYRnGbX/8Q3yNiRgNS5iAXzurJwbyC0eqS0A1EP3Xf zRJfYBZyBkrcsfByiBx4KhTpAZ26FHAQKqRkxHsxLz5w07srJyRAEYC6X6rGdmG+ BSKpBZulvcBRmBYuRNBBoSt73yYkTuK/OTKGnZihmxBlXQ0nJGVYyR4ZsuvHzCRs ycw= =hJ3L -END PGP SIGNATURE-
Re: Salsa-CI featured on GitLab.com blog
Am 24.09.23 um 21:26 schrieb Otto Kekäläinen: Hi all! I just wanted to share that the story about Salsa-CI was featured a couple days ago at https://about.gitlab.com/blog/2023/09/19/debian-customizes-ci-tooling-with-gitlab/ Personally I think Salsa-CI is extremely useful and pleasant to use and understand, and it surely helped make Debian 12 a very solid and high quality release. Nice to see this getting more visibility! +1 OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#1052562: ITP: eza -- Modern replacement for ls
Package: wnpp Severity: wishlist Owner: Sylvestre Ledru X-Debbugs-Cc: debian-devel@lists.debian.org * Package name: eza * URL : https://github.com/eza-community/eza * License : MIT it is a replacement of exa (dead upstream). it will break/replace it. exa is an improved file lister with more features and better defaults. It uses colours to distinguish file types and metadata. It knows about symlinks, extended attributes, and Git. And it’s small, fast, and just one single binary.
Re: lintian.debian.org off ?
On 24/09/23 at 12:16 -0700, Otto Kekäläinen wrote: > > I don't know if it is just me, but even udd gives me a 500 > > when I try to check lintian output for any (existing) package. > > > > For example: https://udd.debian.org/lintian/?packages=nim > > I also just get error 500 when trying to look up LIntian errors for my > own packages.. Hi, Sorry about that, it was caused by a change I pushed a few hours ago to https://udd.debian.org/dmd/ It's fixed now Lucas
Re: lpr/lpd
Hi, On 9/23/23 12:10, Paul Wise wrote: You may be thinking of how-can-i-help, which is recommended to every new person who asks about how to contribute to Debian. There is also the older wnpp-alert in devscripts. What I'd like to see is something like Scanning processes... Scanning processor microcode... Scanning linux images... Running kernel seems to be up-to-date. The processor microcode seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. No installed packages are looking for a new maintainer. And ideally, that check would be quick and use information already on the system at this point, maybe a supplemental file under indices/ that can be downloaded during an update. Simon
Re: lpr/lpd
On Mon, 2023-09-25 at 12:08 +0900, Simon Richter wrote: > What I'd like to see is something like > > No installed packages are looking for a new maintainer. That is what how-can-i-help does, except it doesn't print anything when there have been no changes to the status of packages on the system, nor when there are no packages installed that are known to need work. > And ideally, that check would be quick and use information already on > the system at this point, maybe a supplemental file under indices/ that > can be downloaded during an update. Currently how-can-i-help does online checks, but it should be possible to have it add an apt hook to download the necessary files from UDD. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: lpr/lpd
On Mon, 2023-09-25 at 12:08 +0900, Simon Richter wrote: > What I'd like to see is something like ... > No installed packages are looking for a new maintainer. That is what how-can-i-help does, except it doesn't print anything when there have been no changes to the status of packages on the system, nor when there are no packages installed that are known to need helpers. > And ideally, that check would be quick and use information already on > the system at this point, maybe a supplemental file under indices/ that > can be downloaded during an update. The problem with that approach is that the help needed information changes independently to packages, so the information will get very out of date in between point releases, which is why how-can-i-help does online checks. If desired, it would be easy to have an apt update hook that would download the data from UDD so that how-can-i-help can work offline when the network is available. The main problem with how-can-i-help currently is that the maintainer is not currently active in Debian and no-one else is working on it. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: debian/copyright format and SPDX
On Fri, Sep 22, 2023 at 12:58:10PM +0200, Stephan Lachnit wrote: > On Fri, Sep 22, 2023 at 11:11 AM Steve Langasek wrote: > > SPDX defines an xml format only. They lost before they'd even started. > > debian/copyright is supposed to be human-readable first and foremost. XML > > need not apply. > Not true. From [1]: > > Shall be in a human readable form. > > [...] > > Multiple serialization formats may be used to represent the information > > being exchanged. Current supported formats include: > > [...] > > tag:value flat text file as described in this specification So can you tell me where in that specification this "flat text file" format is actually described? The specification is not on the page that includes this quote. The text does not link to the place in the spec where this format is described. The site's search page (because it's reasonable for a spec to require a server-side search engine in order for users to be able to find information in it...) doesn't return any results for the string 'tag:value', and a search for 'tag value' points first to https://spdx.github.io/spdx-spec/v2.3/file-tags/ which describes embedding tags in a source file, not constructing a tag:value file. Frankly, the fact that the SPDX spec itself is as bad as it is should be disqualifying for using any file format specified within. But I would still be willing to give it a fair shake, if I could actually find it. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Re: Questionable Package Present in Debian: fortune-mod
Hi Am 24.09.23 um 18:41 schrieb Salvo Tomaselli: Without an ftpteam hat on, but my point of view -- I believe the team would absolutely reject a package only based on its name (see: #914179). Not very consistently though: $ apt search penis | grep penis | wc -l 2 Can you please clarify what problem you see with these package names? The results I get for this search are: apt search penis | grep --color penis WARNING: apt does not have a stable CLI interface. Use with caution in scripts. libopeniscsiusr/stable,stable 2.1.8-1 amd64 libopeniscsiusr-dbgsym/stable-debug,stable-debug 2.1.8-1 amd64 debug symbols for libopeniscsiusr libopeniscsiusr-dev/stable,stable 2.1.8-1 all I.e. they don't actually contain the word “penis”, they just happen to contain words that when strung together contain these letters in this order. Regards Sven
