sbuild/schroot: need to get .deb files internally downloaded as package dependencies
When building a package with sbuild, some packages (dependencies of package being built) are internally downloaded and installed by apt. After the build finished, schroot is cleaned again (while everything is done in overlay) I need to cache outside of schroot these .deb files that were downloaded by apt. They are supposed to be used for creaing a local partial debian repository, so that the second build will use this local repo instead of internet one. Currently it's done by parsing dependencies and downloading them independently, but I suppose there should be some more straightforward way to get them using sbuild. I investigated and tried several options for sbuild (like --purge-build=never, --purge-deps=never, --purge-session=never), tried to find .deb files during the build using at some stages (--pre-build-commands="ls -la /var/cache/apt/ archives/", --starting-build-commands="ls -la /var/cache/apt/archives/", -- finished-build-commands="ls -la /var/cache/apt/archives/") - but I couldn't see any .deb files inside the schroot. I previously asked in debian-user maillist (https://lists.debian.org/debian-user/2021/10/msg01044.html), but the tips I got (like using caching proxy) are not what I really need. While sbuild already does the thing (downloading/ installing dependencies), I suppose there should be a way to (at least) see the *.deb files downloaded in '/var/cache/apt/archives' inside schroot. -- Uladzimir Bely Promwad Ltd. External service provider of ilbers GmbH Maria-Merian-Str. 8 85521 Ottobrunn, Germany +49 (89) 122 67 24-0 Commercial register Munich, HRB 214197 General Manager: Baurzhan Ismagulov
Re: sbuild/schroot: need to get .deb files internally downloaded as package dependencies
On Thu, Oct 28, 2021 at 11:58:26AM +0300, Uladzimir Bely wrote: > When building a package with sbuild, some packages (dependencies of package > being built) are internally downloaded and installed by apt. After the build > finished, schroot is cleaned again (while everything is done in overlay) > > I need to cache outside of schroot these .deb files that were downloaded by > apt. A caching proxy is indeed a good answer for this question but if for some reason you don't want to use it, you can bind-mount some directory from outside the chroot as /var/cache/apt/archives, e.g. via /etc/schroot/sbuild/fstab, and set $apt_keep_downloaded_packages=1. -- WBR, wRAR signature.asc Description: PGP signature
Bug#997989: ITP: golang-github-hodgesds-perf-utils -- Perf Utilities for Go
X-Debbugs-CC: debian-devel@lists.debian.org, debian...@lists.debian.org Package: wnpp Severity: wishlist Owner: Benjamin Drung * Package name : golang-github-hodgesds-perf-utils Version : 0.3.1-1 Upstream Author : Daniel Hodges * URL : https://github.com/hodgesds/perf-utils * License : Expat Programming Lang: Go Description : Perf Utilities for Go This package is a Go library for interacting with the perf subsystem in Linux. This library allows you to do things like see how many CPU instructions a function takes (roughly), profile a process for various hardware events, and other interesting things. Note that because the Go scheduler can schedule a goroutine across many OS threads it becomes rather difficult to get an exact profile of an individual goroutine. However, a few tricks can be used; first a call to runtime.LockOSThread (https://golang.org/pkg/runtime/#LockOSThread) to lock the current goroutine to an OS thread. Second a call to unix.SchedSetaffinity (https://godoc.org/golang.org/x/sys/unix#SchedSetaffinity), with a CPU set mask set. Note that if the pid argument is set 0 the calling thread is used (the thread that was just locked). Before using this library you should probably read the perf_event_open man page which this library uses heavily. See the kernel guide https://perf.wiki.kernel.org/index.php/Tutorial for a tutorial how to use perf and some of the limitations. . If you are looking to interact with the perf subsystem directly with perf_event_open syscall than this library is most likely for you. A large number of the utility methods in this package should only be used for testing and/or debugging performance issues. This is due to the nature of the Go runtime being extremely tricky to profile on the goroutine level, with the exception of a long running worker goroutine locked to an OS thread. Eventually this library could be used to implement many of the features of perf but in pure Go. Currently this library is used in node_exporter (https://github.com/prometheus/node_exporter) as well as perf_exporter (https://github.com/hodgesds/perf_exporter), which is a Prometheus exporter for perf related metrics. This package was vendored in prometheus-node-exporter and is packages now separately. I will maintain this package as part of the Go team. -- Benjamin Drung Senior DevOps Engineer and Debian & Ubuntu Developer Compute Platform Operations 1&1 IONOS SE | Greifswalder Str. 207 | 10405 Berlin | Deutschland E-Mail: benjamin.dr...@ionos.com | Web: www.ionos.de Hauptsitz Montabaur, Amtsgericht Montabaur, HRB 24498 Vorstand: Hüseyin Dogan, Dr. Martin Endreß, Claudia Frese, Henning Kettler, Arthur Mai, Matthias Steinberg, Achim Weiß Aufsichtsratsvorsitzender: Markus Kadelke Member of United Internet
Short description disapearing from package tracker
Short descriptions disappeard after a package update. Example, just try some. https://www.gezapig.nl/Software/MaintGroup/Debian_Haskell_Group2021.html Is that suspected? Maybe handy. https://www.gezapig.nl/Free.html
Re: Short description disapearing from package tracker
On Thu, Oct 28, 2021 at 12:50:05PM +0200, Ben Tris wrote: > Short descriptions disappeard after a package update. Do you have any specific examples? > Example, just try some. > https://www.gezapig.nl/Software/MaintGroup/Debian_Haskell_Group2021.html Both packages I tried work as expected. Note that source packages don't have descriptions, only binary ones do. -- WBR, wRAR signature.asc Description: PGP signature
Re: sbuild/schroot: need to get .deb files internally downloaded as package dependencies
Hi, Quoting Uladzimir Bely (2021-10-28 10:58:26) > When building a package with sbuild, some packages (dependencies of package > being built) are internally downloaded and installed by apt. After the build > finished, schroot is cleaned again (while everything is done in overlay) > > I need to cache outside of schroot these .deb files that were downloaded by > apt. They are supposed to be used for creaing a local partial debian > repository, so that the second build will use this local repo instead of > internet one. > > Currently it's done by parsing dependencies and downloading them > independently, but I suppose there should be some more straightforward way to > get them using sbuild. > > I investigated and tried several options for sbuild (like > --purge-build=never, > --purge-deps=never, --purge-session=never), tried to find .deb files during > the build using at some stages (--pre-build-commands="ls -la /var/cache/apt/ > archives/", --starting-build-commands="ls -la /var/cache/apt/archives/", -- > finished-build-commands="ls -la /var/cache/apt/archives/") - but I couldn't > see any .deb files inside the schroot. > > I previously asked in debian-user maillist > (https://lists.debian.org/debian-user/2021/10/msg01044.html), but the tips I > got (like using caching proxy) are > not what I really need. While sbuild already does the thing (downloading/ > installing dependencies), I suppose there should be a way to (at least) see > the *.deb files downloaded in '/var/cache/apt/archives' inside schroot. sbuild maintainer here. You have not explained why you don't want to use a caching proxy like apt-cacher-ng. But using a caching proxy is also what I would suggest you use. If you don't want to use that, here is how you download only the packages you need to satisfy the build depenencies of a particular source package: cat << END > /tmp/apt.conf Apt::Architecture "amd64"; Dir "/tmp/apt"; Dir::Etc::trusted "/etc/apt/trusted.gpg"; Dir::Etc::trustedparts "/etc/apt/trusted.gpg.d/"; Apt::Get::Download-Only true; Apt::Install-Recommends false; END mkdir -p /tmp/apt/etc/apt/apt.conf.d/ /tmp/apt/etc/apt/sources.list.d/ /tmp/apt/var/lib/dpkg /tmp/apt/var/cache/apt/archives/partial /tmp/apt/etc/apt/preferences.d/ touch /tmp/apt/var/lib/dpkg/status cat << END > /tmp/apt/etc/apt/sources.list deb http://deb.debian.org/debian/ unstable main deb-src http://deb.debian.org/debian/ unstable main END APT_CONFIG=/tmp/apt.conf apt update APT_CONFIG=/tmp/apt.conf apt-get dist-upgrade APT_CONFIG=/tmp/apt.conf apt-get build-dep hello Now all your packages to build src:hello are in /tmp/apt/var/cache/apt/archives. No need to waste CPU cycles on a full package build if all you want is to download a certain set of binary packages. Thanks! cheers, josch signature.asc Description: signature
Re: Re: sbuild/schroot: need to get .deb files internally downloaded as package dependencies
Thanks for the answer > You have not explained why you don't want to use a caching proxy like apt- cacher-ng. The question appeared while developing for Isar, a tool build create debian- based distro for embedded systems. In two words, it's similar to Buildroot, Yocto and others, but it uses official Debian packages plus allows to build own customer packages. Currently it uses some own set of scripts to download dependencies, install them is build chroot and build the package. Every .deb that was downloaded during system build is cached internally. So, for the second build it's possible to create a local repo from these stored files and do everything (e.g bootstrapping, building custom packages, rootfs generation) not from some remote repo (e.g. http://ftp.debian.org), but from this local one (file:// foo). I'm currently trying to switch our custom build system to sbuild one, but I faced a problem that the dependencies downloaded by sbuild scripts are not available in form of .deb files. Anyway, your advice uses a bit more native approach that is currently used in Isar, so I'll look at it (or something similar). Also, in parallel thread I was adviced to use $apt_keep_downloaded_packages=1 option that might be also useful for me. -- Uladzimir Bely Promwad Ltd. External service provider of ilbers GmbH Maria-Merian-Str. 8 85521 Ottobrunn, Germany +49 (89) 122 67 24-0 Commercial register Munich, HRB 214197 General Manager: Baurzhan Ismagulov
Bug#998019: ITP: golang-github-go-kit-log -- minimal and extensible structured logger
X-Debbugs-CC: debian-devel@lists.debian.org, debian...@lists.debian.org Package: wnpp Severity: wishlist Owner: Benjamin Drung * Package name : golang-github-go-kit-log Version : 0.2.0-1 Upstream Author : Go kit * URL : https://github.com/go-kit/log * License : Expat Programming Lang: Go Description : minimal and extensible structured logger (Go library) The log package provides a minimal interface for structured logging in services. It may be wrapped to encode conventions, enforce type-safety, provide leveled logging, and so on. It can be used for both typical application log events, and log-structured data streams. . Structured logging is, basically, conceding to the reality that logs are data, and warrant some level of schematic rigor. Using a stricter, key/value-oriented message format for our logs, containing contextual and semantic information, makes it much easier to get insight into the operational activity of the systems we build. Consequently, package log is of the strong belief that "the benefits of structured logging outweigh the minimal effort involved (https://www.thoughtworks.com/radar/techniques/structured-logging)". This library was split from golang-github-go-kit-kit and is used by Prometheus node exporter. -- Benjamin Drung Senior DevOps Engineer and Debian & Ubuntu Developer Compute Platform Operations 1&1 IONOS SE | Greifswalder Str. 207 | 10405 Berlin | Deutschland E-Mail: benjamin.dr...@ionos.com | Web: www.ionos.de Hauptsitz Montabaur, Amtsgericht Montabaur, HRB 24498 Vorstand: Hüseyin Dogan, Dr. Martin Endreß, Claudia Frese, Henning Kettler, Arthur Mai, Matthias Steinberg, Achim Weiß Aufsichtsratsvorsitzender: Markus Kadelke Member of United Internet
Re: Re: sbuild/schroot: need to get .deb files internally downloaded as package dependencies
Hello. Thanks for the idea with $apt_keep_downloaded_packages=1. I'll try to play with it. Previously I tried to mount external directory to /var/cache/apt/archives inside schroot, but finally found it almost empty (only some .lock file was there, AFAIR). As I understand, that it was really used, but later everything downloaded was deleted. I hope, the option above will help me with it. -- Uladzimir Bely Promwad Ltd. External service provider of ilbers GmbH Maria-Merian-Str. 8 85521 Ottobrunn, Germany +49 (89) 122 67 24-0 Commercial register Munich, HRB 214197 General Manager: Baurzhan Ismagulov
Work-needing packages report for Oct 29, 2021
The following is a listing of packages for which help has been requested through the WNPP (Work-Needing and Prospective Packages) system in the last week. Total number of orphaned packages: 1263 (new: 1) Total number of packages offered up for adoption: 198 (new: 1) Total number of packages requested help for: 60 (new: 0) Please refer to https://www.debian.org/devel/wnpp/ for more information. The following packages have been orphaned: mp3check (#997973), orphaned today Description: tool to check mp3 files for consistency Installations reported by Popcon: 364 Bug Report URL: https://bugs.debian.org/997973 1262 older packages have been omitted from this listing, see https://www.debian.org/devel/wnpp/orphaned for a complete list. The following packages have been given up for adoption: diod (#997964), offered yesterday Description: I/O forwarding server for 9P Installations reported by Popcon: 19 Bug Report URL: https://bugs.debian.org/997964 197 older packages have been omitted from this listing, see https://www.debian.org/devel/wnpp/rfa_bypackage for a complete list. For the following packages help is requested: apache2 (#910917), requested days ago Description: Apache HTTP Server Reverse Depends: apache2 apache2-ssl-dev apache2-suexec-custom apache2-suexec-pristine backuppc bfh-container-server courier-webadmin cvsweb debbugs-web doc-central (139 more omitted) Installations reported by Popcon: 98292 Bug Report URL: https://bugs.debian.org/910917 aufs (#963191), requested 495 days ago Description: driver for a union mount for Linux filesystems Reverse Depends: fsprotect Installations reported by Popcon: 10261 Bug Report URL: https://bugs.debian.org/963191 autopkgtest (#846328), requested 1793 days ago Description: automatic as-installed testing for Debian packages Reverse Depends: debci-worker sbuild-qemu Installations reported by Popcon: 1222 Bug Report URL: https://bugs.debian.org/846328 balsa (#642906), requested 3686 days ago Description: An e-mail client for GNOME Reverse Depends: balsa Installations reported by Popcon: 649 Bug Report URL: https://bugs.debian.org/642906 cargo (#860116), requested 1661 days ago Description: Rust package manager Reverse Depends: dh-cargo rust-all Installations reported by Popcon: 2637 Bug Report URL: https://bugs.debian.org/860116 courier (#978755), requested 301 days ago Description: Courier mail server Reverse Depends: courier-faxmail courier-filter-perl courier-imap courier-ldap courier-mlm courier-mta courier-pcp courier-pop courier-webadmin couriergrey (3 more omitted) Installations reported by Popcon: 935 Bug Report URL: https://bugs.debian.org/978755 cron (#984736), requested 235 days ago Description: new maintainer need Reverse Depends: apticron autolog backintime-common btrfsmaintenance buildd checksecurity clamtk cricket email-reminder exim4-base (20 more omitted) Installations reported by Popcon: 207259 Bug Report URL: https://bugs.debian.org/984736 cyrus-imapd (#921717), requested 993 days ago Description: Cyrus mail system - IMAP support Reverse Depends: cyrus-admin cyrus-caldav cyrus-clients cyrus-dev cyrus-imapd cyrus-murder cyrus-nntpd cyrus-pop3d cyrus-replication Installations reported by Popcon: 427 Bug Report URL: https://bugs.debian.org/921717 cyrus-sasl2 (#799864), requested 2227 days ago Description: authentication abstraction library Reverse Depends: 389-ds-base adcli autofs-ldap cyrus-caldav cyrus-clients cyrus-common cyrus-dev cyrus-imapd cyrus-imspd cyrus-murder (79 more omitted) Installations reported by Popcon: 206728 Bug Report URL: https://bugs.debian.org/799864 dbab (#947550), requested 670 days ago Description: dnsmasq-based ad-blocking using pixelserv Installations reported by Popcon: 5 Bug Report URL: https://bugs.debian.org/947550 debtags (#962579), requested 505 days ago Description: Debian Package Tags support tools Reverse Depends: packagesearch Installations reported by Popcon: 1497 Bug Report URL: https://bugs.debian.org/962579 dee (#831388), requested 1931 days ago Description: model to synchronize mutiple instances over DBus Reverse Depends: dee-tools gir1.2-dee-1.0 gir1.2-unity-7.0 libdee-dev libunity-dev libunity-protocol-private0 libunity-tools libunity9 zeitgeist-core Installations reported by Popcon: 38590 Bug Report URL: https://bugs.debian.org/831388 developers-reference (#759995), requested 2616 d
Re: Short description disapearing from package tracker
Sorry, for instance. haskell-hopenpgp-tools > Op 28-10-2021 13:12 schreef Andrey Rahmatullin : > > > On Thu, Oct 28, 2021 at 12:50:05PM +0200, Ben Tris wrote: > > Short descriptions disappeard after a package update. > Do you have any specific examples? > > > Example, just try some. > > https://www.gezapig.nl/Software/MaintGroup/Debian_Haskell_Group2021.html > Both packages I tried work as expected. > Note that source packages don't have descriptions, only binary ones do. > > -- > WBR, wRAR
Re: Re: sbuild/schroot: need to get .deb files internally downloaded as package dependencies
On Thu, Oct 28, 2021 at 07:11:05PM +0300, Uladzimir Bely wrote: > Thanks for the answer > > > You have not explained why you don't want to use a caching proxy like apt- > cacher-ng. > > The question appeared while developing for Isar, a tool build create debian- > based distro for embedded systems. In two words, it's similar to Buildroot, > Yocto and others, but it uses official Debian packages plus allows to build > own customer packages. > > Currently it uses some own set of scripts to download dependencies, install > them is build chroot and build the package. Every .deb that was downloaded > during system build is cached internally. So, for the second build it's > possible to create a local repo from these stored files and do everything > (e.g > bootstrapping, building custom packages, rootfs generation) not from some > remote repo (e.g. http://ftp.debian.org), but from this local one (file:// > foo). > I'm currently trying to switch our custom build system to sbuild one, but I > faced a problem that the dependencies downloaded by sbuild scripts are not > available in form of .deb files. If it does not need to be sbuild, my pbuilder setup has an hook that allows me to have a local directory as apt repo for pbuilder. Any deb placed in this directory will be available during pbuilder build... Kind of https://wiki.debian.org/PbuilderTricks#How_to_include_local_packages_in_the_build, but my setup uses just one directory, not a per-project one.. -- tobi
Re: Short description disapearing from package tracker
On Fri, Oct 29, 2021 at 05:50:02AM +0200, Ben Tris wrote: > Sorry, for instance. > > haskell-hopenpgp-tools Sure, are you saying it had a description before? Because there is no binary package with this name. -- WBR, wRAR signature.asc Description: PGP signature
