Hi fellow developers,
(modified resend, as first attempt didn't arrive)
please allow me to open a can of worms. Package removal from unstable.
Deciding when it is time to remove a package from unstable is difficult.
There may be users still and it is unclear whether keeping the package
imposes a cost. In this mail I want to argue for more aggressive package
removal and seek consensus on a way forward.
What does a package cost?
There are various QA-related teams looking at packages from other
maintainers. When it trips a check, that often incurs time from some QA
person investigating a report or failure. Examples:
* Lucas Nussbaum, Santiago Vila and a few more regularly perform
archive rebuilds and report failures. They review a significant
fraction of reports before sending, but there also is CPU resources
for performing all those builds involved.
* Reproducible builds folks actively investigate packages that fail to
build reproducibly (or fail to build in the first place) and file bug
reports often accompanied by patches.
* Some cross build folks regularly send patches for cross build
failures and also occasionally real FTBFS. About one such patch per
month gets closed by ftp master package removal without ever having
been applied.
* DEP17 support folks send patches. Many of the remaining packages have
accumulated RC bugs such as FTBFS.
* As packages fail to migrate to testing for a long time, a release
team member eventually looks at the package.
* There are many more people doing various forms of QA and sending
patches.
By virtue of being part of Debian, a package receives attention by a
significant number of developers. Assigning a number is non-trivial, but
we can say for sure that it is significant. Especially developers doing
/usr-move NMUS (e.g. Chris Hofstaedler and Michael Biebl) now wonder
how much effort to put into the remaining packages. Removing more
packages would reduce the number of NMUs required there.
I suggest that we are keeping too many packages in unstable and that
they incur a non-trivial cost. It is not clear at all where to draw the
line, but maybe we can shift the line more towards removal?
What does package removal cost?
Before a package can be removed, it needs to be reviewed for reverse
dependencies and if there are any, they have to be switched to something
else or removed as well. The actual package removal first and foremost
is carried out by a ftp master. There may still be people actively using
the package and they have to find some replacement for their task at
hand. Sometimes, packages are reintroduced. Doing so incurs a pass
through NEW (and review by the ftp team). Closed and archived bugs need
to be reopened and reviewed. Sometimes, it is quicker to just NMU a
particular problem that to review a package for removal.
When to remove a package?
I looked at UDD and came up with a suggested query.
SELECT s.source, s.maintainer, b.id, b.title
FROM sources AS s JOIN bugs AS b ON s.source = b.source
WHERE s.release = 'sid'
AND NOT exists(SELECT 1 FROM sources AS t WHERE s.source = t.source AND
t.release IN ('bookworm', 'trixie'))
AND NOT exists(SELECT 1 FROM key_packages AS k WHERE k.source =
s.source)
AND b.affects_unstable = true
AND b.severity >= 'serious'
AND b.last_modified <= now() - interval '1 year'
AND s.source NOT IN ('check-all-the-things', 'debbugs', 'firefox',
'gcc-snapshot', 'gitlab', 'hurd', 'openjdk-19', 'openjdk-20',
'singularity-container', 'virtualbox', 'wine-development')
ORDER BY s.source, b.id;
A very similar query is achievable using the web interface:
https://udd.debian.org/bugs/?release=sid¬bookworm=only¬trixie=only&merged=ign&keypackages=ign&flastmod=ign&flastmodval=366&rc=1&sortby=id&sorto=asc&format=html#results
Human readable explanation:
* Packages in sid
* not in bookworm or trixie
* not a key package
* affected by an RC bug that has been last modified more than a year ago
* not among a few selected exceptions
These results yield 360 or 351 bugs respectively. I am including a
package list from the SQL for those who prefer following offline, but
including more would trip the spam filter.
What do you think about the proposed criteria and suggested set of
source packages? Is it reasonable to remove these packages from
unstable? In a sense, it is extending the idea of the testing auto
remover to unstable. Similarly, a package can be temporarily saved by
mailing the respective bug.
Let us assume that we agree on there being a set of packages to be
removed. What is a reasonable process? Is it ok to just file a pile of
RoQA bugs or is more warning warranted? Should we maybe use a process
similar to salvaging where there is an "ITR" (intent to remove) bug that
is reassigned to ftp after a reasonable timeout?
My personal motivation for looking into this actually is the /usr-move
work and the cross build support work. Please do consider me biased.
Helmut
aboot
adacontrol
aiocoap
aiorwlock
airport-utils
android-platform-external-doclava
anki
ants
apertium-crh-tur
apertium-cy-en
apertium-kaz-tat
apertium-mlt-ara
apertium-sme-nob
arrayfire
asis
aws-shell
backdoor-factory
bdfproxy
beanbag
binutils64
boinc-app-eah-brp
bookkeeper
breezy-loom
bwctl
ca-cacert
chibi-scheme
cpl-plugin-xshoo
crazydiskinfo
cycle-quotes
debian-timeline
devpi-common
dfvfs
dirspec
dltlyse
dogecoin
drmips
dune-pdelab
dxvk
dynamic-motd
ecb
ecere-sdk
eclipse-cdt
eclipse-collections
eclipse-tracecompass
effcee
elektra
enigmail
epigrass
evqueue-core
fakeroot-ng
fava
flask-testing
flightgear-phi
fonts-alegreya-sans
fonts-pecita
freeart
frozen-flask
fruit
gandi-cli
gems
ggcov
giblib
gigedit
github-backup
gli
gnat-gps
golang-github-bsm-pool
golang-github-coreos-go-tspi
golang-github-crewjam-saml
golang-github-form3tech-oss-jwt-go
golang-github-go-redis-redis
golang-github-jesseduffield-asciigraph
golang-github-jesseduffield-gocui
golang-github-jesseduffield-pty
golang-github-jesseduffield-roll
golang-github-jesseduffield-rollrus
golang-github-jesseduffield-termbox-go
golang-github-jesseduffield-yaml
golang-github-manyminds-api2go
golang-github-minio-cli
golang-github-tsenart-tb
golang-gopkg-mcuadros-go-syslog.v2
gr-dab
graftcp
greenbone-security-assistant
guacamole-server
haskell-cborg
haskell98-tutorial
hipspy
hydroffice.bag
i2masschroq
ifscheme
ignore-me
imdbpy
info-beamer
iptables-converter
irssi-plugin-robustirc
itop
jajuk
jodd
jquery-simpletreemenu
jsjac
jsunit
kryo-serializers
laminar
leaflet-image
leap-archive-keyring
libapache2-mod-defensible
libaws
libdevel-bt-perl
libhandy
libmonitoring-availability-perl
libneo4j-client
liboqs
libpam-ssh
libpthread-workqueue
libretro-mupen64plus
libzorpll
lilyterm
literki
lives
lsdb
mahimahi
mate-optimus
matrix-sydent
mbdyn
medialibrary
minify-maven-plugin
mixer.app
mldemos
mlton
mlucas
mod-gnutls
monkeysphere
moviepy
mozillavpn
mppenc
mpqc3
multiplex
mutextrace
mxt-app
myhdl
mysql-8.0
mysql-connector-python
nautilus-scripts-manager
navi2ch
nbibtex
node-lockfile
node-matrix-js-sdk
node-mermaid
node-node-forge
node-node-localstorage
node-puppeteer
node-trust-webcrypto
notcurses
nuitka
nvidia-graphics-drivers-legacy-340xx
nvidia-graphics-drivers-legacy-390xx
nvidia-graphics-drivers-tesla-418
nvidia-graphics-drivers-tesla-450
nvidia-graphics-drivers-tesla-460
obs-ptz
obs-websocket
octave-database
octave-tisean
octicons
openexr-viewers
opengrm-ngram
openmx
openrocket
openscap-daemon
origami
ortools
pafy
pdfrw
perl-doc-html
php-fig-link-util
php-horde-mongo
php-net-ipv6
php-sabre-event
php-sabre-http
php-sabre-uri
php-sabre-xml
php-sparkline
php-webimpress-safe-writer
picprog
platformio
pluginhook
powermock
privbind
profitbricks-sdk-python
pstack
pulseaudio-dlna
pxe-kexec
py-lz4framed
pyfr
pympler
python-dugong
python-gear
python-html-sanitizer
python-opentracing
python-pyramid-multiauth
python-pyramid-zcml
qiskit-aer
qiskit-ibmq-provider
qmenu
rdup
resteasy
resvg
ricochet-im
ruby-coffee-script
ruby-diaspora-federation
ruby-google-cloud-translate
ruby-jekyll-coffeescript
ruby-jekyll-remote-theme
ruby-omniauth-bitbucket
ruby-riddle
ruby-uglifier
ruby-upr
ruby-yaml-db
rust-fxprof-processed-profile
rust-hdrhistogram
rust-librespot-protocol
rust-rspotify
rust-rust-code-analysis
rust-rust-code-analysis-cli
rust-tcmalloc
rust-wasm-bindgen-webidl
scheme2c
shotdetect
snort
socket-activate
soundmanager2
spdx-licenses
spyder-memory-profiler
spyder-reports
squid-deb-proxy
stylish-haskell
subvertpy
swfmill
syncmaildir
sysrepo
telegram-cli
telegram-purple
termtris
thawab
tldjs
tomcatjss
tpot
ttyd
twofish
tycho
umlet
vagrant-bindfs
vg
vibe.d
vlogger
vncterm
vowpal-wabbit
w3-dtd-mathml
watson
win-iconv
xjig
xmms2-scrobbler
xqilla
xrstools
xtide
yap
zeek
zeek-aux
zmodemjs
