Bug#998883: ITP: zsh-histdb -- scalable command history with git versioning and sync across hosts
Package: wnpp Severity: wishlist Owner: Daniel Gröber * Package name: zsh-histdb Version : git HEAD Upstream Author : Tom Hinton * URL : https://github.com/larkery/zsh-histdb * License : MIT Programming Lang: ZSH Description : scalable command history with git versioning and sync across hosts This is a ZSH extension which stores the command history in an sqlite database instead of a text file while adding a number of extremely useful bits of metadata missing from zsh's native history file. A very useful aspect of this approach is that querying the history scales significantly better since sqlite can stream (search) results off the disk rather than having to read everything into memory first as zsh's native history support does. While the database file is binary and thus would ordinarily be hard to keep under version control zsh-histdb provides an automatic merge driver for use with git. This allows not only version control but naturally also syncing history across hosts with regular git push/pull. - I believe the functionality provided by this package is unique in Debian - I plan to maintain this package myself but require a sponsor. --Daniel
Re: merged-/usr transition: debconf or not?
On Mon, Nov 08, 2021 at 12:56:49PM +0100, Marco d'Itri wrote: > On Nov 08, Simon Richter wrote: > > Right now, it is sufficient to preseed debconf to disallow the usrmerge > > package messing with the filesystem tree outside dpkg. Managed installations > > usually have a ready-made method to do so. > This is not really relevant, since the conversion is mandatory. > The CTTE stated that the only exception is "Testing and QA systems > should be able to avoid this transition, but if they do, they cannot be > upgraded beyond Debian 12", and my plan is to arrange for this with > a flag file. As I see it the CTTE decision effectively allows the transition to be deferred until the moment you want to upgrade to 13. Ideally the transition is performed already in the 11→12 upgrade automatically for you, but you could prevent that automatism and do it manually someday while you have 12 installed (as no 12 package can depend on merged /usr as it would not be installable on upgrade from 11 and/or executable on buildds/testing/qa systems at the least). So, wouldn't it make sense to go with an (extreme) low priority debconf question defaulting to 'yes, convert now' which [I think] non-experts aren't bothered with and users/systems wanting to opt-out for the moment (like buildds) have a standard way of preseeding rather than inventing a homegrown flag-file and associated machinery? As a bonus, if I had previously decided to forgo the automatic transition for whatever reason (lets say to test build packages on that box) I also have a standard way of triggering the conversion by calling dpkg-reconfigure on usrmerge at leisure before the 13 upgrade. Best regards David Kalnischkies signature.asc Description: PGP signature
Re: merged-/usr transition: debconf or not?
(Speaking only for myself, not the TC as a whole - but I wrote the first draft of what became the TC resolution we're discussing.) On Tue, 09 Nov 2021 at 15:19:53 +0100, David Kalnischkies wrote: > On Mon, Nov 08, 2021 at 12:56:49PM +0100, Marco d'Itri wrote: > > On Nov 08, Simon Richter wrote: > > > Right now, it is sufficient to preseed debconf to disallow the usrmerge > > > package messing with the filesystem tree outside dpkg. Managed > > > installations > > > usually have a ready-made method to do so. > > This is not really relevant, since the conversion is mandatory. > > The CTTE stated that the only exception is "Testing and QA systems > > should be able to avoid this transition, but if they do, they cannot be > > upgraded beyond Debian 12", and my plan is to arrange for this with > > a flag file. For reference, the wording in the TC resolution is: We anticipate that during the development cycle that will lead to Debian 12, it is likely to be useful for testing and QA infrastructure (such as autopkgtest, piuparts and/or reproducible-builds) to be able to produce an installation of Debian testing/unstable that is not merged-/usr, in order to be able to verify that packages targeted for inclusion in Debian 12 can still be installed and/or built successfully in a non-merged-/usr environment during partial upgrades. As a result, we recommend that if there is an automatic migration from non-merged-/usr to merged-/usr, it should be possible to prevent that migration. **However, systems where that migration has been prevented are not required to be fully-supported**, and in particular, upgrading them to Debian 13 or to the state of testing/unstable during development of Debian 13 should be considered unsupported. (emphasis added in this mail, not present in the TC resolution) As Marco implies, it was not my intention to have this as a general-purpose way for change-averse sysadmins to avoid or defer the automatic transition. I had intended this to be specifically there as a way to do the testing and QA that will ensure that the transition can go as smoothly as possible (for example, reproducible-builds does one build with merged-/usr and one with unmerged /usr, and this should continue, otherwise we'll lose the ability to detect packages that build differently in those two scenarios, which in practice is strongly correlated with packages that won't work on non-merged-/usr systems if they have been built on merged-/usr). I can see that if people are developing dpkg enhancements that allow it to reconcile its view of the filesystem with merged-/usr by having a record of the directories that "should be" merged (perhaps a --paths-aliased analogous to --path-include?), then they would also benefit from the ability to construct up-to-date systems that have and haven't undergone this transition, so they can compare the two. > As I see it the CTTE decision effectively allows the transition to be > deferred until the moment you want to upgrade to 13. I think you mean: until the moment you want to upgrade to testing after Debian 12 release day. That's not Debian 13 *yet*, although you could think of it as some sort of Debian 13~alpha, and the TC resolution allows packages in testing/unstable to start requiring merged-/usr immediately after Debian 12 is released. I tried to make sure the TC resolution distinguished carefully between Debian stable releases, and the states of testing/unstable that will exist at particular points in the stable release cycle. > So, wouldn't it make sense to go with an (extreme) low priority debconf > question defaulting to 'yes, convert now' which [I think] non-experts > aren't bothered with and users/systems wanting to opt-out for the moment > (like buildds) have a standard way of preseeding rather than inventing a > homegrown flag-file and associated machinery? Speaking only for myself and not for the TC, I think a debconf question would be OK as an implementation of this, but the debconf question should indicate that the result of opting out is an unsupported system. I would personally see this as a bit like using dpkg --path-exclude, or downgrading packages: it's allowed, but it might break things immediately, or it might have weird side-effects later on (even after the configuration change has been reverted). > As a bonus, if I had previously decided to forgo the automatic > transition for whatever reason (lets say to test build packages on that > box) I also have a standard way of triggering the conversion by calling > dpkg-reconfigure on usrmerge at leisure before the 13 upgrade. I had intended this to be for the class of systems that you would expect to discard and re-bootstrap rather than upgrading (chroots, lxc/Docker containers, virtual machines, etc. used for autopkgtest, piuparts, reproducible-builds, etc.), where a way to undo the opt-out isn't really necessary because the system is treated as disposabl
Re: merged-/usr transition: debconf or not?
On Tue, Nov 09, 2021 at 03:21:25PM +, Simon McVittie wrote: > > As I see it the CTTE decision effectively allows the transition to be > > deferred until the moment you want to upgrade to 13. > > I think you mean: until the moment you want to upgrade to testing after > Debian 12 release day. That's not Debian 13 *yet*, although you could Yes, I meant that indeed… should have used codenames after all. > > So, wouldn't it make sense to go with an (extreme) low priority debconf > > question defaulting to 'yes, convert now' which [I think] non-experts > > aren't bothered with and users/systems wanting to opt-out for the moment > > (like buildds) have a standard way of preseeding rather than inventing a > > homegrown flag-file and associated machinery? > > Speaking only for myself and not for the TC, I think a debconf question > would be OK as an implementation of this, but the debconf question should > indicate that the result of opting out is an unsupported system. Sure. (Minus that for 12 it is technically still supported as long as it remains 12, but those who have to know will know that and everyone else is better of following the default anyhow) > I had intended this to be for the class of systems that you would expect > to discard and re-bootstrap rather than upgrading (chroots, lxc/Docker > containers, virtual machines, etc. used for autopkgtest, piuparts, > reproducible-builds, etc.), where a way to undo the opt-out isn't really > necessary because the system is treated as disposable. That is likely what happens to most of them, but as we support running the merge somewhere between a few years ago and first unpack of a trixie package anyhow I don't see the harm of having an official opt-out of the opt-out as long as it happens in time. (Perhaps it comes with the job as apt maintainer, but I don't "discard and redo" systems or even chroots… upgrades until hardware failure… my current build chroots are from 2013. So I can totally see me opt-out first and later in… although probably more with apt_preferences) Best regards David Kalnischkies signature.asc Description: PGP signature
Bug#998905: ITP: scikit-misc -- Miscellaneous tools for scientific computing
Package: wnpp Severity: wishlist Owner: Diane Trout X-Debbugs-Cc: debian-devel@lists.debian.org * Package name: scikit-misc Version : 0.1.4 Upstream Author : Hassan Kibirige * URL : https://github.com/has2k1/scikit-misc * License : BSD with non-endorsement clause Programming Lang: Python Description : Miscellaneous tools for scientific computing Miscellaneous tools for data analysis and scientific computing. This package provides a LOESS (locally estimated scatterplot smoothing) module Currently scanpy depends on it which is a widely used bioinformatics toolkit, and LOESS does look like a generally useful technique. I do wish upstream had a better name and description though. I'd like to add it to the python modules team because the python team periodically applies automated packaging updates to python libraries. Diane
Bug#998906: ITP: mdit-py-plugins -- collection of core plugins for python3-markdown-it
Package: wnpp Severity: wishlist Owner: Emmanuel Arias X-Debbugs-Cc: debian-devel@lists.debian.org, eam...@yaerobi.com * Package name: mdit-py-plugins Version : 0.2.8 Upstream Author : Chris Sewell * URL : https://github.com/executablebooks/mdit-py-plugins * License : expat Programming Lang: Python Description : collection of core plugins for python3-markdown-it this package include a collection of core plugins used in the python-markdown-it package. mdit-py-plugins is a build dependency of markdown-it-py and this last one is necessary for myst-parser packaging. I plan maintain it under DPT umbrella.
Re: merged-/usr transition: debconf or not?
On 09.11.21 19:01, David Kalnischkies wrote: (Minus that for 12 it is technically still supported as long as it remains 12, but those who have to know will know that and everyone else is better of following the default anyhow) I'm worried that by saying that unmerged is still supported in 12, we open a can of worms and just punt this down to yet another release cycle. E.g., what exactly does this mean for backports? If I provide a backport from bookworm+1, I don't think it's feasible (and intended) to roll back any changes related to usrmerge, so I think backports should be able to assume a bookworm system is usrmerged? Simon, could you clarify your thoughts regarding this aspect? OpenPGP_signature Description: OpenPGP digital signature
Re: merged-/usr transition: debconf or not?
On Tue, Nov 09, 2021 at 08:19:40PM +0100, Michael Biebl wrote: > I'm worried that by saying that unmerged is still supported in 12, we open a > can of worms and just punt this down to yet another release cycle. No, unmerged will not be supported in 12. Having the ability to create something does not make it supported. > E.g., what exactly does this mean for backports? Stuff from backports is post-release, so requires a merged system. Bastian -- Star Trek Lives!
Re: merged-/usr transition: debconf or not?
On Tue, 09 Nov 2021 at 20:27:24 +0100, Bastian Blank wrote: > On Tue, Nov 09, 2021 at 08:19:40PM +0100, Michael Biebl wrote: > > I'm worried that by saying that unmerged is still supported in 12, we open a > > can of worms and just punt this down to yet another release cycle. > > No, unmerged will not be supported in 12. Having the ability to create > something does not make it supported. That was my intention, yes. My intention when drafting the TC resolution was that unmerged-/usr Debian 12 'bookworm' systems should be technically possible but unsupported - similar to how downgrading packages is possible but unsupported, and configuring dpkg --path-exclude is possible but unsupported. If you contrive to create an unmerged bookworm system and then try to upgrade it, that is likely to fail, and that's on you to resolve. > > E.g., what exactly does this mean for backports? > > Stuff from backports is post-release, so requires a merged system. Yes, I think this is right. IMO you can't validly enable bookworm-backports without first upgrading to bookworm, by which time the automatic transition mechanism called for by the TC resolution should have taken effect. However, I think bookworm-updates and bookworm-security still need to cope with unmerged-/usr systems to the same extent that bookworm itself does, because it's valid to upgrade from bullseye to bookworm + bookworm-updates + bookworm-security in a single step. smcv
Re: merged-/usr transition: debconf or not?
On Tue, 09 Nov 2021 at 19:01:18 +0100, David Kalnischkies wrote:
> On Tue, Nov 09, 2021 at 03:21:25PM +, Simon McVittie wrote:
> > > As I see it the CTTE decision effectively allows the transition to be
> > > deferred until the moment you want to upgrade to 13.
> >
> > I think you mean: until the moment you want to upgrade to testing after
> > Debian 12 release day. That's not Debian 13 *yet*
>
> Yes, I meant that indeed… should have used codenames after all.
To be clear about this: when drafting the TC resolution, I did not intend
for this to "allow the transition to be deferred" and still leave you with
a supported system.
All I was aiming to do there was to make it possible to create an
unsupported, not-quite-Debian system that our QA infra can compare with
the supported Debian system that closely resembles it, so that for example
if a package builds differently on unmerged-/usr and merged-/usr systems,
we can flag it as "this is likely to break upgrades" and look more closely.
> > Speaking only for myself and not for the TC, I think a debconf question
> > would be OK as an implementation of this, but the debconf question should
> > indicate that the result of opting out is an unsupported system.
>
> Sure.
>
> (Minus that for 12 it is technically still supported as long as it
> remains 12
No, it doesn't have to be supported, and the TC resolution explicitly
said that it doesn't have to be supported.
What *does* need to be supported is the upgrade path from 11 to 12,
or from current testing (11-and-a-bit) to 12, with any ordering of apt
transactions that doesn't violate the packages' dependency conditions -
and the TC's reasoning was that the simplest, most conservative, most
robust way to make sure that continues to work was to mandate that all
Debian 12 packages, individually, are installable onto unmerged-/usr
Debian 11 (assuming that "installing a package" implies installing its
dependencies, in any order that apt/dpkg consider to be valid and not
breaking any dependency relationships).
> (Perhaps it comes with the job as apt maintainer, but I don't "discard
> and redo" systems or even chroots… upgrades until hardware failure…
> my current build chroots are from 2013. So I can totally see me opt-out
> first and later in… although probably more with apt_preferences)
For full systems that are managed as full systems ("pets" in the cattle
vs. pets terminology), sure, do that; the Debian installation I'm typing
this into has been copied from several older machines. However, deferring
or avoiding the merged-/usr transition on these systems is not intended
to be something that is considered valid for bookworm.
For installations that are meant to be predictable, like build chroots
and the chroots/containers used for automated QA systems, keeping a 2013
installation and successively upgrading it comes with a significant risk
of ending up with an installation that nobody else (including you, if
the underlying hardware fails!) can replicate without starting from a 1:1
backup of that installation. I would not recommend this approach:
I think installations that are meant to be predictable should be "cattle".
I had intended that preventing the /usr merge (with debconf preseeding or
Marco's flag file or whatever) would be something that people should only
be doing in those "cattle": the steps to reproduce those systems would
include applying the debconf preseeding or creating the flag file.
smcv
Re: merged-/usr transition: debconf or not?
On Tue, Nov 09, 2021 at 08:44:52PM +, Simon McVittie wrote:
> On Tue, 09 Nov 2021 at 19:01:18 +0100, David Kalnischkies wrote:
> > On Tue, Nov 09, 2021 at 03:21:25PM +, Simon McVittie wrote:
> > (Minus that for 12 it is technically still supported as long as it
> > remains 12
>
> No, it doesn't have to be supported, and the TC resolution explicitly
> said that it doesn't have to be supported.
>
> What *does* need to be supported is the upgrade path from 11 to 12,
> or from current testing (11-and-a-bit) to 12, with any ordering of apt
> transactions that doesn't violate the packages' dependency conditions -
> and the TC's reasoning was that the simplest, most conservative, most
> robust way to make sure that continues to work was to mandate that all
> Debian 12 packages, individually, are installable onto unmerged-/usr
> Debian 11 (assuming that "installing a package" implies installing its
> dependencies, in any order that apt/dpkg consider to be valid and not
> breaking any dependency relationships).
Yes, any Debian 12.x package (even the very last security fix build for
it) needs to be installable on a Debian 11.y system as it could be part
of the upgrade from 11 to 12 and as it has no idea if its the first or
last package in that upgrade (within reason) it has to work on 11 as
well as on very-very-close-to-12.
As such, if you promise 11.x to 12.y upgrades I would expect 12.x to
12.y to work just as well as 12.x is very-very-close-to-12(.y).
If you say 12.x to 12.y isn't supported on unmerged it means effectively
that all "cattle" have to be constantly recreated as you can't have
a single package be considered an 'upgrade', they all need to be 'new
install' while e.g. installing build dependencies (as ironically a fully
upgraded 12 system is indistinguishable from an upgrade-in-progress-from-
11 system which just happens to install a bunch of new packages in the
end).
Its also quite a disaster for all systems already technically bookworm
like testing and sid as any upgrade, including to the release 12.0, will
be unsupported in your logic. Unsupported machines (aka our buildds)
building supported packages seems sad and I thought we had talked about
this before.
> > (Perhaps it comes with the job as apt maintainer, but I don't "discard
> > and redo" systems or even chroots… upgrades until hardware failure…
> > my current build chroots are from 2013. So I can totally see me opt-out
> > first and later in… although probably more with apt_preferences)
>
> For full systems that are managed as full systems ("pets" in the cattle
> vs. pets terminology), sure, do that; the Debian installation I'm typing
> this into has been copied from several older machines. However, deferring
> or avoiding the merged-/usr transition on these systems is not intended
> to be something that is considered valid for bookworm.
As the transition hasn't started everyone not already merged is currently
deferring it. That is true for those who upgrade daily as well as for
those people who seemingly only upgrade their sid systems once in a blue
moon. So, at which point have all those systems stopped deferring?
I would say that the first time you can say with absolute certainty that
a given system is no longer deferring the transition is the moment an
unpack of a trixie pkg is attempted as skipping releases is not
supported. All unpacks before that could happened on an unmerged
system as that system might very well be in the process of upgrading
from 11 to 12 at the moment.
(and btw, what I meant with me opting out for a while was delaying the
upgrade of my sid "beasts" to a more exciting problem space than the
first possible moment as that wouldn't be much of a test for apt, /usr
merge and all those other packages installed around here. If I am
asking for an upgrade path its only fair to not take the easiest road
of transitioning to merged before anything could implicitly require
it and hence fail for less lucky people not equipped to deal with it.
My "eldritch horrors" are fine and behave, thanks for asking. 😉)
Best regards
David Kalnischkies
signature.asc
Description: PGP signature
Re: Multiple teams maintaining one package (proposal)
On Sun, Nov 7, 2021 at 5:36 AM Ole Streicher wrote: > One solution here could be to recognize multiple teams: the "primary > one" (by the maintainer's selection) goes into the "Maintainer:" field > in d/control, and all secondary would go into the "Uploaders:" > field. This would imply that the package is conform to all team > policies, except for the salsa location of the package (i.e. a package > that has the Debian Astro Team as primary team would live in the > salsa.d.o/debian-astro/team/). The GNOME and Accessibility Teams co-maintain a few packages. They are hosted on the GNOME team's Salsa. I believe most have the Maintainer set to the Accessibility Team (like orca) but atkmm1.6 has the Maintainer set to the GNOME team. It works well because the GNOME Team has a lot more packaging style tweaks than the Accessibility team so it's easy for these packages to follow GNOME style without conflicting with a different team's style. Thanks, Jeremy Bicha
Re: Proposed mass bug filing: packages without support for build-arch and build-indep
Hi, On 05/11/21 at 21:22 +0100, Lucas Nussbaum wrote: > Hi, > > I'd like to propose a MBF with severity:serious for the above issue. > build-arch and build-indep are required targets according to Debian > Policy section 4.9. This rule was introduced in Policy version 3.9.4, > released in 2012. > https://www.debian.org/doc/debian-policy/ch-source.html#main-building-script-debian-rules > > There are 421 affected packages in unstable (389 in testing as of > 2021-10-01). > The list of affected packages according to lintian is > https://lintian.debian.org/tags/debian-rules-missing-recommended-target > A dd-list is included below. > > Unfortunately this is only a warning in lintian, which might explain > why so many packages are still affected. > > I have no strong feelings about this requirement, but I see it as a good > opportunity to identify packages whose packaging probably need a > refresh. Therefore it is a good target, especially at the beginning of a > release cycle, to either update old cruft or get it removed from the > next stable release. > > This topic was raised back in April on debian-qa@, and saw no > objection back then. See > https://lists.debian.org/debian-qa/2021/04/msg00014.html (the thread > included other topics). > > The bug template I plan to use is included below. > > I would prefer to file bugs directly with severity:serious, but I'm fine > with starting with severity:important and bumping severity after a month > or two if the release team prefers it, of course. > > - Lucas Bugs have been filed: https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=missing-build-arch-indep;users=debian...@lists.debian.org https://udd.debian.org/bugs/?release=na&merged=ign&fnewerval=7&flastmodval=7&fusertag=only&fusertagtag=missing-build-arch-indep&fusertaguser=debian-qa%40lists.debian.org&allbugs=1&cpopcon=1&cseverity=1&ckeypackage=1&ctags=1&caffected=1&clastupload=1&sortby=id&sorto=asc&format=html#results - Lucas signature.asc Description: PGP signature
