Re: autopkgtest/sbuild environment variables: LC_ALL, HOME, XDG_RUNTIME_DIR etc
On Thu, Apr 28, 2022 at 02:32:49PM +0100, Simon McVittie wrote: > On Thu, 28 Apr 2022 at 14:02:46 +0100, Julian Gilbey wrote: > > On Thu, Apr 28, 2022 at 10:16:30AM +0100, Simon McVittie wrote: > > > According to the autopkgtest spec[1], > > > > > > Tests can expect that the $HOME environment variable to be set to > > > a directory that exists and is writeable by the user running the test. > > > > That's interesting; I don't find any code that sets HOME in > > autopkgtest. > > The purpose of autopkgtest is to run the tests in a realistic environment > that resembles a real Debian desktop or server, but it doesn't know much > about how the testbed system is set up, so it relies on the virtualization > backend or the testbed itself to set HOME appropriately. Ideally, it > should be logging into a fully-working Debian system (perhaps under qemu > or lxc, or even on real hardware via ssh) and running tests there. > > As I said on #986665, this seems like either a bug in > autopkgtest-virt-schroot, or misconfiguration in how you and Laurent > are configuring it (the schroot profile that you're telling it to use). > [...] I see; this makes more sense now. So can I suggest that sbuild-setup(7) explains this a bit more and discusses setting up a meaningful HOME directory? And perhaps has the things necessary for a meaningful XDG_RUNTIME_DIR set up by default as well? > > (Incidentally, given that > > typically XDG_RUNTIME_DIR is in the ephemeral /run filesystem, writing > > to it would not violate policy 4.9.) > > Policy §4.9 says that "Required targets must not attempt to write outside > of the unpacked source package tree", then makes a few exceptions for > locations like /tmp. /run is not one of those locations, so writing to > /run is not allowed. Ah, I didn't read it carefully enough; you are right. > > I wonder whether autopkgtest > > could consider setting up a temporary directory for this purpose like > > dh_auto_test does, rather than using the heavy machinery of > > libpam-systemd? > > No, I don't think it should. The purpose of autopkgtest is to run the > [...] Understood. Julian
Re: autopkgtest/sbuild environment variables: LC_ALL, HOME, XDG_RUNTIME_DIR etc
On Thu, Apr 28, 2022 at 02:32:49PM +0100, Simon McVittie wrote: > > That's interesting; I don't find any code that sets HOME in > > autopkgtest. > > The purpose of autopkgtest is to run the tests in a realistic environment > that resembles a real Debian desktop or server, but it doesn't know much > about how the testbed system is set up, so it relies on the virtualization > backend or the testbed itself to set HOME appropriately. Ideally, it > should be logging into a fully-working Debian system (perhaps under qemu > or lxc, or even on real hardware via ssh) and running tests there. > > As I said on #986665, this seems like either a bug in > autopkgtest-virt-schroot, or misconfiguration in how you and Laurent > are configuring it (the schroot profile that you're telling it to use). I've thought a little bit more about this, and I realise that the devil is in the detail, in particular the phrase "a realistic environment that resembles a real Debian desktop or server". A realistic environment is likely to have lots of random packages and services available. It would make some sense, therefore, to be more explicit about what is expected in this environment by default. Here's a very small list to begin with: 1. All essential packages are installed. 2. HOME points to a readable and writeable directory. [What is in this directory? Should it be cleaned before every use?] 3. XDG_RUNTIME_DIR points to ... / maybe this should be: libpam-systemd is installed? 4. Locale LC_ALL ... should be set up in a meaningful way. 5. ... more points? ... This will help developers to set up sensible environments. I now know I have been writing autopkgtest scripts that don't assume this and therefore don't properly replicate a typical desktop environment. But without these assumptions made explicit, it will be hard to guess what can be assumed and what can't be. Best wishes, Julian
Re: autopkgtest/sbuild environment variables: LC_ALL, HOME, XDG_RUNTIME_DIR etc
On Fri, 29 Apr 2022 at 10:43:40 +0800, Paul Wise wrote: > Does that mean that autopkgtests that directly or indirectly use the > dbus-run-session tool to setup a temporary D-Bus session are buggy? There's no single answer to that. For unit-test-style testing, it's often more convenient to start a session bus that's fully under the control of the test suite, particularly if the tests need special configuration (rejecting certain D-Bus messages, or setting limits, or making mock D-Bus services activatable, or something like that). For tests of the overall system (closer to an integration test), the fact that the service is genuinely activatable on a real system seems like something that the autopkgtest framework should not prevent you from testing. You don't necessarily need to actually test this - perhaps your priorities are elsewhere, and 100% test coverage is rarely realistic - but I think it would be a design flaw if autopkgtest prevented you from testing it, by forcing you to use a mock XDG_RUNTIME_DIR or a mock home directory or anything else that would interfere with your tests' access to our real system integration layer. If you run dbus-run-session without the --config-file option, it does use the real /usr/share/dbus-1/services, so you can test traditional (non-systemd) session service activation like this. You can't test systemd service activation this way, because that only works for the dbus-daemon that was launched by systemd and told to integrate with it. dbus has debian/tests/system-bus which does a brief integration test to check that the system bus works, but it doesn't yet have an equivalent for dbus-user-session - contributions welcome, if you are interested in expanding coverage. autopkgtest tries to set up a "real" system login session by running su, even if it's already running as the target uid, to make sure that some of the side-effects of login (in particular a PAM stack) have already happened; but I suspect this doesn't have full coverage. autopkgtest-virt-ssh is probably the most "realistic" of the backends, by running test commands under a ssh session that is genuinely a login entry point (meaning processes have a loginuid and so on), and one of the things on my increasingly intractable to-do list is to add a way to launch qemu VMs on-demand and use ssh to log in to them, rather than using a privileged serial console like virt-qemu does. In some backends it isn't really feasible to run the tests in a "real" system login session, so integration tests for things that rely on the system's session infrastructure might have to have the 'skippable' restriction and detect whether they are running in a login session at runtime. This is also very likely to be something that some autopkgtest backends just cannot provide: -chroot and -schroot don't have their own init systems (and neither would a Docker backend, review/finishing of which is also on my to-do list), so they cannot be expected to run their own system services. > The other thing is testing graphical tools; should that be done in a > mock X11/Wayland session (using xvfb-run), or a real desktop session? At the moment a mock X11 session using xvfb-run or a mock Wayland session using weston --backend=headless-backend.so (see debian/tests/run-with-display in src:gtk4) are the only realistic options for this. Ideally it'd be possible to test graphical tools under a real desktop session, but that's not straightforward to set up even under something like autopkgtest-virt-qemu (the test framework would have to script a login to a display manager using techniques similar to openQA, or configure the display manager to autologin and then reboot to make the autologin happen), and I'm not sure whether this would be feasible to set up under other backends at all. There's a cost/benefit tradeoff here: running tests under a real desktop session is likely to be more difficult, slower, and less robust than xvfb-run, so we should only do it if the benefit is worth those costs. In particular, we can are not going to be able to use this as a QA gate unless someone puts a lot of time and effort into making it completely reliable. GNOME's installed-tests convention[1], which is a frequent source of autopkgtests for Debian, *does* call for tests to be run in a real, active GUI desktop session, although in practice most packages' installed-tests do not actually need this. [1] https://wiki.gnome.org/Initiatives/GnomeGoals/InstalledTests smcv
Re: autopkgtest/sbuild environment variables: LC_ALL, HOME, XDG_RUNTIME_DIR etc
On Fri, 29 Apr 2022 at 08:34:48 +0100, Julian Gilbey wrote: > So can I suggest that > sbuild-setup(7) explains this a bit more and discusses setting up a > meaningful HOME directory? I'm sure patches are accepted, but the problem with this is that what you want for sbuild does not match what you want for autopkgtest. For sbuild, the environment that most closely resembles our real, production buildds involves the /etc/schroot/buildd profile, and a uid whose home directory is /nonexistent. For autopkgtest, one of the more permissive profiles like /etc/schroot/desktop is more realistic, and tests are allowed to assume that they run as a user with a real home directory. I would personally recommend one of the better-isolated autopkgtest backends like -lxc or -qemu for running autopkgtest tests. -qemu doesn't need root, and there are proposed patches adding backends that use unprivileged user namespaces (-unshare and Podman), which I should probably be reviewing instead of replying to this email. > And perhaps has the things necessary for a > meaningful XDG_RUNTIME_DIR set up by default as well? Having a meaningful XDG_RUNTIME_DIR in your local sbuild setup will result in a risk of uploading packages that cannot be built on our official buildds, which do not have a meaningful XDG_RUNTIME_DIR (if I remember correctly, the environment variable is set to a path that is valid outside the chroot, but that path does not exist inside the chroot). This is particularly true for older debhelper compat levels. Again, what you want for sbuild and what you want for autopkgtest are unfortunately not the same. smcv
Re: autopkgtest/sbuild environment variables: LC_ALL, HOME, XDG_RUNTIME_DIR etc
Quoting Simon McVittie (2022-04-29 10:58:34) > On Fri, 29 Apr 2022 at 08:34:48 +0100, Julian Gilbey wrote: > > So can I suggest that > > sbuild-setup(7) explains this a bit more and discusses setting up a > > meaningful HOME directory? > > I'm sure patches are accepted, but the problem with this is that what you > want for sbuild does not match what you want for autopkgtest. For sbuild, > the environment that most closely resembles our real, production buildds > involves the /etc/schroot/buildd profile, and a uid whose home directory > is /nonexistent. For autopkgtest, one of the more permissive profiles > like /etc/schroot/desktop is more realistic, and tests are allowed to > assume that they run as a user with a real home directory. > > I would personally recommend one of the better-isolated autopkgtest > backends like -lxc or -qemu for running autopkgtest tests. -qemu doesn't > need root, and there are proposed patches adding backends that use > unprivileged user namespaces (-unshare and Podman), which I should > probably be reviewing instead of replying to this email. Alas, https://salsa.debian.org/ci-team/autopkgtest/-/merge_requests/138 was merged 14 hours ago. :) Thanks! cheers, josch signature.asc Description: signature
Re: feedback for NEW packages: switch to using the BTS?
Paul Wise wrote: > >During the discussions about NEW on debian-devel in recent times, I had >the idea that instead of the current mechanism of sending REJECT mails, >Debian could switch to using the BTS for most feedback on NEW packages. > >This means that most discussion about NEW packages would become public, >but of course the ftpmasters could opt to send private mail instead if >in some cases if there were sensitive issues to be discussed. > >The ftpmasters could simply file severity serious bug reports against >NEW packages that have issues blocking their entry into Debian. When >there are minor issues noticed at the same time, then file bugs of a >lower severity. Only when a NEW package has not had its serious bugs >fixed in a long time would an eventual removal and REJECT mail happen, >perhaps after a few months of zero action on the bug reports. Just to clarify: is this suggesting that packages from NEW would end up in the archive even with serious bugs? If not, what's the point of the "eventual removal" above? I'm not following you here... -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: feedback for NEW packages: switch to using the BTS?
On 2022-04-29 at 08:36, Steve McIntyre wrote: > Paul Wise wrote: > >> During the discussions about NEW on debian-devel in recent times, I >> had the idea that instead of the current mechanism of sending >> REJECT mails, Debian could switch to using the BTS for most >> feedback on NEW packages. >> >> This means that most discussion about NEW packages would become >> public, but of course the ftpmasters could opt to send private mail >> instead if in some cases if there were sensitive issues to be >> discussed. >> >> The ftpmasters could simply file severity serious bug reports >> against NEW packages that have issues blocking their entry into >> Debian. When there are minor issues noticed at the same time, then >> file bugs of a lower severity. Only when a NEW package has not had >> its serious bugs fixed in a long time would an eventual removal and >> REJECT mail happen, perhaps after a few months of zero action on >> the bug reports. > > Just to clarify: is this suggesting that packages from NEW would end > up in the archive even with serious bugs? If not, what's the point > of the "eventual removal" above? I'm not following you here... I parsed that not as "removal from the archive" but as "removal from the NEW queue", much as now happens (in some order and via some mechanism, perhaps a manual one) when a REJECT mail is sent. -- The Wanderer The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. -- George Bernard Shaw signature.asc Description: OpenPGP digital signature
Help in setting up lxc for autopkgtest - autopkgtest-build-lxc fails
Following Simon's suggestion, I decided to try setting up lxc to use autopkgtest-lxc, mimicking the ci.debian.org setup. I haven't managed to do so yet, and have run into lots of problems. I'd really appreciate some advice on what to try, and them we can record advice somewhere on the Debian wiki. Following the advice in autopkgtest-build-lxc(1), that user containers will not work with many or even most autopkgtests, I ran it as route. Step 1: Install the lxc and autopkgtest packages That went smoothly. (lxc version 1:4.0.11-1, autopkgtest version 5.21) Step 2: Run the command "autopkgtest-build-lxc debian sid" I got various warning messages, and this essentially failed... lxc-create: autopkgtest-sid: storage/btrfs.c: btrfs_create: 938 Inappropriate ioctl for device - Failed to create btrfs subvolume "/var/lib/lxc/autopkgtest-sid/rootfs" lxc-create: autopkgtest-sid: storage/zfs.c: zfs_create: 735 Failed to create zfs dataset "zfs:lxc/autopkgtest-sid": lxc-create: autopkgtest-sid: utils.c: run_command_internal: 1588 lxc-create: autopkgtest-sid: storage/lvm.c: do_lvm_create: 165 Failed to create logical volume "autopkgtest-sid": Volume group "lxc" not found Cannot process volume group lxc lxc-create: autopkgtest-sid: storage/lvm.c: lvm_create: 623 Error creating new logical volume "lvm:/dev/lxc/autopkgtest-sid" of size "1073741824 bytes" debootstrap is /usr/sbin/debootstrap Checking cache download in /var/cache/lxc/debian/rootfs-sid-amd64 ... Downloading debian minimal ... I: Target architecture can be executed I: Retrieving InRelease [downloading and installing base system] I: Base system installed successfully. Download complete. Copying rootfs to /var/lib/lxc/autopkgtest-sid/rootfs...ERROR: ld.so: object 'libeatmydata.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. ERROR: ld.so: object 'libeatmydata.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. Generating locales (this might take a while)... ERROR: ld.so: object 'libeatmydata.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. en_GB.UTF-8ERROR: ld.so: object 'libeatmydata.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. [... lots more libeatmydata.so warnings, interspersed with other information messages ...] ERROR: ld.so: object 'libeatmydata.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. lxc-start: autopkgtest-sid: lxccontainer.c: wait_on_daemonized_start: 867 Received container state "ABORTING" instead of "RUNNING" lxc-start: autopkgtest-sid: tools/lxc_start.c: main: 306 The container failed to start lxc-start: autopkgtest-sid: tools/lxc_start.c: main: 309 To get more details, run the container in foreground mode lxc-start: autopkgtest-sid: tools/lxc_start.c: main: 311 Additional information can be obtained by setting the --logfile and --logpriority options Something weird is going on here, but autopkgtest-build-lxc doesn't seem to allow a --logfile option. I attempted to start it manually, using the command lxc-start -n autopkgtest-sid --logfile /tmp/lxc.log --logpriority INFO and got the following errors in the log file: lxc-start autopkgtest-sid 20220429124743.756 WARN cgfsng - cgroups/cgfsng.c:get_hierarchy:142 - There is no useable devices controller lxc-start autopkgtest-sid 20220429124743.756 ERRORcgfsng - cgroups/cgfsng.c:cg_legacy_set_data:2675 - No such file or directory - Failed to setup limits for the "devices" controller. The controller seems to be unused by "cgfsng" cgroup driver or not enabled on the cgroup hierarchy lxc-start autopkgtest-sid 20220429124743.756 ERRORcgfsng - cgroups/cgfsng.c:cgfsng_setup_limits_legacy:2742 - No such file or directory - Failed to set "devices.deny" to "a" lxc-start autopkgtest-sid 20220429124743.756 ERRORstart - start.c:lxc_spawn:1890 - Failed to setup legacy device cgroup controller limits lxc-start autopkgtest-sid 20220429124743.756 ERRORlxccontainer - lxccontainer.c:wait_on_daemonized_start:867 - Received container state "ABORTING" instead of "RUNNING" [...] I found something like this reported at this GitHub issue against lxc: https://github.com/lxc/lxc/issues/2268 so I followed the advice there and ran the commands: mount -o remount,rw /sys/fs/cgroup mkdir /sys/fs/cgroup/devices mount -t cgroup devices -o devices /sys/fs/cgroup/devices mount -o remount,ro /sys/fs/cgroup But that seems to be really bad, as now systemd-logind.service seems to have broken and cannot be restarted, so I don't recommend doing that! I've restarted my system and started again. The above solution is very bad at least partly because /sys/fs/cgroup is type cgroup2. But I still can't start the LXC container, which makes running autopkgtest impossible. Is this a bug in lxc or in autopkgtest-build-lxc or somewhere else? Any suggestions would be much appreciated! Julian
Re: feedback for NEW packages: switch to using the BTS?
On Wednesday, April 27, 2022 8:54:05 PM EDT Paul Wise wrote: > Hi all, > > During the discussions about NEW on debian-devel in recent times, I had > the idea that instead of the current mechanism of sending REJECT mails, > Debian could switch to using the BTS for most feedback on NEW packages. > > This means that most discussion about NEW packages would become public, > but of course the ftpmasters could opt to send private mail instead if > in some cases if there were sensitive issues to be discussed. > > The ftpmasters could simply file severity serious bug reports against > NEW packages that have issues blocking their entry into Debian. When > there are minor issues noticed at the same time, then file bugs of a > lower severity. Only when a NEW package has not had its serious bugs > fixed in a long time would an eventual removal and REJECT mail happen, > perhaps after a few months of zero action on the bug reports. > > The changes that are needed to make this happen include: > > The community needs to decide that this change is a good idea and be > willing to recieve most feedback on their work in public. > > The BTS and ftpmaster teams need to agree with this change. > One BTS admin said this is feasible in principle and one of the > ftpmasters seemed to like the idea when I mentioned it on IRC. > > Where the bug mail should go to needs to be decided on. Personally I'm > thinking the person who did the upload plus the Maintainer field. > > The people doing processing of NEW packages need to be willing to file > bug reports against them where necessary. > > dak needs to export debian/changelog version lists for NEW packages. > > debbugs needs to import packages/versions from the new.822 file: > > https://ftp-master.debian.org/new.822 > > dak needs to link to the BTS from new.822 and the NEW packages info. > > https://ftp-master.debian.org/new.html > > The technical work needed here is minimal, so if there are no other > volunteers to do that work, then I would be willing to do it. I have > experience with Perl/Python but only a small amount of experience with > working on the debbugs and dak codebases. > > Thoughts welcome, especially from those who don't like this idea. This proposal has a number of parts and I'm not entirely confident I understand what it being proposed. A few thoughts/clarifications. 1. Making reject/prod emails more public: There are actually two different types of messages we can send while reviewing packages in DAK: a. Reject: As indicated by the name, this is an email that accompanies the package being rejected from the New queue. b. Prod: These leave the package in New and are used to ask the maintainer a question. My recollection is that these go to the maintainer and changed by. In many cases the maintainer is a team, so these are already not considered private. Whatever is done in this area should definitely be integrated into DAK as part of process-new. I don't think it would be helpful to try to add steps to the work the FTP Team has to do. New is slow enough that we should definitely not make is slower. Also, if any of this will depend on the existence of a WNPP bug, then policy should be changed to require them. They are currently not required and not everyone files them. If something is going to be required to get a package accepted, it should be in policy. On a related note: It is not unusual for me to notice packaging issues which are not serious enough to reject the package, but should be documented. This is particularly true when an existing package goes through New due to a new binary package. It would be useful to have the option to accept with bug report so that we don't need to go through the steps of filing a bug by hand. When I do this I also send a prod to the uploader/maintainer and it's effectively doing work twice. Making New processing more efficient will help make it faster. 2. Not rejecting packages with serious defects: I'm not sure I understand what it proposed: > The ftpmasters could simply file severity serious bug reports against > NEW packages that have issues blocking their entry into Debian. When > there are minor issues noticed at the same time, then file bugs of a > lower severity. Only when a NEW package has not had its serious bugs > fixed in a long time would an eventual removal and REJECT mail happen, > perhaps after a few months of zero action on the bug reports. I think this proposes to accept all packages regardless of how defective they are and then remove them later if the bugs aren't fixed. If that's what is proposed, my thought is absolutely not. If a package is not suitable for the archive then it should be rejected with appropriate feedback and re-uploaded. Note: Although I am a member of the FTP Team, I am only speaking for myself here, not the team as a whole. Scott K signature.asc Description: This is a digitally signed message part.
Re: feedback for NEW packages: switch to using the BTS?
Hi Scott, thanks a lot for becoming involved into this discussion. Am Fri, Apr 29, 2022 at 11:26:33AM -0400 schrieb Scott Kitterman: > 2. Not rejecting packages with serious defects: > > I'm not sure I understand what it proposed: > > > The ftpmasters could simply file severity serious bug reports against > > NEW packages that have issues blocking their entry into Debian. When > > there are minor issues noticed at the same time, then file bugs of a > > lower severity. Only when a NEW package has not had its serious bugs > > fixed in a long time would an eventual removal and REJECT mail happen, > > perhaps after a few months of zero action on the bug reports. > > I think this proposes to accept all packages regardless of how defective they > are and then remove them later if the bugs aren't fixed. If that's what is > proposed, my thought is absolutely not. > > If a package is not suitable for the archive then it should be rejected with > appropriate feedback and re-uploaded. To give some actual examples that IMHO are candidates for accept + bug report: 1. In case versioneer.py (Creative Commons "Public Domain Dedication" license (CC0-1.0) is missing in d/copyright like in propka[1] 2. Packages which are DFSG free but might miss some single copyright statement. My favourite example would be r-bioc-basilisk[2]. In this specific example I even disagree with ftpmaster[3] but I see no real chance as a maintainer to discuss my point and can only re-re-re-upload what I consider correct. So I gave up and this package is not yet inside Debian. This could be discussed more sensibly in a bug report IMHO. I think Paul was not talking about non-distributable software but rather code that is considered DFSG free but missing proper paragraphs inside d/copyright which can be easily fixed in BTS. > Note: Although I am a member of the FTP Team, I am only speaking for myself > here, not the team as a whole. Thanks a lot for speaking at a "competent yourself" ;-) Kind regards Andreas. [1] https://alioth-lists.debian.net/pipermail/debian-med-packaging/2022-February/098605.html [2] https://alioth-lists.debian.net/pipermail/r-pkg-team/2022-February/024165.html [3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=991859#42 -- http://fam-tille.de
Re: feedback for NEW packages: switch to using the BTS?
On Friday, April 29, 2022 12:08:21 PM EDT Andreas Tille wrote: > Hi Scott, > > thanks a lot for becoming involved into this discussion. > > Am Fri, Apr 29, 2022 at 11:26:33AM -0400 schrieb Scott Kitterman: > > 2. Not rejecting packages with serious defects: > > > > I'm not sure I understand what it proposed: > > > The ftpmasters could simply file severity serious bug reports against > > > NEW packages that have issues blocking their entry into Debian. When > > > there are minor issues noticed at the same time, then file bugs of a > > > lower severity. Only when a NEW package has not had its serious bugs > > > fixed in a long time would an eventual removal and REJECT mail happen, > > > perhaps after a few months of zero action on the bug reports. > > > > I think this proposes to accept all packages regardless of how defective > > they are and then remove them later if the bugs aren't fixed. If that's > > what is proposed, my thought is absolutely not. > > > > If a package is not suitable for the archive then it should be rejected > > with appropriate feedback and re-uploaded. > > To give some actual examples that IMHO are candidates for accept + bug > report: > >1. In case versioneer.py (Creative Commons "Public Domain Dedication" > license (CC0-1.0) is missing in d/copyright like in propka[1] > >2. Packages which are DFSG free but might miss some single copyright > statement. > My favourite example would be r-bioc-basilisk[2]. In this specific > example I even disagree with ftpmaster[3] but I see no real chance > as a maintainer to discuss my point and can only re-re-re-upload > what I consider correct. So I gave up and this package is not yet > inside Debian. This could be discussed more sensibly in a bug > report IMHO. > > I think Paul was not talking about non-distributable software but rather > code that is considered DFSG free but missing proper paragraphs inside > d/copyright which can be easily fixed in BTS. The in that case, it's just a variant of accept plus file a bug. I don't think it's productive to get into a discussion on the distinction between "buggy" and "too buggy to go in the archive". It's very dependent on the specifics of the situation. Also, accept with severe bugs and remove later if not fixed requires someone to track these issues and follow-up on getting them removed. That seems like more work that isn't easily automated (we don't do automatic removals from Unstable now - they are at most semi-automatic where an FTP Team member still needs to review and do the actual removal). To summarize: The reject/prod messages are already not considered private. Any additional addressing of them to whatever location in the BTS is a matter of have the BTS be ready for it and adding it to process-new to send it. If the answer is that ITP bugs are now required, then policy should be updated first. It would be useful to have a mechanism within DAK process-new to file a bug with the prod note of the appropriate severity. I think if this existed you would be inclined to see more accept with bug resolutions. I think accepting something to buggy to remain in unstable and removing it later if not fixed is not something that should be pursued. Still not speaking for the FTP Team. Scott K signature.asc Description: This is a digitally signed message part.
Re: feedback for NEW packages: switch to using the BTS?
On Fri, 2022-04-29 at 13:36 +0100, Steve McIntyre wrote: > Just to clarify: is this suggesting that packages from NEW would end > up in the archive even with serious bugs? If not, what's the point of > the "eventual removal" above? I'm not following you here... No, the bugs I propose to be filed would be filed *while* the package is in NEW and then *after* the serious ones are all fixed, that is a positive indication that the package is probably suitable for an ACCEPT action by the ftpmasters, although of course a final review is needed. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: feedback for NEW packages: switch to using the BTS?
On Fri, 2022-04-29 at 11:26 -0400, Scott Kitterman wrote: > 1. Making reject/prod emails more public: > > There are actually two different types of messages we can send while > reviewing > packages in DAK: > > a. Reject: As indicated by the name, this is an email that accompanies the > package being rejected from the New queue. > > b. Prod: These leave the package in New and are used to ask the maintainer a > question. The proposal essentially turns most Reject mails into Prod mails and turns all Prod mails to bugs. The only Reject mails that will be sent are if the bugs aren't being solved after a certain amount of time, then the packages will be removed from NEW and get a REJECT. > My recollection is that these go to the maintainer and changed by. In many > cases the maintainer is a team, so these are already not considered private. That makes it seem more reasonable to make everything public, also I note the REJECT mails are available to all Debian members. ssh://mirror.ftp-master.debian.org/srv/ftp-master.debian.org/queue/reject/ I have been reading these files and note that many of them are automated, probably the most common one is people mistakenly doing source-only uploads to NEW. I have another idea to add a pre-reject API to dakweb for use by dupload/dput/dput-ng but haven't a proposal yet. > Whatever is done in this area should definitely be integrated into DAK as > part > of process-new. I don't think it would be helpful to try to add steps to the > work the FTP Team has to do. New is slow enough that we should definitely > not > make is slower. That seems reasonable. > Also, if any of this will depend on the existence of a WNPP bug, then policy > should be changed to require them. They are currently not required and not > everyone files them. If something is going to be required to get a package > accepted, it should be in policy. The proposal was created because I think filing WNPP bugs for every NEW upload is very much suboptimal, so I came up with the proposal as an alternative to that, so I'd avoid filing additional WNPP bugs for NEW above the existing ITP/etc bugs. > On a related note: It is not unusual for me to notice packaging issues which > are not serious enough to reject the package, but should be documented. One advantage of this proposal is that it allows filing bugs against any package already in NEW, so you could do that even if you notice a minor issue while processing a package, then have to move on to something else, then come back to the package the next day. > This is particularly true when an existing package goes through New due to a > new > binary package. It would be useful to have the option to accept with bug > report so that we don't need to go through the steps of filing a bug by hand. > > When I do this I also send a prod to the uploader/maintainer and it's > effectively doing work twice. Making New processing more efficient will help > make it faster. That seems like a useful feature, but I think I would separate this into two steps, first file the bug, maybe from dak and then accept. > 2. Not rejecting packages with serious defects: > > I'm not sure I understand what it proposed: > > > The ftpmasters could simply file severity serious bug reports against > > NEW packages that have issues blocking their entry into Debian. When > > there are minor issues noticed at the same time, then file bugs of a > > lower severity. Only when a NEW package has not had its serious bugs > > fixed in a long time would an eventual removal and REJECT mail happen, > > perhaps after a few months of zero action on the bug reports. > > I think this proposes to accept all packages regardless of how defective they > are and then remove them later if the bugs aren't fixed. If that's what is > proposed, my thought is absolutely not. That is definitely *not* what was proposed, and I think it is a bad idea to accept packages that are not suitable for the archive. The proposal is simply to change the NEW packages feedback mechanism and change the procedure for packages that don't yet meet Debian standards. > If a package is not suitable for the archive then it should be rejected with > appropriate feedback and re-uploaded. The proposal is to instead of rejecting the package, keep it in NEW, file a serious bug against the package in NEW, wait for it to be fixed and then reprocess the package. > Note: Although I am a member of the FTP Team, I am only speaking for myself > here, not the team as a whole. Thanks for your response. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: feedback for NEW packages: switch to using the BTS?
On April 29, 2022 11:04:57 PM UTC, Paul Wise wrote: >On Fri, 2022-04-29 at 13:36 +0100, Steve McIntyre wrote: > >> Just to clarify: is this suggesting that packages from NEW would end >> up in the archive even with serious bugs? If not, what's the point of >> the "eventual removal" above? I'm not following you here... > >No, the bugs I propose to be filed would be filed *while* the package >is in NEW and then *after* the serious ones are all fixed, that is a >positive indication that the package is probably suitable for an ACCEPT >action by the ftpmasters, although of course a final review is needed. I don't understand why this is any better than just rejecting the package? Once it's been determined that the upload won't be accepted, I don't see a benefit to having it remain in New. Scott K
Re: feedback for NEW packages: switch to using the BTS?
On Fri, 2022-04-29 at 18:08 +0200, Andreas Tille wrote: > To give some actual examples that IMHO are candidates for accept + bug > report: Packages with incomplete or incorrect debian/copyright information currently would recieve a REJECT rather than acceptance. The proposal would not change that, just turn that REJECT into a bug report, which you could fix in a second upload to NEW and then the package would be reprocessed and get an ACCEPT or another bug. > I think Paul was not talking about non-distributable software but rather > code that is considered DFSG free but missing proper paragraphs inside > d/copyright which can be easily fixed in BTS. I think completely non-distributable software should get a bug against the package in NEW tagged as wontfix and then the NEW package should get removed with a REJECT mail mentioning the reason. There may be cases where it might be possible to fix some issue with a unredistributable file in an otherwise redistributable package, in those cases a serious bug against the package in NEW should be filed and the maintainer be given a chance to fix the issue. The proposal doesn't change anything with regard to packages with incomplete or incorrect debian/copyright information, that is a separate issue to what this proposal aims to solve. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: feedback for NEW packages: switch to using the BTS?
On Fri, 2022-04-29 at 23:32 +, Scott Kitterman wrote: > I don't understand why this is any better than just rejecting the > package? Once it's been determined that the upload won't be > accepted, I don't see a benefit to having it remain in New. The initial upload might not get an ACCEPT, but later revisions of the package might be suitable for inclusion in Debian. The main aim of the proposal is transparency for all problems getting the package into NEW. Often it isn't clear what happened to a package that was rejected, the proposal makes it easy to find out; just check the bug reports for the source package. It also means the ftpmasters can file separate bugs for each problem, rather than combining them all into one mail. It also means the ftpmasters can easily point out other issues they noticed that don't block the package from being accepted and ensure that those issues won't be forgotten. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: feedback for NEW packages: switch to using the BTS?
On April 29, 2022 11:44:54 PM UTC, Paul Wise wrote: >On Fri, 2022-04-29 at 23:32 +, Scott Kitterman wrote: > >> I don't understand why this is any better than just rejecting the >> package? Once it's been determined that the upload won't be >> accepted, I don't see a benefit to having it remain in New. > >The initial upload might not get an ACCEPT, but later revisions of the >package might be suitable for inclusion in Debian. > >The main aim of the proposal is transparency for all problems getting >the package into NEW. Often it isn't clear what happened to a package >that was rejected, the proposal makes it easy to find out; just check >the bug reports for the source package. > >It also means the ftpmasters can file separate bugs for each problem, >rather than combining them all into one mail. > >It also means the ftpmasters can easily point out other issues they >noticed that don't block the package from being accepted and ensure >that those issues won't be forgotten. > Most of that seems mostly reasonable, but I'm still not understanding how any of that needs to have a package we've decided not to accept sitting in New? In my mind if a package is in New it's in one of two states: 1. Not reviewed yet. 2. Partly/completely reviewed, pending resolution of a question (e.g. prod sent to maintainer, waiting for reply). What's the advantage of added a third: 3. Not going to be accepted, waiting for a new upload or for enough time to pass before rejecting. Scott K
Re: feedback for NEW packages: switch to using the BTS?
On Sat, 2022-04-30 at 00:13 +, Scott Kitterman wrote: > I'm still not understanding how any of that needs to have a package > we've decided not to accept sitting in New? My thinking is that debbugs would require a package (imported from new.822) to exist for maintainer addresses. If you file a bug, then remove the package from NEW, then it disappears from the BTS too and subsequent mails to the bug wouldn't have a place to be forwarded to and would go to the address for unknown packages. I suppose debbugs could allow the package state to go out of sync with NEW and instead retain the addresses for a certain period of time after packages are removed from NEW and while there are open bugs on the packages that were removed from NEW. Then some process could regularly close and archive the bugs filed by ftpmasters that were not solved. I think it would be simpler to handle this on the dak side though, by keeping the packages around, perhaps in OLD instead of NEW :) and then rejecting them after a certain time and closing the bugs. > In my mind if a package is in New it's in one of two states: > > 1. Not reviewed yet. > > 2. Partly/completely reviewed, pending resolution of a question > (e.g. prod sent to maintainer, waiting for reply). > > What's the advantage of added a third: > > 3. Not going to be accepted, waiting for a new upload or for enough > time to pass before rejecting. States 2 and 3 are very similar; there is something to be resolved with the package and it is waiting on maintainer/uploader action for that. The only difference is 2 is waiting on a reply, 3 waits on an upload. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Bug#1010382: ITP: mkdocstrings-python-legacy -- Legacy Python handler for mkdocstrings
Package: wnpp Severity: wishlist Owner: Carsten Schoenert X-Debbugs-Cc: debian-devel@lists.debian.org * Package name: mkdocstrings-python-legacy Version : 0.2.2 Upstream Author : Timothée Mazzucotelli * URL : https://github.com/mkdocstrings/python-legacy * License : ISC Programming Lang: Python Description : Legacy Python handler for mkdocstrings MkDocs is a fast, simple and downright gorgeous static site generator that's geared towards building project documentation. Documentation source files are written in Markdown, and configured with a single YAML configuration file. . This package contains an legacy Python handler used within mkdocstrings. This package is a new dependency for mkdocstrings >= 0.18 which due a split of existing legacy Python functions moved into a own library. It will be maintained within the Debian Python Team.
