Re: Any volunteers for lintian co-maintenance?
Hi Nilesh, On 2024-05-10 21:04, Nilesh Patra wrote: On Fri, May 10, 2024 at 05:58:24PM +0300, Andrius Merkys wrote: Do you mean bugs on bugs.d.o, or are there other issues? As you may have seen in the other emails, there are performance issues. Other than that, there are 2 open RC bugs right now (fixed in salsa but not uploaded I don't make uploads for lintian). A pile of MRs are pending for reviews at many points in time. Thanks for the explanation. I personally feel motivated to implement new features in lintian or go after low hanging fruits, but I am somewhat driven away by the need to understand lintian's internals. Is there a documentation on the data/control flow, or class diagrams? This would help me a lot. Not that I know of, I suppose Axel/Bastian may be able answer this. I see. I believe the documentation would help attracting more volunteers. AFAIK, lintian's code is clearly written and nicely organised, and that is enough for fixing localised issues or introducing new features similar to already existing ones. However, before making larger changes one has to familiarise themself with the code, and that takes time, especially for people not well-versed in Perl. Best wishes, Andrius
Re: Make /tmp/ a tmpfs and cleanup /var/tmp/ on a timer by default [was: Re: systemd: tmpfiles.d not cleaning /var/tmp by default]
> I'd like to hear some arguments *in favour* of making this change. > Alignment with systemd-upstream, reduced package maintenance burden > are two that I can think of, but perhaps I've missed more. These two, > IMHO, are significantly outweighed by the risks. Let me see if I understand the arguments being made in favor. 1. Compatibility with upstream. This means all the upstream logic is sort of imported by reference, so the below is mainly the upstream logic, as I understand it. 2. Defend the system against buggy programs that leave debris in /var/tmp/, and against debris left there when programs are terminated prematurely. These are programs which use /var/tmp/ internally, but not as part of their API, so the user would have no particular way of knowing that they are leaving things there, would have no particular reason to check for and delete such files, and might not be able to easily recognize which files should be removed. 3. Defend the system against forgetful users, who create files in /var/tmp/ but neglect to delete them when they're no longer needed. Unfortunately 2 & 3 cannot be mechanically distinguished, so even if you wanted to have separate policies for these two classes of files, it's not really technically possible. So upstream is optimizing for case (2), and suggests that /var/tmp/ be sort of reserved for programs and scripts that are aware of this policy, and users not manually create files there. I hope that's an accurate characterization. I looked at the upstream bug report https://github.com/systemd/systemd/issues/32674 which suggests that deletion of directory trees in /var/tmp/ be atomic, and trigger only when everything in the tree meets criteria for deletion. I added a comment suggesting that the policy be tweaked in two ways. (a) Use system-up time rather than wall-clock time for measuring file age, to address the "suspending or shutting down for over 30 days breaks running data processing scripts that uses /var/tmp/ for intermediate files" issue. I sort of have an invariant in my head, which is that suspending the computer doesn't break things, and also the whole point of /var/tmp/ is that files there are preserved across boots. And (b) check if a file is open by some process, the same way fuser(1) does, and if so, don't delete it. Could also preserve directories which are the current directory of some process, if you want to be even more user friendly. The only response I got was "don't use temporary directories for things that you cannot afford to lose and recreate", which I don't really understand. It's like saying "you should have good backups, so it's not a problem to just delete anything in /home older than two days". Bottom line, it's not clear to me that upstream has really thought this through with users in mind. I'd suggest that Debian may wish to tweak the defaults on this stuff pretty hard to be more user friendly. Here are a couple cheap tweaks I'd suggest. My hope is that these would avoid some of the worst case scenarios discussed while still satisfying the goals 1/2/3 above, and being super easy to implement. - lengthen the reap time for /var/tmp/ to eight weeks, since Europeans often take six-week vacations. - make a "tempdir" command, philosophically similar to tempfile in debianutils, which creates a fresh directory in /var/tmp/ and drops the user into a shell with that as the current directory, directory flocked until that subshell terminates. Could give an optional directory argument to lock so you can get back into a directory, and maybe have options to run a program there before or instead of the subshell. - following a resume-from-suspend or boot, shut off the delete-old-files-in-var-tmp mechanism for a while, maybe eight hours or something like that. Maybe shorten the delay if it doesn't get to run across multiple resume-or-boot cycles for a week or two. - as discussed earlier, add /tmp/00-README and /var/tmp/00-README to explain this old-file-deletion policy
Re: Make /tmp/ a tmpfs and cleanup /var/tmp/ on a timer by default [was: Re: systemd: tmpfiles.d not cleaning /var/tmp by default]
Hi, Quoting Barak A. Pearlmutter (2024-05-13 10:47:43) > > I'd like to hear some arguments *in favour* of making this change. > > Alignment with systemd-upstream, reduced package maintenance burden > > are two that I can think of, but perhaps I've missed more. These two, > > IMHO, are significantly outweighed by the risks. > Let me see if I understand the arguments being made in favor. thank you, I'd also like to understand them. > 1. Compatibility with upstream. This means all the upstream logic is sort of >imported by reference, so the below is mainly the upstream logic, as I >understand it. Yes, I also think that there is value in doing the same thing that upstream or other distros are doing. There is a cost if Debian decides to deviate from what others decided to be the default. > 2. Defend the system against buggy programs that leave debris in /var/tmp/, >and against debris left there when programs are terminated prematurely. >These are programs which use /var/tmp/ internally, but not as part of >their API, so the user would have no particular way of knowing that they >are leaving things there, would have no particular reason to check for and >delete such files, and might not be able to easily recognize which files >should be removed. But I do not understand this as an advantage. In my mind it is quite the opposite. The buggy programs which leave files in /tmp will now have their bugs not noticed anymore because the files get cleaned up by systemd. On the other hand, we are now introducing new bugs in programs which should do an flock on the temporary directory but do not do so yet. Imagine I would not've read debian-devel. How would I as the mmdebstrap author have noticed that my tool as a user of /tmp should set up this flock? I imagine the bug report of somebody who has a weird problem with the chroot but somehow they are unable to reproduce it because it depends on the cleanup timing. Are the bugs we are introducing by regularly cleaning up /tmp not potentially super hard to diagnose and might thus just not get fixed? Is there an effort to go around and identify programs we ship with long lasting use of /tmp and is filing bugs so that they are performing an flock? And at the same time we are now ignoring the bugs in programs who leave files in /tmp and forget to clean them up. Is this not a disadvantage of this change instead of an advantage? > I looked at the upstream bug report > https://github.com/systemd/systemd/issues/32674 which suggests that deletion > of directory trees in /var/tmp/ be atomic, and trigger only when everything > in the tree meets criteria for deletion. I added a comment suggesting that > the policy be tweaked in two ways. (a) Use system-up time rather than > wall-clock time for measuring file age, to address the "suspending or > shutting down for over 30 days breaks running data processing scripts that > uses /var/tmp/ for intermediate files" issue. I sort of have an invariant in > my head, which is that suspending the computer doesn't break things, and also > the whole point of /var/tmp/ is that files there are preserved across boots. > And (b) check if a file is open by some process, the same way fuser(1) does, > and if so, don't delete it. Could also preserve directories which are the > current directory of some process, if you want to be even more user friendly. > > The only response I got was "don't use temporary directories for things that > you cannot afford to lose and recreate", which I don't really understand. > It's like saying "you should have good backups, so it's not a problem to just > delete anything in /home older than two days". Bottom line, it's not clear to > me that upstream has really thought this through with users in mind. I'd > suggest that Debian may wish to tweak the defaults on this stuff pretty hard > to be more user friendly. Thank you, the suspend issue is another issue that is created by this change. If we want to try and weigh cost against benefit, do the benefits really outweigh the cost? How costly is it to carry a patch in Debian and deviate from upstream versus all the problems that participants of this thread now listed? My gut feeling is, that the cost of these hard to debug problems is far greater than continuing to deviate from upstream and carry a Debian-specific patch, no? > - as discussed earlier, add /tmp/00-README and /var/tmp/00-README to explain > this old-file-deletion policy I think this is a really good idea. Thanks! cheers, josch signature.asc Description: signature
Re: Make /tmp/ a tmpfs and cleanup /var/tmp/ on a timer by default [was: Re: systemd: tmpfiles.d not cleaning /var/tmp by default]
Unless somebody's already put it there, I'm going to move these suggestions to a wishlist bug against systemd. Not sure if it should be one bug or a few, one for each suggestion. Currently discussion about reaping /var/tmp/ is in https://bugs.debian.org/966621 and https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1870585 but these are discussing "should we turn on /var/tmp/ reaping" rather than "if we do turn it on, should we take measures to make it safer".
Bug#1071062: ITP: golang-github-mgutz-logxi -- 12-factor app logger (library)
Package: wnpp Severity: wishlist Owner: Francisco Vilmar Cardoso Ruviaro X-Debbugs-Cc: debian-devel@lists.debian.org, debian...@lists.debian.org, vil...@debian.org * Package name: golang-github-mgutz-logxi Version : 1 Upstream Contact: Mario Gutierrez * URL : https://github.com/mgutz/logxi * License : Expat Programming Lang: Go Description : 12-factor app logger (library) Log XI is a structured 12-factor app (http://12factor.net/logs) logger built for speed and happy development. . * Simpler. Sane no-configuration defaults out of the box. * Faster. See benchmarks vs logrus and log15. * Structured. Key-value pairs are enforced. Logs JSON in production. * Configurable. Enable/disable Loggers and levels via env vars. * Friendlier. Happy, colorful and developer friendly logger in terminal. * Helpful. Traces, warnings and errors are emphasized with file, line number and callstack. * Efficient. Has level guards to avoid cost of building complex arguments.
Re: Make /tmp/ a tmpfs and cleanup /var/tmp/ on a timer by default [was: Re: systemd: tmpfiles.d not cleaning /var/tmp by default]
Am 13.05.24 um 11:42 schrieb Johannes Schauer Marin Rodrigues: If we want to try and weigh cost against benefit, do the benefits really outweigh the cost? How costly is it to carry a patch in Debian and deviate from upstream versus all the problems that participants of this thread now listed? My gut feeling is, that the cost of these hard to debug problems is far greater than continuing to deviate from upstream and carry a Debian-specific patch, no? Concerning the /var/tmp (and /tmp) cleaning, the Debian specific patch is https://salsa.debian.org/systemd-team/systemd/-/blob/debian/master/debian/patches/debian/Bring-tmpfiles.d-tmp.conf-in-line-with-Debian-defaul.patch?ref_type=heads This patch/change was added in 2012 and has required very little to no maintenance since that time. So the cost of keeping it is very low maintenance wise. Michael OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#1071068: ITP: golang-github-bettercap-gatt -- building Bluetooth Low Energy peripherals
Package: wnpp Severity: wishlist Owner: Francisco Vilmar Cardoso Ruviaro X-Debbugs-Cc: debian-devel@lists.debian.org, debian...@lists.debian.org, vil...@debian.org * Package name: golang-github-bettercap-gatt Version : 0.0~git20210514.df6e615 Upstream Contact: Simone Margaritelli * URL : https://github.com/bettercap/gatt * License : BSD-3-Clause Programming Lang: Go Description : building Bluetooth Low Energy peripherals This package provides a Bluetooth Low Energy GATT implementation. Gatt (Generic Attribute Profile) is the protocol used to write BLE peripherals (servers) and centrals (clients). . As a peripheral, you can create services, characteristics, and descriptors, advertise, accept connections, and handle requests. . As a central, you can scan, connect, discover services, and make requests.
Bug#1071073: ITP: pydataverse -- Python module for interacting with Dataverse APIs
Package: wnpp Severity: wishlist Owner: Edward Betts X-Debbugs-Cc: debian-devel@lists.debian.org, debian-pyt...@lists.debian.org * Package name: pydataverse Version : 0.3.2 Upstream Author : Jan Range * URL : https://github.com/gdcc/pyDataverse * License : MIT Programming Lang: Python Description : Python module for interacting with Dataverse APIs pyDataverse provides a comprehensive Python interface for interacting with Dataverse's APIs, facilitating the manipulation, validation, import, and export of all Dataverse data-types, including Dataverses, Datasets, and Datafiles. Whether it's for importing large volumes of data into Dataverse, testing instances post-deployment, or executing basic API calls, pyDataverse streamlines these processes. Features include a comprehensive API wrapper covering all Dataverse API endpoints, data models for key Dataverse data types, data conversion tools for Dataverse's JSON format, easy mass imports/exports through CSV templates, utility functions, documented examples, custom exceptions, and a test suite. I plan to maintain this package as part of the Python team.
