Re: Make /tmp/ a tmpfs and cleanup /var/tmp/ on a timer by default [was: Re: systemd: tmpfiles.d not cleaning /var/tmp by default]

[Marc Haber]

On Tue, 07 May 2024 22:29:30 +0100, Richard Lewis
 wrote:
>Holger Levsen  writes:
>> I'm a bit surprised how many people seem to really rely on data in /tmp
>> to survive for weeks or even months. I wonder if they backup /tmp?
>
>I use /tmp for things that fall somewhere between "needs a backup" and
>"unimportant, can be deleted whenever".

For me there is a difference between /tmp and /var/tmp. On all my
systems, /tmp has been a tmpfs for decades, /var/tmp being used for
data that is too large for tmpfs.

Even losing /tmp would probably affect some of my running programs
including the X11 session.

Greetings
Marc
-- 

Marc Haber |   " Questions are the | Mailadresse im Header
Rhein-Neckar, DE   | Beginning of Wisdom " | 
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 6224 1600402

Re: Make /tmp/ a tmpfs and cleanup /var/tmp/ on a timer by default [was: Re: systemd: tmpfiles.d not cleaning /var/tmp by default]

[Emanuele Rocca]

Hi,

On 2024-05-07 09:43, Russ Allbery wrote:
> I understand your point, which is that this pattern is out there in the
> wild and Debian is in danger of breaking existing usage patterns by
> matching the defaults of other distributions.  This is a valid point, and
> I appreciate you making it.

The more general point being that if systems have certain properties,
whether by design or by accident, people tend to rely on them if these
properties are useful for whatever reasons.

In the specific case of /var/tmp in Debian, for a very long time now the
properties have been: (1) persistent, world-writable storage (2) outside
of /home (3) available by default on all systems without any
configuration. To many, these properties make for a good place where
transient-ish work can be done without the risk of losing it upon reboot
(or power loss, or similar). Not being in /home is an important one,
because for instance /home may be regularly backed up, or it may be on a
NFS share, or who knows what else, and you may not want that who knows
why.

All of that being said, I do see the value in uniformity with other
distros, also because it surely makes things easier for maintainers.
And yes, https://xkcd.com/1172/.

It's just that changes are usually a costs/benefits tradeoff -- in the
xkcd a CPU is overheating, whereas in this case the problem to fix seems
a bit less obvious.

Re: Y2038-safe replacements for utmp/wtmp and lastlog

[Jun MO]

On Wed, 8 May 2024 at 09:21:35 +1000, Craig Small  wrote:

> I can only speak for w.  It currently prefers what it gets from 
systemd or

> elogind, effectively
> iterating over the sessions coming from sd_get_sessions() if sd_booted()
> returns true.
>
> If sd_booted() returns false, then it uses the old utmp/utmpx files for
> now. Besides the Y2038
> issue, the utmp "API" is pretty awful with things like errors pretty much
> undetectable. There is also
> the problem about who (e.g. which process) should be writing to those 
files

> as you have pointed out
> in your email.
>
> For now w/uptime will use utmp as a fallback, but I'll be happy if this
> gets updated to something better; it's a low-priority
> for me because systemd/elogind do what I need most of the time.

Thanks for explaining.

For last(1) my concern is that it will be helped to keep the original
last(1) for back-compatibility to read old wtmp files. For w(1), utmp
is only about current sessions, so there is no need to keep years-old
utmp files. Unlike last(1), there is no something like `w -f /run/utmp'.
Actually, one can run `last -f /run/utmp', and it provides output
similar to w(1)'s except missing something like process and CPU times
for each user. And as pointed out by you, w(1) currently already prefer
using infos from systemd/elogind instead of reading from utmp.

So I now think w(1) may be little need to keep the ability to read from
utmp and am also happy to it can change to use something better.

Regards,
Jun MO

Re: new upstream version fails older tests of rdepends packages

[Bill Allombert]

Le Sat, May 04, 2024 at 02:32:22PM +0200, Paul Gevers a écrit :
> Hi,
> 
> On 04-05-2024 11:39 a.m., Jerome BENOIT wrote:
> > What would be the best way to unblock the migration of gap and gap-io ?
> 
> If gap isn't going to change (which might be the easiest solution), then
> file bugs and fix those reverse dependencies. Those bugs are RC and in due
> time will cause autoremoval.

The problem is that all the debs in testing and sid are correct, it is the 
autopkgtest in
testing which are wrong (they are relying on undocumented behaviour).
They are fixed in sid.

Cheers,
-- 
Bill. 

Imagine a large red swirl here.

Re: Tool to build Debian packages not requiring root in containers ?

[Otto Kekäläinen]

Hi!


ti 7. toukok. 2024 klo 23.01 Charles Plessy  kirjoitti:

> Le Tue, May 07, 2024 at 08:17:31PM -0700, Otto Kekäläinen a écrit :
> >
> > Can you give me an example of a package you want to build and what is
> > the starting point, and I can tell you what command to issue to
> > https://salsa.debian.org/otto/debcraft to achieve it?
> >
> > It supports running Podman in user mode (=no root permissions needed),
>
> Hi Otto,
>
> it looks really great!
>
> Do you think you can make it work with Singularity/Apptainer instead of
> Podman?  Our cluster runs only singularity 3.5.2
> (https://docs.sylabs.io/guides/3.5/user-guide/).  Debian has version
> 4.1.2 in the singularity-container package.
>
> The conversion of a Docker container to the Singularity format is
> simple, and Singularity already mounts most of the local storage to make
> it visible and writable from within the container.
>

I read the docs on how Singularity is able to pull Docker images of Debian
Sid and build on top of them, and run and exec just like Docker/Podman.
Unfortunately it has its own Containerfile format (
https://docs.sylabs.io/guides/3.5/user-guide/quick_start.html#singularity-definition-files)
and the commands have their own syntax. I guess Debcraft could be extended
to support it, but that would require at least one Singularity user as
frequent contributor to test and develop Singularity-compatibility.

The entire code base is shell code. Perhaps you want to take a look if it
looks hackable for you?

Bug#1070760: ITP: buteo-syncfw -- Buteo Synchronization Framework

[Mike Gabriel]

Package: wnpp
Severity: wishlist
Owner: Mike Gabriel 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: buteo-syncfw
  Version : 0.11.3
  Upstream Contact: https://sailfishos.org/contact/
* URL : https://github.com/sailfishos/buteo-syncfw
* License : LGPL-2.1
  Programming Lang: C++
  Description : Buteo Synchronization Framework

 In Lomiri, Buteo Sync Framework will be the new backend daemon for
 syncing groupware data between a local device and remote services.
 .
 This package will be maintained by the Debian UBports Packaging Team.

Re: new upstream version fails older tests of rdepends packages

[Graham Inggs]

Hi Bill

On Wed, 8 May 2024 at 13:58, Bill Allombert  wrote:
> The problem is that all the debs in testing and sid are correct, it is the 
> autopkgtest in
> testing which are wrong (they are relying on undocumented behaviour).
> They are fixed in sid.

I think an upload of gap, with Breaks on the versions of the gap-*
packages that are wrong, should allow migration.

Regards
Graham

Re: Make /tmp/ a tmpfs and cleanup /var/tmp/ on a timer by default [was: Re: systemd: tmpfiles.d not cleaning /var/tmp by default]

[Jonathan Dowland]

On Mon May 6, 2024 at 5:01 PM BST, Luca Boccassi wrote:
> On Mon, 6 May 2024 at 16:51, Barak A. Pearlmutter 
> wrote:
> > For whatever reason, a lot of people who process large data use
> > /var/tmp/FOO/ as a place to store information that should not be
> > backed up, but also should not just disappear.
>
> Then such people, assuming they actually exist, can configure their
> custom systems accordingly upon reading the release notes before
> upgrading, as they would do anyway if installing on CentOS or any
> other major OS. Hence, not an issue either.

They actually exist. I'm been one of them, I've worked with many of
them, it's an incredibly common pattern in academic computing at least,
and changing it in Debian should be a very carefully explored decision.

You've pointed out that changing the behaviour from the default, in
either direction, is trivial. The issue is not one of individual
preference but of what is default. The long-established status quo
is not to clean /var/tmp. There is serious risk here: to users data
and correspondingly to Debian's reputation for stability, which
many of us have worked hard to maintain for a very long time.

If you think we need hard data to quantify this practice, then let's
work on a plan for how to gather that going forward, rather than
dismiss this outright because we haven't collected it.

Else-thread, Russ begs people to stop doing this. I agree people
shouldn't! We should also work on education and promotion of the
alternatives.

I'd like to hear some arguments *in favour* of making this change.
Alignment with systemd-upstream, reduced package maintenance burden
are two that I can think of, but perhaps I've missed more. These two,
IMHO, are significantly outweighed by the risks.

Please hold off making this change now and let this discussion continue.


-- 
Please do not CC me for listmail.

👱🏻  Jonathan Dowland
✎j...@debian.org
🔗   https://jmtd.net

Re: new upstream version fails older tests of rdepends packages

[Bill Allombert]

On Wed, May 08, 2024 at 03:21:12PM +, Graham Inggs wrote:
> Hi Bill
> 
> On Wed, 8 May 2024 at 13:58, Bill Allombert  wrote:
> > The problem is that all the debs in testing and sid are correct, it is the 
> > autopkgtest in
> > testing which are wrong (they are relying on undocumented behaviour).
> > They are fixed in sid.
> 
> I think an upload of gap, with Breaks on the versions of the gap-*
> packages that are wrong, should allow migration.

Agreed, but gap does not actually breaks anything, it is just the tests
in testing that are broken. So I can do that but that seems a bit artificial.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Avoiding /var/tmp for long-running compute (was: Make /tmp/ a tmpfs and cleanup /var/tmp/ on a timer by default [was: Re: systemd: tmpfiles.d not cleaning /var/tmp by default])

[Russ Allbery]

"Jonathan Dowland"  writes:

> Else-thread, Russ begs people to stop doing this. I agree people
> shouldn't! We should also work on education and promotion of the
> alternatives.

Also, helping people use better tools for managing workloads like this
that make their lives easier and have better semantics, thus improving
life for everyone.

I'm suggesting solutions that I don't have time to help implement, and of
course it will take a long time for better tools to filter into all those
clusters, so this doesn't address the immediate problem of this thread
(hence the subject change).  But based on my past experience with these
types of systems, I bet a lot of the patterns captured in software are
older ones.  Linux has a *lot* of facilities today that it didn't have, or
at least weren't widely used, five years ago.  It would be great to help
some of those improvements filter down, because they can make a lot of
these problems go away.

For example, take the case of scratch space for batch computing.  The
logical lifespan for temporary files for a batch computing job is the
lifetime of the job, whatever that may be.  (I know there are exceptions,
but here I'm just talking about defaults.)  Previously one would have to
build support into the batch job management system for creating and
managing those per-job temporary directories, and ensure the jobs support
TMPDIR or other environment variables to control where they store data,
and everyone was doing this independently.  (I've done a *lot* of this
kind of thing, once upon a time.)

But now we have mount namespaces, and systemd has PrivateTmp that builds
on top of that.  So if the job is managed by an execution manager, it can
create per-job temporary directories and it may already support (as
systemd does) the semantics of deleting the contents of those directories
on job exit, and it bind-mounts those into the process space and the
process is none the wiser.  I think all of the desirable glue may not
fully be there (controlling what underlying file system is used for
PrivateTmp, ensuring they're also excluded from normal cleanup, etc.), but
this is very close to a much better way of handling this problem that
still exposes /tmp and /var/tmp to the job so that none of the
often-crufty scientific computing software has to change.

The new capabilities that Linux now has due to namespaces are marvellous
and solve a whole lot of problems that I didn't realize were even
solvable, and right now I suspect there are huge opportunities for
substantial improvements without a whole lot of effort by just plumbing
those facilities through to higher-level layers like batch systems.  Whole
classes of long-standing problems would just disappear, or at least be
far, far easier to manage.

Substantial, substantial caveat: I have been out of this world for a
while, and maybe most of this work has already been done?  That would be
amazing.  The best possible response to this post would be for someone to
tell me I'm five years behind and the batch systems have already picked up
this work and we can just point people at them.

-- 
Russ Allbery (r...@debian.org)  

Re: new upstream version fails older tests of rdepends packages

[Paul Gevers]

Hi,

On 08-05-2024 6:06 p.m., Bill Allombert wrote:

Agreed, but gap does not actually breaks anything, it is just the tests
in testing that are broken. So I can do that but that seems a bit artificial.


Aha, that wasn't at all clear to me. If you don't want to do the 
artificial thing (which is fine, except now you depend on members of the 
release team), I'll manually schedule the tests. Maybe tomorrow.


Paul


OpenPGP_signature.asc
Description: OpenPGP digital signature

Bug#1070772: ITP: python-mutf8 -- encoders and decoders for the MUTF-8 character encoding

[Alexandre Detiste]

Package: wnpp
Severity: wishlist
Owner: Alexandre Detiste 
X-Debbugs-Cc: debian-devel@lists.debian.org, debian-pyt...@lists.debian.org

* Package name: python-mutf8
  Version : 1.0.0
  Upstream Contact: Tyler Kennedy
* URL : https://pypi.org/project/mutf8/
* License : MIT
  Programming Lang: Python
  Description : encoders and decoders for the MUTF-8 character encoding

This package contains simple pure-python as well as C encoders
and decoders for the MUTF-8 character encoding.
In most cases, it can also parse the even-rarer CESU-8.

These days, you'll most likely encounter MUTF-8
when working on files or protocols related to the JVM.
Strings in a Java .class file are encoded using MUTF-8,
strings passed by the JNI, as well as strings exported by the object serializer.

This library was extracted from Lawu,
a Python library for working with JVM class files.



I will maintain this inside DPT.

This is a new dependency of androguard

Re: Tool to build Debian packages not requiring root in containers ?

[Charles Plessy]

Le Wed, May 08, 2024 at 08:02:41AM -0700, Otto Kekäläinen a écrit :
> 
> I read the docs on how Singularity is able to pull Docker images of Debian
> Sid and build on top of them, and run and exec just like Docker/Podman.
> Unfortunately it has its own Containerfile format (
> https://docs.sylabs.io/guides/3.5/user-guide/quick_start.html#singularity-definition-files)
> and the commands have their own syntax. I guess Debcraft could be extended
> to support it, but that would require at least one Singularity user as
> frequent contributor to test and develop Singularity-compatibility.
> 
> The entire code base is shell code. Perhaps you want to take a look if it
> looks hackable for you?

Hi Otto,

I looked at the code, and while it would be easy to replace the podman
commands to run containers, I wonder if there isn't a major roadblock:

The main use of Singularity containers is to provide static images for
software.  The default is that the image is read-only and has write
access to the host filesystems.  Thus, running apt upgrade in a
singularity container isn't something that is done usually.  It might
even be impossible, although I am not expert enough to make that
statement firmly.

Is there a chance debcraft can work from a static container provided by
the user?

I think that the key problem I have is that I want to use a build Debian
packages that need no root access and that do not need to install
dependencies that need root access, and I want to do that with user
privileges only.

Have a nice day,

Charles

-- 
Charles Plessy Nagahama, Yomitan, Okinawa, Japan
Debian Med packaging team http://www.debian.org/devel/debian-med
Tooting from home  https://framapiaf.org/@charles_plessy
- You  do not have  my permission  to use  this email  to train  an AI -