tags 943327 patch
thanks
On Wed, 23 Oct 2019 15:52:26 +0200 Benjamin Drung <
benjamin.dr...@cloud.ionos.com> wrote:
> Am Mittwoch, den 23.10.2019, 14:56 +0200 schrieb Johannes Schauer:
> > Hi,
> >
> > Quoting Benjamin Drung (2019-10-23 14:14:15)
> > > one of mmdebstrap benefits over deboostrap is that it is
> > > faster. Creating a
> > > xz tarball as output will take a lot of time, since xz consumes a
> > > lot of
> > > compute power and tar uses only one core.
> > >
> > > pixz is a pallalel version of xz that can speedup the compression a
> > > lot.
> > > It can be simply used by tar by specifying -Ipixz.
> > >
> > > So please support using pixz when creating xz tarballs, maybe even
> > > as default
> > > if pixz can be found.
> >
> > I never heard of pixz. Is it still actively developed? I see that the
> > last
> > release is already four years old and in Debian we have the same
> > version since
> > oldstable. I just don't want to add code to support unmaintained or
> > rarely used
> > software.
>
> I wasn't aware that pixz hasn't have a lot of development recently, but
> I do not see any signs that it is dead. It look like it is being
> feature complete and under maintenance now.
>
> > Currently, mmdebstrap supports all compression file endings supported
> > by tar
> > and tar currently does not know about pixz. Is there a reason why tar
> > maintainers thought they don't add that compressor to their list?
>
> This question triggered me to search the web and I found out, that xz
> gained support for parallel compression files (only compressing, but
> not decompressing):
>
> $ time tar -J -cf root1.tar.xz -C root .
> real 5m19,349s
> user 5m19,775s
> sys 0m4,185s
>
> $ time tar -Ipixz -cf root2.tar.xz -C root .
> real 0m59,250s
> user 10m23,438s
> sys 0m3,734s
>
> $ time tar -I"xz -T 0" -cf root3.tar.xz -C root .
> real 1m0,764s
> user 10m43,783s
> sys 0m3,892s
>
> So it would be nice if mmdebstrap would use -I"xz -T 0" for compressing
> xz in parallel. This does not require addition dependencies!
Attached a simple patch that adds the -I"xz -T 0" option when using xz
as compression.
--
Benjamin Drung
Debian & Ubuntu Developer
Platform Engineering Compute (Enterprise Cloud)
1&1 IONOS SE | Greifswalder Str. 207 | 10405 Berlin | Germany
E-mail: benjamin.dr...@cloud.ionos.com | Web: www.ionos.de
Hauptsitz Montabaur, Amtsgericht Montabaur, HRB 24498
Vorstand: Dr. Christian Böing, Hüseyin Dogan, Hans-Henning Kettler,
Matthias Steinberg, Achim Weiß
Aufsichtsratsvorsitzender: Markus Kadelke
Member of United Internet
From 07e3917529792be9f89083752c5ee887d96d2278 Mon Sep 17 00:00:00 2001
From: Benjamin Drung <benjamin.dr...@cloud.ionos.com>
Date: Tue, 12 Nov 2019 19:17:30 +0100
Subject: [PATCH] Use parallel xz compression
One of mmdebstrap benefits over deboostrap is that it is faster.
Creating a xz tarball as output will take a lot of time, since xz
consumes a lot of compute power and tar uses only one core.
Therefore use parallel xz compression since xz supports it using the -T
parameter.
Closes: #943327
Signed-off-by: Benjamin Drung <benjamin.dr...@cloud.ionos.com>
---
mmdebstrap | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/mmdebstrap b/mmdebstrap
index c8dbbe5..ae8b397 100755
--- a/mmdebstrap
+++ b/mmdebstrap
@@ -2395,6 +2395,11 @@ sub main() {
my $exitstatus = 0;
my @taropts = ('--sort=name', "--mtime=\@$mtime", '--clamp-mtime', '--numeric-owner', '--one-file-system', '-c', '--exclude=./dev');
+ # Use parallel xz compression
+ if (get_tar_compressor($options->{target}) eq 'xz') {
+ push @taropts, "-Ixz -T 0"
+ }
+
# disable signals so that we can fork and change behaviour of the signal
# handler in the parent and child without getting interrupted
my $sigset = POSIX::SigSet->new(SIGINT, SIGHUP, SIGPIPE, SIGTERM);
--
2.20.1
