On Sat, Jul 07, 2012 at 04:22:58PM -0600, Stefano Zacchiroli wrote: > On Sun, Jul 08, 2012 at 12:03:44AM +0200, Cyril Brulebois wrote: > > Ansgar Burchardt <ans...@debian.org> (07/07/2012): > > > Another question is if the release team would consider unblocking > > > updated packages (with just the switch to xz for binaries). I think > > > it would be nice to be able to get a useful desktop system using just > > > the first CD, but I'm not sure if they would make an exception for > > > this. > > > > You may want to actually ask the release team at some point, if you want > > to know. Just saying… > > Thanks for the brilliant idea! :-) > > Oh all mighty release team, > Ansgar has been experimenting with .deb sizes to make the packages > needed for a minimal desktop installation fit in the first CD. It looks > like that's doable by switching to xz compression for the involved > binaries. Would you grant freeze exceptions for packages that only > changes that?
Note that has been raised in LOOOONG threads twice in less than a year.
So let's rehash the findings:
* XZ is almost always better:
• xz -0 is twice as fast as gzip, with 78% the size (tested on a random
92MB unstripped amd64 executable)
• xz -3 is at par with gzip -9's compression speed
• xz -6 (the default) is a lot slower when compressing, fast when
decompressing, needs only 10MB memory, 58% size
• xz -9 has very slow compression, takes gobs of memory, 56% size
(Obviously, the "size" numbers are dragged down by uncompressible files
when you look at the whole archive.)
• It has somewhat slow start: small files may compress better with gzip,
but never to a degree that would justify the complexity of switching
compression algorithms.
* After recompressing the whole archive, it turns out compressing only
biggest packages is not a good idea: the bulk of space is taken by
medium-sized packages, which recompress almost as good. Thus, for best
effects, xz should be dpkg-deb's default.
* Any native ways to install support xz:
• dpkg does (squeeze but not lenny)
• d-i does (via busybox)
* Debootstrap relies on unxz being available on the running system. Which
might be not there for ancient ones.
Ie, the only show-stopper is debootstrap on foreign systems.
Here, there are two approaches:
* making sure all .debs debootstrap has to unpack use gzip.
• explicitely forcing gzip in the packages affected would be error-prone
• dpkg-deb doesn't know which packages are base, as transitive
dependencies make prority itself not enough
On the other hand, checking this by hand and rejecting the package only
during/after upload could work as a short-term solution.
* letting debootstrap handle xz files.
• using arch-dependent executables is probably a bad idea
• unxz would be hard to re-implement in Perl or something else typically
available on bare systems, Mirabilos tells me some BSDs don't even have
Perl. No other reasonable language is likely to be there.
• having a bunch of unxz executables for every architecture is doable, but
would be damn hard to build in an automated way, especially if we want
debootstrap to handle unofficial ports. Currently, gcc needs libc6-dev
for the target system to cross-compile.
• alternatively, one could use stand-alone binaries with some heavy
hackery. Unxz-embedded can be whipped into working with only three
syscalls: read(), write() and _exit(), perhaps also some way to fetch
memory. This could be built without any headers, for any architecture
gcc supports. For an extra bonus, such ultra-static executables would
reduce the number of variants: one file is needed for i386, x32 and
amd64; armel and armhf, etc. Sadly, the ELF header restricts
architectures, or with a proper trampoline you could even have one
executable work on twenty or more archs :p
I did not hear Joey Hess or others chime in: would you accept such an
universal binary into debootstrap? It already ships a tarball (one with
required /dev nodes).
* Repacking existing .debs is not a good idea for the main archive (even if
it works "at home"). I doubt no-recompilation binNMUs would be safe, too.
Also, switching to XZ debs is not just a CD issue, doing so would
drastically reduce the resource usage of mirrors.
--
Copyright and patents were never about promoting culture and innovations;
from the very start they were legalized bribes to give the king some income
and to let businesses get rid of competition. For some history, please read
https://en.wikipedia.org/wiki/Statute_of_Monopolies_1623
signature.asc
Description: Digital signature
