xz instead of bzip2
Dear GCC developers, What happens here ! "Weekly snapshots now use xz compression [2017-05-24] ...instead of bzip2." Are you aware that a better implementation / format exists for this purposes?: http://lzip.nongnu.org/ Review of xz: http://lzip.nongnu.org/xz_inadequate.html Please do the right thing.. pgpZ5F7IIxkox.pgp Description: OpenPGP digital signature
Re: xz instead of bzip2
On Mon, 05 Jun 2017 20:24:50 +0200 Andreas Schwab wrote: > On Jun 05 2017, Matias Fonzo wrote: > > > Dear GCC developers, > > > > What happens here ! > > > > "Weekly snapshots now use xz compression [2017-05-24] > > ...instead of bzip2." > > > > Are you aware that a better implementation / format exists for this > > purposes?: > > > > http://lzip.nongnu.org/ > > > > Review of xz: > > > > http://lzip.nongnu.org/xz_inadequate.html > > > > > > Please do the right thing.. > > http://lists.gnu.org/archive/html/coreutils/2017-01/msg9.html > > Andreas. > I respect Jim Meyering for his career in the GNU project. I do not like that he adopted xz as his toy, but this is a personal opinion (nothing against the person really). If someone can refute the arguments found in the article, I will change my mind about lzip and I will start supporting xz. pgpvydFskj6ru.pgp Description: OpenPGP digital signature
Fw: xz instead of bzip2
Begin forwarded message: Date: Mon, 5 Jun 2017 20:17:57 -0300 From: Matias Fonzo To: R0b0t1 Subject: Re: xz instead of bzip2 On Mon, 5 Jun 2017 15:44:25 -0500 R0b0t1 wrote: > On Mon, Jun 5, 2017 at 1:08 PM, Matias Fonzo > wrote: > > Dear GCC developers, > > > > What happens here ! > > > > "Weekly snapshots now use xz compression [2017-05-24] > > ...instead of bzip2." > > > > Are you aware that a better implementation / format exists for this > > purposes?: > > > > http://lzip.nongnu.org/ > > > > Review of xz: > > > > http://lzip.nongnu.org/xz_inadequate.html > > > > > > Please do the right thing.. > > > > Hello, > > That article is rather interesting but unfortunately it does not > compare and contrast. It only lists facts of XZ but does so mostly in > a vacuum, so I can't really tell whether or not those things are > actually bad without trusting the author. Typically I would have no > trouble doing this and would want to follow up on the author's claims, > but in this case (the article is associated with lzip) I'm more wary > of spending time doing that. > > My personal experience with the XZ format finds it better for > releases, as compression tends to take longer (but give a better > ratio) than decompression. Section 2.7 is kind of interesting in that > case, as it claims that LZMA2 has a lower compression ceiling than > LZMA. I don't know how far typical compression strays from the > ideal. http://lzip.nongnu.org/lzip_benchmark.html > 2.9 is fairly minor but I do agree with it. It and the later sections > lead me to my next point: most of the discussion seems to be on the > failure of XZ's format to provide proper error checking. However in > practice I have not found anyone to rely on the error checking > provided by compressed data formats, and have had it suggested that > doing so is an exercise in futility. That most software projects > provide digests of their source archives seems to agree with this. > Trying to provide error checking within the archive itself falls prey > to the two general's problem > (https://en.wikipedia.org/wiki/Two_Generals'_Problem) and based on the > LZip author's argument error checking should probably be removed from > both formats. > > While I'm not sure this should reflect on the author, I am not able to > understand his conclusion in 2.10, 2.10.3, and 2.10.4. It doesn't seem > to be explained how is he detecting false positives (the given > formulas describe what he is doing with the false positive rate after > it is found). Thus to me it seems entirely likely he might be > interpreting things backwards; CRC algorithms actually pass through a > large number of errors compared to digest algorithms, so figure 3 > makes no sense. Besides that I can't really find out how it is > relevant to the main argument. > > Mr. Fonzo, if you think this is worth the consideration of the GCC > developers, you might consider contacting the author of that web page > so that he is able to explain it. However that is still predicated on > one of the developers wanting to entertain a discussion. > > R0b0t1. pgp9L9b3zcMLi.pgp Description: OpenPGP digital signature
Re: Steering committee, please, consider using lzip instead of xz
On Wed, 7 Jun 2017 15:25:26 -0700 Matthias Klose wrote: > On 07.06.2017 13:25, Antonio Diaz Diaz wrote: > > Dear GCC steering committee, > > > > This has been recently asked in this list[1], but in case you have > > missed it because of a subject line not explicit enough, I would > > like to appeal to you directly. > > > > [1] http://gcc.gnu.org/ml/gcc/2017-06/msg9.html > > > > Since 2017-05-24 weekly snapshots use xz compression instead of > > bzip2. I suppose this means that release tarballs will also use xz > > at some point. > > > > If this is the case, I politely request you to consider using lzip > > instead of xz. I have spent a lot of time during the last 9 years > > developing lzip and studying the xz format, and based on this > > experience I consider that lzip is a better choice than xz, now and > > in the long term. > > > > I have been developing software since the early 80s, and I am a GNU > > maintainer since 2003. You are all experienced developers. All I > > ask is that you read carefully the following references and then > > consider lzip and xz based on their technical merits. > > > > http://www.nongnu.org/lzip/xz_inadequate.html > > http://www.nongnu.org/lzip/lzip_benchmark.html#xz1 > > > > Also note that 'lzip -9' produces a tarball a 1% smaller than xz, > > in spite of lzip using half the RAM to compress and requiring half > > the RAM to decompress than xz. > > > > -rw-r--r-- 1 58765134 2017-06-07 09:13 gcc-8-20170604.tar.lz > > -rw-r--r-- 1 59367680 2017-06-07 09:13 gcc-8-20170604.tar.xz > > I proposed and implemented the change to use xz instead of bzip2 > because of the space savings compared to bzip2. I'm not commenting > on the "inadequateness" of xz, but maybe it would better help lzip to > address some project issues and promoting it as an alternative rather > than appealing to the GCC steering committee. > > - lzip is not a GNU project (afaics), same as for xz. The developer is part of the GNU project and the lzip project lives on Savannah (nongnu). The license of lzip is under the terms of the GPL. While the author of xz is not part of the GNU project, the project lives on tukaani.org, the license is public domain. > - lzip doesn't have a public VCS. The code, license and the releases are a reflection of the "public". A VCS can be a matter of personal choice, working preference. > - lzip doesn't have a documented API, doesn't build as a library, Under "Related projects" (lzip.nongnu.org): Lzlib[1] - A compression library for the lzip file format, written in C. [1] http://lzip.nongnu.org/lzlib.html >and I can't find language bindings for lzip. Well.. this could be done. lzip is *less* complex than xz. See[2]: http://lzip.nongnu.org/lzd.html > - lzip isn't (yet) used for software distributions, while xz is (and > afaics xz is used for GNU projects in addition to gz). > Mainstream distributions provides lzip: Fedora, Debian (and derivatives), Slackware, ... pgp8rwqDJDdao.pgp Description: OpenPGP digital signature
Re: Steering committee, please, consider using lzip instead of xz
On Wed, 7 Jun 2017 17:18:49 -0600 Jeff Law wrote: > On 06/07/2017 02:25 PM, Antonio Diaz Diaz wrote: > > Dear GCC steering committee, > > > > This has been recently asked in this list[1], but in case you have > > missed it because of a subject line not explicit enough, I would > > like to appeal to you directly. > > > > [1] http://gcc.gnu.org/ml/gcc/2017-06/msg9.html > > > > Since 2017-05-24 weekly snapshots use xz compression instead of > > bzip2. I suppose this means that release tarballs will also use xz > > at some point. > > > > If this is the case, I politely request you to consider using lzip > > instead of xz. I have spent a lot of time during the last 9 years > > developing lzip and studying the xz format, and based on this > > experience I consider that lzip is a better choice than xz, now and > > in the long term. > > > > I have been developing software since the early 80s, and I am a GNU > > maintainer since 2003. You are all experienced developers. All I > > ask is that you read carefully the following references and then > > consider lzip and xz based on their technical merits. > > > > http://www.nongnu.org/lzip/xz_inadequate.html > > http://www.nongnu.org/lzip/lzip_benchmark.html#xz1 > > > > Also note that 'lzip -9' produces a tarball a 1% smaller than xz, in > > spite of lzip using half the RAM to compress and requiring half the > > RAM to decompress than xz. > > > > -rw-r--r-- 1 58765134 2017-06-07 09:13 gcc-8-20170604.tar.lz > > -rw-r--r-- 1 59367680 2017-06-07 09:13 gcc-8-20170604.tar.xz > Given the breadth that xz already has in the distribution space, I'd > have a hard time supporting moving to lz over xz. > > We've got far more important items to tackle than this. But if you > want me to bring it up formally with the SC I can. > > jeff > > ps. And just to be clear, I actually don't like xz and I'm always > annoyed when I run into something delivered in xz format. But xz > support at the distro level is pretty ubiquitous at this point. All the important distributions have the lzip tool, besides GNU tar has the support for it. If not, it is easy to add lzip for decompress the source. This does not affect the preference of the distribution for X format in the package redistribution. pgptLJAKLx5ne.pgp Description: OpenPGP digital signature
Re: Steering committee, please, consider using lzip instead of xz
On Thu, 8 Jun 2017 11:42:48 +0200 Jakub Jelinek wrote: > On Thu, Jun 08, 2017 at 11:27:30AM +0200, Antonio Diaz Diaz wrote: > > Gzip was once ubiquituous in distro packages and it was replaced. > > But this time distros won't lead the change because they can work > > around the main defects of xz. As you can read in section 2.2 of > > http://www.nongnu.org/lzip/xz_inadequate.html#fragmented > > You keep referencing the marketing pages of one of the formats > comparing to other formats, that can be hardly considered unbiased. > Most of the compression formats have similar kind of pages, usually > biased as well. > > > "Distributing software in xz format can only be guaranteed to be > > safe if the distributor controls the decompressor run by the user > > (or can force the use of external means of integrity checking)". > > > > Distros control the package manager, which can even verify package > > signatures by default. For them xz, or even lzma-alone, is good > > enough. The only way for distros to change is that a significant > > number of upstream projects change first. This is why upstream > > projects willing and able to compare lzip and xz based on their > > technical merits are required to lead the way. > > For integrity checking, gcc provides the md5.sum, sha512.sum files on > gcc.gnu.org and gpg signatures on ftp.gnu.org. The choice of xz is > that it is used very widely these days, which is not the case of lzip. > This works well as a complement, but this seems to be a mere excuse to palliate the defects of the compressor, in this case of xz. It would be different if the signatures are accompanied with a well-designed compressor (like lzip). pgpj49ucJA8qn.pgp Description: OpenPGP digital signature
Re: CFLAGS for the host in canadian cross build
On Mon, 6 Nov 2017 10:30:14 -0800 Max Filippov wrote: > Hello, > > I'm trying to build a canadian cross compiler with build == > x86_64-linux, host == xtensa-linux and target == xtensa-linux. I need > to specify an xtensa-specific flag in CFLAGS that will be applied to > the host binary. > > I put this flag into CFLAGS and CXXFLAGS. The first question: is that > the right place for it? > > The build process tries to build libcpp for the build system and it > runs x86_64 gcc with that xtensa-specific option. That's where the > build breaks for me, because the option is not recognized by the > build gcc. Is it supposed to work? > > If CFLAGS/CXXFLAGS is the right place for the option, how can the > build tell whether it's intended for the build or for the host? GCC has $CFLAGS_FOR_BUILD and $CFLAGS_FOR_TARGET.
Re: CFLAGS for the host in canadian cross build
On Mon, 6 Nov 2017 11:51:34 -0800 Max Filippov wrote: > On Mon, Nov 6, 2017 at 10:38 AM, Matias Fonzo > wrote: > > On Mon, 6 Nov 2017 10:30:14 -0800 > > Max Filippov wrote: > [..] > >> > >> If CFLAGS/CXXFLAGS is the right place for the option, how can the > >> build tell whether it's intended for the build or for the host? > > > > GCC has $CFLAGS_FOR_BUILD and $CFLAGS_FOR_TARGET. > > Right, but as far as I can see CFLAGS_FOR_TARGET are not applied > when a binary for the host is built. And CFLAGS_FOR_BUILD are applied > when a binary for the build system is built. I don't see what I'd call > CFLAGS_FOR_HOST. > I am trying to find more information about HOST_CFLAGS ...
