Bug#647522: non-deterministic compression results with gzip -n9

2012-03-18 Thread Paul Eggert
Cyril, thanks for the test case. When I used 'valgrind' on it I found where gzip is accessing uninitialized data. I pushed into gzip master the patch at the end of this message; it fixed things for me. The Debian patch, which zeros out a lot of buffers, should work if gzip is compressing regular

Bug#647522: non-deterministic compression results with gzip -n9

2012-02-12 Thread Cyril Brulebois
Paul Eggert (08/02/2012): > Thanks very much for the patch. But can someone who's looked into it > please explain why 'window' needs to be zeroed out? This will save me > time in reviewing the patch. Thanks. Welcome. Here's a slightly more detailed analysis. You can find attached the sample fi

Bug#647522: non-deterministic compression results with gzip -n9

2012-02-08 Thread Paul Eggert
Thanks very much for the patch. But can someone who's looked into it please explain why 'window' needs to be zeroed out? This will save me time in reviewing the patch. Thanks. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Cont

Bug#647522: non-deterministic compression results with gzip -n9

2012-02-08 Thread Riku Voipio
Thanks Cyril for tracking this down. On Wed, Feb 08, 2012 at 01:15:00PM +0100, Cyril Brulebois wrote: > I think at least the attached patch won't hurt (when the DYN_ALLOC part > is fixed; and possibly turning that into a MEMSET-like macro). > And given dh_compress is passing files in an arbitrary

Bug#647522: non-deterministic compression results with gzip -n9

2012-02-08 Thread Fabian Greffrath
I think at least the attached patch won't hurt (when the DYN_ALLOC part is fixed; and possibly turning that into a MEMSET-like macro). Just an idea, but couldn't ZEROIFY in the DYN_ALLOC part be defined as free() and subsequent calloc() of the arrays preserving their size? - Fabian -- To

Bug#647522: non-deterministic compression results with gzip -n9

2012-02-08 Thread Cyril Brulebois
Cyril Brulebois (08/02/2012): > Playing on amd64: > cbrulebois@Cygnus:/tmp/ppl-0.11.2$ cp ../ppl-pristine/{CREDITS,README} . > cbrulebois@Cygnus:/tmp/ppl-0.11.2$ gzip -9nf CREDITS README > cbrulebois@Cygnus:/tmp/ppl-0.11.2$ ls -l *gz > -rw-r--r-- 1 cbrulebois cbrulebois 6343 Feb 8 12:34 CREDITS.g

Bug#647522: non-deterministic compression results with gzip -n9

2012-02-08 Thread Cyril Brulebois
Zack Weinberg (07/02/2012): > I've seen inexplicable nondeterminism like this before, and quite > often it's turned out to be controlled by the total size of the > command line argument area (that is, argv + environ + ELF auxv). FWIW, a quick look on kfreebsd-amd64 with ppl's CREDITS file led me

Bug#647522: non-deterministic compression results with gzip -n9

2012-02-07 Thread Zack Weinberg
I've seen inexplicable nondeterminism like this before, and quite often it's turned out to be controlled by the total size of the command line argument area (that is, argv + environ + ELF auxv). Changes in how big that is change the initial stack pointer address, and while that *shouldn't* matter t

Bug#647522: non-deterministic compression results with gzip -n9

2012-02-07 Thread Uli Martens
Hello there, using a small tool I wrote a while back, it's possible to see that the actuall differences are in the encoding of the last few bytes of cleartext. I've copied the decoding tool to http://youam.net/devel/rfc1952-dec Attached is the difference of the decoded files from <20111208102738

Bug#647522: non-deterministic compression results with gzip -n9

2011-12-09 Thread Paul Eggert
I should add that it's OK (from the point of view of the RFCs) if gzip produces different outputs given the same inputs when compressing. The RFCs allow that and presumably other gzip implementations do that. All that's required is that compress+decompress result in a copy of the original. That

Bug#647522: non-deterministic compression results with gzip -n9

2011-12-09 Thread Paul Eggert
On 12/08/11 02:27, Riku Voipio wrote: > According to gzip RFC, the last 4 bytes are ISIZE, which should be > uncompressed input size. Which leaves me rather baffled how that can > differ on same input files - and how gunzip is completly happy with > both versions of compressed file, producing the s