This seems to me to be a serious problem blocking all
java-helper-using packages
So, I took a look at what's going on and found the following:
The short version is that Archive::Zip is misbehaving as jh_manifest
uses it and making corrupt jars. A patch to workaround that is
attached. It may be that a fix in Archive::Zip is a better solution.
As it took me a while to get there I'll leave my working in in case it
helps anyone else working on the issue:
If I build the package in squeeze everything is fine.
If I build it in wheezy then running it:
terraintool
gives CRC error.
Looking a little more closely. This is OK:
java -jar /usr/share/terraintool/terraintool.jar
It seems that what the binfmt support does is run this:
jarwrapper /usr/share/terraintool/terraintool.jar
That give CRC error.
jarwrapper is just a shell script. The bit that dies is this:
fastjar xf /usr/share/terraintool/terraintool.jar META-INF/MANIFEST.MF
Niels Thykier suggested seeing if LD_PREloading
it was sugested to see if replacing memcpy with memmove using
LD_PRELoAD fixed the situation. I found that you do that like this:
Create a file named "memcpy_hider.c" containing
#include <string.h>
void* memcpy(void *dst, const void *src, size_t size)
{
memmove(dst,src,size);
}
Then compile it with
gcc -fPIC -O2 -c memcpy_hider.c
ld -G memcpy_hider.o -o memcpy_hider.so
Finally, to test it out, run your browser like
LD_PRELOAD=/abolute/path/to/memcpy_hider.so program
So that built OK. I ran:
LD_PRELOAD=/home/wookey/debian/wheezy/memcpy_hider.so fastjar xf
/usr/share/terraintool/terraintool.jar META-INF/MANIFEST.MF
and it still gives a CRC error :-(
So either I've done something wrong or the memcpy thing is not the
problem here?
This suggests that the _hider lib is being used
LD_PRELOAD=/home/wookey/debian/wheezy/memcpy_hider.so ldd /usr/bin/fastjar
linux-vdso.so.1 => (0x00007fffc55ff000)
/home/wookey/debian/wheezy/memcpy_hider.so (0x00007f8819ebf000)
libz.so.1 => /lib/libz.so.1 (0x00007f8819c81000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f88198fc000)
/lib64/ld-linux-x86-64.so.2 (0x00007f881a0c2000)
stracing the fastjar command shows it falling over as soon as it tries
to write some output when unpacking the jar
open("/usr/share/terraintool/terraintool.jar", O_RDONLY) = 3
read(3, "PK\3\4", 4) = 4
read(3, "\24\0\0\0\0\0S\217%@\0\0\0\0\2\0\0\0\0\0\0\0\t\0\4\0", 26) = 26
read(3, "META-INF/", 9) = 9
lseek(3, 4, SEEK_CUR) = 43
read(3, "PK", 2) = 2
write(2, "Error! CRCs do not match! Got 1a"..., 51Error! CRCs do not match! Got
1a6cd7b3, expected 0 ) = 51
So it's opened the jar, found the PKZIp header, found the META-INF
directory entry, skipped 4 bytes, found PK again and died.
Some perusal of the PKZIP header spec:
http://www.pkware.com/documents/casestudies/APPNOTE.TXT
od -tx1 -c /usr/share/terraintool/terraintool.jar |less
show that that header contians:
local file header signature 4 bytes (0x04034b50)
version needed to extract 2 bytes
general purpose bit flag 2 bytes
compression method 2 bytes
last mod file time 2 bytes
last mod file date 2 bytes
crc-32 4 bytes
compressed size 4 bytes
uncompressed size 4 bytes
file name length 2 bytes
extra field length 2 bytes
file name (variable size)
extra field (variable size)
B. File data
Immediately following the local header for a file
is the compressed or stored data for the file.
0000000 50 4b 03 04 14 00 00 00 00 00 53 8f 25 40 00 00
P K 003 004 024 \0 \0 \0 \0 \0 S 217 % @ \0 \0
0000020 00 00 02 00 00 00 00 00 00 00 09 00 04 00 4d 45
\0 \0 002 \0 \0 \0 \0 \0 \0 \0 \t \0 004 \0 M E
0000040 54 41 2d 49 4e 46 2f fe ca 00 00 50 4b 03 04 14
T A - I N F / 376 312 \0 \0 P K 003 004 024
This seems to say that this entry has uncompressed size 0 and
compressed size 2 - is that right?
and this is a header for another directory:
50 4b 03 04 0a 00 00 00 00 00
P K 003 004 \n \0 \0 \0 \0 \0
0002120 4e 8f 25 40 00 00 00 00 00 00 00 00 00 00 00 00
N 217 % @ \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
0002140 08 00 00 00 6d 63 63 6f 6d 62 65 2f 50 4b 03 04
\b \0 \0 \0 m c c o m b e /
this has size 0 for both compressed and uncompressed.
doing unzip -t /usr/share/terraintool/terraintool.jar
Archive: /usr/share/terraintool/terraintool.jar
META-INF/: ucsize 0 <> csize 2 for STORED entry
continuing with "compressed" size value
testing: META-INF/ bad CRC 1a6cd7b3 (should be 00000000)
testing: Tester.class OK
testing: mccombe/ OK
hmm, so the jar is corrupt. with the wrong size for the compressed
file. So fastjar reads 2 extra bytes after the file entry and gets
'PK'. And the crc of "PK" is 1a6cd7b3 which is why you always get that
number no matter what .jar is used.
OK, we're getting somewhere now. And it's nothing to do with
memcpy/memmove - that was a red herring.
Earlier mails say jh_manifest is at fault. That's a perl script that
is responsible for making this jar (zip archive). It uses
libarchive-zip-perl to do the zipifying.
So why is it now making corrupt archives when it wasn't before?
The version in squeeze is 1.30-3, and in wheezy 1.30.4. But perl
itself has gone from 5.10.1-17 to 5.14.2-6. Could that be it?
Javahelper has gone from 0.32 to 0.37 and jh_manfest has changed a
little but not obviously in areas that matter.
jar if jh_manifest not run:
0000000 50 4b 03 04 14 00 08 00 08 00 c1 18 26 40 00 00
P K 003 004 024 \0 \b \0 \b \0 301 030 & @ \0 \0
0000020 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
\0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \t \0 004 \0 M E
0000040 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
T A - I N F / 376 312 \0 \0 003 \0
jar after jh_manifest is run:
0000000 50 4b 03 04 14 00 00 00 00 00 c1 18 26 40 00 00
P K 003 004 024 \0 \0 \0 \0 \0 301 030 & @ \0 \0
0000020 00 00 02 00 00 00 00 00 00 00 09 00 04 00 4d 45
\0 \0 002 \0 \0 \0 \0 \0 \0 \0 \t \0 004 \0 M E
0000040 54 41 2d 49 4e 46 2f fe ca 00 00 50 4b 03 04 0a
T A - I N F / 376 312 \0 \0
The relevant code in jh_manifest is the end of 'update_jar':
verbose_print("Updating manifest in $jar");
$zip->removeMember( 'META-INF/MANIFEST.MF' ) unless($new_manifest);
$mem = $zip->addString($var, 'META-INF/MANIFEST.MF');
$mem->desiredCompressionMethod(COMPRESSION_DEFLATED);
# This on the other hand may fail.
$zip->overwrite() == AZ_OK or error("Writing modified jar ($jar) failed$
so the issue, which may well actually be a bug in Archive::Zip given
that this presumably used to work is that nothing explicit is done
with the META-INF/ directory entry member in the archive, but by
creating a new 'META-INF/MANIFEST.MF' member is gets corrupted.
We can fix it by explicitly removing and recreating it:
verbose_print("Updating manifest in $jar");
$zip->removeMember( 'META-INF/MANIFEST.MF' ) unless($new_manifest);
$zip->removeMember( 'META-INF/' );
$mem = $zip->addString($var, 'META-INF/MANIFEST.MF');
$mem->desiredCompressionMethod(COMPRESSION_DEFLATED);
$zip->addDirectory( 'META-INF/' );
# This on the other hand may fail.
$zip->overwrite() == AZ_OK or error("Writing modified jar ($jar) failed$
which I hope is an acceptable fix. Patch attached.
Wookey
--
Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM
http://wookware.org/
--- jh_manifest 2012-01-06 04:02:57.226942289 +0000
+++ jh_manifest.fixed 2012-01-06 04:10:22.873152146 +0000
@@ -334,8 +334,10 @@
close($fd);
verbose_print("Updating manifest in $jar");
$zip->removeMember( 'META-INF/MANIFEST.MF' ) unless($new_manifest);
+ $zip->removeMember( 'META-INF/' );
$mem = $zip->addString($var, 'META-INF/MANIFEST.MF');
$mem->desiredCompressionMethod(COMPRESSION_DEFLATED);
+ $zip->addDirectory( 'META-INF/' );
# This on the other hand may fail.
$zip->overwrite() == AZ_OK or error("Writing modified jar ($jar) failed: $!");
} else {
