On Mon, 03 Jul 2023 at 09:12:08 +0300, Dmitry Baryshkov wrote: > I tried building the quake shareware packages on my fresh iMX53 install > (Debian bookworm, armhf port). However I faced an issue with > game-data-packager being unable to identify any source file, because it > miscomputes sha1 sums. As you can see in the following fragment, md5 and > sha256 sums match, but sha1 doesn't: > > $ game-data-packager quake > INFO:game_data_packager.download:downloading > ftp://ftp.fu-berlin.de/pc/msdos/games/idgames/idstuff/quake/quake106.zip > WARNING:game_data_packager.build:found quake106.zip but it is not one of the > expected versions: > file: /tmp/gdptmp.0aow0955/quake106.zip > size: 9094045 bytes > md5: 8cee4d03ee092909fdb6a4f84f0c1357 > sha1: e24e7b1c5030bf38d6caaa71dcd54469087aef0f > sha256: ec6c9d34b1ae0252ac0066045b6611a7919c2a0d78a3a66d9387a8f597553239 > expected: > quake106.zip: > size: 9094045 bytes > md5: 8cee4d03ee092909fdb6a4f84f0c1357 > sha1: f8a1a509b094ccdbed3c54b96f7d9b351c0898f5 > sha256: ec6c9d34b1ae0252ac0066045b6611a7919c2a0d78a3a66d9387a8f597553239
If I'm understanding you correctly, your use of Quake and quake106.zip is just an example, and you get similar results (md5 and sha256 correct, sha1 wrong) for other files and other games? Is that true? Is this a general problem with Python on this hardware? game-data-packager doesn't do anything particularly special to calculate hashes, it just uses the Python standard library to do it. Possible steps to reproduce (for simplicity this just reads the whole file into RAM, so it will briefly require a few MB): $ wget ftp://ftp.fu-berlin.de/pc/msdos/games/idgames/idstuff/quake/quake106.zip ... $ sha1sum quake106.zip f8a1a509b094ccdbed3c54b96f7d9b351c0898f5 quake106.zip $ python3 >>> import hashlib >>> hasher = hashlib.new('sha1') >>> with open('quake106.zip', 'rb') as reader: ... hasher.update(reader.read()) ... >>> hasher.hexdigest() 'f8a1a509b094ccdbed3c54b96f7d9b351c0898f5' If this gives a different answer for you on the affected hardware, then this is a general problem with hashlib and should be reassigned to python3. Or if that code reports f8a1... for you too, the next thing to try would be reading it a block at a time. The fact that gdp is reporting the correct md5 and sha256 suggests that hashlib (and gdp's code that interfaces with hashlib) is basically working correctly, but the sha1 implementation might be doing something that is only valid for newer CPUs or something like that? Thanks, smcv
