Source: python3.7 Severity: minor User: reproducible-bui...@lists.alioth.debian.org Usertags: toolchain
Hi, I noticed that the pyc files generated by Python 3.7 (but not by earlier Python versions) is different depending on how the package is installed. I use __init__.cpython-37.pyc as an example but other .pyc files also differ. With debootstrap --include: $ sudo SOURCE_DATE_EPOCH=1545769394 LC_ALL=C.UTF-8 debootstrap --variant=minbase --include=python3 unstable debian-unstable $ md5sum debian-unstable/usr/lib/python3.7/collections/__pycache__/__init__.cpython-37.pyc df6fe61fe176e4858ce2062233d2280e debian-unstable/usr/lib/python3.7/collections/__pycache__/__init__.cpython-37.pyc Installing with apt after a minbase debootstrap: $ sudo SOURCE_DATE_EPOCH=1545769394 LC_ALL=C.UTF-8 debootstrap --variant=minbase unstable debian-unstable $ sudo SOURCE_DATE_EPOCH=1545769394 LC_ALL=C.UTF-8 chroot debian-unstable apt-get install python3 $ md5sum debian-unstable/usr/lib/python3.7/collections/__pycache__/__init__.cpython-37.pyc 2c6fdc51b035428a3881f3eef70e3a5b debian-unstable/usr/lib/python3.7/collections/__pycache__/__init__.cpython-37.pyc Manually installing with dpkg (in an order that respects Pre-Depends): $ sudo SOURCE_DATE_EPOCH=1545769394 LC_ALL=C.UTF-8 debootstrap --variant=minbase unstable debian-unstable $ sudo SOURCE_DATE_EPOCH=1545769394 LC_ALL=C.UTF-8 chroot debian-unstable apt install bzip2 file libexpat1 libmagic-mgc libmagic1 libmpdec2 libreadline7 libsqlite3-0 libssl1.1 mime-support readline-common xz-utils $ sudo SOURCE_DATE_EPOCH=1545769394 LC_ALL=C.UTF-8 chroot debian-unstable apt-get install --download-only python3 $ sudo SOURCE_DATE_EPOCH=1545769394 LC_ALL=C.UTF-8 chroot debian-unstable sh -c "cd /var/cache/apt/archives; dpkg -i libpython3.7-minimal_3.7.2~rc1-1_amd64.deb python3.7-minimal_3.7.2~rc1-1_amd64.deb" $ sudo SOURCE_DATE_EPOCH=1545769394 LC_ALL=C.UTF-8 chroot debian-unstable sh -c "cd /var/cache/apt/archives; dpkg -i python3-minimal_3.7.1-3_amd64.deb" $ sudo SOURCE_DATE_EPOCH=1545769394 LC_ALL=C.UTF-8 chroot debian-unstable sh -c "cd /var/cache/apt/archives; dpkg -i libpython3.7-stdlib_3.7.2~rc1-1_amd64.deb libpython3-stdlib_3.7.1-3_amd64.deb python3_3.7.1-3_amd64.deb python3.7_3.7.2~rc1-1_amd64.deb" $ md5sum debian-unstable/usr/lib/python3.7/collections/__pycache__/__init__.cpython-37.pyc 2c6fdc51b035428a3881f3eef70e3a5b debian-unstable/usr/lib/python3.7/collections/__pycache__/__init__.cpython-37.pyc So manually installing with dpkg in the right order yields the same hash as with apt. So despite bug #917386 this might not be an apt bug. I found that depending on the postinstall that is executed, the hash is different. So I removed __init__.cpython-37.pyc and then ran dpkg-reconfigure for several packages. Here are the hashes of __init__.cpython-37.pyc depending on the postinst that got run: python3.7: c06080cb5a72be8e409814025236f097 python3.7-minimal: df6fe61fe176e4858ce2062233d2280e python3-minimal: b940cc549d39dd3e521a41b8eba3a295 python3: b940cc549d39dd3e521a41b8eba3a295 Debian policy ยง6.2 requires idempotency for individual maintainer scripts which is not violated here. Nevertheless I would argue, that it would make sense for different Python3 maintainer scripts not to leave the same files with different content depending on which maintainer script is being run. As by the analysis above, there exist at least five different ways that pyc files can end up. For the sake of reproducibility, there should probably only be one result independent on how Python3 is installed. Thanks! cheers, josch
