Control: clone 1055919 -1
Control: reassign -1 dh-python 5.20230130+deb12u1
Making a clone of this bug for dh-python to track it getting fixed
there; MR to come.
Context:
On Thu, 16 Nov 2023 17:33:58 +1100 Stuart Prescott <[email protected]>
wrote:
> tldr: smells like a dh-python bug - I'll look at it more and reassign
> etc later.
>
>
> Staring at some build logs some more:
>
> * the dirs are generated always
> * they get copied from .../.pybuild to ../debian/$package/ always
> * they are supposed to get removed by dh_python3
> * that removal is not always working
>
> A common theme of the failures is that there are _two_
> /usr/lib/python3.11/dist-packages/.foo directories to remove and only
> one of them is being removed. For python-ansible-pygments, .pytest_cache
> was being removed by dh-python3 but .test-results was not.
>
> Adding in PYBUILD_VERBOSE=1 and some breakpoints into dh-python
> (specifically /usr/share/dh-python/dhpython/fs.py), I think there's a
> subtle bug about altering `dirs` while inside a loop from `os.walk`:
>
> for name in dirs:
> if name in ('test', 'tests') or name.startswith('.'):
> log.debug('removing dist-packages/%s', name)
> rmtree(join(root, name))
> dirs.remove(name)
>
> Removing `name` from `dirs` means that the next item is accidentally
> skipped. A classic "don't change the object you're iterating through
> while you are iterating through it" issue:
>
> In [1]: L = list(range(0, 10))
>
> In [2]: for i in L:
> ...: print(i)
> ...: L.remove(i)
> ...:
> 0
> 2
> 4
> 6
> 8
>
> Which item is not processed in the next iteration by dh_python3 now
> depends on the order in which `os.walk` lists the directories. That is
> presumably dependent on all manner of things (filesystem? collation?
> luck?). On the r-b setup and building by hand I get different results to
> within sbuild (and on the buildd).
>
> In sbuild on ext4, `find -type d` (I have memory that this reflects disk
> order?) has an order in
> ./debian/python3-ansible-pygments/usr/lib/python3.11/dist-packages of:
>
> .test-results ansible_pygments .pytest_cache
>
> while building by hand on tmpfs, I get
>
> ansible_pygments .test-results .pytest_cache
>
> For the former, accidentally skipping the directory after the one that
> gets removed isn't an issue, but for the latter it is.
--
Stuart Prescott http://www.nanonanonano.net/ [email protected]
Debian Developer http://www.debian.org/ [email protected]
GPG fingerprint 90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7
