On 2024-08-24 11:28, Étienne Mollier wrote:
Hi Reinhard,
Reinhard Tartler, on 2024-08-24:
Actually, I've been able to compile pystack on the machine
(see https://github.com/bloomberg/pystack, and amazing tool)
and was able to get a trace of what the stuck process is doing:
This is a very handy tool and I need to recall about it when
I'll next need it, probably too soon for my taste. :)
[…]
(Python) File
"/home/siretart/python-biopython/.pybuild/cpython3_3.12/build/Bio/Align/analysis.py",
line 484, in _yn00
P = expm(Q * t)
(Python) File
"/usr/lib/python3/dist-packages/scipy/linalg/_matfuncs.py", line 358,
in
expm
eAw = eAw @ eAw
I think that indicates that the problematic / looping test is indeed
Align_condonalign.py?
This is highly plausible. It was a bold assumption from me to
believe the standard output would be flushed to let us see
whether the test started.
You know what, I tried something crazy. On the porterbox that I used to
reproduce it, I pressed Ctrl-C to interrupt the hanging process. This is
the output:
======================================
(sid_riscv64-dchroot)siretart@ricci:~/python-biopython$
test_Align_codonalign ... Traceback (most recent call last):
File
"/home/siretart/python-biopython/.pybuild/cpython3_3.12/build/Tests/run_tests.py",
line 343, in <module>
errors = main(sys.argv[1:])
^^^^^^^^^^^^^^^^^^
File
"/home/siretart/python-biopython/.pybuild/cpython3_3.12/build/Tests/run_tests.py",
line 187, in main
return runner.run()
^^^^^^^^^^^^
File
"/home/siretart/python-biopython/.pybuild/cpython3_3.12/build/Tests/run_tests.py",
line 324, in run
ok = self.runTest(test)
^^^^^^^^^^^^^^^^^^
File
"/home/siretart/python-biopython/.pybuild/cpython3_3.12/build/Tests/run_tests.py",
line 281, in runTest
suite.run(result)
File "/usr/lib/python3.12/unittest/suite.py", line 122, in run
test(result)
File "/usr/lib/python3.12/unittest/suite.py", line 84, in __call__
return self.run(*args, **kwds)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/unittest/suite.py", line 122, in run
test(result)
File "/usr/lib/python3.12/unittest/case.py", line 690, in __call__
return self.run(*args, **kwds)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/unittest/case.py", line 634, in run
self._callTestMethod(testMethod)
File "/usr/lib/python3.12/unittest/case.py", line 589, in
_callTestMethod
if method() is not None:
^^^^^^^^
File
"/home/siretart/python-biopython/.pybuild/cpython3_3.12/build/Tests/test_Align_codonalign.py",
line 8242, in test_dn_ds
dN, dS = calculate_dn_ds(pairwise_alignment, method="YN00")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/siretart/python-biopython/.pybuild/cpython3_3.12/build/Bio/Align/analysis.py",
line 88, in calculate_dn_ds
return _yn00(codons1, codons2, codon_table)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/siretart/python-biopython/.pybuild/cpython3_3.12/build/Bio/Align/analysis.py",
line 484, in _yn00
P = expm(Q * t)
^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/scipy/linalg/_matfuncs.py", line
358, in expm
eAw = eAw @ eAw
^^^
KeyboardInterrupt
================================
Note the 'test_Align_codonalign ... ' part at the beginning. That means
that python's
unittest library indeed wrote that to stdout before running the test,
but because
of the missing '\n' (newline), it didn't get printed because the buffer
didn't get
flushed yet. That's probably not something that could be fixed in the
biopython
package, and would probably affect other python software equally as
well.
The stack trace is even pointing to
something off in scipy/linalg/_matfunc.py, even more so as scipy
is only suggested by python3-biopython, so it is not available
in the lightweight variant of the autopkgtest, which does run
the test_Align_codonalign.py but doesn't hangup. And indeed,
here is what I see in test_Align_codonalign.py, starting line
8235:
try:
import scipy
except ImportError:
# Silently skip the rest of the test
return
My current conclusion is: it seems to me that Biopython's test
suite revealed something off in SciPy specifically on Risc-V.
I tend to agree. Looking at a previous testrun where this test passes,
https://ci.debian.net/packages/p/python-biopython/testing/riscv64/50850003/
we see this output:
744s test_Align_clustal ... ok
778s test_Align_codonalign ... ok
779s test_Align_emboss ... ok
781s test_Align_exonerate ... ok
Look at the timestamps. There is no way that test_Align_codonalign
actually
passes in less than 2s, in particular not on riscv64. How did it pass so
fast?
I believe the code that you showed above could have something to do with
this.
Maybe we can disable test_Align_codonalign, at least on riscv64, and see
if
that improves things? -- I'm also skeptical about the "silent skip"
above.
Maybe that code above could be changed to print on stderr to provide
better
diagnostics?
best,
-rt
