Your message dated Fri, 12 Apr 2019 17:18:39 +0000
with message-id <e1hezp1-0004oa...@fasolo.debian.org>
and subject line Bug#926930: fixed in joblib 0.13.0-2
has caused the Debian Bug report #926930,
regarding joblib: FTBFS (test_nested_parallelism_limit does not always work)
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ow...@bugs.debian.org
immediately.)
--
926930: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926930
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
--- Begin Message ---
Package: src:joblib
Version: 0.13.0-1
Severity: serious
Tags: ftbfs patch
Dear maintainer:
I tried to build this package in buster but it failed:
--------------------------------------------------------------------------------
[...]
debian/rules build-indep
dh build-indep --buildsystem=python_distutils --with=python2,python3
dh_update_autotools_config -i -O--buildsystem=python_distutils
dh_auto_configure -i -O--buildsystem=python_distutils
dh_auto_configure: Please use the third-party "pybuild" build system instead of
python-distutils
dh_auto_configure: This feature will be removed in compat 12.
dh_auto_build -i -O--buildsystem=python_distutils
dh_auto_build: Please use the third-party "pybuild" build system instead of
python-distutils
dh_auto_build: This feature will be removed in compat 12.
python setup.py build --force
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
[... snipped ...]
joblib/test/test_testing.py::test_check_subprocess_call PASSED
joblib/test/test_testing.py::test_check_subprocess_call_non_matching_regex
PASSED
joblib/test/test_testing.py::test_check_subprocess_call_wrong_command PASSED
joblib/test/test_testing.py::test_check_subprocess_call_non_zero_return_code
PASSED
joblib/test/test_testing.py::test_check_subprocess_call_timeout
/<<PKGBUILDDIR>>/joblib/testing.py:50: UserWarning: Timeout running
['/usr/bin/python2.7', '-c', 'import time\nimport sys\nprint("before sleep on
stdout")\nsys.stdout.flush()\nsys.stderr.write("before sleep on
stderr")\nsys.stderr.flush()\ntime.sleep(1.1)\nprint("process should have be
killed before")\nsys.stdout.flush()']
warnings.warn("Timeout running {}".format(cmd))
PASSED
joblib/test/data/create_numpy_pickle.py::joblib.test.data.create_numpy_pickle.get_joblib_version
PASSED
=================================== FAILURES ===================================
_____________________ test_nested_parallelism_limit[loky] ______________________
backend = 'loky'
@with_multiprocessing
@parametrize('backend', ['loky', 'threading'])
def test_nested_parallelism_limit(backend):
with parallel_backend(backend, n_jobs=2):
backend_types_and_levels = _recursive_backend_info()
if cpu_count() == 1:
second_level_backend_type = 'SequentialBackend'
max_level = 1
else:
second_level_backend_type = 'ThreadingBackend'
max_level = 2
top_level_backend_type = backend.title() + 'Backend'
expected_types_and_levels = [
(top_level_backend_type, 0),
(second_level_backend_type, 1),
('SequentialBackend', max_level),
('SequentialBackend', max_level)
]
> assert backend_types_and_levels == expected_types_and_levels
E AssertionError: assert [('LokyBacken...lBackend', 2)] ==
[('LokyBackend...lBackend', 1)]
E At index 1 diff: ('ThreadingBackend', 1) != ('SequentialBackend', 1)
E Full diff:
E [('LokyBackend', 0),
E - ('ThreadingBackend', 1),
E - ('SequentialBackend', 2),
E ? ^
E + ('SequentialBackend', 1),...
E
E ...Full output truncated (7 lines hidden), use '-vv' to show
joblib/test/test_parallel.py:1452: AssertionError
___________________ test_nested_parallelism_limit[threading] ___________________
backend = 'threading'
@with_multiprocessing
@parametrize('backend', ['loky', 'threading'])
def test_nested_parallelism_limit(backend):
with parallel_backend(backend, n_jobs=2):
backend_types_and_levels = _recursive_backend_info()
if cpu_count() == 1:
second_level_backend_type = 'SequentialBackend'
max_level = 1
else:
second_level_backend_type = 'ThreadingBackend'
max_level = 2
top_level_backend_type = backend.title() + 'Backend'
expected_types_and_levels = [
(top_level_backend_type, 0),
(second_level_backend_type, 1),
('SequentialBackend', max_level),
('SequentialBackend', max_level)
]
> assert backend_types_and_levels == expected_types_and_levels
E AssertionError: assert [('ThreadingB...lBackend', 2)] ==
[('ThreadingBa...lBackend', 1)]
E At index 1 diff: ('ThreadingBackend', 1) != ('SequentialBackend', 1)
E Full diff:
E [('ThreadingBackend', 0),
E - ('ThreadingBackend', 1),
E - ('SequentialBackend', 2),
E ? ^
E + ('SequentialBackend', 1),...
E
E ...Full output truncated (7 lines hidden), use '-vv' to show
joblib/test/test_parallel.py:1452: AssertionError
============== 2 failed, 1099 passed, 19 skipped in 70.93 seconds ==============
[INFO:MainProcess:Dummy-463] process shutting down
[DEBUG:MainProcess:Dummy-463] running all "atexit" finalizers with priority >= 0
[DEBUG:MainProcess:Dummy-463] Interpreter shutting down. Waking up
queue_manager_threads [(<Thread(QueueManagerThread, started daemon
139974192133888)>, <joblib.externals.loky.process_executor._ThreadWakeup
instance at 0x7f4e544197a0>)]
[DEBUG:MainProcess:QueueManagerThread] queue management thread shutting down
[DEBUG:MainProcess:QueueManagerThread] closing call_queue
[DEBUG:MainProcess:QueueManagerThread] telling queue thread to quit
[DEBUG:MainProcess:QueueManagerThread] joining processes
[DEBUG:MainProcess:QueueFeederThread] feeder thread got sentinel -- exiting
[INFO:LokyProcess-239:MainThread] Shutting down worker on sentinel
[INFO:LokyProcess-239:MainThread] process shutting down
[DEBUG:LokyProcess-239:MainThread] running all "atexit" finalizers with
priority >= 0
[DEBUG:LokyProcess-239:MainThread] running the remaining "atexit" finalizers
[INFO:LokyProcess-239:MainThread] process exiting with exitcode 0
[INFO:LokyProcess-239:Dummy-1] process shutting down
[DEBUG:LokyProcess-239:Dummy-1] running all "atexit" finalizers with priority
>= 0
[DEBUG:LokyProcess-239:Dummy-1] running the remaining "atexit" finalizers
[INFO:LokyProcess-240:MainThread] Shutting down worker on sentinel
[INFO:LokyProcess-240:MainThread] process shutting down
[DEBUG:LokyProcess-240:MainThread] running all "atexit" finalizers with
priority >= 0
[DEBUG:LokyProcess-240:MainThread] running the remaining "atexit" finalizers
[INFO:LokyProcess-240:MainThread] process exiting with exitcode 0
[INFO:LokyProcess-240:Dummy-1] process shutting down
[DEBUG:LokyProcess-240:Dummy-1] running all "atexit" finalizers with priority
>= 0
[DEBUG:LokyProcess-240:Dummy-1] running the remaining "atexit" finalizers
[DEBUG:MainProcess:QueueManagerThread] queue management thread clean shutdown
of worker processes: []
[DEBUG:MainProcess:Dummy-463] finalizing pool
[DEBUG:MainProcess:Dummy-463] helping task handler/workers to finish
[DEBUG:MainProcess:Dummy-463] removing tasks from inqueue until task handler
finished
[DEBUG:MainProcess:Thread-40] worker handler exiting
[DEBUG:MainProcess:Thread-41] task handler got sentinel
[DEBUG:MainProcess:Thread-41] task handler sending sentinel to result handler
[DEBUG:MainProcess:Thread-41] task handler sending sentinel to workers
[DEBUG:MainProcess:Thread-41] task handler exiting
[DEBUG:MainProcess:Thread-42] result handler got sentinel
[DEBUG:MainProcess:Thread-42] ensuring that outqueue is not full
[DEBUG:MainProcess:Thread-42] result handler exiting: len(cache)=0,
thread._state=0
[DEBUG:PoolWorker-78:MainThread] worker got sentinel -- exiting
[DEBUG:PoolWorker-78:MainThread] worker exiting after 1 tasks
[INFO:PoolWorker-78:MainThread] process shutting down
[DEBUG:PoolWorker-78:MainThread] running all "atexit" finalizers with priority
>= 0
[DEBUG:PoolWorker-78:MainThread] running the remaining "atexit" finalizers
[INFO:PoolWorker-78:MainThread] process exiting with exitcode 0
[DEBUG:PoolWorker-79:MainThread] worker got sentinel -- exiting
[DEBUG:PoolWorker-79:MainThread] worker exiting after 0 tasks
[INFO:PoolWorker-79:MainThread] process shutting down
[DEBUG:PoolWorker-79:MainThread] running all "atexit" finalizers with priority
>= 0
[DEBUG:PoolWorker-79:MainThread] running the remaining "atexit" finalizers
[INFO:PoolWorker-79:MainThread] process exiting with exitcode 0
[DEBUG:MainProcess:Dummy-463] joining worker handler
[DEBUG:MainProcess:Dummy-463] terminating workers
[DEBUG:MainProcess:Dummy-463] joining task handler
[DEBUG:MainProcess:Dummy-463] joining result handler
[DEBUG:MainProcess:Dummy-463] joining pool workers
[DEBUG:MainProcess:Dummy-463] running the remaining "atexit" finalizers
joblib/externals/loky/backend/semaphore_tracker.py:198: UserWarning:
semaphore_tracker: There appear to be 16 leaked semaphores to clean up at
shutdown
len(cache))
make[1]: *** [debian/rules:25: python-test2.7] Error 1
make[1]: Leaving directory '/<<PKGBUILDDIR>>'
make: *** [debian/rules:10: build-indep] Error 2
dpkg-buildpackage: error: debian/rules build-indep subprocess returned exit
status 2
--------------------------------------------------------------------------------
Version 0.12.5-1 used to work ok in my autobuilders. I see that there
is some code to detect the number of CPUs, so this is clearly intended
to work everywhere, but apparently fails when there is only one CPU.
I've put a bunch of build logs here:
https://people.debian.org/~sanvila/build-logs/joblib/
Assuming the package is useful "as is" on every system and this is
really a bug in the test suite, the patch below works for me.
Note: I've checked and this failure may be reproduced easily on any
system by doing "taskset -c 0 dpkg-buildpackage", but if for whatever
reason you need a single-CPU system to test, please contact me
privately and I will gladly provide one.
Thanks.
--- a/joblib/test/test_parallel.py
+++ b/joblib/test/test_parallel.py
@@ -1429,6 +1429,7 @@ def _recursive_backend_info(limit=3, **kwargs):
return this_level + results[0]
+@pytest.mark.xfail
@with_multiprocessing
@parametrize('backend', ['loky', 'threading'])
def test_nested_parallelism_limit(backend):
--- End Message ---
--- Begin Message ---
Source: joblib
Source-Version: 0.13.0-2
We believe that the bug you reported is fixed in the latest version of
joblib, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 926...@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Yaroslav Halchenko <deb...@onerussian.com> (supplier of updated joblib package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmas...@ftp-master.debian.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Format: 1.8
Date: Fri, 12 Apr 2019 10:54:23 -0400
Source: joblib
Binary: python-joblib python3-joblib
Architecture: source all
Version: 0.13.0-2
Distribution: unstable
Urgency: medium
Maintainer: Yaroslav Halchenko <deb...@onerussian.com>
Changed-By: Yaroslav Halchenko <deb...@onerussian.com>
Description:
python-joblib - tools to provide lightweight pipelining in Python
python3-joblib - tools to provide lightweight pipelining in Python
Closes: 926930
Changes:
joblib (0.13.0-2) unstable; urgency=medium
.
* debian/patches/deb_xfail_test_nested_parallelism_limit (Closes: #926930)
- xfail a test failing on single core boxes. Thanks Santiago Vila
Checksums-Sha1:
d68779f28f3829c128015c8686568204aa0045f8 2106 joblib_0.13.0-2.dsc
15bcd876875ce370b4b52ca42e999cd45107920c 6492 joblib_0.13.0-2.debian.tar.xz
97340b989b9a3afd72e16e2510996b6fb3b0fe8a 7905 joblib_0.13.0-2_amd64.buildinfo
7ad3df40e8c0715e6921c38e71c5219b7858787d 189944 python-joblib_0.13.0-2_all.deb
0729c721ee79282d5f10adb4a7526b5aae0fce26 186956 python3-joblib_0.13.0-2_all.deb
Checksums-Sha256:
fa850e9a5a3cd95610581ecf7f973a9e3fea56607c32bedac614085678c7c9da 2106
joblib_0.13.0-2.dsc
f231b81ec10699b4150e03d0515929d280c58fb9427c7df5aa80df02c71e9479 6492
joblib_0.13.0-2.debian.tar.xz
36ad0d839bf573074ee15760feadf51882f728bf8cb8bb5191aa2bb4b7e00cce 7905
joblib_0.13.0-2_amd64.buildinfo
d6b8d71856835fc5fff91cb2eab1aa3d6adac0c709bfa680113d40b3b78152d0 189944
python-joblib_0.13.0-2_all.deb
5a322101b8e72a40284fbf359d52a68c5359d82ab27c3c96eeb1a080ca3a3cc3 186956
python3-joblib_0.13.0-2_all.deb
Files:
0b3cd4b293ea75c9d5c40f3ab0dab51a 2106 python optional joblib_0.13.0-2.dsc
fcbf9eb5025d2888e308040bf6516c19 6492 python optional
joblib_0.13.0-2.debian.tar.xz
52ef37f42968dc9a0efb076588fce135 7905 python optional
joblib_0.13.0-2_amd64.buildinfo
1272fbba9b8fc931128d3201bde620e9 189944 python optional
python-joblib_0.13.0-2_all.deb
e8df9c67b698d8ae8640c1e433004cae 186956 python optional
python3-joblib_0.13.0-2_all.deb
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEExbkF8OjZ/ZZo/zZvot4jUGLaM/oFAlyww/QACgkQot4jUGLa
M/oIZA//fc8/YuquXP0tuYeBO291HcidS5ufH9NvOiI0jLGZqxKc0tcYQ0KRmgum
1JJ5wrJyczQkMUWYUldzgD6dmB1kjO9cdqmdFhFoH4acbm/6l1Jt2T320ZDRAGEw
kTPW416+THw1uMjrJKzm6Jz3QAsUpPUhDqfi/ymOwgKeY6rBsrSRUg0zOU/rptFm
MvOdAOTEHjurxtT+vhDQykdvJw5KwK35mouASiNoEU6W3ABKYk0F0M6Q/n0/7P8b
9nUS6tATEazca3XX1SI5Oskj3FpFG3eyfy28ySann1yAsOZ00IlKuIeCh0Roq7fj
94SfEzmNqGDADwYCdyEfZ/CJ99WOVDUqz9lDyTRndZus+7VMe+7TP3NyhsJAEwVM
xcoJyv/vNizvurRbEysin6ANCUX7piS/X7IGDNGfGJBvkBVQbqAR7HSwIZRxiykp
IKoQvZvFjjRUjzqeaWWS0Qw+BnAAbBxFfeRk5il1C33UE4iO2Wk4ykdfHcQgAPSY
ig9dwV5xZSTUoZqSxC3EtOtZocJhoQDxevpo5lX7qrW25K8djCWbJcsepXVE3JQP
hdH/GcXc7/qYC0QsXfDErdKVrPmbzl8dh/Md5ONOJ4Qz7pphytw65lLcEm9ymqTC
1Jd/FqpDYwI1EuamJPEwJQJydhXqATMsB0NuaUV+RP/GAZMEVeE=
=lw9B
-----END PGP SIGNATURE-----
--- End Message ---
