Package: src:pyeapi
Version: 0.6.1-2
Severity: serious
Dear maintainer:
I tried to build this package in stretch with "dpkg-buildpackage -A"
(which is what the "Arch: all" autobuilder would do to build it)
but it failed:
--------------------------------------------------------------------------------
[...]
debian/rules build-indep
dh build-indep --with python3,python2,sphinxdoc --buildsystem=pybuild
dh_testdir -i -O--buildsystem=pybuild
dh_update_autotools_config -i -O--buildsystem=pybuild
dh_auto_configure -i -O--buildsystem=pybuild
I: pybuild base:184: python2.7 setup.py config
running config
I: pybuild base:184: python3.5 setup.py config
running config
debian/rules override_dh_auto_build
make[1]: Entering directory '/<<PKGBUILDDIR>>'
dh_auto_build
I: pybuild base:184: /usr/bin/python setup.py build
[... snipped ...]
test_make_connection_raises_typeerror (test_client.TestClient) ... ok
test_missing_connection_raises_attribute_error (test_client.TestClient) ... ok
test_node_hasattr_connection (test_client.TestClient) ... ok
test_node_repr_returns (test_client.TestClient) ... ok
test_node_returns_cached_startup_confgi (test_client.TestClient) ... ok
test_node_returns_running_config (test_client.TestClient) ... ok
test_node_returns_startup_config (test_client.TestClient) ... ok
test_node_str_returns (test_client.TestClient) ... ok
test_api_autoloader (test_client.TestNode) ... ok
test_config_with_multiple_commands (test_client.TestNode) ... ok
test_config_with_multiple_multilines (test_client.TestNode) ... ok
test_config_with_single_command (test_client.TestNode) ... ok
test_config_with_single_multiline (test_client.TestNode) ... ok
test_enable_authentication (test_client.TestNode) ... ok
test_enable_with_config_statement (test_client.TestNode) ... ok
test_enable_with_multiple_commands (test_client.TestNode) ... ok
test_enable_with_single_command (test_client.TestNode) ... ok
test_get_config (test_client.TestNode) ... ok
test_get_config_as_string (test_client.TestNode) ... ok
test_get_config_raises_type_error (test_client.TestNode) ... ok
test_collapse_mixed (test_utils.TestUtils) ... ok
test_collapse_range (test_utils.TestUtils) ... ok
test_collapse_singles (test_utils.TestUtils) ... ok
test_debug (test_utils.TestUtils) ... ok
test_expand_mixed (test_utils.TestUtils) ... ok
test_expand_range (test_utils.TestUtils) ... ok
test_expand_singles (test_utils.TestUtils) ... ok
test_import_module (test_utils.TestUtils) ... ok
test_import_module_raises_import_error (test_utils.TestUtils) ... ok
test_load_module (test_utils.TestUtils) ... ok
test_load_module_raises_import_error (test_utils.TestUtils) ... ok
test_make_iterable_from_iterable (test_utils.TestUtils) ... ok
test_make_iterable_from_string (test_utils.TestUtils) ... ok
test_make_iterable_raises_type_error (test_utils.TestUtils) ... ok
======================================================================
FAIL: test_isvlan_with_string (test_api_vlans.TestApiVlans)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/<<PKGBUILDDIR>>/.pybuild/pythonX.Y_2.7/build/test/unit/test_api_vlans.py",
line 52, in test_isvlan_with_string
self.assertFalse(pyeapi.api.vlans.isvlan(random_string()))
AssertionError: True is not false
----------------------------------------------------------------------
Ran 333 tests in 0.233s
FAILED (failures=1)
E: pybuild pybuild:276: test: plugin distutils failed with: exit code=1: cd
/<<PKGBUILDDIR>>/.pybuild/pythonX.Y_2.7/build; python2.7 -m unittest discover
-v test/unit
dh_auto_test: pybuild --test -i python{version} -p 2.7 returned exit code 13
debian/rules:6: recipe for target 'build-indep' failed
make: *** [build-indep] Error 25
dpkg-buildpackage: error: debian/rules build-indep gave error exit status 2
--------------------------------------------------------------------------------
The failing test creates a random string and then ensures that it's
not a valid vlan value (a number between 1 and 4094).
So this is what happens:
1 out of 50 times the string will have length 1.
Of those, 9 out of 10+26 times the string will be a valid vlan.
(from 1 to 9)
1 out of 50 times the string will have length 2.
Of those, 90 out of (10+26)**2 times the string will be a valid vlan.
(from 10 to 99)
1 out of 50 times the string will have length 3.
Of those, 900 out of (10+26)**3 times the string will be a valid vlan.
(from 100 to 999)
1 out of 50 times the string will have length 4.
Of those, 3094 out of (10+26)**4 times the string will be a valid vlan.
(from 1000 to 4094)
So I estimate that the probability that this fails is:
1/50*(9/b+90/b**2+900/b**3+3094/b**4) = 286019/41990400,
where b=10+26, i.e. approximately once every 147 times,
which of course is > 0 and not good enough.
To see how easy this is to happen, just try the program below.
TLDR: Please disable such test, we want tests to always succeed,
not just "most of the time".
Thanks.
#!/usr/bin/python
import random
import string
def isvlan(value):
try:
value = int(value)
return value in range(1, 4095)
except ValueError:
return False
def random_string(minchar=1, maxchar=50):
return ''.join(random.choice(string.ascii_uppercase + string.digits)
for _ in range(random.randint(minchar, maxchar)))
while True:
r=random_string()
if isvlan(r):
print r
