[issue29047] Where are the test results stored?

[Patrik Iselind]

New submission from Patrik Iselind:

I cannot find any documentation on where the test results are stored.

Looked at https://docs.python.org/devguide/runtests.html

--
assignee: docs@python
components: Documentation
messages: 283825
nosy: docs@python, patriki
priority: normal
severity: normal
status: open
title: Where are the test results stored?
versions: Python 3.7

___
Python tracker 
<http://bugs.python.org/issue29047>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29046] Coverage related documentation missing

[Patrik Iselind]

New submission from Patrik Iselind:

Is it possible to do coverage -j8 or similar? Cannot find any documentation on 
this. Coverage takes so long on the tests...

I checked https://docs.python.org/devguide/coverage.html

--
assignee: docs@python
components: Documentation
messages: 283824
nosy: docs@python, patriki
priority: normal
severity: normal
status: open
title: Coverage related documentation missing
versions: Python 3.7

___
Python tracker 
<http://bugs.python.org/issue29046>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29048] Coverage influence tests, make some of them fail

[Patrik Iselind]

New submission from Patrik Iselind:

I checkout the latest tip from scratch.

When i run the tests i get the following results:
% ./python -m test -j
[...]
376 tests OK.

1 test altered the execution environment:
test_site

27 tests skipped:
test_bz2 test_ctypes test_curses test_dbm_gnu test_dbm_ndbm
test_devpoll test_gdb test_kqueue test_lzma test_msilib
test_ossaudiodev test_readline test_smtpnet test_socketserver
test_sqlite test_startfile test_timeout test_tix test_tk
test_ttk_guionly test_urllib2net test_urllibnet test_winconsoleio
test_winreg test_winsound test_xmlrpc_net test_zipfile64

Total duration: 2 min 57 sec
Tests result: SUCCESS

Without any changes i run the tests again but this time under coverage. I get a 
completely different result:
% ./python ../coveragepy run --pylib Lib/test/regrtest.py
[...]
328 tests OK.

8 tests failed:
test_exceptions test_frame test_ssl test_subprocess test_super
test_traceback test_unittest test_xml_etree_c

41 tests altered the execution environment:
test_asynchat test_asyncio test_asyncore test_capi
test_concurrent_futures test_decimal test_distutils test_docxmlrpc
test_email test_fork1 test_ftplib test_functools test_gc
test_hashlib test_httplib test_httpservers test_imaplib
test_import test_io test_json test_logging test_mailbox
test_multiprocessing_fork test_multiprocessing_forkserver
test_multiprocessing_spawn test_nntplib test_os test_poll
test_poplib test_pydoc test_queue test_robotparser test_sched
test_smtplib test_socket test_sys test_telnetlib
test_threaded_import test_threadedtempfile test_threading
test_xml_etree

27 tests skipped:
test_bz2 test_ctypes test_curses test_dbm_gnu test_dbm_ndbm
test_devpoll test_gdb test_kqueue test_lzma test_msilib
test_ossaudiodev test_readline test_smtpnet test_socketserver
test_sqlite test_startfile test_timeout test_tix test_tk
test_ttk_guionly test_urllib2net test_urllibnet test_winconsoleio
test_winreg test_winsound test_xmlrpc_net test_zipfile64

Total duration: 92 min 34 sec
Tests result: FAILURE

Is the reason for this difference known? Coverage shouldn't affect the outcome 
of the tests right?

--
components: Tests
messages: 283828
nosy: patriki
priority: normal
severity: normal
status: open
title: Coverage influence tests, make some of them fail
type: behavior
versions: Python 3.7

___
Python tracker 
<http://bugs.python.org/issue29048>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29046] Coverage related documentation missing

[Patrik Iselind]

Patrik Iselind added the comment:

Agree

--
resolution:  -> not a bug
status: open -> closed

___
Python tracker 
<http://bugs.python.org/issue29046>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29047] Where are the test results stored?

[Patrik Iselind]

Patrik Iselind added the comment:

Is this something we could clarify in the docs?

I'd like to try and dip my toes in this.

Where can i find the source for
https://docs.python.org/devguide/runtests.html
?

--

___
Python tracker 
<http://bugs.python.org/issue29047>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29048] Coverage influence tests, make some of them fail

[Patrik Iselind]

Patrik Iselind added the comment:

Are the tests that altered the execution environment something to bother with 
at all? Perhaps something for another issue?

--

___
Python tracker 
<http://bugs.python.org/issue29048>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29048] Coverage influence tests, make some of them fail

[Patrik Iselind]

Patrik Iselind added the comment:

The fixes for test_ssl and test_subprocess seem strange to me. All they do is 
sweep the messages under the rug. Then there is not really any point with the 
'with' block, is there?

Without coverage those tests passed so i cannot really see how this adds up.

--

___
Python tracker 
<http://bugs.python.org/issue29048>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29047] Where are the test results stored?

[Patrik Iselind]

Patrik Iselind added the comment:

Great, thanks!

Patrik

Den 22 dec 2016 15:01 skrev "Eric V. Smith" :

>
> Eric V. Smith added the comment:
>
> https://github.com/python/devguide/blob/github/runtests.rst
>
> --
> nosy: +eric.smith
>
> ___
> Python tracker 
> <http://bugs.python.org/issue29047>
> ___
>

--

___
Python tracker 
<http://bugs.python.org/issue29047>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29047] Where are the test results stored?

[Patrik Iselind]

Patrik Iselind added the comment:

Made a pull request for this

--
pull_requests: +7

___
Python tracker 
<http://bugs.python.org/issue29047>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29047] Where are the test results stored?

[Patrik Iselind]

Changes by Patrik Iselind :


--
pull_requests:  -7

___
Python tracker 
<http://bugs.python.org/issue29047>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29047] Where are the test results stored?

[Patrik Iselind]

Patrik Iselind added the comment:

Cannot figure out how to add the link to the GitHub PR field. Adding link here 
instead.

https://github.com/python/devguide/pull/88

--

___
Python tracker 
<http://bugs.python.org/issue29047>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29048] Coverage influence tests, make some of them fail

[Patrik Iselind]

Patrik Iselind added the comment:

As an example, the difference for test_exceptions:

$ ./python ../coveragepy run --pylib --source=exceptions Lib/test/regrtest.py 
test_exceptions
Run tests sequentially
0:00:00 [1/1] test_exceptions
Exception ignored in: .BrokenDel.__del__ of 
.BrokenDel object 
at 0x7f9feefc1c88>>
Traceback (most recent call last):
  File "/home/irina/patrik/cpython/Lib/test/test_exceptions.py", line 1024, in 
__del__
raise exc
ValueError: del is broken
Exception ignored in: 
Traceback (most recent call last):
  File "/home/irina/patrik/cpython/Lib/test/test_exceptions.py", line 1024, in 
__del__
raise exc
ValueError: del is broken
Exception ignored in: .BrokenExceptionDel.__del__ of 
.BrokenExceptionDel
 object at 0x7f9feefc3d58>>
Traceback (most recent call last):
  File "/home/irina/patrik/cpython/Lib/test/test_exceptions.py", line 1034, in 
__del__
raise exc
test.test_exceptions.BrokenStrException: 
test test_exceptions failed -- multiple errors occurred; run in verbose mode 
for details
test_exceptions failed

1 test failed:
test_exceptions

Total duration: 1 sec
Tests result: FAILURE
Coverage.py warning: Module exceptions was never imported.
Coverage.py warning: No data was collected.



$ ./python -m test -v test_exceptions
== CPython 3.7.0a0 (default:8e311f109b22, Dec 24 2016, 12:26:43) [GCC 5.4.0 
20160609]
==   Linux-4.4.0-53-generic-x86_64-with-debian-stretch-sid little-endian
==   hash algorithm: siphash24 64bit
==  cwd: /home/irina/patrik/cpython/build/test_python_17521
==  encodings: locale=UTF-8, FS=utf-8
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, 
dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, 
verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
Run tests sequentially
0:00:00 [1/1] test_exceptions
testAttributes (test.test_exceptions.ExceptionTests) ... ok
testChainingAttrs (test.test_exceptions.ExceptionTests) ... ok
testChainingDescriptors (test.test_exceptions.ExceptionTests) ... ok
testExceptionCleanupNames (test.test_exceptions.ExceptionTests) ... ok
testExceptionCleanupState (test.test_exceptions.ExceptionTests) ... ok
testInfiniteRecursion (test.test_exceptions.ExceptionTests) ... ok
testInvalidAttrs (test.test_exceptions.ExceptionTests) ... ok
testInvalidTraceback (test.test_exceptions.ExceptionTests) ... ok
testKeywordArgs (test.test_exceptions.ExceptionTests) ... ok
testNoneClearsTracebackAttr (test.test_exceptions.ExceptionTests) ... ok
testRaising (test.test_exceptions.ExceptionTests) ... ok
testSettingException (test.test_exceptions.ExceptionTests) ... ok
testSyntaxErrorMessage (test.test_exceptions.ExceptionTests) ... ok
testSyntaxErrorOffset (test.test_exceptions.ExceptionTests) ... ok
testWithTraceback (test.test_exceptions.ExceptionTests) ... ok
test_3114 (test.test_exceptions.ExceptionTests) ... ok
test_MemoryError (test.test_exceptions.ExceptionTests) ... ok
test_WindowsError (test.test_exceptions.ExceptionTests) ... ok
test_badisinstance (test.test_exceptions.ExceptionTests) ... ok
test_errno_ENOTDIR (test.test_exceptions.ExceptionTests) ... ok
test_exception_target_in_nested_scope (test.test_exceptions.ExceptionTests) ... 
ok
test_exception_with_doc (test.test_exceptions.ExceptionTests) ... ok
test_generator_close_cleanup_exc_state (test.test_exceptions.ExceptionTests) 
... ok
test_generator_del_cleanup_exc_state (test.test_exceptions.ExceptionTests) ... 
ok
test_generator_doesnt_retain_old_exc (test.test_exceptions.ExceptionTests) ... 
ok
test_generator_finalizing_and_exc_info (test.test_exceptions.ExceptionTests) 
... ok
test_generator_leaking (test.test_exceptions.ExceptionTests) ... ok
test_generator_leaking2 (test.test_exceptions.ExceptionTests) ... ok
test_generator_leaking3 (test.test_exceptions.ExceptionTests) ... ok
test_generator_leaking4 (test.test_exceptions.ExceptionTests) ... ok
test_generator_next_cleanup_exc_state (test.test_exceptions.ExceptionTests) ... 
ok
test_generator_send_cleanup_exc_state (test.test_exceptions.ExceptionTests) ... 
ok
test_generator_throw_cleanup_exc_state (test.test_exceptions.ExceptionTests) 
... ok
test_memory_error_cleanup (test.test_exceptions.ExceptionTests) ... ok
test_recursion_error_cleanup (test.test_exceptions.ExceptionTests) ... ok
test_str (test.test_exceptions.ExceptionTests) ... ok
test_unhandled (test.test_exceptions.ExceptionTests) ... ok
test_unicode_change_attributes (test.test_exceptions.ExceptionTests) ... ok
test_unicode_errors_no_object (test.test_exceptions.ExceptionTests) ... ok
test_unraisable (test.test_exceptions.ExceptionTests) ... ok
test_windows_message (test.test_exceptions.ExceptionTests)
Should fill in unknown error code in Windows error message ... skipped 'test 
specific to Windows'
test_attributes (test.test_exceptions.ImportErrorTests) ... ok
test_non_str_argument (test.test_exceptions.ImportErrorTests) ... ok
test_reset_attri

[issue29048] Coverage influence tests, make some of them fail

[Patrik Iselind]

Patrik Iselind added the comment:

I've been looking into test_exceptions, why it fails. The reason seems to be 
that when the tests are executed under coverage, then they get nothing on the 
line that says the following in test_unraisable().

```python
report = stderr.getvalue()
```
So coverage steal strerr in this case, and the test's thunder.

A way to get around this would be to not let the test rely in stderr but 
instead catch the exception and unfold it as a string for testing.

--

___
Python tracker 
<http://bugs.python.org/issue29048>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29048] Coverage influence tests, make some of them fail

[Patrik Iselind]

Patrik Iselind added the comment:

i use the latest code. perhaps something for introduced after 3.6?

Patrik

Den 24 dec 2016 13:51 skrev "Ned Batchelder" :

>
> Ned Batchelder added the comment:
>
> I doubt very very much that coverage.py is stealing stderr.  Many many
> test suites would have this problem if that were true.  More likely is that
> somehow __del__ is not being invoked when you need it to be.
>
> I tried running the tests under coverage also, and did not see this
> behavior. I don't have a build tree of the latest code, but I ran it on
> 3.6.0.  The tests passed both with and without coverage measurement (output
> below).  I would like to understand why you are seeing the results you are.
>
> $ mkvirtualenv -p python3.6 issue29048
> Running virtualenv with interpreter /usr/local/bin/python3.6
> Using base prefix '/usr/local/pythonz/pythons/CPython-3.6.0'
> New python executable in issue29048/bin/python3.6
> Also creating executable in issue29048/bin/python
> Installing setuptools, pip, wheel...done.
> Collecting pip
>   Using cached pip-9.0.1-py2.py3-none-any.whl
> Collecting setuptools
>   Using cached setuptools-32.2.0-py2.py3-none-any.whl
> Installing collected packages: pip, setuptools
>   Found existing installation: pip 7.1.2
> Uninstalling pip-7.1.2:
>   Successfully uninstalled pip-7.1.2
>   Found existing installation: setuptools 18.2
> Uninstalling setuptools-18.2:
>   Successfully uninstalled setuptools-18.2
> Successfully installed pip-9.0.1 setuptools-32.2.0
> (issue29048)$ pip install coverage
> Collecting coverage
> Installing collected packages: coverage
> Successfully installed coverage-4.2
> (issue29048)$ python3.6 -m test -v test_exceptions
> == CPython 3.6.0 (default, Dec 23 2016, 07:11:25) [GCC 4.2.1 Compatible
> Apple LLVM 7.0.2 (clang-700.1.81)]
> ==   Darwin-14.5.0-x86_64-i386-64bit little-endian
> ==   hash algorithm: siphash24 64bit
> ==  cwd: /private/var/folders/j2/gr3cj3jn63s5q8g3bjvw57hmgp
> /T/test_python_20266
> ==  encodings: locale=UTF-8, FS=utf-8
> Testing with flags: sys.flags(debug=0, inspect=0, interactive=0,
> optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0,
> ignore_environment=0, verbose=0, bytes_warning=0, quiet=0,
> hash_randomization=1, isolated=0)
> Run tests sequentially
> 0:00:00 [1/1] test_exceptions
> testAttributes (test.test_exceptions.ExceptionTests) ... ok
> testChainingAttrs (test.test_exceptions.ExceptionTests) ... ok
> testChainingDescriptors (test.test_exceptions.ExceptionTests) ... ok
> testExceptionCleanupNames (test.test_exceptions.ExceptionTests) ... ok
> testExceptionCleanupState (test.test_exceptions.ExceptionTests) ... ok
> testInfiniteRecursion (test.test_exceptions.ExceptionTests) ... ok
> testInvalidAttrs (test.test_exceptions.ExceptionTests) ... ok
> testInvalidTraceback (test.test_exceptions.ExceptionTests) ... ok
> testKeywordArgs (test.test_exceptions.ExceptionTests) ... ok
> testNoneClearsTracebackAttr (test.test_exceptions.ExceptionTests) ... ok
> testRaising (test.test_exceptions.ExceptionTests) ... ok
> testSettingException (test.test_exceptions.ExceptionTests) ... ok
> testSyntaxErrorMessage (test.test_exceptions.ExceptionTests) ... ok
> testSyntaxErrorOffset (test.test_exceptions.ExceptionTests) ... ok
> testWithTraceback (test.test_exceptions.ExceptionTests) ... ok
> test_3114 (test.test_exceptions.ExceptionTests) ... ok
> test_MemoryError (test.test_exceptions.ExceptionTests) ... ok
> test_WindowsError (test.test_exceptions.ExceptionTests) ... ok
> test_badisinstance (test.test_exceptions.ExceptionTests) ... ok
> test_errno_ENOTDIR (test.test_exceptions.ExceptionTests) ... ok
> test_exception_target_in_nested_scope (test.test_exceptions.ExceptionTests)
> ... ok
> test_exception_with_doc (test.test_exceptions.ExceptionTests) ... ok
> test_generator_close_cleanup_exc_state (test.test_exceptions.ExceptionTests)
> ... ok
> test_generator_del_cleanup_exc_state (test.test_exceptions.ExceptionTests)
> ... ok
> test_generator_doesnt_retain_old_exc (test.test_exceptions.ExceptionTests)
> ... ok
> test_generator_finalizing_and_exc_info (test.test_exceptions.ExceptionTests)
> ... ok
> test_generator_leaking (test.test_exceptions.ExceptionTests) ... ok
> test_generator_leaking2 (test.test_exceptions.ExceptionTests) ... ok
> test_generator_leaking3 (test.test_exceptions.ExceptionTests) ... ok
> test_generator_leaking4 (test.test_exceptions.ExceptionTests) ... ok
> test_generator_next_cleanup_exc_state (test.test_exceptions.ExceptionTests)
> ... ok
> test_generator_send_cleanup_exc_state (test.test_exceptions.ExceptionTests)
> ... ok
> test_generator_throw_cleanup_exc_state (test.test_exceptions.Ex

[issue29048] Coverage influence tests, make some of them fail

[Patrik Iselind]

Patrik Iselind added the comment:

report doesn't contain any text when run under coverage. something made the
text not appear...

Patrik

Den 24 dec 2016 16:39 skrev "Patrik Iselind" :

>
> Patrik Iselind added the comment:
>
> i use the latest code. perhaps something for introduced after 3.6?
>
> Patrik
>
> Den 24 dec 2016 13:51 skrev "Ned Batchelder" :
>
> >
> > Ned Batchelder added the comment:
> >
> > I doubt very very much that coverage.py is stealing stderr.  Many many
> > test suites would have this problem if that were true.  More likely is
> that
> > somehow __del__ is not being invoked when you need it to be.
> >
> > I tried running the tests under coverage also, and did not see this
> > behavior. I don't have a build tree of the latest code, but I ran it on
> > 3.6.0.  The tests passed both with and without coverage measurement
> (output
> > below).  I would like to understand why you are seeing the results you
> are.
> >
> > $ mkvirtualenv -p python3.6 issue29048
> > Running virtualenv with interpreter /usr/local/bin/python3.6
> > Using base prefix '/usr/local/pythonz/pythons/CPython-3.6.0'
> > New python executable in issue29048/bin/python3.6
> > Also creating executable in issue29048/bin/python
> > Installing setuptools, pip, wheel...done.
> > Collecting pip
> >   Using cached pip-9.0.1-py2.py3-none-any.whl
> > Collecting setuptools
> >   Using cached setuptools-32.2.0-py2.py3-none-any.whl
> > Installing collected packages: pip, setuptools
> >   Found existing installation: pip 7.1.2
> > Uninstalling pip-7.1.2:
> >   Successfully uninstalled pip-7.1.2
> >   Found existing installation: setuptools 18.2
> > Uninstalling setuptools-18.2:
> >   Successfully uninstalled setuptools-18.2
> > Successfully installed pip-9.0.1 setuptools-32.2.0
> > (issue29048)$ pip install coverage
> > Collecting coverage
> > Installing collected packages: coverage
> > Successfully installed coverage-4.2
> > (issue29048)$ python3.6 -m test -v test_exceptions
> > == CPython 3.6.0 (default, Dec 23 2016, 07:11:25) [GCC 4.2.1 Compatible
> > Apple LLVM 7.0.2 (clang-700.1.81)]
> > ==   Darwin-14.5.0-x86_64-i386-64bit little-endian
> > ==   hash algorithm: siphash24 64bit
> > ==  cwd: /private/var/folders/j2/gr3cj3jn63s5q8g3bjvw57hmgp
> > /T/test_python_20266
> > ==  encodings: locale=UTF-8, FS=utf-8
> > Testing with flags: sys.flags(debug=0, inspect=0, interactive=0,
> > optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0,
> > ignore_environment=0, verbose=0, bytes_warning=0, quiet=0,
> > hash_randomization=1, isolated=0)
> > Run tests sequentially
> > 0:00:00 [1/1] test_exceptions
> > testAttributes (test.test_exceptions.ExceptionTests) ... ok
> > testChainingAttrs (test.test_exceptions.ExceptionTests) ... ok
> > testChainingDescriptors (test.test_exceptions.ExceptionTests) ... ok
> > testExceptionCleanupNames (test.test_exceptions.ExceptionTests) ... ok
> > testExceptionCleanupState (test.test_exceptions.ExceptionTests) ... ok
> > testInfiniteRecursion (test.test_exceptions.ExceptionTests) ... ok
> > testInvalidAttrs (test.test_exceptions.ExceptionTests) ... ok
> > testInvalidTraceback (test.test_exceptions.ExceptionTests) ... ok
> > testKeywordArgs (test.test_exceptions.ExceptionTests) ... ok
> > testNoneClearsTracebackAttr (test.test_exceptions.ExceptionTests) ... ok
> > testRaising (test.test_exceptions.ExceptionTests) ... ok
> > testSettingException (test.test_exceptions.ExceptionTests) ... ok
> > testSyntaxErrorMessage (test.test_exceptions.ExceptionTests) ... ok
> > testSyntaxErrorOffset (test.test_exceptions.ExceptionTests) ... ok
> > testWithTraceback (test.test_exceptions.ExceptionTests) ... ok
> > test_3114 (test.test_exceptions.ExceptionTests) ... ok
> > test_MemoryError (test.test_exceptions.ExceptionTests) ... ok
> > test_WindowsError (test.test_exceptions.ExceptionTests) ... ok
> > test_badisinstance (test.test_exceptions.ExceptionTests) ... ok
> > test_errno_ENOTDIR (test.test_exceptions.ExceptionTests) ... ok
> > test_exception_target_in_nested_scope (test.test_exceptions.
> ExceptionTests)
> > ... ok
> > test_exception_with_doc (test.test_exceptions.ExceptionTests) ... ok
> > test_generator_close_cleanup_exc_state (test.test_exceptions.
> ExceptionTests)
> > ... ok
> > test_generator_del_cleanup_exc_state (test.test_exceptions.
> ExceptionTests)
> > ... ok
> > test_generator_doesnt_retain_old_exc (test.test_exceptions.
> Exce

[issue29048] Coverage influence tests, make some of them fail

[Patrik Iselind]

Patrik Iselind added the comment:

yes, I'm sure. i printed markings around where the text should have been.

Patrik

Den 24 dec 2016 17:37 skrev "Ned Batchelder" :

>
> Ned Batchelder added the comment:
>
> Are you sure the code that tried to print to stderr was even run? It's in
> a __del__ method, which is notorious for being affected by surprising
> things.
>
> --
>
> ___
> Python tracker 
> <http://bugs.python.org/issue29048>
> ___
>

--

___
Python tracker 
<http://bugs.python.org/issue29048>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29048] Coverage influence tests, make some of them fail

[Patrik Iselind]

Patrik Iselind added the comment:

For clarity, this is what i get in the test_exceptions test when i've applied 
the attached diff.

$ ./python -m test -v test_exceptions
== CPython 3.7.0a0 (default:8e311f109b22, Dec 24 2016, 12:26:43) [GCC 5.4.0 
20160609]
==   Linux-4.4.0-53-generic-x86_64-with-debian-stretch-sid little-endian
==   hash algorithm: siphash24 64bit
==  cwd: /home/irina/patrik/cpython/build/test_python_19104
==  encodings: locale=UTF-8, FS=utf-8
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, 
dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, 
verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
Run tests sequentially
0:00:00 [1/1] test_exceptions
testAttributes (test.test_exceptions.ExceptionTests) ... ok
testChainingAttrs (test.test_exceptions.ExceptionTests) ... ok
testChainingDescriptors (test.test_exceptions.ExceptionTests) ... ok
testExceptionCleanupNames (test.test_exceptions.ExceptionTests) ... ok
testExceptionCleanupState (test.test_exceptions.ExceptionTests) ... ok
testInfiniteRecursion (test.test_exceptions.ExceptionTests) ... ok
testInvalidAttrs (test.test_exceptions.ExceptionTests) ... ok
testInvalidTraceback (test.test_exceptions.ExceptionTests) ... ok
testKeywordArgs (test.test_exceptions.ExceptionTests) ... ok
testNoneClearsTracebackAttr (test.test_exceptions.ExceptionTests) ... ok
testRaising (test.test_exceptions.ExceptionTests) ... ok
testSettingException (test.test_exceptions.ExceptionTests) ... ok
testSyntaxErrorMessage (test.test_exceptions.ExceptionTests) ... ok
testSyntaxErrorOffset (test.test_exceptions.ExceptionTests) ... ok
testWithTraceback (test.test_exceptions.ExceptionTests) ... ok
test_3114 (test.test_exceptions.ExceptionTests) ... ok
test_MemoryError (test.test_exceptions.ExceptionTests) ... ok
test_WindowsError (test.test_exceptions.ExceptionTests) ... ok
test_badisinstance (test.test_exceptions.ExceptionTests) ... ok
test_errno_ENOTDIR (test.test_exceptions.ExceptionTests) ... ok
test_exception_target_in_nested_scope (test.test_exceptions.ExceptionTests) ... 
ok
test_exception_with_doc (test.test_exceptions.ExceptionTests) ... ok
test_generator_close_cleanup_exc_state (test.test_exceptions.ExceptionTests) 
... ok
test_generator_del_cleanup_exc_state (test.test_exceptions.ExceptionTests) ... 
ok
test_generator_doesnt_retain_old_exc (test.test_exceptions.ExceptionTests) ... 
ok
test_generator_finalizing_and_exc_info (test.test_exceptions.ExceptionTests) 
... ok
test_generator_leaking (test.test_exceptions.ExceptionTests) ... ok
test_generator_leaking2 (test.test_exceptions.ExceptionTests) ... ok
test_generator_leaking3 (test.test_exceptions.ExceptionTests) ... ok
test_generator_leaking4 (test.test_exceptions.ExceptionTests) ... ok
test_generator_next_cleanup_exc_state (test.test_exceptions.ExceptionTests) ... 
ok
test_generator_send_cleanup_exc_state (test.test_exceptions.ExceptionTests) ... 
ok
test_generator_throw_cleanup_exc_state (test.test_exceptions.ExceptionTests) 
... ok
test_memory_error_cleanup (test.test_exceptions.ExceptionTests) ... ok
test_recursion_error_cleanup (test.test_exceptions.ExceptionTests) ... ok
test_str (test.test_exceptions.ExceptionTests) ... ok
test_unhandled (test.test_exceptions.ExceptionTests) ... ok
test_unicode_change_attributes (test.test_exceptions.ExceptionTests) ... ok
test_unicode_errors_no_object (test.test_exceptions.ExceptionTests) ... ok
test_unraisable (test.test_exceptions.ExceptionTests) ... 
BEGINBEGINBEGINBEGINBEGIN
Exception ignored in: .BrokenDel.__del__ of 
.BrokenDel object 
at 0x7f4313a41ef8>>
Traceback (most recent call last):
  File "/home/irina/patrik/cpython/Lib/test/test_exceptions.py", line 1024, in 
__del__
raise exc
ValueError: del is broken

ENDENDENDENDENDENDENDEND
BEGINBEGINBEGINBEGINBEGIN
Exception ignored in: 
Traceback (most recent call last):
  File "/home/irina/patrik/cpython/Lib/test/test_exceptions.py", line 1024, in 
__del__
raise exc
ValueError: del is broken

ENDENDENDENDENDENDENDEND
BEGINBEGINBEGINBEGINBEGIN
Exception ignored in: .BrokenExceptionDel.__del__ of 
.BrokenExceptionDel
 object at 0x7f4313a41ef8>>
Traceback (most recent call last):
  File "/home/irina/patrik/cpython/Lib/test/test_exceptions.py", line 1034, in 
__del__
raise exc
test.test_exceptions.BrokenStrException: 

ENDENDENDENDENDENDENDEND
ok
test_windows_message (test.test_exceptions.ExceptionTests)
Should fill in unknown error code in Windows error message ... skipped 'test 
specific to Windows'
test_attributes (test.test_exceptions.ImportErrorTests) ... ok
test_non_str_argument (test.test_exceptions.ImportErrorTests) ... ok
test_reset_attributes (test.test_exceptions.ImportErrorTests) ... ok

--
Ran 44 tests in 0.158s

OK (skipped=1)
1 test OK.

Total duration: 363 ms
Tests result: SUCCESS




$ ./pyth

[issue29048] Coverage influence tests, make some of them fail

[Patrik Iselind]

Patrik Iselind added the comment:

I'd like to clarify that the file i just attached, no-stderr.diff, is in no way 
a suggestion for a fix. It's just my way of conveying what i see.

--

___
Python tracker 
<http://bugs.python.org/issue29048>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1116520] Prefix search is filesystem-centric

[Patrik Iselind]

Patrik Iselind added the comment:

How much smaller would the stdlib for 3.5 become if you compress it with zip?

--
nosy: +patriki

___
Python tracker 
<http://bugs.python.org/issue1116520>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1116520] Prefix search is filesystem-centric

[Patrik Iselind]

Patrik Iselind added the comment:

What about the performance penalty for zipping stdlib? is it significant?

When would you like to zip stdlib? For embedded systems with limited disk
space?

Patrik Iselind

On Sat, Dec 24, 2016 at 6:34 PM, Patrik Iselind 
wrote:

>
> Patrik Iselind added the comment:
>
> How much smaller would the stdlib for 3.5 become if you compress it with
> zip?
>
> --
> nosy: +patriki
>
> ___
> Python tracker 
> <http://bugs.python.org/issue1116520>
> ___
>

--

___
Python tracker 
<http://bugs.python.org/issue1116520>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29047] Where are the test results stored?

[Patrik Iselind]

Patrik Iselind added the comment:

My GitHub PR got closed. So closing this issue as well.

--
resolution:  -> rejected
status: open -> closed

___
Python tracker 
<http://bugs.python.org/issue29047>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29048] Coverage influence tests, make some of them fail

[Patrik Iselind]

Patrik Iselind added the comment:

Sure, here you go.

$ ./python ../coveragepy debug sys
-- sys ---
  version: 4.3a0
 coverage: ../coveragepy/coverage/__init__.py
   cover_dirs: /home/irina/patrik/coveragepy/coverage
   pylib_dirs: /home/irina/patrik/cpython/Lib
   tracer: PyTracer
 plugins.file_tracers: -none-
 config_files: .coveragerc
   setup.cfg
   tox.ini
 configs_read: -none-
data_path: /home/irina/patrik/cpython/.coverage
   python: 3.7.0a0 (default:7ceacac48cd2+, Dec 24 2016, 19:08:24) 
[GCC 5.4.0 20160609]
 platform: Linux-4.4.0-53-generic-x86_64-with-debian-stretch-sid
   implementation: CPython
   executable: /home/irina/patrik/cpython/python
  cwd: /home/irina/patrik/cpython
 path: 
   /usr/local/lib/python37.zip
   /home/irina/patrik/cpython/Lib
   /home/irina/patrik/cpython/build/lib.linux-x86_64-3.7
  environment: -none-
 command_line: ../coveragepy/coverage/__main__.py debug sys
 source_match: -none-
source_pkgs_match: -none-
include_match: -none-
   omit_match: -none-
  cover_match: /home/irina/patrik/coveragepy/coverage
  pylib_match: /home/irina/patrik/cpython/Lib

--

___
Python tracker 
<http://bugs.python.org/issue29048>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29048] Coverage influence tests, make some of them fail

[Patrik Iselind]

Patrik Iselind added the comment:

@nedbat, do you know of a revision where coverage doesn't influence the tests? 
If so, which one? Perhaps i can perform a bisect search to find when this went 
bad.

I'm testing the latest tip/head. So that's one part of the bisect.

--

___
Python tracker 
<http://bugs.python.org/issue29048>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29048] Coverage influence tests, make some of them fail

[Patrik Iselind]

Patrik Iselind added the comment:

OK, for now. Let's disregard my idea that stderr is swallowed. At some
point these test might have worked under coverage.

I was more thinking to bisect cpython. Perhaps i can find a revision that
works.

Patrik

Den 24 dec 2016 20:26 skrev "Ned Batchelder" :

>
> Ned Batchelder added the comment:
>
> I don't believe this is coverage.py's fault, so I don't have an answer to
> your question.  There is no revision of coverage.py that *does* swallow
> stderr.  Also, as you can see from the test output, it isn't swallowed,
> it's not being captured where it should be captured, but it is appearing on
> the terminal.  I see more than 60 uses of captured_stderr in the Python
> test suite. If coverage.py were swallowing stderr, I would expect dozens of
> test failures.  The problem is more subtle.
>
> If you want, you can bisect against any revision of coverage.py you like,
> perhaps the released 4.2?
>
> --
>
> ___
> Python tracker 
> <http://bugs.python.org/issue29048>
> ___
>

--

___
Python tracker 
<http://bugs.python.org/issue29048>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1116520] Prefix search is filesystem-centric

[Patrik Iselind]

Patrik Iselind added the comment:

Is it enough to include everything in the Lib folder, excluding 
__pycache__, site-packages and the test folder in Lib? Would that be 
representative enough?

Patrik Iselind

Den 2016-12-25 kl. 17:31, skrev Brett Cannon:
> Brett Cannon added the comment:
>
> Originally zip file importing was faster than standard importing from disk 
> because of the fewer stat calls, but importlib caches such things so I don't 
> know if it's still beneficial. As for space savings, I have no idea; you can 
> try zipping the files yourself to find out the space savings.
>
> --
>
> ___
> Python tracker 
> <http://bugs.python.org/issue1116520>
> ___

--

___
Python tracker 
<http://bugs.python.org/issue1116520>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com