[issue38029] Should io.TextIOWrapper raise an error at instantiation if a StringIO is passed as 'buffer'?
New submission from Brian Skinn :
If I read the docs correctly, io.TextIOWrapper is meant to provide a str-typed
interface to an underlying bytes stream.
If a TextIOWrapper is instantiated with the underlying buffer=io.StringIO(), it
breaks:
>>> import io
>>> tw = io.TextIOWrapper(io.StringIO())
>>> tw.write(b'abcd\n')
Traceback (most recent call last):
File "", line 1, in
TypeError: write() argument must be str, not bytes
>>> tw.write('abcd\n')
5
>>> tw.read()
Traceback (most recent call last):
File "", line 1, in
TypeError: string argument expected, got 'bytes'
>>> tw.read(1)
Traceback (most recent call last):
File "", line 1, in
TypeError: underlying read() should have returned a bytes-like object, not 'str'
Would it be better for TextIOWrapper to fail earlier, at instantiation-time,
for this kind of (unrecoverably?) broken type mismatch?
--
components: Library (Lib)
messages: 351139
nosy: bskinn
priority: normal
severity: normal
status: open
title: Should io.TextIOWrapper raise an error at instantiation if a StringIO is
passed as 'buffer'?
type: behavior
versions: Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue38029>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45391] 3.10 objects.inv classifies UnionType as data
Brian Skinn added the comment: If I understand the problem correctly, these mis-attributions of roles (to 'data' instead of 'class' come about when a thing that is technically a class is defined in source using simple assignment, as with UnionType. Problematic entries will thus have 'data' as role, and their identifiers will be camel-cased. So, as a quick search to identify likely candidates: >>> import re, sphobjinv as soi >>> from pprint import pprint >>> inv = soi.Inventory(url="https://docs.python.org/3.10/objects.inv";) # Find entries where the first character after the final period # is uppercase, and the second character after the final period # is lowercase. >>> pat = re.compile(r"([.][A-Z][a-z])[^.]*$") >>> pprint([obj.name for obj in inv.objects if obj.role == "data" and >>> pat.search(obj.name)]) ['_thread.LockType', 'ast.PyCF_ALLOW_TOP_LEVEL_AWAIT', 'ast.PyCF_ONLY_AST', 'ast.PyCF_TYPE_COMMENTS', 'importlib.resources.Package', 'importlib.resources.Resource', 'socket.SocketType', 'types.AsyncGeneratorType', 'types.BuiltinFunctionType', 'types.BuiltinMethodType', 'types.CellType', 'types.ClassMethodDescriptorType', 'types.CoroutineType', 'types.EllipsisType', 'types.FrameType', 'types.FunctionType', 'types.GeneratorType', 'types.GetSetDescriptorType', 'types.LambdaType', 'types.MemberDescriptorType', 'types.MethodDescriptorType', 'types.MethodType', 'types.MethodWrapperType', 'types.NoneType', 'types.NotImplementedType', 'types.UnionType', 'types.WrapperDescriptorType', 'typing.Annotated', 'typing.Any', 'typing.AnyStr', 'typing.Callable', 'typing.ClassVar', 'typing.Concatenate', 'typing.Final', 'typing.Literal', 'typing.NoReturn', 'typing.Optional', 'typing.ParamSpecArgs', 'typing.ParamSpecKwargs', 'typing.Tuple', 'typing.TypeAlias', 'typing.TypeGuard', 'typing.Union', 'weakref.CallableProxyType', 'weakref.ProxyType', 'weakref.ProxyTypes', 'weakref.ReferenceType'] I would guess those 'ast.PyCF...' objects can be ignored, they appear to be constants? -- nosy: +bskinn ___ Python tracker <https://bugs.python.org/issue45391> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45391] 3.10 objects.inv classifies UnionType as data
Brian Skinn added the comment: Identifiers starting with two uppercase letters returns a HUGE list. >>> pat2 = re.compile(r"([.][A-Z][A-Z])[^.]*$") Filtering down by only those that contain.lower() "type": >>> pprint([obj.name for obj in inv.objects if obj.role == "data" and >>> pat2.search(obj.name) and "type" in obj.name.lower()]) ['errno.EPROTOTYPE', 'locale.LC_CTYPE', 'sqlite3.PARSE_DECLTYPES', 'ssl.CHANNEL_BINDING_TYPES', 'token.TYPE_COMMENT', 'token.TYPE_IGNORE', 'typing.TYPE_CHECKING', 'xml.parsers.expat.XMLParserType'] Of these, only 'xml.parsers.expat.XMLParserType' seems to me a likely problem entry. -- ___ Python tracker <https://bugs.python.org/issue45391> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37699] Explicit mention of raised ValueError's after .detach() of underlying IO buffer
Brian Skinn added the comment: Indeed, I hadn't been thinking about the testing/maintenance burden to CPython or other implementations when I made the suggestion. I no longer have a strong opinion about this change, so I am happy to reject/close. -- resolution: -> rejected stage: -> resolved status: pending -> closed ___ Python tracker <https://bugs.python.org/issue37699> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37134] [EASY] Use PEP570 syntax in the documentation
Change by Brian Skinn : -- nosy: +bskinn ___ Python tracker <https://bugs.python.org/issue37134> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37134] Use PEP570 syntax in the documentation
Brian Skinn added the comment: First, for anyone interested, there are screenshots and links to docs versions at the SC GH issue (https://github.com/python/steering-council/issues/12#issuecomment-498856524, and following) where we're exploring what the tabbed approach to the PEP570 docs might look like. Second, I'd like to suggest an additional aspect of the docs situation for consideration. Regardless of which specific approach is decided upon here, it seems likely the consensus will be a course of action requiring touching the majority (entirety?) of the stdlib API docs. Currently, all of the API docs are manually curated in the .rst, at minimum for the reasons laid out by Victor in bpo25461 (https://bugs.python.org/issue25461#msg253381). If there's any chance that the balance of factors has changed such that using autodoc (or similar) *would* now be preferable, IWSTM that now is a good time to have that discussion, so that an autodoc conversion could be done at the same time as the PEP570 rework. -- ___ Python tracker <https://bugs.python.org/issue37134> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37134] Use PEP570 syntax in the documentation
Brian Skinn added the comment: Brett, to be clear, this sounds like the tabbed solution is not going to be used at this point? If so, I'll pull down the tabbed docs I'm hosting. -- ___ Python tracker <https://bugs.python.org/issue37134> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37134] Use PEP570 syntax in the documentation
Brian Skinn added the comment: :thumbsup: Glad I happened to be in the right place at the right time to put it together. I'll leave the tabslash repo up for future reference. -- ___ Python tracker <https://bugs.python.org/issue37134> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37699] Explicit mention of raised ValueError's after .detach() of underlying IO buffer
Change by Brian Skinn : -- type: enhancement -> behavior ___ Python tracker <https://bugs.python.org/issue37699> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37699] Explicit mention of raised ValueError's after .detach() of underlying IO buffer
New submission from Brian Skinn : Once the underlying buffer/stream is .detach()ed from an instance of a subclass of TextIOBase or BufferedIOBase, accession of most attributes defined on TextIOBase/BufferedIOBase or the IOBase parent, as well as calling of most methods defined on TextIOBase/BufferedIOBase/IOBase, results in raising of a ValueError. Currently, the documentation of both .detach() methods states simply: > After the raw stream has been detached, the buffer is in an unusable state. I propose augmenting the above to something like the following in the docs for both .detach() methods, to make this behavior more explicit: > After the raw stream has been detached, the buffer > is in an unusable state. As a result, accessing/calling most > attributes/methods of either :class:`BufferedIOBase` or its > :class:`IOBase` parent will raise :exc:`ValueError`. I confirm that the error raised for both `BufferedReader` and `TextIOWrapper` after `.detach()` *is* ValueError in all of 3.5.7, 3.6.8, 3.7.3, and 3.8.0b1. -- assignee: docs@python components: Documentation messages: 348594 nosy: bskinn, docs@python priority: normal severity: normal status: open title: Explicit mention of raised ValueError's after .detach() of underlying IO buffer type: enhancement versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue37699> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36714] Tweak doctest 'example' regex to allow a leading ellipsis in 'want' line
Brian Skinn added the comment: Mm, agreed--that regex wouldn't be hard to write. The problem is, AFAICT there's irresolvable syntactic ambiguity in a line starting with exactly three periods, if the doctest PS2 specification is not constrained to be exactly "... ". In such a case, "..." could mark either (1) an ellipsis standing in for an entire line of 'want', or (2) a PS2, marking a blank line in 'source'. I don't really think aggressive lookahead would help much -- an arbitrary number of following lines could contain exactly "...", and the intended transition from 'source' to 'want' could lie at any one of them. The nonrecursive nature of regex is unhelpful here, but I don't think one could even write a recursive-descent parser, or similar, that could be 100% reliable on a single comparison. It would have to test the string against all the various splits between 'source' and 'want' along those "..." lines, and see if any match. Hairy mess. AFAICT, defining "... " as PS2, and "..." as 'ellipsis representing a whole line' is the cleanest solution from a logical point of view. Of course, then it's *visually* confusing, because trailing space. ¯\_(ツ)_/¯ -- ___ Python tracker <https://bugs.python.org/issue36714> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36714] Tweak doctest 'example' regex to allow a leading ellipsis in 'want' line
Brian Skinn added the comment:
I suppose one alternative solution might be to tweak the ELLIPSIS feature of
doctest, such that it would interpret a run of >=3 periods in a row (matching
regex pattern of "[.]{3,}") as 'ellipsis'.
The regex for PS2 could then have a negative lookahead added, so that it *only*
matches three periods, plus optionally other content: '\.\.\.(?!\.)'
That way, a line like "... foo" would retain the current meaning of "'source'
line, consisting of PS2 plus the identifier 'foo'", but the meaning of
"arbitrary content followed by ' foo'" could be achieved by " foo", since
the leading "" would NOT match the negative lookahead for PS2.
In other situations, where "..." is *not* the leading non-whitespace content,
the old behavior suffices: the PS2 regex won't match anyways, so it'll be left
for ELLIPSIS to process.
--
___
Python tracker
<https://bugs.python.org/issue36714>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36714] Tweak doctest 'example' regex to allow a leading ellipsis in 'want' line
Brian Skinn added the comment:
On reflection, it would probably be better to limit the ELLIPSIS to 3 or 4
periods ('[.]{3,4}'); otherwise, it would be impossible to express an ellipsis
followed by a period in a 'want'.
--
___
Python tracker
<https://bugs.python.org/issue36714>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning
New submission from Brian Skinn :
In [this project](https://github.com/bskinn/stdio-mgr) of mine, I have a tox
matrix set up with Pythons from 3.3. to 3.8. I have pytest set up to run
doctest on my
[`README.rst`](https://github.com/bskinn/stdio-mgr/blob/6444cce8e5866e2d519c1c0630551d8867f30c9a/README.rst).
For Pythons 3.4 to 3.7 (3.4.10, 3.5.7, 3.6.8, 3.7.2), the following doctest
example passes:
```
>>> import warnings
>>> with stdio_mgr() as (in_, out_, err_):
... warnings.warn("'foo' has no 'bar'")
... err_cap = err_.getvalue()
>>> err_cap
"...UserWarning: 'foo' has no 'bar'\n..."
```
Under Python 3.8.0a3, though, it fails (actual local paths elided):
```
$ tox -re py38-attrs_latest
.package recreate: .../.tox/.package
.package installdeps: wheel, setuptools, attrs>=17.1
py38-attrs_latest recreate: .../.tox/py38-attrs_latest
py38-attrs_latest installdeps: attrs, pytest
py38-attrs_latest inst: .../.tox/.tmp/package/1/stdio-mgr-1.0.2.dev1.tar.gz
py38-attrs_latest installed:
atomicwrites==1.3.0,attrs==19.1.0,more-itertools==7.0.0,pluggy==0.9.0,py==1.8.0,pytest==4.4.1,six==1.12.0,stdio-mgr==1.0.2.dev1
py38-attrs_latest run-test-pre: PYTHONHASHSEED='2720295779'
py38-attrs_latest run-test: commands[0] | pytest
===
test session starts
platform linux -- Python 3.8.0a3, pytest-4.4.1, py-1.8.0, pluggy-0.9.0
cachedir: .tox/py38-attrs_latest/.pytest_cache
rootdir: ..., inifile: tox.ini
collected 6 items
README.rst F
[ 16%]
tests/test_stdiomgr_base.py .
[100%]
=
FAILURES
=
___
[doctest] README.rst
___
077
078 **Mock** ``stderr``\ **:**
079
080 .. code ::
081
082 >>> import warnings
083 >>> with stdio_mgr() as (in_, out_, err_):
084 ... warnings.warn("'foo' has no 'bar'")
085 ... err_cap = err_.getvalue()
086 >>> err_cap
Expected:
"...UserWarning: 'foo' has no 'bar'\n..."
Got:
':2: UserWarning: \'foo\' has no \'bar\'\n
warnings.warn("\'foo\' has no \'bar\'")\n'
.../README.rst:86: DocTestFailure
1 failed, 5 passed in 0.06 seconds
ERROR: InvocationError for command .../.tox/py38-attrs_latest/bin/pytest
(exited with code 1)
_
summary
__
ERROR: py38-attrs_latest: commands failed
```
If I change the doctest in README to the following, where the expected output
is surrounded by single-quotes instead of double-quotes, and the internal
single quotes are escaped, it passes fine in 3.8.0a3:
```
>>> import warnings
>>> with stdio_mgr() as (in_, out_, err_):
... warnings.warn("'foo' has no 'bar'")
... err_cap = err_.getvalue()
>>> err_cap
'...UserWarning: \'foo\' has no \'bar\'\n...'
```
But, naturally, it fails in 3.7 and below.
It *looks* like this is probably a glitch somewhere in 3.8.0a3, where this
string containing single quotes is rendered (at the REPL?) using enclosing
single quotes and escaped internal single quotes, rather than enclosing
double-quotes and non-escaped internal single-quotes?
--
components: Library (Lib)
messages: 340637
nosy: bskinn
priority: normal
severity: normal
status: open
title: Change (regression?) in v3.8.0a3 doctest output after capturing the
stderr out
[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning
Brian Skinn added the comment: Karthikeyan, my apologies for the slow reply -- I posted this right before I went to bed. To emphasize, the change to the formatting of the string contents, by adding the filename, I think is not problematic: I'm using ellipses to elide everything before and after my custom "warning" message. Rather, I think the problem is that the string is being rendered as a '' string, instead of as a "" string; IOW: 'Test string with \'enclosing\' single quotes' vs "Test string with 'enclosing' double quotes" --- In the interim, as you suggest, Karthikeyan, I can just conditionally skip the doctests on 3.8 with a suitable pytest -k flag. -- ___ Python tracker <https://bugs.python.org/issue36695> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning
Brian Skinn added the comment:
Here is warn.py, a minimal no-dependency repro script:
```
import doctest
class Tester:
r"""Provide docstring for testing.
>>> import warnings
>>> from contextlib import redirect_stderr
>>> from io import StringIO
>>> sio = StringIO()
>>> with redirect_stderr(sio):
... warnings.warn("Warning 'containing' single quotes")
>>> sio.getvalue()
"...UserWarning: Warning 'containing' single quotes\n..."
"""
doctest.run_docstring_examples(
Tester(),
{},
optionflags=doctest.ELLIPSIS,
)
```
`python3.7 warn.py` (3.7.3) gives no output.
`python3.8 warn.py` (3.8.0a3) gives:
```
$ python3.8 warn.py
File "warn.py", line ?, in NoName
Failed example:
sio.getvalue()
Expected:
"...UserWarning: Warning 'containing' single quotes\n..."
Got:
':2: UserWarning: Warning \'containing\' single quotes\n
warnings.warn("Warning \'containing\' single quotes")\n'
```
The problem appears to be centered around *doctest*, as the following script
DOES NOT raise AssertionError with either of 3.7 or 3.8:
```
import warnings
from contextlib import redirect_stderr
from io import StringIO
sio = StringIO()
with redirect_stderr(sio):
warnings.warn("Warning 'containing' single quotes")
assert " 'containing' " in sio.getvalue()
```
--
___
Python tracker
<https://bugs.python.org/issue36695>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning
Brian Skinn added the comment:
, it seems like the problem must somehow stem from the new commit using
frame.f_code.co_filename (or the C equivalent), instead of using __file__ as
previously.
Consider this warn2.py, similar to the other but with no single quotes in the
warning message:
```
import doctest
class Tester:
r"""Provide docstring for testing.
>>> import warnings
>>> from contextlib import redirect_stderr
>>> from io import StringIO
>>> sio = StringIO()
>>> with redirect_stderr(sio):
... warnings.warn("Warning with no quotes")
>>> sio.getvalue()
'...'
"""
doctest.run_docstring_examples(
Tester(),
{},
optionflags=doctest.ELLIPSIS,
)
```
This doctest PASSES for me in both 3.7 and 3.8; note that the expected doctest
output from `sio.getvalue()` is *single-quote delimited*. This implies to me
that REPL string output is single-quote delimited by default, and that there's
some sort of "smart string formatter" functionality involved that is working
correctly in 3.7 but not in 3.8, which REPL-prints the single-quote-containing
string using enclosing double-quotes, so as to avoid escaping the internal
single quotes.
Why 11a8966 would break this in this way is ... baffling to me.
---
Unfortunately, I don't think it will work to fix the doctest on my end simply
by using `print(sio.getvalue())`, as the resulting message is one line long in
3.7, but two lines long in 3.8. Further, doctest gets confused when you try to
start a line of output with an ellipsis, as it thinks it's a continuation of
the prior command.
--
___
Python tracker
<https://bugs.python.org/issue36695>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning
Brian Skinn added the comment: TBH, now that I've tweaked tox and CI just not to run the doctests on 3.8, I don't really need this to be fixed. This seems like such an edge case -- a doctest catching a warning with a message containing single quotes -- it might not really be worth the effort to figure out. Unless someone is really invested in tracking this down, I would be content to close. -- ___ Python tracker <https://bugs.python.org/issue36695> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning
Brian Skinn added the comment:
It looks to me like it's a standard feature of the CPython string rendering
routines, where if single and double quotes are present in any string, the
preferred rendering is enclosure with single quotes with escaped internal
single quotes.
On 3.6.6, regardless how I enter the following, it always returns enclosed in
single quotes:
>>> """ ' " """
' \' " '
>>> ''' ' " '''
' \' " '
>>> ' \' " '
' \' " '
>>> " ' \" "
' \' " '
For my particular situation, then, the problem is that my warning message, as
it sits in the source, consists of a double-quoted string that contains single
quotes. Then, when 3.8 doctest goes to print the source line, it has to print
a string containing both single and double quotes, so the above default
rendering rule kicks in and it gets printed with enclosing single-quotes. For
3.7 doctest, where the regex doesn't match, the source line doesn't get
printed, and so the resulting string contains no double quotes, and thus the
string gets printed with enclosing double quotes.
Clearly, the solution is just for me to change the warning message! And indeed,
changing to `warnings.warn("foo has no bar")` and updating the expected result
to `'...UserWarning: foo has no bar\n...'` yields a passing test on both 3.7
and 3.8 now.
--
___
Python tracker
<https://bugs.python.org/issue36695>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning
Change by Brian Skinn : -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue36695> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning
Brian Skinn added the comment: Thank you for taking the time to dig into it so deeply! -- ___ Python tracker <https://bugs.python.org/issue36695> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36714] Tweak doctest 'example' regex to allow a leading ellipsis in 'want' line
New submission from Brian Skinn :
doctest requires code examples have PS1 as ">>> " and PS2 as "... " -- that is,
each is three printed characters, followed by a space:
```
$ cat ell_err.py
import doctest
class Foo:
"""Test docstring.
>>>print("This is a test sentence.")
...a test...
"""
doctest.run_docstring_examples(
Foo(),
{},
optionflags=doctest.ELLIPSIS,
)
$ python3.8 --version
Python 3.8.0a3
$ python3.8 ell_err.py
Traceback (most recent call last):
...
ValueError: line 3 of the docstring for NoName lacks blank after >>>: '
>>>print("This is a test sentence.")'
$ cat ell_print.py
import doctest
class Foo:
"""Test docstring.
>>> print("This is a test sentence.")
...a test...
"""
doctest.run_docstring_examples(
Foo(),
{},
optionflags=doctest.ELLIPSIS,
)
$ python3.8 ell_print.py
Traceback (most recent call last):
...
ValueError: line 4 of the docstring for NoName lacks blank after ...: '...a
test...'
```
AFAICT, this behavior is consistent across 3.4.10, 3.5.7, 3.6.8, 3.7.3, and
3.8.0a3.
**However**, in this `ell_print.py` above, that "PS2" line isn't actually meant
to be a continuation of the 'source' portion of the example; it's meant to be
the *output* (the 'want') of the example, with a leading ellipsis to be matched
per `doctest.ELLIPSIS` rules.
The regex currently used to look for the 'source' of an example is
(https://github.com/python/cpython/blob/4f5a3493b534a95fbb01d593b1ffe320db6b395e/Lib/doctest.py#L583-L586):
```
(?P
(?:^(?P [ ]*) >>>.*)# PS1 line
(?:\n [ ]* \.\.\. .*)*) # PS2 lines
\n?
```
Since this pattern is compiled with re.VERBOSE
(https://github.com/python/cpython/blob/4f5a3493b534a95fbb01d593b1ffe320db6b395e/Lib/doctest.py#L592),
the space-as-fourth-character in PS1/PS2 is not explicitly matched.
I propose changing the regex to:
```
(?P
(?:^(?P [ ]*) >>>[ ].*)# PS1 line
(?:\n [ ]* \.\.\.[ ] .*)*) # PS2 lines
\n?
```
This will then *explicitly* match the trailing space of PS1; it *shouldn't*
break any existing doctests, because the parsing code lower down has already
been requiring that space to be present in PS1, as shown for `ell_err.py` above.
This will also require an *explicit trailing space* to be present in order for
a line starting with three periods to be interpreted as a PS2 line of 'source';
otherwise, it will be treated as part of the 'want'. I made this change in my
local user install of 3.8's doctest.py, and it works as I expect on
`ell_print.py`, passing the test:
```
$ python3.8 ell_print.py
$
$ cat ell_wrongprint.py
import doctest
class Foo:
"""Test docstring.
>>> print("This is a test sentence.")
...a foo test...
"""
doctest.run_docstring_examples(
Foo(),
{},
optionflags=doctest.ELLIPSIS,
)
$ python3.8 ell_wrongprint.py
**
File "ell_wrongprint.py", line ?, in NoName
Failed example:
print("This is a test sentence.")
Expected:
...a foo test...
Got:
This is a test sentence.
```
For completeness, the following piece of regex in the 'want' section
(https://github.com/python/cpython/blob/4f5a3493b534a95fbb01d593b1ffe320db6b395e/Lib/doctest.py#L589):
```
(?![ ]*>>>) # Not a line starting with PS1
```
should probably also be changed to:
```
(?![ ]*>>>[ ]) # Not a line starting with PS1
```
I would be happy to put together a PR for this; I would plan to take a ~TDD
style approach, implementing a few tests first and then making the regex change.
--
components: Library (Lib)
messages: 340788
nosy: bskinn
priority: normal
severity: normal
status: open
title: Tweak doctest 'example' regex to allow a leading ellipsis in 'want' line
type: enhancement
versions: Python 3.8
___
Python tracker
<https://bugs.python.org/issue36714>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning
Brian Skinn added the comment:
The application of repr() (or a repr()-equivalent) appears to occur as some
part of the exec(compile(...)) call within doctest
(https://github.com/python/cpython/blob/4f5a3493b534a95fbb01d593b1ffe320db6b395e/Lib/doctest.py#L1328-L1329).
On 3.6.6, in REPL:
```
>>> from contextlib import redirect_stdout
>>> from io import StringIO
>>> sio = StringIO()
>>> with redirect_stdout(sio):
... exec(compile('""" \' " """', 'dummyfile', 'single'))
...
>>> output = sio.getvalue()
>>> output
'\' \\\' " \'\n'
```
Also 3.6.6, at Win cmd:
```
>type exec_compile.py
from contextlib import redirect_stdout
from io import StringIO
exec(compile('""" \' " """', 'dummyfile', 'single'))
sio = StringIO()
with redirect_stdout(sio):
exec(compile('""" \' " """', 'dummyfile', 'single'))
output = sio.getvalue()
assert output == '\' \\\' " \'\n'
>python exec_compile.py
' \' " '
>
```
It *looks* like exec() executes the compile()'d source as if it were typed into
a REPL -- IOW, any unassigned non-None return value X gets pushed to stdout as
repr(X). This is then what the doctest self._fakeout captures for comparison to
the 'want' of the example.
--
___
Python tracker
<https://bugs.python.org/issue36695>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning
Brian Skinn added the comment: Well, the warning content *itself* may not get passed through the displayhook at raise-time, in the process of being run through stderr and displayed by the REPL. But, when you capture the warning content with redirect_stderr(sio) and then ">>> sio.getvalue()", the contents of the capture from stderr, as produced by .getvalue(), *will* get passed through the displayhook, and thus be escaped. In theory, I could have obtained a consistent 'want' by using print() as you've done. However, for my particular example (see OP), I wanted to elide the first part of the warning message, which is messy, irrelevant to my code example, and can change from Python version to Python version. However, as doctest is currently implemented, a 'want' can't start with an ellipsis because it collides with the regex that detects PS2 prompts (https://github.com/python/cpython/blob/4f5a3493b534a95fbb01d593b1ffe320db6b395e/Lib/doctest.py#L583-L586). See #36714 (https://bugs.python.org/issue36714) for more information and a proposed enhancement/fix. -- ___ Python tracker <https://bugs.python.org/issue36695> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning
Brian Skinn added the comment: LOL. No special thanks necessary, that last post only turned into something coherent (and possibly correct, it seems...) after a LOT of diving into the source, fiddling with the code, and (((re-)re-)re-)writing! Believe me, it reads as a lot more knowledgeable and confident than I actually felt while writing it. :-D Thanks to all of you for coming along with me on this dive into the CPython internals! -- ___ Python tracker <https://bugs.python.org/issue36695> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36714] Tweak doctest 'example' regex to allow a leading ellipsis in 'want' line
Brian Skinn added the comment:
Ahh, this *will* break some doctests: any with blank PS2 lines in the 'source'
portion without the explicit trailing space:
1] >>> def foo():
2] ...print("bar")
3] ...
4] ...print("baz")
5] >>> foo()
6] bar
7] baz
If line 3 contains exactly "..." instead of starting with "... ", it will not
be recognized as a PS2 line and the example will be parsed as:
'source'
>>> def foo():
...print("bar")
'want'
...
...print("baz")
IMO this isn't a *terribly* unreasonable tradeoff, though -- it would enable
the specific ellipsis use-case as in the OP, at the cost of breaking some
doctests, which shouldn't(?) be in any critical paths?
--
___
Python tracker
<https://bugs.python.org/issue36714>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33109] argparse: make new 'required' argument to add_subparsers default to False instead of True
Brian Skinn added the comment: I second Wolfgang's recommendation to change the default back to `False`. I started developing CLI apps &c. in Python ~4yrs ago; I dabbled briefly in 2.7, then switched firmly to Python 3. When I started, I was aimed at supporting 3.3 to 3.5; now I'm specifically supporting 3.4 to 3.6, but starting to test my code against the 3.7b versions. Thus, all I've ever known is the default `required=False` behavior. On my one tool currently using subparsers (https://github.com/bskinn/sphobjinv), I made the subparser choice optional, to allow a `sphobjinv --version` invocation. So, when 3.7 failed that test (https://github.com/bskinn/sphobjinv/blob/6c1f22e40dc3d129485462aec05adbed2ff40ab8/sphobjinv/test/sphobjinv_cli.py#L419-L422), it seemed like a regression to me. All that said, given that the `subparsers.required = False` fix is a clean, single line, I have no beef with a `True` default if that's preferred. However, please include this change in the 3.7 CHANGELOG, at least. -- nosy: +bskinn ___ Python tracker <https://bugs.python.org/issue33109> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
