Re: [Python-Dev] [Python-checkins] Daily reference leaks (b78574cb00ab): sum=1120
I'm trying to fix refleaks in 3.6. So far: On 2016-11-09 4:02 AM, solip...@pitrou.net wrote: results for b78574cb00ab on branch "default" test_ast leaked [98, 98, 98] references, sum=294 test_ast leaked [98, 98, 98] memory blocks, sum=294 test_asyncio leaked [3, 0, 0] memory blocks, sum=3 test_code leaked [2, 2, 2] references, sum=6 test_code leaked [2, 2, 2] memory blocks, sum=6 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_pydoc leaked [106, 106, 106] references, sum=318 test_pydoc leaked [42, 42, 42] memory blocks, sum=126 test_trace leaked [12, 12, 12] references, sum=36 test_trace leaked [11, 11, 11] memory blocks, sum=33 test_ast, test_code and test_trace were fixed by https://hg.python.org/cpython/rev/2c6825c9ecfd test_pydoc leaks in test_typing_pydoc. I tried git bisect and it looks like that the first commit that introduced the refleak was the one that added test_typing_pydoc! 62127e60e7b0 doesn't modify any CPython internals, so it looks like that test_typing_pydoc exposed some bug that has existed before it. Any help tracking that down is welcome :) Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] Daily reference leaks (b78574cb00ab): sum=1120
On 2016-11-09 10:16 AM, Yury Selivanov wrote: I'm trying to fix refleaks in 3.6. So far: On 2016-11-09 4:02 AM, solip...@pitrou.net wrote: results for b78574cb00ab on branch "default" test_ast leaked [98, 98, 98] references, sum=294 test_ast leaked [98, 98, 98] memory blocks, sum=294 test_asyncio leaked [3, 0, 0] memory blocks, sum=3 test_code leaked [2, 2, 2] references, sum=6 test_code leaked [2, 2, 2] memory blocks, sum=6 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_pydoc leaked [106, 106, 106] references, sum=318 test_pydoc leaked [42, 42, 42] memory blocks, sum=126 test_trace leaked [12, 12, 12] references, sum=36 test_trace leaked [11, 11, 11] memory blocks, sum=33 test_ast, test_code and test_trace were fixed by https://hg.python.org/cpython/rev/2c6825c9ecfd test_pydoc leaks in test_typing_pydoc. I tried git bisect and it looks like that the first commit that introduced the refleak was the one that added test_typing_pydoc! 62127e60e7b0 doesn't modify any CPython internals, so it looks like that test_typing_pydoc exposed some bug that has existed before it. Any help tracking that down is welcome :) I've created an issue to track the leak in pydocs/typing: http://bugs.python.org/issue28649 Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Parsing f-strings from PEP 498 -- Literal String Interpolation
On Sat, Nov 5, 2016 at 10:36 AM, Nick Coghlan wrote:
> On 5 November 2016 at 04:03, Fabio Zadrozny wrote:
> > On Fri, Nov 4, 2016 at 3:15 PM, Eric V. Smith
> wrote:
> >> Using PyParser_ASTFromString is the easiest possible way to do this.
> Given
> >> a string, it returns an AST node. What could be simpler?
> >
> >
> > I think that for implementation purposes, given the python
> infrastructure,
> > it's fine, but for specification purposes, probably incorrect... As I
> don't
> > think f-strings should accept:
> >
> > f"start {import sys; sys.version_info[0];} end" (i.e.:
> > PyParser_ASTFromString doesn't just return an expression, it accepts any
> > valid Python code, even code which can't be used in an f-string).
>
> f-strings use the "eval" parsing mode, which starts from the
> "eval_input" node in the grammar (which is only a couple of nodes
> higher than 'test', allowing tuples via 'testlist' as well as trailing
> newlines and EOF):
>
> >>> ast.parse("import sys; sys.version_info[0];", mode="eval")
> Traceback (most recent call last):
> File "", line 1, in
> File "/usr/lib64/python3.5/ast.py", line 35, in parse
>return compile(source, filename, mode, PyCF_ONLY_AST)
> File "", line 1
>import sys; sys.version_info[0];
> ^
> SyntaxError: invalid syntax
>
> You have to use "exec" mode to get the parser to allow statements,
> which is why f-strings don't do that:
>
> >>> ast.dump(ast.parse("import sys; sys.version_info[0];",
> mode="exec"))
> "Module(body=[Import(names=[alias(name='sys', asname=None)]),
> Expr(value=Subscript(value=Attribute(value=Name(id='sys', ctx=Load()),
> attr='version_info', ctx=Load()), slice=Index(value=Num(n=0)),
> ctx=Load()))])"
>
> The unique aspect for f-strings that means they don't permit some
> otherwise valid Python expressions is that it also does the initial
> pre-tokenisation based on:
>
> 1. Look for an opening '{'
> 2. Look for a closing '!', ':' or '}' accounting for balanced string
> quotes, parentheses, brackets and braces
>
> Ignoring the surrounding quotes, and using the `atom` node from
> Python's grammar to represent the nesting tracking, and TEXT to stand
> in for arbitrary text, it's something akin to:
>
> fstring: (TEXT ['{' maybe_pyexpr ('!' | ':' | '}')])+
> maybe_pyexpr: (atom | TEXT)+
>
> That isn't quite right, since it doesn't properly account for brace
> nesting, but it gives the general idea - there's an initial really
> simple tokenising pass that picks out the potential Python
> expressions, and then those are run through the AST parser's
> equivalent of eval().
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
>
Hi Nick and Eric,
Just wanted to say thanks for the feedback and point to a grammar I ended
up doing on my side (in JavaCC), just in case someone else decides to do a
formal grammar later on it can probably be used as a reference (shouldn't
be hard to convert it to a bnf grammar):
https://github.com/fabioz/Pydev/blob/master/plugins/org.python.pydev.parser/src/org/python/pydev/parser/grammar_fstrings/grammar_fstrings.jjt
Also, as a feedback, I found it a bit odd that there can't be any space nor
new line between the last format specifiers and '}'
I.e.:
f'''{
dict(
a = 10
)
!r
}
'''
is not valid, whereas
f'''{
dict(
a = 10
)
!r}
'''
is valid -- as a note, this means my grammar has a bug as both versions are
accepted -- and I currently don't care enough about that change from the
implementation to fix it ;)
Cheers,
Fabio
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Parsing f-strings from PEP 498 -- Literal String Interpolation
On 9 November 2016 at 16:20, Fabio Zadrozny wrote:
> Also, as a feedback, I found it a bit odd that there can't be any space nor
> new line between the last format specifiers and '}'
FWIW, that is the case for normal format strings, as well:
>>> print("{!r\n}".format(12))
Traceback (most recent call last):
File "", line 1, in
ValueError: expected ':' after conversion specifier
I guess the behaviour is simply inherited from there.
Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Parsing f-strings from PEP 498 -- Literal String Interpolation
On 11/9/2016 11:35 AM, Paul Moore wrote:
On 9 November 2016 at 16:20, Fabio Zadrozny wrote:
Also, as a feedback, I found it a bit odd that there can't be any space nor
new line between the last format specifiers and '}'
FWIW, that is the case for normal format strings, as well:
print("{!r\n}".format(12))
Traceback (most recent call last):
File "", line 1, in
ValueError: expected ':' after conversion specifier
I guess the behaviour is simply inherited from there.
Right. That and the fact that whitespace is significant inside the
format specifier portion:
>>> '{:%H:%M:%S\n}'.format(datetime.datetime.now())
'17:14:10\n'
I don't think it's worth changing this to allow whitespace after the
optional one-character conversion flag.
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
