[issue35777] mismatched eval() and ast.literal_eval() behavior with unicode_literals

[jez]

jez  added the comment:

I take the point about breaking backward compatibility. The problem, then, is 
in the `ast` documentation, and may not be confined to Python 2.7

The `literal_eval` doc <https://docs.python.org/2/library/ast.html#ast-helpers> 
states no more than that the function evaluates a "string containing a Python 
literal".  The future statement effectively changes the definition of "a Python 
literal" but of course, as the official `__future__` doc 
<https://docs.python.org/2/reference/simple_stmts.html#future> explains, this 
is done for the compilation of each "particular module" in isolation. The 
question is left open whether the occurrence of this term in `ast` 
documentation should be read as "a Python literal (as defined by the ast module 
itself at compile time)",  or "a Python literal (as defined by the caller at 
compile time)".

More generally, future statements effectively change the definition of the term 
"Python syntax", sometimes also simply referred to as "Python".  Unfortunately, 
the `ast` documentation uses these terms loosely throughout, without 
acknowledging that they are mutable. I propose that the `ast` module 
documentation flag cases in which the `ast` module's own definitions of 
"Python" and "Python syntax" may not match those of a caller who has included 
`__future__` statements, leading to unexpected behavior.

I am insufficiently familiar with the full functionality of `ast` to be able to 
identify where this is and is not an issue, except in `literal_eval`, but I can 
see it could in principle affect Python 3.x too.

--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python
versions:  -Python 2.7

___
Python tracker 
<https://bugs.python.org/issue35777>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35777] mismatched eval() and ast.literal_eval() behavior with unicode_literals

[Jez Hill]

New submission from Jez Hill :

Following  `from __future__ import unicode_literals`   the expression `eval(" 
'foo' ")` will return a `unicode` instance.  However, using the same input, 
`ast.literal_eval(" 'foo' ")` will return a `str` instance. The caller's 
preference, that those plain single-quotes should a denote unicode literal, is 
respected by `eval()` but not by `ast.literal_eval()`.   I propose that 
`ast.literal_eval()` be made sensitive to this preference, to bring it in line 
with `eval()`.

--
messages: 334011
nosy: jez
priority: normal
severity: normal
status: open
title: mismatched eval() and ast.literal_eval() behavior with unicode_literals
type: behavior
versions: Python 2.7

___
Python tracker 
<https://bugs.python.org/issue35777>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com