[issue36300] eval of generator expressions cannot access local variables
New submission from Martin Hosken :
The following code fails:
>>> lcls = {'w': 100}
>>> eval('[w for x in ("hello", "world")]', None, lcls)
Traceback (most recent call last):
File "", line 1, in
File "", line 1, in
File "", line 1, in
NameError: name 'w' is not defined
>>> eval('[w, w, w]', None, lcls)
[100, 100, 100]
whereas in python2 it succeeds
>>> lcls = {'w': 100}
>>> eval('[w for x in ("hello", "world")]', None, lcls)
[100, 100]
--
messages: 337980
nosy: Martin Hosken
priority: normal
severity: normal
status: open
title: eval of generator expressions cannot access local variables
___
Python tracker
<https://bugs.python.org/issue36300>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36300] eval of generator expressions cannot access local variables
Change by Martin Hosken : -- versions: +Python 3.6 ___ Python tracker <https://bugs.python.org/issue36300> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34600] python3 regression ElementTree.iterparse() unable to capture comments
New submission from Martin Hosken : This is a regression from python2 by being forced to use cElementTree. I have code that uses iterparse to process an XML file, but I also want to process comments and so I have a comment handling function called by the parser during iterparse. Under python3 I can find no way to achieve the same thing: ``` parser = et.XMLParser(target=et.TreeBuilder()) parser.parser.CommentHandler = myCommentHandler for event, elem in et.iterparse(fh, parser=parser): ... ``` Somewhat ugly but works in python2, but I can find no way to set a comment handler on the parser in python3. 1. There is no way(?) to get to xml.etree.ElementTree.XMLParser since the C implementation completely masks the python versions. 2. It is possible to create a subclass of TreeBuilder to add a comment method. But the C version XMLParser requires that its TreeBuilder not be a subclass, when used in iterparse. The only solution I found was to copy the XMLParser code out of ElementTree into a private module and use that pure python implementation. Suggested solutions: 1. Allow access to all the python implementations in ElementTree and not just Element. 2. Allow a comments method to be passed to the XMLParser on creation. Thank you. -- components: XML messages: 324719 nosy: Martin Hosken priority: normal severity: normal status: open title: python3 regression ElementTree.iterparse() unable to capture comments type: behavior versions: Python 3.6 ___ Python tracker <https://bugs.python.org/issue34600> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34600] python3 regression ElementTree.iterparse() unable to capture comments
Martin Hosken added the comment:
Sorry. This test is rather long because it is 3 tests:
from __future__ import print_function
import sys
import xml.etree.ElementTree as et
import xml.etree.cElementTree as cet
from io import StringIO
teststr = u"""
Hello World
"""
testf = StringIO(teststr)
if len(sys.argv) >= 2 and 'a' in sys.argv[1]:
testf.seek(0)
for event, elem in et.iterparse(testf, events=["end", "comment"]):
if event == 'end':
print(elem.tag + ": " + str(elem.text))
elif event == 'comment':
print("comment: " + elem.text)
if len(sys.argv) < 2 or 'b' in sys.argv[1]:
testf.seek(0)
def doComment(data):
parser.parser.StartElementHandler("!--", ('text', data))
parser.parser.EndElementHandler("!--")
parser = et.XMLParser()
parser.parser.CommentHandler = doComment
for event, elem in et.iterparse(testf, parser=parser):
if hasattr(elem, 'text'):
print(elem.tag + ": " + str(elem.text))
else:
print(elem.tag + ": " + elem.get('text', ""))
if len(sys.argv) < 2 or 'c' in sys.argv[1] or 'd' in sys.argv[1]:
testf.seek(0)
useet = et if len(sys.argv) < 2 or 'c' in sys.argv[1] else cet
class CommentingTb(useet.TreeBuilder):
def __init__(self):
self.parser = None
def comment(self, data):
self.parser.parser.StartElementHandler("!--", ('text', data))
self.parser.parser.EndElementHandler("!--")
tb = CommentingTb()
parser = useet.XMLParser(target=tb)
tb.parser = parser
kw = {'parser': parser} if len(sys.argv) < 2 or 'c' in sys.argv[1] else {}
for event, elem in useet.iterparse(testf, **kw):
if hasattr(elem, 'text'):
print(elem.tag + ": " + str(elem.text))
else:
print(elem.tag + ": " + elem.get('text', ""))
Test 'a' is how I would like to write the solution to my problem. Not sure why
'comment' isn't supported by iterparse directly, but hey.
Test 'b' is how I solved in it python2
Test 'c' is how I would have to solve it in python3 if it worked
Test 'd' is the same as 'c' but uses cElementTree rather than ElementTree.
Results:
Success output for a test is:
```
!--: None
child:
Hello
root:
```
Python2:
aFails (obviously)
bSucceeds
cSucceeds
dFails: can't inherit from cElementTree.TreeBuilder
Python3:
aFails (obviously)
bFails: XMLParser has no attribute 'parser'
cFails: event handling only supported for ElementTree.TreeBuilder targets
dFails: Gives output but no initial comment component (line 1)
The key failure here is Python3 'c'. This is what stops any hope of comment
handling using the et.XMLParser. The only way I could get around it was to use
my own copy from the source code.
--
___
Python tracker
<https://bugs.python.org/issue34600>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34600] python3 regression ElementTree.iterparse() unable to capture comments
Martin Hosken added the comment:
Blast. Bugs. Sorry. Missing superclass init call in CommentingTb. I enclose the
whole thing again to save editing. Also fixes comment output to give text.
from __future__ import print_function
import sys
import xml.etree.ElementTree as et
import xml.etree.cElementTree as cet
from io import StringIO
teststr = u"""
Hello World
"""
testf = StringIO(teststr)
if len(sys.argv) >= 2 and 'a' in sys.argv[1]:
testf.seek(0)
for event, elem in et.iterparse(testf, events=["end", "comment"]):
if event == 'end':
print(elem.tag + ": " + str(elem.text))
elif event == 'comment':
print("comment: " + elem.text)
if len(sys.argv) < 2 or 'b' in sys.argv[1]:
testf.seek(0)
def doComment(data):
parser.parser.StartElementHandler("!--", ('text', data))
parser.parser.EndElementHandler("!--")
parser = et.XMLParser()
parser.parser.CommentHandler = doComment
for event, elem in et.iterparse(testf, parser=parser):
if elem.tag == "!--":
print(elem.tag + ": " + elem.get('text', ""))
else:
print(elem.tag + ": " + str(elem.text))
if len(sys.argv) < 2 or 'c' in sys.argv[1] or 'd' in sys.argv[1]:
testf.seek(0)
useet = et if len(sys.argv) < 2 or 'c' in sys.argv[1] else cet
class CommentingTb(useet.TreeBuilder):
def __init__(self):
useet.TreeBuilder.__init__(self)
self.parser = None
def comment(self, data):
self.parser.parser.StartElementHandler("!--", ('text', data))
self.parser.parser.EndElementHandler("!--")
tb = CommentingTb()
parser = useet.XMLParser(target=tb)
tb.parser = parser
kw = {'parser': parser} if len(sys.argv) < 2 or 'c' in sys.argv[1] else {}
for event, elem in useet.iterparse(testf, **kw):
if elem.tag == "!--":
print(elem.tag + ": " + elem.get('text', ""))
else:
print(elem.tag + ": " + str(elem.text))
--
___
Python tracker
<https://bugs.python.org/issue34600>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
