[Python-Dev] Fwd: Re: ANN compiler2 : Produce bytecode from Python 2.5 AST
Martin v. Löwis wrote: > Georg Brandl schrieb: >> Perhaps you can bring up a discussion on python-dev about your improvements >> and how they could be integrated into the standard library... > > Let me second this. The compiler package is largely unmaintained and > was known to be broken (and perhaps still is). A replacement > implementation, especially if it comes with a new maintainer, would > be welcome. > > Regards, > Martin Hello python-dev. I use AST-based code inspection and manipulation, and I've been looking forward to using v2.5 ASTs for their increased accuracy, consistency and speed. However, there is as yet no Python-exposed mechanism for compiling v2.5 ASTs to bytecode. So to meet my own need and interest I've been implementing 'compiler2', similar in scope to the stblib compiler package, but generating code from Python 2.5 _ast.ASTs. The code has evolved considerably from the compiler package: in aggregate the changes amount to a re-write. More about the package and its status below. I'm introducing this project here to discuss whether and how these changes should be integrated with the stdlib. I believe there is a prima facie need to have a builtin/stdlib capability for compiling v2.5 ASTs from Python, and there is some advantage to having that be implemented in Python. There is also a case for deprecating the v2.4 ASTs to ease maintenance and reduce the confusion associated with two different AST formats. If there is interest, I'm willing make compiler2 stdlib-ready. I'm also open to alternative approaches, including doing nothing. compiler2 Objectives and Status === My goal is to get compiler2 to produce identical output to __builtin__.compile (at least optionally), while also providing an accessible framework for AST-manipulation, experimental compiler optimizations and customization. compiler2 is not finished - there are some unresolved bugs, and open questions on interface design - but already it produces identical output to __builtin__.compile for all of the stdlib modules and their tests (except for the stackdepth attribute which is different in 12 cases). All but three stdlib modules pass their tests after being compiled using compiler2. More on goals, status, known issues etc... in the project readme.txt at: http://svn.brownspencer.com/pycompiler/branches/new_ast/readme.txt Code is available in Subversion at http://svn.brownspencer.com/pycompiler/branches/new_ast/ The main test script is test/test_compiler.py which compiles all the modules in /Lib and /Lib/test and compares the output with __builtin__.compile. Best regards Michael Spencer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SoC AST generation question
Jay Parlar wrote: > I'm considering applying to be a student in this year's SoC, and the > AST code generation in particular looks interesting to me (listed > here: http://wiki.python.org/moin/CodingProjectIdeas/PythonCore). > > I was wondering a few things: > > 1) Who would most likely mentor this project? > 2) I've never worked in the core before (but have been using Python as > my primary language for about 6 years), so I'm wondering if the > potential mentor thinks it'd even be feasible for me to jump at a > project like this without prior knowledge. > > I'm interested in this project for two reasons. The first is that I'm > still trying to pick my PhD thesis, and I'm leaning in the direction > of automated code generation for embedded systems. I feel like working > on this project would at least push me one way or another in terms of > picking. I've done a major code generation tool before, but it was for > a problem domain I was already an "expert" in, so I didn't develop any > generic methods. > > Also, I've been wanting to contribute to Python core for awhile now, > and just haven't had the opportunity to get my feet wet with the code. > > Thanks, > Jay P. > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org > FWIW, I've already implemented bytecode generation from AST, http://svn.brownspencer.com/pycompiler/branches/new_ast/ as I reported to this list previously. http://mail.python.org/pipermail/python-dev/2006-October/069589.html Cheers Michael Spencer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SoC AST generation question
Brett Cannon wrote: > On 3/19/07, Michael Spencer <[EMAIL PROTECTED]> wrote: ... >>> >> FWIW, I've already implemented bytecode generation from AST, >> http://svn.brownspencer.com/pycompiler/branches/new_ast/ >> as I reported to this list previously. >> http://mail.python.org/pipermail/python-dev/2006-October/069589.html >> > > But the idea being put forth is to use the built-in AST compiler > directly, not for an external library like yours, Michael. > Thanks for the clarification, Brett. I did not understand exactly what is being proposed in the wiki entry, but I thought the reference could be useful. Michael ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 340: syntax suggestion - try opening(filename) as f:
I don't know whether it's true for all the PEP 340 use cases, but the all the
current examples would read very naturally if the block-template could be
specified in an extended try statement:
1. A template for ensuring that a lock, acquired at the start of a
block, is released when the block is left:
try with_lock(myLock):
# Code here executes with myLock held. The lock is
# guaranteed to be released when the block is left (even
# if by an uncaught exception).
2. A template for opening a file that ensures the file is closed
when the block is left:
try opening("/etc/passwd") as f:
for line in f:
print line.rstrip()
3. A template for committing or rolling back a database
transaction:
try transaction(mydb):
4. A template that tries something up to n times:
try auto_retry(3):
f = urllib.urlopen("http://python.org/peps/pep-0340.html";)
print f.read()
5. It is possible to nest blocks and combine templates:
try with_lock(myLock):
try opening("/etc/passwd") as f:
for line in f:
print line.rstrip()
Michael
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
