Re: [Python-Dev] Policy for making changes to the AST

2011-04-07 Thread fwierzbi...@gmail.com
On Tue, Apr 5, 2011 at 6:37 AM, Nick Coghlan wrote: > 1. Making "docstring" an attribute of the Function node rather than > leaving it embedded as the first statement in the suite (this avoids > issues where AST-based constant folding could potentially corrupt the > docstring) > 2. Collapsing Num,

Re: [Python-Dev] Policy for making changes to the AST

2011-04-07 Thread Benjamin Peterson
2011/4/7 Maciej Fijalkowski : >> AFAIK the AST is >> CPython-specific so should be treated with the same attitude as >> changes to the bytecode. That means, do it conservatively, since there >> *are* people who like to write tools that manipulate or analyze this, >> and while they know they're doin

Re: [Python-Dev] Policy for making changes to the AST

2011-04-07 Thread Maciej Fijalkowski
> AFAIK the AST is > CPython-specific so should be treated with the same attitude as > changes to the bytecode. That means, do it conservatively, since there > *are* people who like to write tools that manipulate or analyze this, > and while they know they're doing something CPython and > version-s

Re: [Python-Dev] Policy for making changes to the AST

2011-04-05 Thread Greg Ewing
Nick Coghlan wrote: 1. Making "docstring" an attribute of the Function node rather than leaving it embedded as the first statement in the suite (this avoids issues where AST-based constant folding could potentially corrupt the docstring) 2. Collapsing Num, Str, Bytes, Ellipsis into a single Lite

Re: [Python-Dev] Policy for making changes to the AST

2011-04-05 Thread Nick Coghlan
On Tue, Apr 5, 2011 at 7:49 PM, "Martin v. Löwis" wrote: > Historically, that's incorrect. The AST structure was introduced to > simplify the implementation of (C)Python. Exposing it to Python modules > was done primarily because it's neat and was easy to do - not because > any specific use was ex

Re: [Python-Dev] Policy for making changes to the AST

2011-04-05 Thread Martin v. Löwis
> Add py.test as an application that uses the AST to support Jython, > PyPy and CPython in a portable way. I always assumed AST was created > *because* bytecode was too CPython specific (but then I've never > implemented a language). Historically, that's incorrect. The AST structure was introduce

Re: [Python-Dev] Policy for making changes to the AST

2011-04-05 Thread Martin v. Löwis
> Ok, so it sounds like ast is *not* limited to CPython? That makes it > harder to justify changing it just so as to ease the compilation > process in CPython (as opposed to add new language features). I propose a different view: the AST *is* implementation specific, although implementations are c

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Eugene Toder
> Ok, so it sounds like ast is *not* limited to CPython? That makes it > harder to justify changing it just so as to ease the compilation > process in CPython (as opposed to add new language features). The changes above are not just for CPython, but to simplify processing of AST in general, by red

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Terry Reedy
On 4/4/2011 4:05 PM, Dino Viehland wrote: "reduce" to the core DLR nodes on-demand). We already do a huge amount of manipulation of those ASTs from optimizations (constant folding being the primary one) to re-writing them completely for things like generators or sys.settrace support and other

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Guido van Rossum
On Mon, Apr 4, 2011 at 12:56 PM, Terry Reedy wrote: > Moving optimizations from bytecode (where they > are demonstrably a bit fragile) to ast manipulations (where we presume they > will be more robust and can be broader) should be a win in itself I am still doubtful of that. While in theory it is

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Dino Viehland
Terry wrote: > Are at least some of the implementation methods similar enough that they > could use the same AST? It is, after all, a *semantic* translation into > another > language, and that need not depend on subsequent transforation and > compilation to the ultimate target. A multiple-implemen

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Terry Reedy
On 4/4/2011 2:00 PM, Guido van Rossum wrote: On Mon, Apr 4, 2011 at 10:05 AM, fwierzbi...@gmail.com wrote: As a re-implementor of ast.py that tries to be node for node compatible, I'm fine with #1 but would really like to have tests that will fail in test_ast.py to alert me! [and] On Mon, A

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Floris Bruynooghe
On 4 April 2011 19:07, Glyph Lefkowitz wrote: > > On Apr 4, 2011, at 2:00 PM, Guido van Rossum wrote: > >> On Mon, Apr 4, 2011 at 10:05 AM, fwierzbi...@gmail.com >> wrote: >>> As a re-implementor of ast.py that tries to be node for node >>> compatible, I'm fine with #1 but would really like to ha

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Dino Viehland
Guido wrote: > On Mon, Apr 4, 2011 at 10:05 AM, fwierzbi...@gmail.com > wrote: > > As a re-implementor of ast.py that tries to be node for node > > compatible, I'm fine with #1 but would really like to have tests that > > will fail in test_ast.py to alert me! > > [and] > > On Mon, Apr 4, 2011 a

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Glyph Lefkowitz
On Apr 4, 2011, at 2:00 PM, Guido van Rossum wrote: > On Mon, Apr 4, 2011 at 10:05 AM, fwierzbi...@gmail.com > wrote: >> As a re-implementor of ast.py that tries to be node for node >> compatible, I'm fine with #1 but would really like to have tests that >> will fail in test_ast.py to alert me!

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Guido van Rossum
On Mon, Apr 4, 2011 at 10:05 AM, fwierzbi...@gmail.com wrote: > As a re-implementor of ast.py that tries to be node for node > compatible, I'm fine with #1 but would really like to have tests that > will fail in test_ast.py to alert me! [and] On Mon, Apr 4, 2011 at 10:38 AM, Michael Foord wrote

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Michael Foord
On 04/04/2011 18:05, fwierzbi...@gmail.com wrote: As a re-implementor of ast.py that tries to be node for node compatible, I'm fine with #1 but would really like to have tests that will fail in test_ast.py to alert me! A lot of tools that work with Python source code use ast - so even though

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread fwierzbi...@gmail.com
As a re-implementor of ast.py that tries to be node for node compatible, I'm fine with #1 but would really like to have tests that will fail in test_ast.py to alert me! -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailm

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Terry Reedy
On 4/3/2011 10:02 PM, Guido van Rossum wrote: Sure, but do we have any indication that the warnings for dis make a difference? I think there had been a few grumbles about bytecode not being stable. Without that, it is part of the newish effort to specify in the docs what is CPython specific.

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Neil Schemenauer
As a user of the AST, I as well favor just changing the AST and the version. IMHO, it is not intended to be stable between Python releases (similar to bytecode). Neil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/li

Re: [Python-Dev] Policy for making changes to the AST

2011-04-03 Thread Terry Reedy
On 4/2/2011 9:55 PM, Eugene Toder wrote: Documentation for ast module does not warn about possible changes, The current boxed warning at the top of the dis doc is fairly recent. The ast doc should gain something similar. It currently does say: "__version__ which is the decimal Subversion revi

Re: [Python-Dev] Policy for making changes to the AST

2011-04-03 Thread Nick Coghlan
On Mon, Apr 4, 2011 at 12:02 PM, Guido van Rossum wrote: >> Perhaps we should add a warning to the ast module docs similar to the >> one we have for the dis module, and use it to explicitly remind people >> to check ast.__version__ before proceeding with AST manipulation? > > Sure, but do we have

Re: [Python-Dev] Policy for making changes to the AST

2011-04-03 Thread Guido van Rossum
On Sun, Apr 3, 2011 at 6:43 PM, Nick Coghlan wrote: > On Mon, Apr 4, 2011 at 5:11 AM, Guido van Rossum wrote: >> In the mean time, until we hear differently, I'm also in favor of #1 >> (do nothing). I would (perhaps redundantly) say that such changes >> should only go into new major releases (i.e

Re: [Python-Dev] Policy for making changes to the AST

2011-04-03 Thread Nick Coghlan
On Mon, Apr 4, 2011 at 5:11 AM, Guido van Rossum wrote: > In the mean time, until we hear differently, I'm also in favor of #1 > (do nothing). I would (perhaps redundantly) say that such changes > should only go into new major releases (i.e. 3.3 right now), not > backported into bugfix releases (e

Re: [Python-Dev] Policy for making changes to the AST

2011-04-03 Thread Brett Cannon
On Sat, Apr 2, 2011 at 23:55, "Martin v. Löwis" wrote: > > 1. Do nothing. This will break code that currently uses AST, but doesn't > add > > any complexity to cpython. > > I'm in favor of this approach as well. Notice that there is > ast.__version__ precisely so that applications can support mul

Re: [Python-Dev] Policy for making changes to the AST

2011-04-03 Thread Guido van Rossum
On Sun, Apr 3, 2011 at 4:17 AM, Paul Moore wrote: > On 3 April 2011 07:55, "Martin v. Löwis" wrote: >>> 1. Do nothing. This will break code that currently uses AST, but doesn't add >>> any complexity to cpython. >> >> I'm in favor of this approach as well. Notice that there is >> ast.__version__

Re: [Python-Dev] Policy for making changes to the AST

2011-04-03 Thread Paul Moore
On 3 April 2011 07:55, "Martin v. Löwis" wrote: >> 1. Do nothing. This will break code that currently uses AST, but doesn't add >> any complexity to cpython. > > I'm in favor of this approach as well. Notice that there is > ast.__version__ precisely so that applications can support multiple AST >

Re: [Python-Dev] Policy for making changes to the AST

2011-04-02 Thread Martin v. Löwis
> 1. Do nothing. This will break code that currently uses AST, but doesn't add > any complexity to cpython. I'm in favor of this approach as well. Notice that there is ast.__version__ precisely so that applications can support multiple AST versions. Regards, Martin ___

Re: [Python-Dev] Policy for making changes to the AST

2011-04-02 Thread Nick Coghlan
On Sun, Apr 3, 2011 at 3:42 PM, Eugene Toder wrote: >> If it's do-able, your option 2 is probably the way to go. Out of the >> box, it may just need to raise an exception if asked to down-convert >> code that uses new constructs that can't readily be expressed using >> the old AST (I'm specificall

Re: [Python-Dev] Policy for making changes to the AST

2011-04-02 Thread Eugene Toder
> If it's do-able, your option 2 is probably the way to go. Out of the > box, it may just need to raise an exception if asked to down-convert > code that uses new constructs that can't readily be expressed using > the old AST (I'm specifically thinking of the challenge of converting > PEP 380's yie

Re: [Python-Dev] Policy for making changes to the AST

2011-04-02 Thread Nick Coghlan
On Sun, Apr 3, 2011 at 2:24 PM, Eugene Toder wrote: >> However, I'm not sure we *can* do a general-purpose AST transformation >> that handles both new nodes and changes to existing nodes correctly >> for all applications. > > As long as both versions contain the same information we can write a > t

Re: [Python-Dev] Policy for making changes to the AST

2011-04-02 Thread Eugene Toder
> However, I'm not sure we *can* do a general-purpose AST transformation > that handles both new nodes and changes to existing nodes correctly > for all applications. As long as both versions contain the same information we can write a transformation that does a near-perfect job. E.g. for my chang

Re: [Python-Dev] Policy for making changes to the AST

2011-04-02 Thread Nick Coghlan
On Sun, Apr 3, 2011 at 12:07 PM, Benjamin Peterson wrote: >> Assuming we do want to make changes (not just extensions for new language >> features), I see 3 options for handling them: >> >> 1. Do nothing. This will break code that currently uses AST, but doesn't add >> any complexity to cpython. >

Re: [Python-Dev] Policy for making changes to the AST

2011-04-02 Thread Benjamin Peterson
2011/4/2 Eugene Toder : > Hello, > > While working on a rewrite of peephole optimizer that works on AST > (http://bugs.python.org/issue11549) I had to make a few changes to the > structure of AST. The changes are described in the issue. This raises the > question -- what is the policy for making ch

[Python-Dev] Policy for making changes to the AST

2011-04-02 Thread Eugene Toder
Hello, While working on a rewrite of peephole optimizer that works on AST (http://bugs.python.org/issue11549) I had to make a few changes to the structure of AST. The changes are described in the issue. This raises the question -- what is the policy for making changes to the AST? Documentation for