Martin v. Löwis wrote:
> Nick Coghlan wrote:
> > The ast C structs are already auto-generated by a Python script
> (asdl_c.py, to
> > be precise). The trick is to make that script generate full PyObjects
> rather
> > than the simple C structures that it generates now.
>
> See the ast-object b
Sure. If they're immutable sharing is fine, but you end up making a
copy anyway if you want to make changes, right?
Jeremy
On 11/30/05, Greg Ewing <[EMAIL PROTECTED]> wrote:
> Jeremy Hylton wrote:
>
> > I still think passing copies is better than sharing live
> > objects between Python and C,
>
Jeremy Hylton wrote:
> I still think passing copies is better than sharing live
> objects between Python and C,
Even if the objects are immutable?
--
Greg Ewing, Computer Science Dept, +--+
University of Canterbury, | A citizen of NewZealandCorp, a
On 11/30/05, Neal Norwitz <[EMAIL PROTECTED]> wrote:
> On 11/30/05, Thomas Lee <[EMAIL PROTECTED]> wrote:
> >
> > Quick semi-related question: where are the marshal_* functions called?
> > They're all static in Python-ast.c and don't seem to be actually called
> > anywhere. Can we ditch them?
>
> I
Nick Coghlan wrote:
> The ast C structs are already auto-generated by a Python script
(asdl_c.py, to
> be precise). The trick is to make that script generate full PyObjects
rather
> than the simple C structures that it generates now.
See the ast-object branch.
> The second step is to then m
Neal Norwitz <[EMAIL PROTECTED]> wrote:
> If everything is a PyObject, wouldn't [the marshal functions] be
> redundant?
You could be right. Spending time to kept them working is probably
wasted effort.
Neil
___
Python-Dev mailing list
Python-Dev@pyt
On 11/30/05, Neil Schemenauer <[EMAIL PROTECTED]> wrote:
> Thomas Lee <[EMAIL PROTECTED]> wrote:
> > Quick semi-related question: where are the marshal_* functions called?
> > They're all static in Python-ast.c and don't seem to be actually called
> > anywhere. Can we ditch them?
>
> They are inten
Thomas Lee <[EMAIL PROTECTED]> wrote:
> Quick semi-related question: where are the marshal_* functions called?
> They're all static in Python-ast.c and don't seem to be actually called
> anywhere. Can we ditch them?
They are intended to be used to make the AST available to Python
code. It would
On 11/30/05, Thomas Lee <[EMAIL PROTECTED]> wrote:
>
> Quick semi-related question: where are the marshal_* functions called?
> They're all static in Python-ast.c and don't seem to be actually called
> anywhere. Can we ditch them?
I *think* they are not necessary. My guess is that they were there
On Wed, Nov 30, 2005 at 07:42:20PM +1000, Nick Coghlan wrote:
> The second step is to then modify ast.c to use the new structures. A branch
> probably wouldn't help much with initial development (this is a "break the
> world, check in when stuff compiles again" kind of change, which is hard to
>
Nick Coghlan wrote:
>Greg Ewing wrote:
>
>
>>Neal Norwitz wrote:
>>
>>
>>
>>>I'm mostly convinced that using PyObjects would be a good thing.
>>>However, making the change isn't free as all the types need to be
>>>created and this is likely quite a bit of code.
>>>
>>>
>>Since they're
Greg Ewing <[EMAIL PROTECTED]> writes:
> Guido van Rossum wrote:
>
>>>To me, that's an argument in favour of always generating
>>>a .pyc, even for scripts.
>>
>> I'm not sure I follow the connection.
>
> You were saying that if the parser and compiler were
> slow, it would slow down single-file s
Greg Ewing wrote:
> Neal Norwitz wrote:
>
>> I'm mostly convinced that using PyObjects would be a good thing.
>> However, making the change isn't free as all the types need to be
>> created and this is likely quite a bit of code.
>
> Since they're all so similar, perhaps they could be
> auto-gen
Neal Norwitz wrote:
> I'm mostly convinced that using PyObjects would be a good thing.
> However, making the change isn't free as all the types need to be
> created and this is likely quite a bit of code.
Since they're all so similar, perhaps they could be
auto-generated by a fairly simple scrip
Neal Norwitz wrote:
> On 11/28/05, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
>
> > he assumed that FunctionDef would *consume* the references
> > being passed (whether it is successful or not).
>
> I keep resisting this solution, though I'm not sure why.
One reason for not liking it is that i
Guido van Rossum wrote:
>>To me, that's an argument in favour of always generating
>>a .pyc, even for scripts.
>
> I'm not sure I follow the connection.
You were saying that if the parser and compiler were
slow, it would slow down single-file scripts that
didn't have a .pyc (or at least that's w
Nick Coghlan wrote:
> Declaring that none of the AST node creation methods steal references would
> be
> consistent with most of the existing C API (e.g. PySequence_SetItem,
> PySequence_Tuple, PySequence_List),
Agreed, although the rest of your proposal (while
admirably cunning) requires that
Neal Norwitz wrote:
> For those watching, Greg's and Martin's version were almost the same.
> However, Greg's version left in the memory leak, while Martin fixed it
> by letting the result fall through.
I addressed the memory leak by stipulating that FunctionDef
should steal references to its ar
On 11/29/05, Neal Norwitz <[EMAIL PROTECTED]> wrote:
> On 11/29/05, Nick Coghlan <[EMAIL PROTECTED]> wrote:
> >
> > When working on the CST->AST parser, there were only a few things I found to
> > be seriously painful about the memory management:
> >
> >1. Remembering which free_* variant to ca
On 11/29/05, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> On 11/28/05, Greg Ewing <[EMAIL PROTECTED]> wrote:
> > [...] My mental model
> > of parsing & compiling in the presence of a parse tree
> > is like this:
> >
> >[source] -> scanner -> [tokens]
> > -> parser -> [AST] -> code_generat
On 11/28/05, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> Neal Norwitz wrote:
> > For those watching, Greg's and Martin's version were almost the same.
> > However, Greg's version left in the memory leak, while Martin fixed it
> > by letting the result fall through.
>
> Actually, Greg said (corre
On 11/29/05, Nick Coghlan <[EMAIL PROTECTED]> wrote:
>
> When working on the CST->AST parser, there were only a few things I found to
> be seriously painful about the memory management:
>
>1. Remembering which free_* variant to call for AST nodes
>2. Remembering which asdl_seq_*_free varian
On 11/28/05, Greg Ewing <[EMAIL PROTECTED]> wrote:
> [...] My mental model
> of parsing & compiling in the presence of a parse tree
> is like this:
>
>[source] -> scanner -> [tokens]
> -> parser -> [AST] -> code_generator -> [code]
>
> The fact that there still seems to be another kind of
Neal Norwitz wrote:
> On 11/28/05, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
>> Neal Norwitz wrote:
>>> Hope this helps explain a bit. Please speak up with how this can be
>>> improved. Gotta run.
>> I would rewrite it as
>
> [code snipped]
>
> For those watching, Greg's and Martin's version
Jeremy Hylton wrote:
> > Me neither. Adding yet another memory allocation scheme to Python's
> > already staggering number of memory allocation strategies sounds like
> > a bad idea.
>
> The reason this thread started was the complaint that reference
> counting in the compiler is really difficult.
Neal Norwitz wrote:
> For those watching, Greg's and Martin's version were almost the same.
> However, Greg's version left in the memory leak, while Martin fixed it
> by letting the result fall through.
Actually, Greg said (correctly) that his version also fixes the
leak: he assumed that Function
On 11/28/05, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> Neal Norwitz wrote:
> > Hope this helps explain a bit. Please speak up with how this can be
> > improved. Gotta run.
>
> I would rewrite it as
[code snipped]
For those watching, Greg's and Martin's version were almost the same.
Howeve
Brett Cannon wrote:
> Is there a specific reason you are leaving out the AST, Greg, or do
> you count that as part of the bytecode compiler
No, I consider it part of the parser. My mental model
of parsing & compiling in the presence of a parse tree
is like this:
[source] -> scanner -> [tokens
On 11/28/05, Guido van Rossum <[EMAIL PROTECTED]> wrote:
>
> I guess I don't understand the AST compiler code enough to participate
> in this discussion.
I hope everyone while chime in here. This is important to improve and
learn from others.
Let me try to describe the current situation with a s
On 11/28/05, Greg Ewing <[EMAIL PROTECTED]> wrote:
> Here's a somewhat radical idea:
>
> Why not write the parser and bytecode compiler in Python?
>
> A .pyc could be bootstrapped from it and frozen into
> the executable.
>
Is there a specific reason you are leaving out the AST, Greg, or do
you co
Neal Norwitz wrote:
> Hope this helps explain a bit. Please speak up with how this can be
> improved. Gotta run.
I would rewrite it as
static PyObject*
ast_for_funcdef(struct compiling *c, const node *n)
{
/* funcdef: [decorators] 'def' NAME parameters ':' suite */
PyObject *name = NU
Here's a somewhat radical idea:
Why not write the parser and bytecode compiler in Python?
A .pyc could be bootstrapped from it and frozen into
the executable.
--
Greg Ewing, Computer Science Dept, +--+
University of Canterbury, | A citizen of NewZeal
Neal Norwitz wrote:
> This is an entire function from Python/ast.c.
> Sequences do not know what type they hold, so there needs to be
> different dealloc functions to free them properly (asdl_*_seq_free()).
Well, that's one complication that would go away if
the nodes were PyObjects.
> The memo
Jeremy Hylton wrote:
> Almost every line of
> code can lead to an error exit. The code becomes quite cluttered when
> it uses reference counting.
I don't see why very many more error exits should become
possible just by introducing refcounting. Errors are possible
whenever you allocate something
On 11/28/05, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> [Guido]
> > > Then I don't understand why there was discussion of alloca() earlier
> > > on -- surely the lifetime of a node should not be limited by the stack
> > > frame that allocated it?
>
> [Jeremy]
> > Actually this is a pretty good l
[Guido]
> > Then I don't understand why there was discussion of alloca() earlier
> > on -- surely the lifetime of a node should not be limited by the stack
> > frame that allocated it?
[Jeremy]
> Actually this is a pretty good limit, because all these data
> structures are temporaries used by the
On Mon, Nov 28, 2005 at 03:47:07PM -0500, Jeremy Hylton wrote:
> The reason this thread started was the complaint that reference
> counting in the compiler is really difficult.
I don't think that's exactly right. The problem is that the AST
compiler mixes its own memory management strategy with r
Jeremy Hylton wrote:
> The reason this thread started was the complaint that reference
> counting in the compiler is really difficult. Almost every line of
> code can lead to an error exit. The code becomes quite cluttered when
> it uses reference counting. Right now, the AST is created with
On 11/28/05, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> > The code becomes quite cluttered when
> > it uses reference counting. Right now, the AST is created with
> > malloc/free, but that makes it hard to free the ast at the right time.
>
> Would fixing the code to add free() calls in all the
On 11/28/05, Jeremy Hylton <[EMAIL PROTECTED]> wrote:
> On 11/28/05, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> > On 11/18/05, Neil Schemenauer <[EMAIL PROTECTED]> wrote:
> > > Perhaps we should use the memory management technique that the rest
> > > of Python uses: reference counting. I don't
On 11/28/05, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> On 11/18/05, Neil Schemenauer <[EMAIL PROTECTED]> wrote:
> > Perhaps we should use the memory management technique that the rest
> > of Python uses: reference counting. I don't see why the AST
> > structures couldn't be PyObjects.
>
> Me n
On 11/18/05, Neil Schemenauer <[EMAIL PROTECTED]> wrote:
> Perhaps we should use the memory management technique that the rest
> of Python uses: reference counting. I don't see why the AST
> structures couldn't be PyObjects.
Me neither. Adding yet another memory allocation scheme to Python's
alre
Neil Schemenauer wrote:
>Fredrik Lundh <[EMAIL PROTECTED]> wrote:
>
>
>>Thomas Lee wrote:
>>
>>
>>
>>>Even if it meant we had just one function call - one, safe function call
>>>that deallocated all the memory allocated within a function - that we
>>>had to put before each and every return,
Fredrik Lundh <[EMAIL PROTECTED]> wrote:
> Thomas Lee wrote:
>
>> Even if it meant we had just one function call - one, safe function call
>> that deallocated all the memory allocated within a function - that we
>> had to put before each and every return, that's better than what we
>> have.
>
> all
The behavior of libiberty's alloca() replacement might be interesting as well:
http://gcc.gnu.org/onlinedocs/libiberty/Functions.html#index-alloca-59
Regards,
Michael
On 11/18/05, Alex Martelli <[EMAIL PROTECTED]> wrote:
>
> On Nov 17, 2005, at 5:00 PM, Thomas Lee wrote:
>
> > Portability may al
On Nov 17, 2005, at 5:00 PM, Thomas Lee wrote:
> Portability may also be an issue to take into consideration:
Of course -- but so is anno domini... the eskimo.com FAQ is (C) 1995,
and the neohapsis.com page just points to the eskimo.com one:
> http://www.eskimo.com/~scs/C-faq/q7.32.html
> htt
Portability may also be an issue to take into consideration:
http://www.eskimo.com/~scs/C-faq/q7.32.html
http://archives.neohapsis.com/archives/postfix/2001-05/1305.html
Cheers,
Tom
Alex Martelli wrote:
>On Nov 17, 2005, at 12:46 PM, Brett Cannon wrote:
>...
>
>
>>>alloca?
>>>
>>>(duck)
>
On Nov 17, 2005, at 12:46 PM, Brett Cannon wrote:
...
>> alloca?
>>
>> (duck)
>>
>
> But how widespread is its support (e.g., does Windows have it)?
Yep, spelled with a leading underscore:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/
vclib/html/_crt__alloca.asp
Alex
_
On 11/16/05, Thomas Lee <[EMAIL PROTECTED]> wrote:
> Just messing around with some ideas. I was trying to avoid the ugly
> macros (note my earlier whinge about a learning curve) but they're the
> cleanest way I could think of to get around the problem without
> resorting to a mass deallocation righ
On 11/16/05, Fredrik Lundh <[EMAIL PROTECTED]> wrote:
> Thomas Lee wrote:
>
> > Even if it meant we had just one function call - one, safe function call
> > that deallocated all the memory allocated within a function - that we
> > had to put before each and every return, that's better than what we
Just messing around with some ideas. I was trying to avoid the ugly
macros (note my earlier whinge about a learning curve) but they're the
cleanest way I could think of to get around the problem without
resorting to a mass deallocation right at the end of the AST run. Which
may not be all that
Thomas Lee wrote:
> As the writer of the crappy code that sparked this conversation, I feel
> I should say something :)
Don't feel bad about it. It turned out the 'helpful' review comments from Neal
and I didn't originally work out very well either ;)
With the AST compiler being so new, this is
On 11/16/05, Niko Matsakis <[EMAIL PROTECTED]> wrote:
> - Another idea is to have trees of arenas: the idea is that when an
> arena is created, it is assigned a parent. When an arena is freed,
> an arenas in its subtree are also freed. This way you can have one
> master arena for exception handli
Thomas Lee wrote:
> Even if it meant we had just one function call - one, safe function call
> that deallocated all the memory allocated within a function - that we
> had to put before each and every return, that's better than what we
> have.
alloca?
(duck)
By the way, I liked the sound of the arena/pool tree - really good idea.
Thomas Lee wrote:
>Niko Matsakis wrote:
>
>
>
>>>Boy am I wanting RAII from C++ for automatic freeing when scope is
>>>left. Maybe we need to come up with a similar thing, like all memory
>>>that should be freed once a sc
Niko Matsakis wrote:
>>Boy am I wanting RAII from C++ for automatic freeing when scope is
>>left. Maybe we need to come up with a similar thing, like all memory
>>that should be freed once a scope is left must use some special struct
>>that stores references to all created memory locally and then
> Boy am I wanting RAII from C++ for automatic freeing when scope is
> left. Maybe we need to come up with a similar thing, like all memory
> that should be freed once a scope is left must use some special struct
> that stores references to all created memory locally and then a free
> call must be
Nick Coghlan wrote:
> Marek Baczek Baczyński wrote:
>> 2005/11/15, Nick Coghlan <[EMAIL PROTECTED]>:
>>> It avoids the potential for labelling problems that arises when goto's are
>>> used for resource cleanup. It's a far cry from real exception handling, but
>>> it's the best solution I've seen wi
As the writer of the crappy code that sparked this conversation, I feel
I should say something :)
Brett Cannon wrote:
>On 11/15/05, Neal Norwitz <[EMAIL PROTECTED]> wrote:
>
>
>>On 11/15/05, Jeremy Hylton <[EMAIL PROTECTED]> wrote:
>>
>>
>>>Thanks for the message. I was going to suggest t
On 11/15/05, Neal Norwitz <[EMAIL PROTECTED]> wrote:
> On 11/15/05, Jeremy Hylton <[EMAIL PROTECTED]> wrote:
> >
> > Thanks for the message. I was going to suggest the same thing. I
> > think it's primarily a question of how to add an arena layer. The AST
> > phase has a mixture of malloc/free a
On 11/15/05, Jeremy Hylton <[EMAIL PROTECTED]> wrote:
>
> Thanks for the message. I was going to suggest the same thing. I
> think it's primarily a question of how to add an arena layer. The AST
> phase has a mixture of malloc/free and Python object allocation. It
> should be straightforward to
On 11/15/05, Jeremy Hylton <[EMAIL PROTECTED]> wrote:
> On 11/15/05, Niko Matsakis <[EMAIL PROTECTED]> wrote:
> > > As Neal pointed out, it's tricky to write code for the AST parser
> > > and compiler
> > > without accidentally letting memory leak when the parser or
> > > compiler runs into
> > > a
On 11/15/05, Niko Matsakis <[EMAIL PROTECTED]> wrote:
> > As Neal pointed out, it's tricky to write code for the AST parser
> > and compiler
> > without accidentally letting memory leak when the parser or
> > compiler runs into
> > a problem and has to bail out on whatever it was doing. Thomas's
>
> As Neal pointed out, it's tricky to write code for the AST parser
> and compiler
> without accidentally letting memory leak when the parser or
> compiler runs into
> a problem and has to bail out on whatever it was doing. Thomas's
> patch got to
> v5 (based on Neal's review comments) with m
Marek Baczek Baczyński wrote:
> 2005/11/15, Nick Coghlan <[EMAIL PROTECTED]>:
>> It avoids the potential for labelling problems that arises when goto's are
>> used for resource cleanup. It's a far cry from real exception handling, but
>> it's the best solution I've seen within the limits of C.
>
>
2005/11/15, Nick Coghlan <[EMAIL PROTECTED]>:
> Specifically, the body of the entire function is written inside a switch
> statement, with 'break' then used as the equivalent of "raise Exception". For
> example:
>
>PyObject* switchAsTry()
>{
> switch(0) {
>default:
> /
66 matches
Mail list logo
