Re: [Python-Dev] Official citation for Python

[Jeremy Hylton]

I wanted to start with an easy answer that is surely unsatisfying:
http://blog.apastyle.org/apastyle/2015/01/how-to-cite-software-in-apa-style.html

APA style is pretty popular, and it says that standard software doesn't
need to be specified. Standard software includes "Microsoft Word, Java, and
Adobe Photoshop." So I'd say Python fits well in that category, and doesn't
need to be cited.

I said you wouldn't be satisfied...

On Sat, Sep 15, 2018 at 11:02 AM Jacqueline Kazil 
wrote:

> I just got caught up on the thread. This is a really great discussion.
> Thank you for all the contributions.
>
> Before we get into the details, let's go back to the main use case we are
> trying to solve.
> *As a user, I am writing an academic paper and I need to cite Python. *
>

The goal here is ambiguous. Python means many things--a language described
by the language specification, the source code of a particular
implementation of the language (Python often refers to C Python), a
particular binary release of the implementation of the language (Python
1.5.2 for Windows). Which one is relevant in the context of the paper? If
you're talking about a bug in timsort in a particular version of C Python,
then you probably want to cite that specific version of the implementation.

I suspect the most common goal for a citation is just to describe the
language "in general" where 1.5.2 or 3.7.0 and Jython or CPython are all
details that don't matter. In that case I'd cite the language
specification. We're talking about putting a citation in a paper (a written
source) and the written language specification captures what we think of as
essential for the language. If you want to cite Turing's proof of the
undecidability of the halting problem, you'd cite the paper where he wrote
it down (in Proceedings of the London Mathematical Society). If you want to
cite a programming language in the abstract, cite the specification that
describes it.

I think style guides are relevant here. They give guidance on how to cite
an item based on its category. For example, the MLA style guide describes
how to cite a digital file, a physical object, and many other things. My
favorite example under "physical object" is "Physical objects found
online." (Think about it :-).

There's some discussion of how to cite source code here:
http://integrity.mit.edu/handbook/writing-code. Notably this is talking
about citing source code in the context of other source code, and it mostly
recommends using URLs. If you wanted to cite a particular piece of source
code in an written article, you'd probably follow one of the approaches for
citing online resources. Try to identify who / when / what / where. For
example MLA style for a blog post would be : Editor, screen name, author,
or compiler name (if available). “Posting Title.” Name of Site, Version
number (if available), Name of institution/organization affiliated with the
site (sponsor or publisher), URL. Date of access. You could cite a
particular source file this way or a particular source release.

The date usually refers to the original publication date. I think that was
with the 1.0 release, although I'm not sure. I'd probably pick that date,
but someone can correct me if there's an earlier date. It would suggest
somehow that current Python and the original Python were mostly the same
thing, which is an idea I like.

van Rossum, Guido (1994). "The Python Language Reference". Python Software
Foundation, https://docs.python.org/reference/index.html. Retrieved 16
September 2018.

I'd say that's all settled. If anyone asks you, "How can you be sure that
settles it?" You can answer, "Some guy said it on a mailing list." And then
you can site the message:

Jeremy Hylton. "[Python-Dev] Official citation for Python." Sep. 17, 2018.
python-dev, https://mail.python.org/mailman/listinfo/python-dev. Accessed
18 September 2018.

Jeremy


> Let's throw reproducibility out the window for now (<--- something I never
> thought I would say), because that should be captured in the code, not in
> the citations.
>
> So, if we don't need the specific version of Python, then maybe creating
> one citation is all we need.
> And that gives it some good Google juice as well.
>
> Thoughts?
>
> (Once we nail down one or many, I think we can then move into the details
> of the content of the citation.)
>
> -Jackie
>
> On Thu, Sep 13, 2018 at 12:47 AM Wes Turner  wrote:
>
>> There was a thread about adding __cite__ to things and a tool to collect
>> those citations awhile back.
>>
>> "[Python-ideas] Add a __cite__ method for scientific packages"
>> http://markmail.org/thread/rekmbmh64qxwcind
>>
>> Which CPython source file should contain thi

Re: [Python-Dev] Does anyone use CALL_PROFILE special build?

[Jeremy Hylton]

I don't think there's any need to keep this feature. I hope we have better
ways to profile function and method calls now than we did when this code
was submitted.

Jeremy

On Sat, Nov 26, 2016 at 3:34 AM Raymond Hettinger <
raymond.hettin...@gmail.com> wrote:

>
> > On Nov 25, 2016, at 1:28 AM, Victor Stinner 
> wrote:
> >
> > I would like to completely remove the feature to be able to implement
> > further optimizations.
>
> This seems reasonable to me.  I've never used or needed this special
> build; StackOverflow has no mention of it; and a Google search comes up
> nearly empty.   That said, it might be worthwhile to check with Jeremy to
> get his thoughts before removing his code.
>
>
> Raymond
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue5578 - explanation

[Jeremy Hylton]

I posted in the bug report, but repeating here:  I don't remember why
exec in a nested function changed either.  It would help if someone
could summarize why we made the change.  (I hope I didn't do it <0.2
wink>.)

Jeremy

On Tue, Mar 31, 2009 at 11:36 PM, Maciej Fijalkowski  wrote:
> Because classes have now it's own local scope (according to Martin)
>
> It's not about exec in class, it's about exec in class in nested function.
>
> On Wed, Apr 1, 2009 at 5:25 AM, Guido van Rossum  wrote:
>> Well hold on for a minute, I remember we used to have an exec
>> statement in a class body in the standard library, to define some file
>> methods in socket.py IIRC.  It's a totally different case than exec in
>> a nested function, and I don't believe it should be turned into a
>> syntax error at all. An exec in a class body is probably meant to
>> define some methods or other class attributes. I actually think the
>> 2.5 behavior is correct, and I don't know why it changed in 2.6.
>>
>> --Guido
>>
>> On Tue, Mar 31, 2009 at 8:15 PM, Maciej Fijalkowski  wrote:
>>> So. The issue was closed and I suppose it was closed by not entirely
>>> understanding
>>> the problem (or I didn't get it completely).
>>>
>>> The question is - what the following code should do?
>>>
>>> def f():
>>>  a = 2
>>>  class C:
>>>    exec 'a = 42'
>>>    abc = a
>>>  return C
>>>
>>> print f().abc
>>>
>>> (quick answer - on python2.5 it return 42, on python 2.6 and up it
>>> returns 2, the patch changes
>>> it to syntax error).
>>>
>>> I would say that returning 2 is the less obvious thing to do. The
>>> reason why IMO this should
>>> be a syntax error is this code:
>>>
>>> def f():
>>>  a = 2
>>>  def g():
>>>    exec 'a = 42'
>>>    abc = a
>>>
>>> which throws syntax error.
>>>
>>> Cheers,
>>> fijal
>>> ___
>>> Python-Dev mailing list
>>> Python-Dev@python.org
>>> http://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe: 
>>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>
>>
>>
>>
>> --
>> --Guido van Rossum (home page: http://www.python.org/~guido/)
>>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue5578 - explanation

[Jeremy Hylton]

Eeek, I think it was me.  Part of the AST changes involved raising a
SyntaxError when exec was used in a scope that had a free variable,
since the behavior is pretty much undefined.  If the compiler decides
a variable is free, then it can't be assigned to in the function body.
 The compiled exec code can't know whether the variable is local or
free in the exec context, only that it should generate a STORE_NAME
opcode.  The STORE_NAME can't possibly work.

It looks like I did a bad job of documenting the change, though.  I
had forgotton about it ,because it was three or four years ago.

It looks like the same exception should be raised for the class statement.

Jeremy



On Wed, Apr 1, 2009 at 11:00 AM, R. David Murray  wrote:
> On Wed, 1 Apr 2009 at 13:12, Chris Withers wrote:
>>
>> Guido van Rossum wrote:
>>>
>>>  Well hold on for a minute, I remember we used to have an exec
>>>  statement in a class body in the standard library, to define some file
>>>  methods in socket.py IIRC.
>>
>> But why an exec?! Surely there must be some other way to do this than an
>> exec?
>
> Maybe, but this sure is gnarly code:
>
>    _s = ("def %s(self, *args): return self._sock.%s(*args)\n\n"
>          "%s.__doc__ = _realsocket.%s.__doc__\n")
>    for _m in _socketmethods:
>        exec _s % (_m, _m, _m, _m)
>    del _m, _s
>
> Guido's memory is good, that's from the _socketobject class in
> socket.py.
>
> --David
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] question about docstring formatting

[Jeremy Hylton]

A question came up at work about docstring formatting.  It relates to
the description of the summary line in PEP 257.

http://www.python.org/dev/peps/pep-0257/
"""Multi-line docstrings consist of a summary line just like a
one-line docstring, followed by a blank line, followed by a more
elaborate description. The summary line may be used by automatic
indexing tools; it is important that it fits on one line and is
separated from the rest of the docstring by a blank line. The summary
line may be on the same line as the opening quotes or on the next
line. The entire docstring is indented the same as the quotes at its
first line (see example below)."""

It says that the summary line may be used by automatic indexing tools,
but is there any evidence that such a tool actually exists?  Or was
there once upon a time?  If there are no such tools, do we still think
that it is important that it fits on line line?

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Modifying Grammar/grammar and other foul acts

[Jeremy Hylton]

On Sat, Mar 6, 2010 at 11:27 AM, Gregg Lind  wrote:
> Python-devs,
>
> I'm writing to you for some help in understanding the Python grammar.  As an
> excuse to deep dive into Python's tokenizer / grammar, I decided (as a
> hideous, hideous joke) to want to allow braces where colons are allowed (as
> flow control).
>
> Starting from PEP 306 (and branch r311), I hacked on Grammar/Grammer
>
> As a first example:
>
> funcdef: ('def' NAME parameters ['->' test] ':' suite |
>   'def' NAME parameters ['->' test] '{' suite '}' )
>
> I reran Parser/pgen and the dfa changes, but python (3.1) when recompiled,
> throws errors on things like:
>
> def a() { None }
>
> Strangely enough:
>
> lambdef: ( 'lambda' [varargslist] ':' test  |
>    'lambda' [varargslist] '{' test '}' )
>
> works fine!  I this simplely some difference between "test" and "suite".
>
> I have tried tackling this with gdb, looking at err_input clearly isn't
> enough.
>
>     (gdb) break err_input
>     (gdb) break PyParser_ASTFromString
>     import sys
>     b = compile("def a() {pass}","sys.stdout","single")
>     # yet a simple grammar fix is enough for this!
>     c = compile("lambda x {None}","sys.stdout","single")
>
> I'm in over my head!

You don't say what errors occur when you try to compile strings in
your new language.  You may have changed the Grammar, which allows you
to tokenize the input.  That isn't enough to get the input to compile.
 You also need to change the compiler to understand the new tokens.

Jeremy

> Any insights / help would be appreciated.  Full-on flaming is also
> appropriate, but would be less appreciated.
>
> Specific questions
>
> 1.)  I assume the Grammar/grammar is read top to bottom.  Confirm?
> 2.)  More help figuring out how to debug what python *thinks* it's seeing
> when it see "def a() {pass}".  It's not getting to the ast construction
> stage, as near as I can tell.  What additional breakpoints can I set to see
> where it's failing.
>
> Gregg L.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Explicit Lexical Scoping (pre-PEP?)

[Jeremy Hylton]

On 7/10/06, Ka-Ping Yee <[EMAIL PROTECTED]> wrote:
> On Mon, 10 Jul 2006 [EMAIL PROTECTED] wrote:
> > I think Talin's got a point though.  It seems hard to find one short English
> > word that captures the essence of the desired behavior.  None of the words
> > in his list seem strongly suggestive of the meaning to me.  I suspect that
> > means one's ultimately as good (or as bad) as the rest.
>
> What's wrong with "nonlocal"?  I don't think i've seen an argument
> against that one so far (from Talin or others).

It's a made-up word.  You won't find it in the dictionary and the
google define: query sends me to a wikipedia page about quantum
mechanics.  It also expresses itself in the negative form "not local"
as opposed to the positive form like global "this is a global."
Finally, I think it sounds yucky.

To express this email in the positive form:
1. Reserved words should be real words.
2. The meaning of the word should be clear.
3. "Put statements in positive form."  (Strunk & White)
4. The word should sound good.

global meets all of these requirements.  "free" was the word I
remember preferring from earlier discussions, but I think it fails #2.
 (Too much confusion about freeing memory, for example.)

Jeremy

>
>
> -- ?!ng
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Explicit Lexical Scoping (pre-PEP?)

[Jeremy Hylton]

On 7/10/06, Ka-Ping Yee <[EMAIL PROTECTED]> wrote:
> On 7/10/06, Ka-Ping Yee <[EMAIL PROTECTED]> wrote:
> > What's wrong with "nonlocal"?  I don't think i've seen an argument
> > against that one so far (from Talin or others).
>
> On Mon, 10 Jul 2006, Jeremy Hylton wrote:
> > It's a made-up word.  You won't find it in the dictionary and the
> > google define: query sends me to a wikipedia page about quantum
> > mechanics.
>
> Two million Google hits for "nonlocal" seems like plenty.
> The explanation of "nonlocal" is pretty straightforward --
>
> If the definition of f() contains an assignment to x, then
> x is a local variable in f, unless x is declared nonlocal.
>
> > To express this email in the positive form:
> > 1. Reserved words should be real words.
> > 2. The meaning of the word should be clear.
> > 3. "Put statements in positive form."  (Strunk & White)
> > 4. The word should sound good.
> >
> > global meets all of these requirements.
>
> But it's the wrong word.  "Purple" also meets all of these requirements.

No.  Don't be silly.

The current use of global meets the requirements.  The meaning there
is clear, because global means global namespace.  If purple were a
keyword, I wouldn't know what it means.  I was not proposing the use
of global for some other meaning (and thought I made that clear in the
remainder of the message).

Jeremy

>
> I'd rather accurately express the concept in the negative (the
> true meaning really is in the negative: "don't make a new binding"),
> than choose a simple-sounding word that is essentially a lie.
>
> x = 1
>
> def f(x):
> print x
>
> def g():
> global x
> x = 3
> print x
>
> The "x" used in g is not global at all -- it belongs to f.
>
>
> -- ?!ng
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Explicit Lexical Scoping (pre-PEP?)

[Jeremy Hylton]

On 7/12/06, Fredrik Lundh <[EMAIL PROTECTED]> wrote:
> Boris Borcic wrote:
>
> >> note that most examples of this type already work, if the target type is
> >> mutable, and implement the right operations:
> >>
> >>   def counter(num):
> >>   num = mutable_int(num)
> >>   def inc():
> >>   num += 1
> >>   return num
> >>   return inc
> >
> > I agree with you (and argued it in "scopes vs augmented assignment vs sets"
> > recently) that mutating would be sufficient /if/ the compiler would view
> > augmented assignment as mutations operators
>
> feel free to replace that += with an .add(1) method call; the point
> wasn't the behaviour of augmented assigment, the point was that that the
> most common use pattern involves *mutation* of the target object.
>
> the syntax isn't that important, really.

Mutation is different from rebinding.  A tuple is immutable, but you
can rebind the variable that refers to the tuple.  I think we will
confuse users if we use the term mutation to refer to name binding.
Name binding is already a subtle issue, so I think the risk is
significant.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Community buildbots (was Re: User's complaints)

[Jeremy Hylton]

On 7/14/06, Anthony Baxter <[EMAIL PROTECTED]> wrote:
> On Friday 14 July 2006 16:39, Neal Norwitz wrote:
> > Remember I also tried to push for more features to go in early?
> > That would have given more time for external testing.  Still
> > features are coming in.  Python developers weren't happy about
> > having to get things in earlier.  I don't see a practical way to
> > implement what you propose (see Anthony's comments).
>
> Following up on this point:
> Given the number of "just-this-one-more-thing-please" we've _already_
> had since the b1 feature freeze, do you really except that 90 days of
> feature freeze is feasible? And if there's not going to be a feature
> freeze, there's hardly any point to people doing testing until there
> _is_ a feature freeze, is there? Oh, great, my code works with 2.5b1.
> Oops. 2.5b9 added a new feature that broke my code, but I didn't test
> with that.

Maybe the basic question is right, but the emphasis needs to be
changed.  If we had a rule that said the final release was 90 days
after the last submission that wasn't to fix a regression, we'd ask
"Is this feature important enough to warrant delaying the release
until three months from now?"  I'm not sure what I think, but it
doesn't seem like an implausible policy.

Jeremy

>
> Anthony
> --
> Anthony Baxter <[EMAIL PROTECTED]>
> It's never too late to have a happy childhood.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Community buildbots (was Re: User's complaints)

[Jeremy Hylton]

On 7/14/06, Nick Coghlan <[EMAIL PROTECTED]> wrote:
> Anthony Baxter wrote:
> > On Friday 14 July 2006 16:39, Neal Norwitz wrote:
> >> Remember I also tried to push for more features to go in early?
> >> That would have given more time for external testing.  Still
> >> features are coming in.  Python developers weren't happy about
> >> having to get things in earlier.  I don't see a practical way to
> >> implement what you propose (see Anthony's comments).
> >
> > Following up on this point:
> > Given the number of "just-this-one-more-thing-please" we've _already_
> > had since the b1 feature freeze, do you really except that 90 days of
> > feature freeze is feasible? And if there's not going to be a feature
> > freeze, there's hardly any point to people doing testing until there
> > _is_ a feature freeze, is there? Oh, great, my code works with 2.5b1.
> > Oops. 2.5b9 added a new feature that broke my code, but I didn't test
> > with that.
>
> I think this is where release candidates can come into their own - the beta's
> can flush out the "just-one-more-thing" pleases, the maintenance branch is
> forked for rc1, and then any changes on the branch are purely to fix 
> regressions.
>
> As far as the idea of a suite of buildbots running the unit tests of large
> Python applications against the current SVN trunk goes, one problem is that
> failures in those buildbots will come from two sources:
>- Python itself fails (broken checkin)
>- the application unit tests fail (backwards incompatibility)
>
> To weed out the false alarms, the slaves will need to be set up to run the
> Python unit tests first and then the application unit tests only if the Python
> test run is successful.
>
> When the slave suffers a real failure due to a backwards incompatibility, it
> will take a developer of the application to figure out what it was that broke
> the application's tests.
>
> So while I think it's a great idea, I also think it will need significant
> support from the application developers in debugging any buildbot failures to
> really make it work.

These buildbots should run the tests from stable, released versions of
external packages, assuming that those packages don't ship releases
with failing tests.  If you ran the test suite for a Zope release and
had a test failure, I think there would be a reasonable expectation
that it was a Python bug.

I'd definitely avoiding mixing a development version of Python and a
development version of the test software in the automated build
environment.

Jeremy

>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
> ---
>  http://www.boredomandlaziness.org
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python sprint in NY and CA, Aug. 21-24

[Jeremy Hylton]

I'd like to repeat my invitation to spend a week at Google in
California or New York for a Python sprint.  We are hosting sprints at
our offices in Mountain View and New York City the week of Aug. 21,
Monday through Thursday.

We're planning to work broadly on Python 2.6 and Python 3000.  If
you're interested in the core implementation, the standard library,
the build process, or documentation, they're all in scope.  Working on
a test process using large community projects like Twisted and Zope
would be a great activity, too.

There's a wiki page with some more details:
 http://wiki.python.org/moin/GoogleSprint

Feel free to follow up with Neal or me if you have questions.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] AST structure and maintenance branches

[Jeremy Hylton]

On 9/23/06, Anthony Baxter <[EMAIL PROTECTED]> wrote:
> I'd like to propose that the AST format returned by passing PyCF_ONLY_AST to
> compile() get the same guarantee in maintenance branches as the bytecode
> format - that is, unless it's absolutely necessary, we'll keep it the same.
> Otherwise anyone trying to write tools to manipulate the AST is in for a
> massive world of hurt.
>
> Anyone have any problems with this, or can it be added to PEP 6?

It's possible we should poll developers of other Python
implementations and find out if anyone has objections to supporting
this AST format.  But in principle, it sounds like a good idea to me.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] AST structure and maintenance branches

[Jeremy Hylton]

On 9/28/06, Anthony Baxter <[EMAIL PROTECTED]> wrote:
> On Friday 29 September 2006 00:30, Jeremy Hylton wrote:
> > On 9/23/06, Anthony Baxter <[EMAIL PROTECTED]> wrote:
> > > I'd like to propose that the AST format returned by passing PyCF_ONLY_AST
> > > to compile() get the same guarantee in maintenance branches as the
> > > bytecode format - that is, unless it's absolutely necessary, we'll keep
> > > it the same. Otherwise anyone trying to write tools to manipulate the AST
> > > is in for a massive world of hurt.
> > >
> > > Anyone have any problems with this, or can it be added to PEP 6?
> >
> > It's possible we should poll developers of other Python
> > implementations and find out if anyone has objections to supporting
> > this AST format.  But in principle, it sounds like a good idea to me.
>
> I think it's extremely likely that the AST format will change over time -
> with major releases. I'd just like to guarantee that we won't mess with it
> other than that.

Good point.  I'm fine with the change, then.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 315 - do while

[Jeremy Hylton]

On 10/4/06, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> You are all wasting your time on this. It won't go in.

+1 from me.  Should you mark PEP 315 as rejected?

Jeremy

>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] classes and cell variables question

[Jeremy Hylton]

On 12/19/06, tomer filiba <[EMAIL PROTECTED]> wrote:
> to my understanding of the object model, the code of snippet 1
> and snippet 2 should be equivalent. a class is just a "special function"
> that returns its locals automatically and passes them to the metaclass
> constructor:
>
> --- snippet 1 ---
> class foo(object):
> x = 5
> def f(self):
> print "f", x
>
> --- snippet 2 ---
> def bar():
> x = 5
> def g(self):
> print "g", x
> return locals()
> barcls = type("barcls", (object,), bar())
>
> but there's one big difference. classes don't create cell variables
> to hold bound external variables. the "bar" version works, because
> "x" is a bound cell variable, but the "foo" version fails, as it attempts
> to access "x" as a global.

Others have explained the rationale for this design choice, and the
PEP is a good source for what we were thinking at the time.

I thought it was worth adding that a) class variables can never be
access as free variables, as your example foo() shows, and b) you
could access a variable bound in a function scope in a method.
Example of b):

def spam():
  x = 5  # provides the binding for the free variable x in methods of eggs
  class eggs(object):
x = 6  # irrelevant for purposes of resolving the free variable in
the method spam
def spam(self):  # can't have too much spam
  return x
  return eggs

spam()().spam() should return 5.

Jeremy

>
> .>>> barcls().g()
> g 5
>
> .>>> foo().f()
> f
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "", line 4, in f
> NameError: global name 'x' is not defined
>
> for reference, i attached the code of all four functions below.
>
> my question is, how come classes don't create cell variables, like
> normal functions? was this done on purpose? does it have to
> do with inheritance? if so, what's wrong with my "bar" version?
>
>
> [1]
> # code of class foo
> 
> #  2   0 LOAD_NAME0 (__name__)
> #  3 STORE_NAME   1 (__module__)
> #  3   6 LOAD_CONST   0 (5)
> #  9 STORE_NAME   2 (x)
> #
> #  4  12 LOAD_CONST   1 ( 009E5608, file "", line 4>)
> # 15 MAKE_FUNCTION0
> # 18 STORE_NAME   3 (f)
> # 21 LOAD_LOCALS
> # 22 RETURN_VALUE
>
> [2]
> # code of foo.f:
> 
> #  5   0 LOAD_CONST   1 ('f')
> #  3 PRINT_ITEM
> #  4 LOAD_GLOBAL  0 (x)
> #  7 PRINT_ITEM
> #  8 PRINT_NEWLINE
> #  9 LOAD_CONST   0 (None)
> # 12 RETURN_VALUE
>
> [3]
> # code of bar:
> 
> #  2   0 LOAD_CONST   1 (5)
> #  3 STORE_DEREF  0 (x)
> #
> #  3   6 LOAD_CLOSURE 0 (x)
> #  9 BUILD_TUPLE  1
> # 12 LOAD_CONST   2 ( 009F6698, file "", line 3>)
> # 15 MAKE_CLOSURE 0
> # 18 STORE_FAST   0 (g)
> #
> #  5  21 LOAD_GLOBAL  0 (locals)
> # 24 CALL_FUNCTION0
> # 27 RETURN_VALUE
>
> [4]
> # code of bar.g:
> 
> #  4   0 LOAD_CONST   1 ('g')
> #  3 PRINT_ITEM
> #  4 LOAD_DEREF   0 (x)
> #  7 PRINT_ITEM
> #  8 PRINT_NEWLINE
> #  9 LOAD_CONST   0 (None)
> # 12 RETURN_VALUE
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] safety of Py_CLEAR and self

[Jeremy Hylton]

I was wondering today how I could convince myself that a sequence of
Py_CLEAR() calls in a tp_clear function was safe.  Take for example a
really trivial sequence of code on frame_clear():

Py_CLEAR(f->f_exc_type);
Py_CLEAR(f->f_exc_value);
Py_CLEAR(f->f_exc_traceback);
Py_CLEAR(f->f_trace);

We use Py_CLEAR() so that multiple calls to frame_clear() are safe.
The first time this runs it sets exc_type to NULL before calling
DECREF.  This guarantees that a subsequent frame_clear() or
frame_dealloc() won't attempt to DECREF it a second time.  I think I
understand why that's desireable and why it works.  The primary risk
is that via DECREF we may execute an arbitrary amount of Python code
via weakref callbacks, finalizers, and code in other threads that gets
resumed while the callbacks and finalizers are running.

My question, specifically, then: Why it is safe to assume that f
doesn't point to trash after a particular call to Py_CLEAR()?  Any
particular call to Py_CLEAR() could break the cycle that the object is
involved in an lead to a call to frame_dealloc().  The frame could get
returned to an obmalloc pool, returned to the OS, or just re-used by
another object before we get back to Py_CLEAR().  It seems like such
behavior would be exceedingly unlikely, but I can't convince myself
that it is impossible.  Which is it: improbable or impossible?  If it
is only improbable, should we figure out how to write code that is
safe against such an improbable event?

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] safety of Py_CLEAR and self

[Jeremy Hylton]

On 2/12/07, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> Looking for where tp_clear() is being called, the only caller is line
> 713 in gmodule.c, which explicitly surrounds the call with an
> INCREF/DECREF pair. Perhaps that's the clue you're looking for?

Yes, of course.  The INCREF guarantees that the object can't be
collected until the tp_clear() returns.  Thanks.

Jeremy

>
> --Guido
>
> On 2/12/07, Jeremy Hylton <[EMAIL PROTECTED]> wrote:
> > I was wondering today how I could convince myself that a sequence of
> > Py_CLEAR() calls in a tp_clear function was safe.  Take for example a
> > really trivial sequence of code on frame_clear():
> >
> > Py_CLEAR(f->f_exc_type);
> > Py_CLEAR(f->f_exc_value);
> > Py_CLEAR(f->f_exc_traceback);
> > Py_CLEAR(f->f_trace);
> >
> > We use Py_CLEAR() so that multiple calls to frame_clear() are safe.
> > The first time this runs it sets exc_type to NULL before calling
> > DECREF.  This guarantees that a subsequent frame_clear() or
> > frame_dealloc() won't attempt to DECREF it a second time.  I think I
> > understand why that's desireable and why it works.  The primary risk
> > is that via DECREF we may execute an arbitrary amount of Python code
> > via weakref callbacks, finalizers, and code in other threads that gets
> > resumed while the callbacks and finalizers are running.
> >
> > My question, specifically, then: Why it is safe to assume that f
> > doesn't point to trash after a particular call to Py_CLEAR()?  Any
> > particular call to Py_CLEAR() could break the cycle that the object is
> > involved in an lead to a call to frame_dealloc().  The frame could get
> > returned to an obmalloc pool, returned to the OS, or just re-used by
> > another object before we get back to Py_CLEAR().  It seems like such
> > behavior would be exceedingly unlikely, but I can't convince myself
> > that it is impossible.  Which is it: improbable or impossible?  If it
> > is only improbable, should we figure out how to write code that is
> > safe against such an improbable event?
> >
> > Jeremy
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: 
> > http://mail.python.org/mailman/options/python-dev/guido%40python.org
> >
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] dinner at Standard in Dallas

[Jeremy Hylton]

I'm organizing a trip to Standard in downtown Dallas for dinner
tonight (Sunday night).  It's about a 10 minute cab ride to Standard.
We can share cabs and get there without too much trouble.  The
restaurant is on the expensive side.  I'm thinking we should leave
from the hotal around 6:30pm.

http://www.standarddallas.com/
http://restaurants.dallasobserver.com/search/restaurants.php?oid=27399

A large group of us went for dinner last year and had a great time.
The food was excellent, short ribs and mashed potatoes with truffles
stand out in my mind.  They also serve a tangerita cocktail--margerita
made with Tang--that was memorable.

Let me know if you'd like to come and I'll make reservations and
arrange for cabs.

Jeremy

[Apologies if you get two copies of this message.  The first one looks
like it got lost or stuck.]
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] dinner at Standard in Dallas

[Jeremy Hylton]

I'm organizing a trip to Standard in downtown Dallas for dinner
tonight (Sunday night).  It's about a 10 minute cab ride to Standard.
We can share cabs and get there without too much trouble.  The
restaurant is on the expensive side.  I'm thinking we should leave
from the hotal around 6:30pm.

http://www.standarddallas.com/
http://restaurants.dallasobserver.com/search/restaurants.php?oid=27399

A large group of us went for dinner last year and had a great time.
The food was excellent, short ribs and mashed potatoes with truffles
stand out in my mind.  They also serve a tangerita cocktail--margerita
made with Tang--that was memorable.

Let me know if you'd like to come and I'll make reservations and
arrange for cabs.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Renaming Include/object.h

[Jeremy Hylton]

On 1/3/07, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> In #1626545, Anton Tropashko requests that object.h should be
> renamed, because it causes conflicts with other software.
>
> I would like to comply with this requests for 2.6, assuming there
> shouldn't be many problems with existing software as object.h
> shouldn't be included directly, anyway.

I like the idea of renaming the header files, but I expect there is a
lot more opportunity for breaking code that you estimate.  I did a
search on google.com/codesearch and found seven external packages that
include Python.h and object.h (before I gave up).

I assume we expect people won't include it, because it is also
included in object.h.  But they do it.  Is there documentation that
says you shouldn't import it?

Jeremy


> What do you think?
>
> Regards,
> Martin
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] locals(), closures, and IronPython...

[Jeremy Hylton]

On 3/5/07, Dino Viehland <[EMAIL PROTECTED]> wrote:
> Thanks Guido.  It might take some time (and someone may very well beat me to 
> it if they care a lot) but I'll see if we can get the PEP started.

Dino,

One of the questions I was puzzling over was what locals() should
return in a class scope.  It's confusing in CPython, because the same
dictionary is used to stored names that are destined to become class
attributes and names that are used to created closures for methods
with free variables.  The implementation was causing the latter names
to show up as class attributes when you called locals().

Terry is right that lifetime of variables is a key issue.  It would
probably be bad if all the local variables of a function were kept
alive because a nested function used locals().

One idea was discussed at PyCon was having a different representation
for an environment so that it would be easier to introspect on it but
still possible to pass the environment to exec.  One possibility is
just a third dictionary--globals, locals, nonlocals.  Another
possibility is an object where you can ask about the scope of each
variable but also extract a dictionary to pass to locals.

Jeremy

>
> -Original Message-
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Guido van 
> Rossum
> Sent: Monday, March 05, 2007 2:14 PM
> To: Dino Viehland
> Cc: python-dev@python.org
> Subject: Re: [Python-Dev] locals(), closures, and IronPython...
>
> Jeremy Hylton has been asking questions about this too at the sprint
> after PyCon. I'm tempted to accept that the exact behavior of locals()
> is implementation-defined (IOW undefined :-) as long as it includes
> the locals defined in the current scope; whether it also includes free
> variables could be debatable. I don't know too many good use cases for
> locals() apart from "learning about the implementation" I think this
> might be okay. Though a PEP might be in order to get agreement between
> users, developers and other implementation efforts (e.g. PyPy,
> Jython).
>
> On 3/5/07, Dino Viehland <[EMAIL PROTECTED]> wrote:
> >
> >
> >
> >
> > def a():
> >
> > x = 4
> >
> > y = 2
> >
> > def b():
> >
> > print y, locals()
> >
> > print locals()
> >
> > b()
> >
> >
> >
> > a()
> >
> >
> >
> > in CPython prints:
> >
> >
> >
> > {'y': 2, 'x': 4, 'b': }
> >
> > 2 {'y': 2}
> >
> >
> >
> > I'm wondering if it's intentional that these don't print dictionaries w/ the
> > same contents or if it's more an accident of the implementation.   In other
> > words would it be reasonable for IronPython to promote all of the locals of
> > a into b's dictionary when both a and b call locals?
> >
> >
> >
> > We currently match CPython's behavior here - at least in what we display
> > although possibly not in the lifetimes of objects.  We're considering an
> > internal change which might alter the behavior here though and end up
> > displaying all the members.
> >
> >
> >
> >
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> > http://mail.python.org/mailman/options/python-dev/guido%40python.org
> >
> >
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Encouraging developers

[Jeremy Hylton]

On 3/6/07, Georg Brandl <[EMAIL PROTECTED]> wrote:
> Raymond Hettinger schrieb:
> > [Phil Thompson]
> >> I think a lot of people care, but many can't
> >> do anything about because the barrier to entry is too great.
> >
> > Do you mean commit priviledges?  ISTM, those tend to be
> > handed out readily to people who assert that they have good use for them.
> > Ask the Georg-bot how readily he was accepted and coached.  IMO,
> > his acceptance was a model that all open source projects should aspire to.
>
> Indeed. For me, it wasn't "hard" to get tracker rights. I reviewed some 
> patches,
> commented on bugs, posted suggestions to python-dev etc. When I asked about
> tracker rights on python-dev, they were given to me.
> Then, it wasn't "hard" to get commit rights. I contributed some stuff, and
> after a while I asked about commit rights on python-dev, and they were given
> to me on condition that I still let a core dev review inteded changes.
>
> As far as I recall, there has been nearly no one who asked for commit rights
> recently, so why complain that the entry barrier is too great? Surely you
> cannot expect python-dev to got out and say "would you like to have commit
> privileges?"...

You can ask whether we should have a plan for increasing the number of
developers, actively seeking out new developers, and mentoring people
who express interest.  Would the code be better if we had more good
developers working on it?  Would we get more bugs fixed and patches
closed?  If so, it wouldn't hurt to have some deliberate strategy for
bringing new developers in.  I can easily imagine someone spending a
lot of time mentoring and a little time coding, but having a bigger
impact that someone who only wrote code.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal to revert r54204 (splitext change)

[Jeremy Hylton]

On 3/15/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> On 05:51 pm, [EMAIL PROTECTED] wrote:
> >At 07:45 AM 3/15/2007 +0100, Martin v. Löwis wrote:
> >>I apparently took the same position that you now take back then,
> >>whereas I'm now leaning towards (or going beyond) the position
> >>Tim had back then, who wrote "BTW, if it *weren't* for the code breakage,
> >>I'd be in favor of doing this."
> >
> >If it weren't for the code breakage, I'd be in favor too.  That's not the
> >point.
> >
> >The point is that how can Python be stable as a language if precedents can
> >be reversed without a migration plan, just because somebody changes their
> >mind?  In another five years, will you change your mind again, and decide
> >to put this back the way it was?
>
> Hear, hear.  Python is _not_ stable as a language.  I have Java programs
> that I wrote almost ten years ago which still run perfectly on the latest
> runtime.  There is python software I wrote two years ago which doesn't work
> right on 2.5, and some of the Python stuff contemporary with that Java code
> won't even import.

I think the problem has less to do with bug fixing than with lack of
any clear specifications or documentation about what developers can
depend on.You could probably make a case that any change that
doesn't fix a crash bug is likely to cause some particular program to
behave differently.

Take bug 1504333 which lead to a change in sgmllib behavior for angle
brackets in quoted attribute values.  Did the sgmllib documentation
explain that the fixed behavior was incorrect?  Might a programmer
working with sgmllib have written code that depended on this bug?  Do
you object to this bug fix?

For many of these bugs, some people will have written code against the
documentation and some people against the implementation or behavior.
(In this case, the documentation is vague or conflicting.)  I don't
think I know how to balance the important of these two classes of
users.  Some code is going to break the first time they run into the
under-specific edge case, some code is going to break when the
specification and implementation are clarified.  You have to weigh
which you think is more likely and which will benefit users the most.

I think everyone wants to do the "right thing" by Python's users, but
it's not clear what that right thing is.

> >Speaking as a business person, that seems to me... unwise.  When I found
> >out that this change had been checked in despite all the opposition, my gut
> >reaction was, "I guess I can't rely on Python any more", despite 10 years
> >of working with it, developing open source software with it, and
> >contributing to its development.  Because from a *business* perspective,
> >this sort of flip-flopping means that moving from one "minor" Python
> >version to another is potentially *very* costly.
>
> And indeed it is.  Python's advantages in terms of rapidity of development
> have, thus far, made up the difference for me, but it is threatening to
> become a close thing.  This is a severe problem and something needs to be
> done about it.

Could you point out a few such programs that people on python-dev can
look at?  I think it would be useful to gather some data about the
kind of migration pains real users are having.  I believe Martin and
others are trying to do the right thing.  Real data is more likely to
convince them than passionate arguments on python-dev.

> >But as you are so fond of pointing out, there is no "many people".  There
> >are only individual people.  That a majority want it one way, means that
> >there is a minority who want it another.  If next year, it becomes more
> >popular to have it the other way, will we switch again?  If a majority of
> >people want braces and required type declarations, will we add them?
>
> And, in fact, there is not even a majority.  There is a *perception* of a
> majority.  There isn't even a *perception* of a majority of Python users,
> but a perception of a majority of python-dev readers, who are almost by
> definition less risk-averse when it comes to language change than anyone
> else!

I think you missed the point here.  The hypothetical question was not
about any particular majority, but rather that regardless of which
group you poll, the majority decision may not be the right one.  Even
a majority of Twised users :-).

Jeremy

> If we actually care about majorities, let's set up a voting application and
> allow Python users to vote on each and every feature, and publicize it each
> time such a debate comes up.  Here, I'll get it started:
> http://jyte.com/cl/python-should-have-a-strict-backward-compatibility-policy-to-guide-its-development
>
> According to that highly scientific study, at this point in time, "Nobody
> disagrees" :).  (One in favor, zero against.)
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.pyt

Re: [Python-Dev] HTTP responses and errors

[Jeremy Hylton]

On 4/10/07, Facundo Batista <[EMAIL PROTECTED]> wrote:
> Facundo Batista wrote:
>
> > Martin v. Löwis wrote:
> > ...
> >
> >> think it should treat all 2xx responses as success. Callers can
> >> then still check the response code themselves if they need to.
> >
> > The same I think. If nobody has a conflic with this decission, I'll fix
> > this.
>
> Nobody raised any objection, I'll fix this these days.

+1

Jeremy
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GC Changes

[Jeremy Hylton]

On 10/1/07, Justin Tulloss <[EMAIL PROTECTED]> wrote:
> Hello,
>
> I've been doing some tests on removing the GIL, and it's becoming clear that
> some basic changes to the garbage collector may be needed in order for this
> to happen efficiently. Reference counting as it stands today is not very
> scalable.
>
> I've been looking into a few options, and I'm leaning towards the
> implementing IBMs recycler GC (
> http://www.research.ibm.com/people/d/dfb/recycler-publications.html
> ) since it is very similar to what is in place now from the users'
> perspective. However, I haven't been around the list long enough to really
> understand the feeling in the community on GC in the future of the
> interpreter. It seems that a full GC might have a lot of benefits in terms
> of performance and scalability, and I think that the current gc module is of
> the mark-and-sweep variety. Is the trend going to be to move away from
> reference counting and towards the mark-and-sweep implementation that
> currently exists, or is reference counting a firmly ingrained tradition?
>
> On a more immediately relevant note, I'm not certain I understand the full
> extent of the gc module. From what I've read, it sounds like it's fairly
> close to a fully functional GC, yet it seems to exist only as a
> cycle-detecting backup to the reference counting mechanism. Would somebody
> care to give me a brief overview on how the current gc module interacts with
> the interpreter, or point me to a place where that is done? Why isn't the
> mark-and-sweep mechanism used for all memory management?

There are probably lots of interesting things to say about the gc
cycle collector, but I'll just pick a few things that seem relevant.
First off, the gc doesn't have a root set.  It traces all the
container objects, subtracting known references from the refcount, and
is left with a root set, i.e. those objects that have some references
that can't be accounted for among the known container objects.  (see
update_refs and substract_refs)  In the end, we make three traversals
of the heap to detect the objects that appear to be unreachable.
(move_unreachable is the third.)

The cycle detection relies on having the reference counts correct, so
it doesn't really represent a move away from refcounting.

I skipped the generations.  The GC divides the heap into three
generations and tends to focus on the youngest generation.  So that
limits the portion of the heap that is scanned, but I don't understand
the magnitude of that effect in practice.

The current collector works in collaboration with ref counting.  In
particular, refcounting probably handles the majority of
deallocations.  If the cycle detection system were responsible for all
deallocations, the gc module would have a lot more work to do.

I do think it would be interesting to experiment with the recycler
approach, but I think it would take a lot of work to do a decent
experiment.  But please give it a shot!

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

[Jeremy Hylton]

On Fri, May 2, 2008 at 1:38 PM, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> Thomas Lee wrote:
>  > Martin v. Löwis wrote:
>  >>> This leaves us with a few options:
>  >>>
>  >>
>  >> 5. Reuse/Abuse Num(object) for arbitrary constants.
>  >>AFAICT, this should work out of the box.
>  >>
>  >>
>  > Eek. It *does* seem like Num would work out of the box, but would this
>  > be a good idea?
>
>  No. I suggested it just for completeness.
>
>
>  > What about *replacing* Num with Const? Might make optimizations
>  > specifically for numeric values slightly hairier, and semantically I
>  > think they might be different enough to warrant separate AST nodes
>  > despite the similarity in implementation at the compiler level.
>
>  I think they should be separate. Const would be a mere addition;
>  for compatibility with other uses of the AST, that's actually better
>  than simultaneous removal of Num.

Adding Const sounds good to me.

Jeremy

>
>  Regards,
>  Martin
>
>
>
>  ___
>  Python-Dev mailing list
>  Python-Dev@python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] AST Optimization: Branch Elimination in Generator Functions

[Jeremy Hylton]

I'm not sure I understand the problem exactly.  If we have one pass
converting the concrete syntax to the AST, we can mark functions as
generators as part of that pass.  In a later pass, you can remove the
unreachable code.

Jeremy

On Sat, May 3, 2008 at 12:51 AM, Thomas Lee <[EMAIL PROTECTED]> wrote:
> The next problem that cropped up during the implementation of the AST code
> optimizer is related to branch elimination and the elimination of any code
> after a return.
>
>  Within a FunctionDef node, we would (ideally) like to blow away If nodes
> with a constant - but false - test expression. e.g.:
>
>  def foo():
>  if False:
>   # ... stuff ...
>
>  For most functions, this will cause no problems and the code will behave as
> expected. However, if the eliminated branch contains a "yield" expression,
> the function is actually a generator function - even if the yield expression
> can never be reached:
>
>  def foo():
>  if False:
>   yield 5
>
>  In addition to this, the following should also be treated as a generator
> even though we'd like to be able to get rid of all the code following the
> "return" statement:
>
>  def foo():
>  return
>  yield 5
>
>  Again, blowing away the yield results in a normal function instead of a
> generator. Not what we want: we need to preserve the generator semantics.
>
>  Upon revisiting this, it's actually made me reconsider the use of a Const
> node for the earlier problem relating to arbitrary constants. We may be
> better off with annotations after all ... then we could mark FunctionDef
> nodes as being generators at the AST level to force the compiler to produce
> code for a generator, but eliminate the branches anyway.
>
>  The other alternative I can think of is injecting a yield node somewhere
> unreachable and ensuring it doesn't get optimized away, but this seems
> pretty hacky in comparison.
>
>  Any other ideas?
>
>  Cheers,
>  Tom
>  ___
>  Python-Dev mailing list
>  Python-Dev@python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

[Jeremy Hylton]

On Wed, May 7, 2008 at 11:43 AM, Thomas Lee <[EMAIL PROTECTED]> wrote:
> Nick Coghlan wrote:
>
> >
> > As Thomas mentions in a later message, making it possible to annotate
> nodes would permit Functions to be annotated as being a generator at the AST
> stage (currently it is left to the bytecode compiler's symtable generation
> pass to make that determination).
> >
> > Although I guess an alternative solution to that would be to have separate
> AST nodes for Functions and Generators as well...
> >
> >
>  I've actually backtracked a little and gone back down the Const path again.
> I know this is the third time I've changed my mind, but it's primarily
> because annotations tend to get a little clunky (or my implementation was,
> at least). Using Const nodes feels a lot more natural inside the optimizer.
> I think it's going to stick, at least in the short term.
>
>  Rather than separate FunctionDef and GeneratorDef nodes, I think a new bool
> attribute (is_generator?) on FunctionDef should do the job nicely. Further,
> I'm thinking we can move the "generator detection code" from the symtable
> into Python/ast.c so the generator/function information is available to the
> optimizer.

Does the optimizer run after the symtable?  I'd expect the symbol
table information to be essential for almost all analysis, so it
should be fine to wait until that pass to mark the generators.

Jeremy

>  This is made a little tricky by the absence of the contextual information
> available that is normally available when flagging generators in the
> symtable. When generating AST nodes for a suite, we know nothing about the
> parent node in which the suite resides. Still, it might be doable. If this
> winds up being ugly, we might need to fall back to the original plan of a
> separate pass over function bodies to detect yield expressions.
>
>  I'll look into all this tomorrow night, along with any other crazy
> suggestions. For now I need to sleep a few hours. :)
>
>  Thanks for the feedback, it's much appreciated.
>
>  Cheers,
>  T
>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

[Jeremy Hylton]

On Thu, May 8, 2008 at 8:00 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote:
> Jeremy Hylton wrote:
>
> > On Wed, May 7, 2008 at 11:43 AM, Thomas Lee <[EMAIL PROTECTED]> wrote:
> >
> > > Nick Coghlan wrote:
> > >
> > >
> > > > As Thomas mentions in a later message, making it possible to annotate
> > > >
> > > nodes would permit Functions to be annotated as being a generator at the
> AST
> > > stage (currently it is left to the bytecode compiler's symtable
> generation
> > > pass to make that determination).
> > >
> > > > Although I guess an alternative solution to that would be to have
> separate
> > > >
> > > AST nodes for Functions and Generators as well...
> > >
> > > >
> > > >
> > >  I've actually backtracked a little and gone back down the Const path
> again.
> > > I know this is the third time I've changed my mind, but it's primarily
> > > because annotations tend to get a little clunky (or my implementation
> was,
> > > at least). Using Const nodes feels a lot more natural inside the
> optimizer.
> > > I think it's going to stick, at least in the short term.
> > >
> > >  Rather than separate FunctionDef and GeneratorDef nodes, I think a new
> bool
> > > attribute (is_generator?) on FunctionDef should do the job nicely.
> Further,
> > > I'm thinking we can move the "generator detection code" from the
> symtable
> > > into Python/ast.c so the generator/function information is available to
> the
> > > optimizer.
> > >
> >
> > Does the optimizer run after the symtable?  I'd expect the symbol
> > table information to be essential for almost all analysis, so it
> > should be fine to wait until that pass to mark the generators.
> >
>
>  There are a lot of micro-optimisations that are actually context
> independent, so moving them before the symtable pass should be quite
> feasible - e.g. replacing "return None" with "return", stripping dead code
> after a return statement, changing a "if not" statement into an "if"
> statement with the two suites reversed, changing "(1, 2, 3)" into a stored
> constant, folding "1 + 2" into the constant "3".
>
>  I believe the goal is to see how many of the current bytecode optimisations
> can actually be brought forward to the AST generation stage, rather than
> waiting until after the bytecode symtable calculation and compilation
> passes.
>
>  The current structure goes:
>
>  tokenisation->AST construction->symtable construction->bytecode
> compilation->bytecode optimisation
>
>  My understanding of what Thomas is trying to do is to make it look more
> like this:
>
>  tokenisation->AST construction->AST optimisation->symtable
> construction->bytecode compilation
>
>  There may still be a direct bytecode optimisation step left at the end if
> it turns out something relies on information that isn't worked out until the
> symtable construction pass (and that can't easily be worked out earlier,
> such as adding an 'is_generator' flag to the FunctionDef AST node).

The symbol table pass is not important for deciding where to place the
optimizations.  Either the optimization operates on the AST itself, in
which case it needs to run before bytecode compilation or it operators
on the lowered bytecode representation, in which case it needs to run
after bytecode compilation.  Since the symbol table passes doesn't
change the representation, you ought to be able to put the
optimizations before or after it.

But I haven't looked closely at the actual code, so maybe I'm missing something.

Jeremy

>
>
>
>  Cheers,
>  Nick.
>
>  --
>  Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
>  ---
> http://www.boredomandlaziness.org
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Trickery with moving urllib

[Jeremy Hylton]

On Sat, May 10, 2008 at 11:38 PM, Brett Cannon <[EMAIL PROTECTED]> wrote:
> There is going to be an issue with the current proposal for keeping
> around urllib. Since the package is to be named the same thing as the
> module, to handle the new name that means urllib.__init__ will need to
> gain the Py3K warning for the new name. But that doesn't quite work as
> the package's module will be imported automatically any time the
> module within the package is used.
>
> I see three solutions for dealing with this.
>
> 1. Have stubs for the entire urllib API in urllib.__init__ that raise
> a DeprecationWarning either specifying the new name or saying the
> function/class is deprecated.
>
> 2. Rename urllib to urllib.fetch or urllib.old_request to get people
> to move over to urllib.request (aka urllib2) at some point.
>
> 3. Ditch urllib completely and have people move to urllib.request
> directly from 2.6 going into 3.0.
>
> I was a fan of 3 in the stdlib SIG, but MAL argued well for 2 since so
> much code uses urllib (although I have always wondered how many
> *really* use urllib and how many could easily just move over to
> urllib2). The current solution of keeping around urllib's classes was
> a compromise, but I had not thought out the transition yet (and Jeremy
> Hylton will be running into this as he has volunteered to handle
> this).
>
> What do people think we should do?

I am migrating all the stdlib code to use urllib2 APIs when possible.
It seems straightforward enough to keep the old urllib classes in
urllib.request, but change the standard API functions (urlopen,
urlretrieve) to use the urllib2 implementation.

We could probably have a big collection of deprecation warnings in the
urllib/__init__ without much trouble.  It doesn't have any code at
this point.

Jeremy

>
> -Brett
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] buildbot failure in sparc Debian 3.0

[Jeremy Hylton]

Does anyone have a clue about why this test fails only on this
platform?  The test is question is verifying that URLError gets
raised.  From the traceback, it appears that there is an uncaught
exception (URLError) but it fails in an assertRaises() check for
URLError.  That doesn't make much sense unless the variable URLError
refers to different objects, but both modules use "from urllib.error
import URLError".  And, of course, the test works fine on other
platforms.

Jeremy

On Wed, Jul 2, 2008 at 4:23 PM,  <[EMAIL PROTECTED]> wrote:
> The Buildbot has detected a new failure of sparc Debian 3.0.
> Full details are available at:
>  http://www.python.org/dev/buildbot/all/sparc%20Debian%203.0/builds/330
>
> Buildbot URL: http://www.python.org/dev/buildbot/all/
>
> Buildslave for this Build: klose-debian-sparc
>
> Build Reason:
> Build Source Stamp: [branch branches/py3k] HEAD
> Blamelist: benjamin.peterson
>
> BUILD FAILED: failed test
>
> Excerpt from the test logfile:
> 1 test failed:
>test_urllib2
>
> ==
> ERROR: test_badly_named_methods (test.test_urllib2.OpenerDirectorTests)
> --
>
> Traceback (most recent call last):
>  File 
> "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/test/test_urllib2.py",
>  line 408, in test_badly_named_methods
>self.assertRaises(URLError, o.open, scheme+"://example.com/")
>  File 
> "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/unittest.py", 
> line 311, in failUnlessRaises
>callableObj(*args, **kwargs)
>  File 
> "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/urllib/request.py",
>  line 356, in open
>response = self._open(req, data)
>  File 
> "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/urllib/request.py",
>  line 379, in _open
>'unknown_open', req)
>  File 
> "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/urllib/request.py",
>  line 334, in _call_chain
>result = func(*args)
>  File 
> "/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/urllib/request.py",
>  line 1102, in unknown_open
>raise URLError('unknown url type: %s' % type)
> urllib.error.URLError: 
>
> make: *** [buildbottest] Error 1
>
> sincerely,
>  -The Buildbot
>
> ___
> Python-checkins mailing list
> [EMAIL PROTECTED]
> http://mail.python.org/mailman/listinfo/python-checkins
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Summary of Python tracker Issues

[Jeremy Hylton]

I wanted to ask a policy question on the bug tracker.  What are we
doing with bugs filed against Python 2.4?
This bug http://bugs.python.org/issue1208304 reports a fd leak in
Python 2.4, which doesn't exist in the head.  Since Python 2.4 is in
security-fix only mode, is it fair to close this with a won't fix?

A meta-question:  Is there a document that has any tips on how to
handle bug reports?  It might be helpful to have a list of hints or
policy decisions to help people decide what to do with bugs.

Jeremy

On Fri, Nov 21, 2008 at 12:06 PM, Python tracker <[EMAIL PROTECTED]> wrote:
>
> ACTIVITY SUMMARY (11/14/08 - 11/21/08)
> Python tracker at http://bugs.python.org/
>
> To view or respond to any of the issues listed below, click on the issue
> number.  Do NOT respond to this message.
>
>
>  2194 open (+35) / 14007 closed (+18) / 16201 total (+53)
>
> Open issues with patches:   728
>
> Average duration of open issues: 712 days.
> Median duration of open issues: 2043 days.
>
> Open Issues Breakdown
>   open  2177 (+35)
> pending17 ( +0)
>
> Issues Created Or Reopened (56)
> ___
>
> Tkinter binding involving Control-spacebar raises unicode error  11/18/08
>   http://bugs.python.org/issue1028reopened ocean-city
>   patch
>
> Byte/string inconsistencies between different dbm modules11/21/08
>   http://bugs.python.org/issue3799reopened brett.cannon
>   patch, needs review
>
> type of UserList instance returns class instead of instance  11/14/08
> CLOSED http://bugs.python.org/issue4326created  chafporte
>
>
> Patch: simplify complex constant assignment statements   11/14/08
>   http://bugs.python.org/issue4327created  novalis_dt
>   patch
>
> "à" in u"foo" raises a misleading error 11/15/08
>   http://bugs.python.org/issue4328created  ezio.melotti
>
>
> base64 does not properly handle unicode strings  11/15/08
>   http://bugs.python.org/issue4329created  mbecker
>
>
> wsgiref.validate doesn't accept arguments to readline11/16/08
> CLOSED http://bugs.python.org/issue4330created  ianb
>
>
> Can't use _functools.partial() created function as method11/16/08
>   http://bugs.python.org/issue4331created  ssadler
>
>
> asyncore.file_dispatcher does not use dup()'ed fd11/17/08
> CLOSED http://bugs.python.org/issue4332created  christianbecke
>   patch
>
> Reworked Dialog.py   11/17/08
>   http://bugs.python.org/issue4333created  gpolo
>   patch
>
> Mac Build Script is broken for 2.6 release   11/17/08
> CLOSED http://bugs.python.org/issue4334created  ngift
>
>
> inspect.getsourcelines ignores last line in module   11/17/08
>   http://bugs.python.org/issue4335created  RafeSacks
>
>
> Fix performance issues in xmlrpclib  11/17/08
>   http://bugs.python.org/issue4336created  krisvale
>   patch, patch, easy
>
> Iteration over a map object with list()  11/17/08
> CLOSED http://bugs.python.org/issue4337created  jmfauth
>
>
> TypeError (bytes/str) in distutils command "upload"  11/17/08
> CLOSED http://bugs.python.org/issue4338created  hagen
>   patch
>
> Fix set-like dictview doc11/17/08
> CLOSED http://bugs.python.org/issue4339created  tjreedy
>
>
> xmlrpc.client - default 'SlowParser' not defined 11/17/08
>   http://bugs.python.org/issue4340created  mwatkins
>
>
> Update __hash__ doc  11/17/08
> CLOSED http://bugs.python.org/issue4341created  tjreedy
>
>
> (Tkinter) Please backport these  11/18/08
>   http://bugs.python.org/issue4342created  gpolo
>
>
> New function in Tkinter.py: setup_master 11/18/08
>   http://bugs.python.org/issue4343created  gpolo
>   patch
>
> crash upon launch11/18/08
> CLOSED http://bugs.python.org/issue4344created  source.mod
>
>
> Implement nb_nonzero for PyTclObject 11/18/08
>   http://bugs.python.org/issue4345created  gpolo
>   patch
>
> PyObject_CallMethod changes the exception message already set by 11/18/08
>   http://bugs.python.org/issue4346created  gpolo
>   patch
>
> Dependencies of graminit.h are not rebuilt when the file is rege 11/18/08
>   http://bugs.python.org/issue4347created  thomas.lee
>   patch
>
> bytearray methods returning self 11/18/08
> CLOSED http://bugs.python.org/issue4348created  DinoV
>   patch, needs review
>
> sys.path includes extraneous junk11/18/08
> CLOSED http://bugs.python.org/issue4349created  ex

Re: [Python-Dev] Summary of Python tracker Issues

[Jeremy Hylton]

On Mon, Nov 24, 2008 at 12:54 PM, Brett Cannon <[EMAIL PROTECTED]> wrote:
> On Mon, Nov 24, 2008 at 08:58, Jeremy Hylton <[EMAIL PROTECTED]> wrote:
>> I wanted to ask a policy question on the bug tracker.  What are we
>> doing with bugs filed against Python 2.4?
>> This bug http://bugs.python.org/issue1208304 reports a fd leak in
>> Python 2.4, which doesn't exist in the head.  Since Python 2.4 is in
>> security-fix only mode, is it fair to close this with a won't fix?
>>
>
> I think so.
>
>> A meta-question:  Is there a document that has any tips on how to
>> handle bug reports?  It might be helpful to have a list of hints or
>> policy decisions to help people decide what to do with bugs.
>
> It's coming. I just have not gotten around to it yet. I plan to
> outline the basic steps in the life of an issue and that includes
> closing an issue and the various reasons this can happen.

If you put up skeleton doc, I can add a note to this effect.

Jeremy

>
> -Brett
>
>
>>
>> Jeremy
>>
>> On Fri, Nov 21, 2008 at 12:06 PM, Python tracker <[EMAIL PROTECTED]> wrote:
>>>
>>> ACTIVITY SUMMARY (11/14/08 - 11/21/08)
>>> Python tracker at http://bugs.python.org/
>>>
>>> To view or respond to any of the issues listed below, click on the issue
>>> number.  Do NOT respond to this message.
>>>
>>>
>>>  2194 open (+35) / 14007 closed (+18) / 16201 total (+53)
>>>
>>> Open issues with patches:   728
>>>
>>> Average duration of open issues: 712 days.
>>> Median duration of open issues: 2043 days.
>>>
>>> Open Issues Breakdown
>>>   open  2177 (+35)
>>> pending17 ( +0)
>>>
>>> Issues Created Or Reopened (56)
>>> ___
>>>
>>> Tkinter binding involving Control-spacebar raises unicode error  11/18/08
>>>   http://bugs.python.org/issue1028reopened ocean-city
>>>   patch
>>>
>>> Byte/string inconsistencies between different dbm modules11/21/08
>>>   http://bugs.python.org/issue3799reopened brett.cannon
>>>   patch, needs review
>>>
>>> type of UserList instance returns class instead of instance  11/14/08
>>> CLOSED http://bugs.python.org/issue4326created  chafporte
>>>
>>>
>>> Patch: simplify complex constant assignment statements   11/14/08
>>>   http://bugs.python.org/issue4327created  novalis_dt
>>>   patch
>>>
>>> "à" in u"foo" raises a misleading error 11/15/08
>>>   http://bugs.python.org/issue4328created  ezio.melotti
>>>
>>>
>>> base64 does not properly handle unicode strings  11/15/08
>>>   http://bugs.python.org/issue4329created  mbecker
>>>
>>>
>>> wsgiref.validate doesn't accept arguments to readline11/16/08
>>> CLOSED http://bugs.python.org/issue4330created  ianb
>>>
>>>
>>> Can't use _functools.partial() created function as method11/16/08
>>>   http://bugs.python.org/issue4331created  ssadler
>>>
>>>
>>> asyncore.file_dispatcher does not use dup()'ed fd11/17/08
>>> CLOSED http://bugs.python.org/issue4332created  christianbecke
>>>   patch
>>>
>>> Reworked Dialog.py   11/17/08
>>>   http://bugs.python.org/issue4333created  gpolo
>>>   patch
>>>
>>> Mac Build Script is broken for 2.6 release   11/17/08
>>> CLOSED http://bugs.python.org/issue4334created  ngift
>>>
>>>
>>> inspect.getsourcelines ignores last line in module   11/17/08
>>>   http://bugs.python.org/issue4335created  RafeSacks
>>>
>>>
>>> Fix performance issues in xmlrpclib  11/17/08
>>>   http://bugs.python.org/issue4336created  krisvale
>>>   patch, patch, easy
>>>
>>> Iteration over a map object with list()  11/17/08
>>> CLOSED http://bugs.python.org/issue4337created  jmfauth
>>>
>>>
>>> TypeError (bytes/str) in distutils command "upload"  11/17/08
>>> CLOSED http://bugs.python.org/issue4338created  hagen
>>>   patch
>>>
>>> Fix se

Re: [Python-Dev] [Python-3000] Merging mailing lists

[Jeremy Hylton]

On Thu, Dec 4, 2008 at 2:36 AM, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> I would like to merge mailing lists, now that the design and first
> implementation of Python 3000 is complete. In particular, I would
> like to merge the python-3000 mailing list back into python-dev,
> and the python-3000-checkins mailing list back into python-checkins.
> The rationale is to simplify usage of the lists, and to avoid
> cross-postings.

+1

> To implement this, all subscribers of the 3000 mailing lists would
> be added to the trunk mailing lists (avoiding duplicates, of course),
> and all automated messages going to python-3000-checkins would then
> be directed to the trunk lists. The 3000 mailing lists would change
> into read-only mode (i.e. primarily leaving the archives behind).
>
> Any objections?

No

Jeremy


> Regards,
> Martin
> ___
> Python-3000 mailing list
> [EMAIL PROTECTED]
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-3000/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Merging flow

[Jeremy Hylton]

On Thu, Dec 4, 2008 at 10:12 AM, Christian Heimes <[EMAIL PROTECTED]> wrote:
> Several people have asked about the patch and merge flow. Now that Python
> 3.0 is out it's a bit more complicated.
>
> Flow diagram
> 
>
> trunk ---> release26-maint
>   \->  py3k   ---> release30-maint
>
>
> Patches for all versions of Python should land in the trunk. They are then
> merged into release26-maint and py3k branches. Changes for Python 3.0 are
> merged via the py3k branch.

You say "they are then merged."  Does that mean if I commit something
on the trunk, someone else will merge it for me?  Or do I need to do
it?

The library is vastly different between 2.x and 3.x.  I'm personally
aware of the many changes related to httplib / urllib / xmlrpclib.
I'm worried that it will be hard to decide how to "merge" things
between the two versions.

Jeremy

> Christian
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python 3.0 urllib fails with chunked HTTP responses

[Jeremy Hylton]

This bug is pretty serious, because urllib will insert garbage into
the application-visible data for a chunked response.  It simply
ignores the fact that it's reading a chunked response and includes the
chunked header data is payload data.  The original bug was reported in
September, but no one noticed it.  It was reported again recently.

http://bugs.python.org/issue3761
http://bugs.python.org/issue4631

I suspect we'd want to get a 3.0.1 out as soon as this is fixed, but
that's not my call.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3.0 urllib fails with chunked HTTP responses

[Jeremy Hylton]

I have a patch that appears to fix this bug
http://bugs.python.org/file12361/urllib-chunked.diff
but I'm not sure about its interaction with the io module and
RawIOBase.  Is there a new IO expert who could take a look at it for
me?

Jeremy

On Sun, Dec 14, 2008 at 11:06 PM, Jeremy Hylton  wrote:
> This bug is pretty serious, because urllib will insert garbage into
> the application-visible data for a chunked response.  It simply
> ignores the fact that it's reading a chunked response and includes the
> chunked header data is payload data.  The original bug was reported in
> September, but no one noticed it.  It was reported again recently.
>
> http://bugs.python.org/issue3761
> http://bugs.python.org/issue4631
>
> I suspect we'd want to get a 3.0.1 out as soon as this is fixed, but
> that's not my call.
>
> Jeremy
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3.0 urllib fails with chunked HTTP responses

[Jeremy Hylton]

On Wed, Dec 17, 2008 at 1:05 PM, Guido van Rossum  wrote:
> The inheritance from io.RawIOBase seems fine.

There is a small problem with the interaction between HTTPResponse and
RawIOBase, but I think the problem is more on the http side.  You may
recall that the HTTP code has a habit of closing the connection for
you.  In a variety of cases, once you've read the last bytes of the
response, the HTTPResponse object calls its own close() method.  This
interacts poorly with RawIOBase, because it raises a ValueError for
any operation on a closed io object.  This prevents iterators from
working correctly.  The iterator implementation expects the final call
to readline() to return an empty string and converts that to a
StopIteration.  Instead, it's seeing a ValueError that propagates out.

It's always been odd to me that the connection closed itself.  It's
going to be tricky to fix the current bug (chunked responses) and keep
the self-closing behavior, but I worry that change the self-closing
behavior too dramatically isn't appropriate for a bug fix.  Will look
some more at this tomorrow.

Jeremy

> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
>
>
> On Mon, Dec 15, 2008 at 11:19 AM, Jeremy Hylton  wrote:
>> I have a patch that appears to fix this bug
>> http://bugs.python.org/file12361/urllib-chunked.diff
>> but I'm not sure about its interaction with the io module and
>> RawIOBase.  Is there a new IO expert who could take a look at it for
>> me?
>>
>> Jeremy
>>
>> On Sun, Dec 14, 2008 at 11:06 PM, Jeremy Hylton  wrote:
>>> This bug is pretty serious, because urllib will insert garbage into
>>> the application-visible data for a chunked response.  It simply
>>> ignores the fact that it's reading a chunked response and includes the
>>> chunked header data is payload data.  The original bug was reported in
>>> September, but no one noticed it.  It was reported again recently.
>>>
>>> http://bugs.python.org/issue3761
>>> http://bugs.python.org/issue4631
>>>
>>> I suspect we'd want to get a 3.0.1 out as soon as this is fixed, but
>>> that's not my call.
>>>
>>> Jeremy
>>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3.0 urllib fails with chunked HTTP responses

[Jeremy Hylton]

On Thu, Dec 18, 2008 at 12:27 PM, Guido van Rossum  wrote:
> It sounds like the self-closing is an implementation detail, meant to
> make sure the socket is closed as early as possible (which I suppose
> is a good thing if there's a server waiting for the final ACK on the
> other side). Perhaps it should not use close() but something slightly
> lower level that affects the socket directly?

That's what I'm thinking, too.  I had 10 minutes last night after the
kids went to bed, and my first attempt didn't work :-).

Jeremy

>
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
>
>
> On Thu, Dec 18, 2008 at 5:22 AM, Jeremy Hylton  wrote:
>> On Wed, Dec 17, 2008 at 1:05 PM, Guido van Rossum  wrote:
>>> The inheritance from io.RawIOBase seems fine.
>>
>> There is a small problem with the interaction between HTTPResponse and
>> RawIOBase, but I think the problem is more on the http side.  You may
>> recall that the HTTP code has a habit of closing the connection for
>> you.  In a variety of cases, once you've read the last bytes of the
>> response, the HTTPResponse object calls its own close() method.  This
>> interacts poorly with RawIOBase, because it raises a ValueError for
>> any operation on a closed io object.  This prevents iterators from
>> working correctly.  The iterator implementation expects the final call
>> to readline() to return an empty string and converts that to a
>> StopIteration.  Instead, it's seeing a ValueError that propagates out.
>>
>> It's always been odd to me that the connection closed itself.  It's
>> going to be tricky to fix the current bug (chunked responses) and keep
>> the self-closing behavior, but I worry that change the self-closing
>> behavior too dramatically isn't appropriate for a bug fix.  Will look
>> some more at this tomorrow.
>>
>> Jeremy
>>
>>> --Guido van Rossum (home page: http://www.python.org/~guido/)
>>>
>>>
>>>
>>> On Mon, Dec 15, 2008 at 11:19 AM, Jeremy Hylton  wrote:
>>>> I have a patch that appears to fix this bug
>>>> http://bugs.python.org/file12361/urllib-chunked.diff
>>>> but I'm not sure about its interaction with the io module and
>>>> RawIOBase.  Is there a new IO expert who could take a look at it for
>>>> me?
>>>>
>>>> Jeremy
>>>>
>>>> On Sun, Dec 14, 2008 at 11:06 PM, Jeremy Hylton  
>>>> wrote:
>>>>> This bug is pretty serious, because urllib will insert garbage into
>>>>> the application-visible data for a chunked response.  It simply
>>>>> ignores the fact that it's reading a chunked response and includes the
>>>>> chunked header data is payload data.  The original bug was reported in
>>>>> September, but no one noticed it.  It was reported again recently.
>>>>>
>>>>> http://bugs.python.org/issue3761
>>>>> http://bugs.python.org/issue4631
>>>>>
>>>>> I suspect we'd want to get a 3.0.1 out as soon as this is fixed, but
>>>>> that's not my call.
>>>>>
>>>>> Jeremy
>>>>>
>>>> ___
>>>> Python-Dev mailing list
>>>> Python-Dev@python.org
>>>> http://mail.python.org/mailman/listinfo/python-dev
>>>> Unsubscribe: 
>>>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>>
>>>
>>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3.0.1

[Jeremy Hylton]

4631 should be a release blocker.  I'll have a bit of time on Monday
and Tuesday to wrap it up.

Jeremy

On Fri, Dec 19, 2008 at 5:28 PM, Barry Warsaw  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> I'd like to get Python 3.0.1 out before the end of the year.  There are no
> showstoppers, but I haven't yet looked at the deferred blockers or the
> buildbots.
>
> Do you think we can get 3.0.1 out on December 24th?  Or should we wait until
> after Christmas and get it out, say on the 29th?  Do we need an rc?
>
> This question goes mostly to Martin and Georg.  What would work for you
> guys?
>
> - -Barry
>
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.9 (Darwin)
>
> iQCVAwUBSUwgEXEjvBPtnXfVAQIthgP7BDS6xfBHhADKc50ANvZ5aAfWhGSU9GH/
> DR+IRduVmvosu9gm92hupCOaLCN4IbtyFx27A8LQuPNVc4BVrhWfDKDSzpxO2MJu
> xLJntkF2BRWODSbdrLGdZ6H6WDT0ZAhn6ZjlWXwxhGxQ5FwEJb7moMuY7jAIEeor
> 5n6Ag5zT+e8=
> =oU/g
> -END PGP SIGNATURE-
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.4 news reaches interesting places

[Jeremy Hylton]

On Thu, 9 Dec 2004 14:16:18 -0500, James Y Knight <[EMAIL PROTECTED]> wrote:
> On Dec 9, 2004, at 1:31 PM, Phillip J. Eby wrote:
> > Isn't it funny, by the way, that people don't go around talking about
> > how slow Jython is?  At least, I haven't seen it to the extent that
> > I've seen with CPython.
> 
> People talk about how slow CPython is, is because they are writing code
> targeted for it that traditionally would have been written in C. Python
> *is* slower than C. You can try to deny it, but it is simple fact.
> However, python is a really nice language to program in. So people use
> it anyways. It is "fast enough" for many things.

I think you are exactly right.  It's easy to get the impression that
Python is slow when many programs written in Python *are* slowly than
the same program written in C, C++, or Java.  If you look are other
"dynamic" languages, you'll find that IronPython is often faster than
CPython and that Smalltalk and Common Lisp are often substantially
faster than Python.  Python isn't remarkably slow for a scripting
language, but it is not fast.

> There is currently a lot of work going on in the python community to
> make Python faster, which indicates both that Python is too slow for
> some people, and that it will be faster in the future. This is a good
> thing, and not something that should be shoved under the rug to try to
> pretend python is ultra mega super fast already.

I agree, although it's not clear to me how much faster it will be in
the future.  Making a *fast* Python based on our own virtual execution
environment (as opposed to piggybacking a JVM or CLR) is a big
project.  It's not clear than anyone has enough resources to make
credible effort, so we're left to piecemeal improvements.  The
improvement I see from 2.2 to 2.4 on my box is about 55% (measured
using pystone) over almost three years.

I think the right thing to do with PR is frame the question
differently.  We need to say that people use Python because it lets
them right clear, simple code or that it is easy to maintain or that
dynamic languages are excellent for prototyping -- whatever the
message is -- and simply avoid talking about speed.  There are a lot
of issues that affect the selection of a programming language, and
speed is not always the most important one.  It is easy for a
journalist to write about and it is easy to report (possibly
meaningless) measurements of speed.

Jeremy
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: 2.4 news reaches interesting places

[Jeremy Hylton]

On Fri, 10 Dec 2004 14:01:55 -0500, A.M. Kuchling <[EMAIL PROTECTED]> wrote:
> On Thu, Dec 09, 2004 at 05:11:04PM +0300, Oleg Broytmann wrote:
> > some popular areas. Let's add another topic, "Making things fast". Let's
> > even make it the first topic, though I personnaly dont see a need for
> > this.
> 
> The topic guides are migrating into the Wiki, and there's already a Wiki page
> about this: http://www.python.org/moin/PythonSpeed

The Wiki entry seems to reinforce the impression that bugged Guido to
begin with.  It provides a bunch of "but ..." explanations about why
Python's speed isn't that important.  Python is slow, but "speed of
development is far more important."

If you want to avoid people saying "Python is slow, but ..." you need
to give them a different message.  "Writing code quickly is crucial in
today's marketplace."  (whatever, that's not much of a message.) 
David's dynamic languages whitepaper has more of message, for example.

Jeremy
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: Re: 2.4 news reaches interesting places

[Jeremy Hylton]

On Mon, 13 Dec 2004 11:30:45 -0500 (EST), Stephan Deibel
<[EMAIL PROTECTED]> wrote:
> For example, a September article in InfoWorld said "But the big winner
> this time around is the object-oriented scripting language Python, which
> saw a 6 percent gain in popularity, almost doubling last year's results."
> 
> http://www.infoworld.com/article/04/09/24/39FErrdev_1.html?s=feature

Can we extrapolate from the numbers here to get an estimate of how
many Python developers there are?  I was asked for that number at
workshop a few months ago and I didn't have any idea how to answer. 
Is there a good answer?

Two possibilities come to mind.  1) 14% of developers in the survey
work at companies that use Python.  How many developers are there? 
Assume that 14% of them use Python.  But what's a good estimate for
"number of developers."  Pretty rough -- number of survey respondents
at company != number of Python programmers at company, and %age
companies != %age of programmers.  2) 64% of companies use Java, 4.5
times more than Python.  Find out how many Java programmers there are,
divide by 4.5.

Jeremy
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: Re: 2.4 news reaches interesting places

[Jeremy Hylton]

On Mon, 13 Dec 2004 13:11:38 -0500 (EST), Stephan Deibel
<[EMAIL PROTECTED]> wrote:
> On Mon, 13 Dec 2004, Jeremy Hylton wrote:
> > Two possibilities come to mind.  1) 14% of developers in the survey
> > work at companies that use Python.  How many developers are there?
> > Assume that 14% of them use Python.  But what's a good estimate for
> > "number of developers."  Pretty rough -- number of survey respondents
> > at company != number of Python programmers at company, and %age
> > companies != %age of programmers.
> 
> Supposedly there are 5-6 million developers world-wide (by various
> estimates; I've no idea whether to believe them).  If you just multiply
> out naively you get 700K-840K Python programmers.  There certainly are
> vastly more one person Python projects than large ones, so this may not be
> all that far off.
> 
> > 2) 64% of companies use Java, 4.5
> > times more than Python.  Find out how many Java programmers there are,
> > divide by 4.5.
> 
> I've seen claims of 3-4 million java programmers so that's 666K-888K
> Python programmers.  Interestingly, this matches up with the above abuse
> of statistics.

Thanks!  I didn't know the numbers from which to guestimate, and it's
nice that they are in agreement.  This seems like a satisfactory
back-of-the-envelope answer.

Jeremy
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80

[Jeremy Hylton]

On Mon, 20 Dec 2004 15:03:13 -0500, Barry Warsaw <[EMAIL PROTECTED]> wrote:
> On Mon, 2004-12-20 at 12:48, Raymond Hettinger wrote:
> 
> > Perhaps a rather quick Py2.4.1 would be in order.
> 
> +1

Nothing wrong with an incremental release, but none of these sound
like critical bugs to me.

Jeremy
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80

[Jeremy Hylton]

On Mon, 20 Dec 2004 18:25:39 -0500, Aahz <[EMAIL PROTECTED]> wrote:
> On Mon, Dec 20, 2004, Jeremy Hylton wrote:
> > On Mon, 20 Dec 2004 15:03:13 -0500, Barry Warsaw <[EMAIL PROTECTED]> wrote:
> >> On Mon, 2004-12-20 at 12:48, Raymond Hettinger wrote:
> >>>
> >>> Perhaps a rather quick Py2.4.1 would be in order.
> >>
> >> +1
> >
> > Nothing wrong with an incremental release, but none of these sound
> > like critical bugs to me.
> 
> You don't think a blowup in marshal is critical?  Mind expanding on
> that?

An undocumented extension to marshal causes a segfault.  It's
certainly a bug worth fixing.  It doesn't sound like a critical bug to
me.

Jeremy
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Patches: 1 for the price of 10.

[Jeremy Hylton]

On Sun, 19 Dec 2004 02:13:24 -0800, Titus Brown <[EMAIL PROTECTED]> wrote:
> Hello all,
> 
> per previous discussion,
> 
> http://mail.python.org/pipermail/python-dev/2004-October/049495.html
> 
> I'd like to push a trivial little patch to sgmllib (#1087808) on you
> gents, in exchange for my reviews & effort etc. on 10 other patches.

I got started on these this morning, will likely finish them tomorrow.
 It would be perverse to apply your patch last, wouldn't it?

Jeremy
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please help complete the AST branch

[Jeremy Hylton]

On Mon, 03 Jan 2005 18:02:52 -0800, Brett C. <[EMAIL PROTECTED]> wrote:
> Plus there is the running tradition of sprinting on the AST branch at PyCon.  
> I
> was planning on shedding my bug fixing drive at PyCon this year and sprinting
> with (hopefully) Jeremy, Neal, Tim, and Neil on the AST branch as a prep for
> working on it afterwards for my class credit.

I'd like to sprint on it before PyCon; we'll have to see what my
schedule allows.

> If anyone would like to see the current code, check out ast-branch from CVS
> (read the dev FAQ on how to check out a branch from CVS).  Read
> Python/compile.txt for an overview of how the thing works and such.
> 
> It will get done, just don't push for a 2.5 release within a month.  =)

I think the branch is in an awkward state, because of the new features
added to Python 2.4 after the AST branch work ceased.  The ast branch
doesn't handle generator expressions or decorators; extending the ast
to support them would be a good first step.

There are also the simple logistical questions of integrating changes.
 Since most of the AST branch changes are confined to a few files, I
suspect the best thing to do is to merge all the changes from the head
except for compile.c.  I haven't done a major CVS branch integrate in
at least nine months; if someone feels more comfortable with that, it
would also be a good step.

Perhaps interested parties should take up the discussion on the
compiler-sig.  I think we can recover the state of last May's effort
pretty quickly, and I can help outline the remaining work even if I
can't help much.  (Although I hope I can help, too.)

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please help complete the AST branch

[Jeremy Hylton]

The list archives look like they are mostly full of spam, but it's
also the only list we've used to discuss the ast work.  I haven't
really worried whether the sig was "active," as long as the list was
around.  I don't mind if you want to resurrect it.  Is there some way
to delete the spam from the archives?

By "discussion on" I meant a discussion of the remaining work.  I'm
not sure why you quoted just that part.  I was suggesting that there
is an ongoing discussion that should continue on the compiler-sig.

Jeremy



On Tue, 04 Jan 2005 07:43:28 -0500, Barry Warsaw <[EMAIL PROTECTED]> wrote:
> On Mon, 2005-01-03 at 23:17, Tony Meyer wrote:
> > > Perhaps interested parties should take up the discussion on
> > > the compiler-sig.
> >
> > This isn't listed in the 'currently active' SIGs list on
> >  - is it still active, or will it now be?  If so,
> > perhaps it should be added to the list?
> >
> > By 'discussion on', do you mean via the wiki at
> > ?
> 
> If compiler-sig is where ASTers want to hang out, I'd be happy to
> resurrect it.
> 
> -Barry
> 
> 
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please help complete the AST branch

[Jeremy Hylton]

That's fine with me.  We had taken it to the compiler-sig when it
wasn't clear there was interest in the ast branch :-).

Jeremy


On Tue, 4 Jan 2005 07:31:30 -0800, Guido van Rossum
<[EMAIL PROTECTED]> wrote:
> >I was suggesting that there
> > is an ongoing discussion that should continue on the compiler-sig.
> 
> I'd be fine with keeping this on python-dev too.
> 
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please help complete the AST branch

[Jeremy Hylton]

Does anyone want to volunteer to integrate the current head to the
branch?  I think that's a pretty important near-term step.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please help complete the AST branch

[Jeremy Hylton]

On Thu, 6 Jan 2005 01:32:44 +0900, Jiwon Seo <[EMAIL PROTECTED]> wrote:
> I'd like to help here on the AST branch, if it's not too late.
> (Especially I'm interested with the generator expression part.)

Great!  It's not too late.

> If I want to volunteer, do I just begin to work with it? Or do I need
> to read something or discuss with someone?

The file Python/compile.txt on the ast-branch has a brief overview of
the project:

http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Python/Attic/compile.txt?rev=1.1.2.8&only_with_tag=ast-branch&view=auto

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] ast branch pragmatics

[Jeremy Hylton]

The existing ast-branch mostly works, but it does not include most of
the new features of Python 2.4.  There is a substantial integration
effort, perhaps easy for someone who does a lot of CVS branch merges. 
(In particular, the head has already been merged to this branch once.)

I think it would be easier to create a new branch from the current
head, integrate the small number of changed files from ast-branch, and
work with that branch instead.  The idea is that it's an end-run
around doing an automatic CVS merge and relying on someone to manually
merge the changes.

At the same time, since there is a groundswell of support for
finishing the AST work, I'd like to propose that we stop making
compiler / bytecode changes until it is done.  Every change to
compile.c or the bytecode ends up creating a new incompatibilty that
needs to be merged.

If these two plans sound good, I'll get started on the new branch.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please help complete the AST branch

[Jeremy Hylton]

On Wed, 5 Jan 2005 13:28:11 -0500, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> Would it be helpful for me to move the peepholer out of compile.c into a
> separate source file?

It doesn't really matter.  There are two reasons.  1) We've been
working on the new compiler code in newcompile.c, rather than
compile.c.  When it is finished, we'll replace compile.c with
newcompile.c, but it was helpful to have both around at first.
2) Peephole optimizations would be done on the basic block
intermediate representation rather than code objects.  So we'll need
to rewrite it anyway to use the new IR.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please help complete the AST branch

[Jeremy Hylton]

On Wed, 05 Jan 2005 13:20:51 -0500, Kurt B. Kaiser <[EMAIL PROTECTED]> wrote:
> Jeremy Hylton <[EMAIL PROTECTED]> writes:
> 
> > Does anyone want to volunteer to integrate the current head to the
> > branch?  I think that's a pretty important near-term step.
> 
> I'll take a shot at it.

Great!  I say this after reading your other message in response to my
suggestion to create a new branch.  If you can manage to do the
integration, it's simpler for everyone to stick to a single branch. 
(For example, there will be no opportunity for someone to work on the
wrong branch.)
 
> 29 conflicts

Oh.  That's not as bad as I expected.
 
> Major conflict:
> Python/compile.c(Probably not merged during 1st merge)

I think that's right.  I didn't merge any of the changes, then.

> Lib/test/test_compile.c (ditto)

Probably.

> Lib/test/test_os.py (AST?)
> Lib/test/test_re.py (AST?)

I wonder if these two were edited to worm around some bugs in early
versions of newcompile.c.  You could check the revision history.  If
that's the case, it's safe to drop the changes.

> Major conflict probably not AST related:
> Lib/test/test_bool.py
> Lib/test/test_urllib.py
> Lib/test/output/test_profile
> Python/pythonrun.c (check brackets!)

There are actually a lot of AST-related changes in pythonrun.c,
because it is the gunk between files and stdin and the actual compiler
and runtime.
 
Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: [Python-checkins] python/dist/src/Python pythonrun.c, 2.161.2.15, 2.161.2.16

[Jeremy Hylton]

How's the merge going?

And if I haven't already said thanks, then, thanks for doing it!

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Python pythonrun.c, 2.161.2.15, 2.161.2.16

[Jeremy Hylton]

On Fri, 07 Jan 2005 16:18:11 -0500, Kurt B. Kaiser <[EMAIL PROTECTED]> wrote:
> Looks like it's done.  I tagged ast-branch when I finished:
> 
> merged_from_MAIN_07JAN05
> 
> Right now I'm trying to get Python-ast.c to compile.  It wasn't
> modified by the merge, so there's some other issue.

I'm getting a compilation failure in symtable.c:

 gcc -pthread -c -fno-strict-aliasing -g -Wall -Wstrict-prototypes -I.
-I../Include   -DPy_BUILD_CORE -o Python/symtable.o
../Python/symtable.c
../Python/symtable.c: In function `symtable_new':
../Python/symtable.c:193: structure has no member named `st_tmpname'

Do you see that?

There is this one ugly corner of Python-ast.c.  There's a routine that
expects to take a pointer to a node, but instead gets passed an int. 
The generated code is bogus, and I haven't decided if it needs to be
worried about.  You need to manually edit the generated code to add a
cast.

> > And if I haven't already said thanks, then, thanks for doing it!
> 
> You're welcome!  I volunteer to keep ast-branch synch'd, how often
> do you want to do it?

I don't think we'll need to merge again.  This last merge got all the
language changes that were made for 2.4.  Since we've agreed to a
moratorium on more compiler/bytecode changes, we shouldn't need to
merge from the head again.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: [Python-checkins] python/dist/src/Python ceval.c, 2.420, 2.421

[Jeremy Hylton]

On Tue, 18 Jan 2005 07:56:19 -0800, [EMAIL PROTECTED]
<[EMAIL PROTECTED]> wrote:
> Update of /cvsroot/python/python/dist/src/Python
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4034/Python
> 
> Modified Files:
> ceval.c
> Log Message:
> Change the name of the macro used by --with-tsc builds to the less
> inscrutable READ_TIMESTAMP.

An obvious improvement.  Thanks!

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Misc NEWS, 1.1237, 1.1238

[Jeremy Hylton]

On Fri, 4 Feb 2005 10:46:52 -0800, Guido van Rossum
<[EMAIL PROTECTED]> wrote:
> [EMAIL PROTECTED]
> > Log Message:
> > Add NEWS item about future parser bug.
> 
> Give back the time machine!

I already will have by the time you needed it.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Python future.c, 2.14, 2.15

[Jeremy Hylton]

On Sat, 5 Feb 2005 02:31:26 -0500, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> [Anthony]
> > While this is undoubtedly a bug fix, I'm not sure that it should be
> > backported - it will break people's code that is "working" now (albeit
> > in a faulty way). What do people think?
> 
> I concur -- the balance of risks is towards the patch causing more harm
> than good.

I would not backport it to Python 2.3.  People have been using it for
a long time.  I'd be inclined to backport it to Python 2.4, which is
still relatively new.  If someone has buggy code, an upgrade is going
to cause a problem for them at some point.  Given how unlikely the
risk is -- particularly given that division is the only useful future
now -- I'd say the risk is acceptable for Python 2.4.1.  (Unlike, say,
Python 2.4.2.)

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] license issues with profiler.py and md5.h/md5c.c

[Jeremy Hylton]

Maybe some ambitious PSF activitst could contact Roskind and Steve
Kirsch and see if they know who at Disney to talk to...  Or maybe the
Disney guys who were at PyCon last year could help.

Jeremy


On Tue, 8 Feb 2005 15:37:50 -0500, Tim Peters <[EMAIL PROTECTED]> wrote:
> [Matthias Klose]
> > A Debian user pointed out (http://bugs.debian.org/293932), that the
> > current license for the Python profiler is not conforming to the DFSG
> > (Debian free software guidelines).
> >
> > http://www.python.org/doc/current/lib/node829.html states
> >
> >  "This permission is explicitly restricted to the copying and
> >  modification of the software to remain in Python, compiled Python,
> >  or other languages (such as C) wherein the modified or derived code
> >  is exclusively imported into a Python module."
> ...
> > - Does somebody knows about the history of this license, why it is
> >  more restricted than the Python license?
> 
> Simply because that's the license Jim Roskind slapped on it when he
> contributed this code 10 years ago.  I imagine (but don't know) that
> Guido looked at it, thought "hmm -- shouldn't be a problem for
> Python's users", and so accepted it.
> 
> > - Is there a chance to change the license for these two modules
> >  (profile.py, pstats.py)?
> 
> Not unless some remnant of InfoSeek Corp can be found, since they're
> the copyright holder (their work, their license).  Alas, Jim Roskind
> hasn't been seen in the Python world this century.
> 
> OTOH, if InfoSeek has vanished, it's unlikely they'll be suing anyone.
>  Given how Python-specific profile.py and pstats.py are, it's hard for
> me to imagine anyone wanting to make a derivative that isn't imported
> into a Python module.  In that respect it seems like a license clause
> that forbids you to run the software while the tip of your tongue is
> licking the back of your own neck.
> 
> Still, if that matters, perhaps Debian will need to leave these
> modules out.  Bold  users will still be able to grab them from
> any number of other places.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] license issues with profiler.py and md5.h/md5c.c

[Jeremy Hylton]

On Fri, 11 Feb 2005 12:55:02 +0100, Matthias Klose <[EMAIL PROTECTED]> wrote:
> > Currently md5c.c is included in the python sources. The libmd
> > implementation has a drop in replacement for md5c.c. The openssl
> > implementation is a complicated tangle of Makefile expanded template
> > code that would be harder to include in the Python sources.
> 
> I would prefer that one as a short term solution. Patch at #1118602.

Unfortunately a license that says it is in the public domain is
unacceptable (and should be for Debian, too).  That is to say, it's
not possible for someone to claim that something they produce is in
the public domain.  See http://www.linuxjournal.com/article/6225

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adding any() and all()

[Jeremy Hylton]

On Fri, 11 Mar 2005 07:19:56 -0800, Guido van Rossum
<[EMAIL PROTECTED]> wrote:
> > Personally, I think Python has too many builtins already.
> 
> It has fewer than most dynamic languages; and remember that I'm
> trading product(), any(), all() for reduce(), map() and filter().
> There are others slated for disappearance, too.

I think I'd miss reduce, but I'd be happy to have it moved to an
extension module.  There's no need for reduce to be a builtin.  I'd be
very happy to have any and all around.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: RELEASED Python 2.4.1, release candidate 1

[Jeremy Hylton]

I seem to have a problem with the install on XP SP1.  Python itself is
installed, but IDLE won't start.  The error says: "IDLE's subprocess
didn't make connection.  Either IDLE can't start a subprocess or
personal firewall software is blocking the connection."  I believe the
problem is the firewall, but I'm not sure if it is related to the
install.  The previous install (Python 2.3) worked fine.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] thread semantics for file objects

[Jeremy Hylton]

Are the thread semantics for file objecst documented anywhere?  I
don't see anything in the library manual, which is where I expected to
find it.  It looks like read and write are atomic by virtue of fread
and fwrite being atomic.

I'm less sure what guarantees, if any, the other methods attempt to
provide.  For example, it looks like concurrent calls to writelines()
will interleave entire lines, but not parts of lines.  Concurrent
calls to readlines() provide insane results, but I don't know if
that's a bug or a feature.  Specifically, if your file has a line that
is longer than the internal buffer size SMALLCHUNK you're likely to
get parts of that line chopped up into different lines in the
resulting return values.

If we can come up with intended semantics, I'd be willing to prepare a
patch for the documentation.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] thread semantics for file objects

[Jeremy Hylton]

On Thu, 17 Mar 2005 16:25:44 -0500, Aahz <[EMAIL PROTECTED]> wrote:
> On Thu, Mar 17, 2005, Jeremy Hylton wrote:
> >
> > Are the thread semantics for file objecst documented anywhere?  I
> > don't see anything in the library manual, which is where I expected to
> > find it.  It looks like read and write are atomic by virtue of fread
> > and fwrite being atomic.
> 
> Uncle Timmy will no doubt agree with me: the semantics don't matter.
> NEVER, NEVER access the same file object from multiple threads, unless
> you're using a lock.  And even using a lock is stupid.

I'm not looking for your permission or approval.  I just want to know
what semantics are intended.  If the documentation wants to say that
the semantics are undefined that okay, although I think we need to say
more because some behavior has been provided by the implementation for
a long time.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] thread semantics for file objects

[Jeremy Hylton]

On Thu, 17 Mar 2005 23:04:16 +0100, "Martin v. Löwis"
<[EMAIL PROTECTED]> wrote:
> Jeremy Hylton wrote:
> >>>Are the thread semantics for file objecst documented anywhere?  I
> >>>don't see anything in the library manual, which is where I expected to
> >>>find it.  It looks like read and write are atomic by virtue of fread
> >>>and fwrite being atomic.
> >>
> >>Uncle Timmy will no doubt agree with me: the semantics don't matter.
> >>NEVER, NEVER access the same file object from multiple threads, unless
> >>you're using a lock.  And even using a lock is stupid.
> >
> >
> > I'm not looking for your permission or approval.
> 
> Literally, the answer to your question is "no". In fact, Python does not
> specify *any* interleaving semantics for threads whatsoever. The only
> statement to this respect is

I'm surprised that it does not, for example, guarantee that reads and
writes are atomic, since CPython relies on fread and fwrite which are
atomic.

Also, there are other operations that go to the trouble of calling
flockfile().  What's the point if we don't provide any guarantees?
<0.6 wink>.  If it is not part of the specified behavior, then I
suppose it's a quality of implementation issue.  Either way it would
be helpful if the Python documentation said something, e.g. you can
rely on readline() being threadsafe or you can't but the current
CPython implementation happens to be.

readline() seemed like an interesting case because readlines() doesn't
have the same implementation and the behavior is different.  So, as
another example, you could ask whether readlines() has a bug or not.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] thread semantics for file objects

[Jeremy Hylton]

On Thu, 17 Mar 2005 17:13:05 -0500, Tim Peters <[EMAIL PROTECTED]> wrote:
> [Jeremy Hylton]
> > Are the thread semantics for file objecst documented anywhere?
> 
> No.  At base level, they're inherited from the C stdio implementation.
>  Since the C standard doesn't even mention threads, that's all
> platform-dependent.  POSIX defines thread semantics for file I/O, but
> fat lot of good that does you on Windows, etc.

Fair enough.  I didn't consider Windows at all or other non-POSIX platforms.  

> 
> > I don't see anything in the library manual, which is where I expected to
> > find it.  It looks like read and write are atomic by virtue of fread
> > and fwrite being atomic.
> 
> I wouldn't consider this as more than CPython implementation accidents
> in the cases it appears to apply.  For example, in universal-newlines
> mode, are you sure f.read(n) always maps to exactly one fread() call?

Universal newline reads and get_line() both lock the stream if the
platform supports it.  So I expect that they are atomic on those
platforms.

But it certainly seems safe to conclude this is a quality of
implementation issue.  Otherwise, why bother with the flockfile() at
all, right?  Or is there some correctness issue I'm not seeing that
requires the locking for some basic safety in the implementation.

> And even using a lock is stupid.
> 
> ZODB's FileStorage is bristling with locks protecting multi-threaded
> access to file objects, therefore that can't be stupid.  QED

Using a lock seemed like a good idea there and still seems like a good
idea now :-).

jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] thread semantics for file objects

[Jeremy Hylton]

On Thu, 17 Mar 2005 23:57:52 +0100, "Martin v. Löwis"
<[EMAIL PROTECTED]> wrote:
> Remember, you were asking what behaviour is *documented*, not what
> behaviour is guaranteed by the implementation (in a specific version
> of the implementation).

Martin,

I think you're trying to find more finesse in my question than I ever
intended.  I intended to ask -- hey, what are the semantics we intend
in this case?  since the documentation doesn't say, we could improve
them by capturing the intended semantics.

> > Also, there are other operations that go to the trouble of calling
> > flockfile().  What's the point if we don't provide any guarantees?
> 
> Because nobody cares about guarantees in the documentation. Instead,
> people care about observable behaviour. So if you get a crash due to a
> race condition, you care, you report a bug, the Python developer agrees
> its a bug, and fixes it by adding synchronization.

As Tim later reported this wasn't to address a crash, but to appease a
pig headed developer :-).  I'm surprised by your claim that whether
something is a bug depends on the person who reviews it.  In practice,
this may be the case, but I've always been under the impression that
there was rough consensus about what constituted a bug and what a
feature.  I'd certainly say its a goal to strive for.

It sounds like the weakest intended behavior we have is the one Tim
reported:  "provided the platform C stdio wasn't thread-braindead,
then if you had N threads all simultaneously reading a file object
containing B bytes, while nobody wrote to that file object, then the
total number of bytes seen by all N threads would sum
to B at the time they all saw EOF."  It seems to me like a good idea
to document this intended behavior somewhere.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] thread semantics for file objects

[Jeremy Hylton]

On Fri, 18 Mar 2005 07:57:25 +0100, "Martin v. Löwis"
<[EMAIL PROTECTED]> wrote:
> Writing down all these properties does little good, IMO. This includes
> your proposed property of file reads: anybody reading your statement
> will think "of course it works this way - why even mention it".

The thingsa that are so obvious they don't need to be written down are
often the most interesting things to write down.  In fact, you started
the thread by saying there were no guarantees whatsoever and chiding
me for asking if there were any.  But it seems there are some intended
semantics that are strong than what you would find in C or Perl. 
Hence, I don't think they would be obvious to anyone who comes to
Python from one of those languages.

I agree that the semantics of multi-threaded Python programs is an
enormous domain and we're discussing a tiny corner of it.  I agree
that it would be quite challenging to get better documentation or
specifications here.  But I also think that every little bit helps.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/distutils/tests test_dist.py, 1.1, 1.2

[Jeremy Hylton]

On Mon, 21 Mar 2005 16:08:57 +0100, Thomas Heller <[EMAIL PROTECTED]> wrote:
> [EMAIL PROTECTED] writes:
> 
> > PEP 314 implementation (client side):
> 
> I'm not sure where I should post this, but shouldn't there be a way to
> specify the encoding of the metadata?  There are people (not me,
> fortunately), with umlauts in their names, for example.

UTF-8 as the default would accommodate most uses, but it does seem
essential to have some way of specifying an encoding.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Shorthand for lambda

[Jeremy Hylton]

For filter and map, list comprehensions and generator expressions are
the answer.

>>> numbers = [5, 9, 56, 34, 1, 24, 37, 89]
>>> [x for x in numbers if x < 30]
[5, 9, 1, 24]
>>> (x for x in numbers if x < 30)

>>> list(_)
[5, 9, 1, 24]

Jeremy

On Wed, 23 Mar 2005 10:33:53 -0600 (CST), Ka-Ping Yee
<[EMAIL PROTECTED]> wrote:
> Hey folks,
> 
> I'm sitting over here in the AppleScript talk and Jacob is explaining a
> module called 'appscript' that interfaces to the Apple Events system.
> 
> What caught my eye was this example:
> 
> from appscript import *
> ab = app('Address Book')
> people = ab.people.filter(its.emails != [])
> 
> That last line asks the Address Book to select only entries with
> e-mail addresses.  The weird 'its' object comes from the appscript
> module -- asking for its properties and using operators causes it
> to set up thunks for you.
> 
> It dawned on me that you could use this idea to make the whole
> filter/lambda experience vastly more pleasant.  I whipped up a quick
> implementation:
> 
> >>> from placeholder import _
> >>> numbers = [5, 9, 56, 34, 1, 24, 37, 89]
> >>> filter(_ < 30, numbers)
> [5, 9, 1, 24]
> >>> map(_ + 10, numbers)
> [15, 19, 66, 44, 11, 34, 47, 99]
> >>>
> 
> Look ma, no lambdas!
> 
> I bet someone has already done this before, right?
> 
> -- ?!ng
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Developer list update

[Jeremy Hylton]

On Apr 8, 2005 9:31 AM, Fred Drake <[EMAIL PROTECTED]> wrote:
> On Thursday 07 April 2005 10:58, Raymond Hettinger wrote:
>  >   Eric Price
> 
> Eric Price was an intern at CNRI; I think it's safe to remove him from the
> list, as I've not seen anything from him in a *long* time.

Eric Price did some of the work on the decimal package, which was only
two summers ago.  He wasn't an intern at CNRI.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] AST manipulation and source code generation

[Jeremy Hylton]

On 5/24/05, Ka-Ping Yee <[EMAIL PROTECTED]> wrote:
> Would there be any interest in extending the compiler package with tools
> for AST transformations and for emitting Python source code from ASTs?

Sure.  Eventually, we'll have to figure out how to unify the compiler
package AST and the ast-branch AST, but don't let that delay you now.

> I was experimenting with possible translations for exception chaining
> and wanted to run some automated tests, so i started playing around
> with the compiler package to do source-to-source transformations.
> Then i started working on a way to do template-based substitution of
> ASTs and a way to spit source code back out, and i'm wondering if
> that might be good for experimenting with future Python features.
> 
> (If there's already stuff out there for doing this, let me know --
> i don't intend to duplicate existing work.)

I don't know of any existing work, but it certainly sounds useful.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] AST manipulation and source code generation

[Jeremy Hylton]

On 5/26/05, Chad Whitacre <[EMAIL PROTECTED]> wrote:
> > Would there be any interest in extending the compiler package with tools
> > for AST transformations and for emitting Python source code from ASTs?
> 
> Heh, so I guess the answer is "yes."
> 
> BTW, how does the concept of AST transformations relate to the concept
> of (Lisp) macros? Am I right to think that they are similar?

I think they are similar, but two key differences are:

 - An AST transformation can transform existing syntax but doesn't allow you
   to create new syntax.

 - An AST transformation has to be explicitly invoked.  A macro is
part of the language
   proper and has a semantics for how and when macros are evaluated.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] AST manipulation and source code generation

[Jeremy Hylton]

On 5/26/05, Chad Whitacre <[EMAIL PROTECTED]> wrote:
> chad: Hmmm ... I don't follow python-dev closely but hasn't there been
> resistance to macros in Python? Are we saying macros may be a good idea
> after all?
> 
> ?!ng: resistance -> Yes.
> ?!ng: good idea -> Not really.  AST transformations are useful for
> experimenting with the language, but i don't think there is much
> enthusiasm for making these transformations a normal part of the way
> most programs are written.

Right.  We fear macros and prefer to suffer getattr hooks,
descriptors, and decorators .

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Baffled by PyArg_ParseTupleAndKeywords modification

[Jeremy Hylton]

It sounds like the right answer for Python is to change the signature
of PyArg_ParseTupleAndKeywords() back.  We'll fix it when C fixes its
const rules .

Jeremy

On 2/13/06, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> On 2/12/06, Greg Ewing <[EMAIL PROTECTED]> wrote:
> >  > [A large head-exploding set of rules]
> >
> > Blarg.
> >
> > Const - Just Say No.
>
> +1
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Baffled by PyArg_ParseTupleAndKeywords modification

[Jeremy Hylton]

On 2/10/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> Jeremy Hylton wrote:
> > Ok.  I reviewed the original problem and you're right, the problem was
> > not that it failed outright but that it produced a warning about the
> > deprecated conversion:
> > warning: deprecated conversion from string constant to 'char*''
> >
> > I work at a place that takes the same attitude as python-dev about
> > warnings:  They're treated as errors and you can't check in code that
> > the compiler generates warnings for.
>
> In that specific case, I think the compiler's warning should be turned
> off; it is a bug in the compiler if that specific warning cannot be
> turned off separately.

The compiler in question is gcc and the warning can be turned off with
-Wno-write-strings.  I think we'd be better off leaving that option
on, though.  This warning will help me find places where I'm passing a
string literal to a function that does not take a const char*.  That's
valuable, not insensate.

Jeremy

> While it is true that the conversion is deprecated, the C++ standard
> defines this as
>
> "Normative for the current edition of the Standard, but not guaranteed
> to be part of the Standard in future revisions."
>
> The current version is from 1998. I haven't been following closely,
> but I believe there are no plans to actually remove the feature
> in the next revision.
>
> FWIW, Annex D also defines these features as deprecated:
> - the use of "static" for objects in namespace scope (AFAICT
>   including C file-level static variables and functions)
> - C library headers (i.e. )
>
> Don't you get a warning when including Python.h, because that
> include ?
>
> > Nonetheless, the consensus on the c++ sig and python-dev at the time
> > was to fix Python.  If we don't allow warnings in our compilations, we
> > shouldn't require our users at accept warnings in theirs.
>
> We don't allow warnings for "major compilers". This specific compiler
> appears flawed (or your configuration of it).
>
> Regards,
> Martin
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] http://www.python.org/dev/doc/devel still available

[Jeremy Hylton]

On 2/13/06, Fred L. Drake, Jr. <[EMAIL PROTECTED]> wrote:
> On Monday 13 February 2006 15:40, Guido van Rossum wrote:
>  > Shouldn't docs.python.org be removed? It seems to add mroe confusion
>  > than anything, especially since most links on python.org continue to
>  > point to python.org/doc/.
>
> docs.python.org was created specifically to make searching the most recent
> "stable" version of the docs easier (using Google's site: modifier, no less).
> I don't know what the link count statistics say (other than what you
> mention), and don't know which gets hit more often, but I still think it's a
> reasonable approach.

Why not do a query like this?
http://www.google.com/search?q=site%3Apython.org/doc/current%20urllib&hl=en

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Baffled by PyArg_ParseTupleAndKeywords modification

[Jeremy Hylton]

On 2/14/06, M.-A. Lemburg <[EMAIL PROTECTED]> wrote:
> Martin v. Löwis wrote:
> > M.-A. Lemburg wrote:
> >>> It's the consequences:  nobody complains about tacking "const" on to a
> >>> former honest-to-God "char *" argument that was in fact not modified,
> >>> because that's not only helpful for C++ programmers, it's _harmless_
> >>> for all programmers.  For example, nobody could sanely object (and
> >>> nobody did :-)) to adding const to the attribute-name argument in
> >>> PyObject_SetAttrString().  Sticking to that creates no new problems
> >>> for anyone, so that's as far as I ever went.
> >>
> >> Well, it broke my C extensions... I now have this in my code:
> >>
> >> /* The keyword array changed to const char* in Python 2.5 */
> >> #if PY_VERSION_HEX >= 0x0205
> >> # define Py_KEYWORDS_STRING_TYPE const char
> >> #else
> >> # define Py_KEYWORDS_STRING_TYPE char
> >> #endif
> >> ...
> >> static Py_KEYWORDS_STRING_TYPE *kwslist[] = {"yada", NULL};
> >> ...
> >
> > You did not read Tim's message carefully enough. He wasn't talking
> > about PyArg_ParseTupleAndKeywords *at all*. He only talked about
> > changing char* arguments to const char*, e.g. in
> > PyObject_SetAttrString. Did that break your C extensions also?
>
> I did read Tim's post: sorry for phrasing the reply the way I did.
>
> I was referring to his statement "nobody complains about tacking "const"
> on to a former honest-to-God "char *" argument that was in fact not
> modified".
>
> Also: it's not me complaining, it's the compilers !

Tim was talking about adding const to a char* not adding const to a
char** (note the two stars).  The subsequent discussion has been about
the different way those are handled in C and C++ and a general
agreement that the "const char**" has been a bother for people.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Baffled by PyArg_ParseTupleAndKeywords modification

[Jeremy Hylton]

On 2/14/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> Jeremy Hylton wrote:
> > The compiler in question is gcc and the warning can be turned off with
> > -Wno-write-strings.  I think we'd be better off leaving that option
> > on, though.  This warning will help me find places where I'm passing a
> > string literal to a function that does not take a const char*.  That's
> > valuable, not insensate.
>
> Hmm. I'd say this depends on what your reaction to the warning is.
> If you sprinkle const_casts in the code, nothing is gained.

Except for the Python APIs, we would declare the function as taking a
const char* if took a const char*.  If the function legitimately takes
a char*, then you have to change the code to avoid a segfault.

> Perhaps there is some value in finding functions which ought to expect
> const char*. For that, occasional checks should be sufficient; I cannot
> see a point in having code permanently pass with that option. In
> particular not if you are interfacing with C libraries.

I don't understand what you mean:  I'm not sure what you mean by
"occasional checks" or "permanently pass".  The compiler flags are
always the same.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] C AST to Python discussion

[Jeremy Hylton]

I am still -1 on the ast-objects branch.  It adds a lot of boilerplate
code and its makes complicated what is now simple.  I'll see if I can
get a rough cut of the marshal code ready today, so there will be a
complete implementation of my original plan.

I also think we should keep the transformation api simple.  If we
provide an extension module, along the lines of the parser module,
users can write transformations with that module.  They can also write
their own wrapper script that runs a script after applying
transformations.

I agree that the question of saved bytecode files still needs to be
resolved.  I'm not sure that extending the bytecode format to record
modifications is enough, since you also have a filename problem:  How
do you manage two versions of a module, one compiled with
transformation and one compiled without?

How about we arrange for some open space time at PyCon to discuss? 
Unfortunately, the compiler talk isn't until the last day and I can't
stay for sprints.  It would be better to have the talk, then the open
space, then the sprint.

Jeremy

On 2/15/06, Simon Burton <[EMAIL PROTECTED]> wrote:
> On Wed, 15 Feb 2006 00:34:35 -0800
> Brett Cannon <[EMAIL PROTECTED]> wrote:
>
> > As per Neal's prodding email, here is a thread to discuss where we
> > want to go with the C AST to Python stuff and what I think are the
> > core issues at the moment.
> >
> > First issue is the ast-objects branch.  Work is being done on it, but
> > it still leaks some references (Neal or Martin can correct me if I am
> > wrong).
>
> I've been doing the heavy lifting on ast-objects the last few weeks.
> Today it finally passed the python test suite. The last thing to do is
> the addition of XDECREF's, so yes, it is leaking a lot of references.
>
> I won't make it to PyCon (it's a long way for me to come), but gee I've left
> all the fun stuff for you to do !
> :)
>
> Even if AST transforms are not allowed, I see it as the strongest form of
> code reflection, and long over-due in python.
>
> Simon.
>
>
> --
> Simon Burton, B.Sc.
> Licensed PO Box 8066
> ANU Canberra 2601
> Australia
> Ph. 61 02 6249 6940
> http://arrowtheory.com
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] http://www.python.org/dev/doc/devel still available

[Jeremy Hylton]

As I said in an earlier message, there's no need to have a separate
domain to restrict queries to just the doc/current part of python.org.
 Just type
"site:python.org/doc/current your query here"

If there isn't any other rationale, maybe we can redirects
docs.python.org back to www.python.org?

Jeremy

On 2/15/06, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> On 2/15/06, Tim Parkin <[EMAIL PROTECTED]> wrote:
> > Guido van Rossum wrote:
> >
> > > (Now that I work for Google I realize more than ever before the
> > > importance of keeping URLs stable; PageRank(tm) numbers don't get
> > > transferred as quickly as contents. I have this worry too in the
> > > context of the python.org redesign; 301 permanent redirect is *not*
> > > going to help PageRank of the new page.)
>
> > Could you expand on why 301 redirects won't help with the transfer of
> > page rank (if you're allowed)? We've done exactly this on many sites and
> > the pagerank (or more relevantly the search rankings on specific terms)
> > has transferred almost overnight. The bigger pagerank updates (both
> > algorithm changes and overhauls in approach) seem to only happen every
> > few months and these also seem to take notice of 301 redirects (they
> > generally clear up any supplemental results).
>
> OK, perhaps I stand corrected. I don't actually know that much about PageRank!
>
> I still don't like docs.python.org, and adding more like it seems a
> mistake; but it's possible that this is because of a poor execution of
> the idea (there's no "search docs" button near the search button on
> the old python.org).
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] http://www.python.org/dev/doc/devel still available

[Jeremy Hylton]

I don't think this message is on-topic for python-dev.  There are lots
of great places to discuss the design of the python web site, but the
list for developers doesn't seem like a good place for it.  Do we need
a different list for people to gripe^H^H^H^H^H discuss the web site?

Jeremy

On 2/15/06, Fredrik Lundh <[EMAIL PROTECTED]> wrote:
> Tim Parkin wrote:
>
> > As for cutting edge, it's using twisted, restructured text, nevow, clean
> > urls, xhtml, semantic markup, css2, interfaces, adaption, eggs, the path
> > module, moinmoin, yaml (to avoid xml),
>
> that's not cutting edge, that's buzzword bingo.
>
> > something I don't think I would be qualified to judge,never mind receive).
>
> no, you're not qualified.  yet, someone gave you total control over the
> future of python.org, and there's no way to make you give it up, despite
> the fact that you're over a year late and the stuff you've delivered this
> far is massively underwhelming.  that's the problem.
>
> 
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] C AST to Python discussion

[Jeremy Hylton]

On 2/16/06, Greg Ewing <[EMAIL PROTECTED]> wrote:
> Whereas if there weren't any funky coding style in the
> first place, more potential compiler hackers might be
> encouraged to have a go.

I'm trying to make the code simple.  The style of code is different
than other parts of Python, but a compiler is different than a
bytecode engine or implementations of basic types.  Different problem
domains lead to different program structure.

> Also I'm still wondering why we're going to all this effort
> to build a whole new AST and compiler structure if the
> purpose isn't to *avoid* all this transformation between
> different representations.

The goal is to get the right representation for the problem.  It was
harder to understand and modify the compiler when it worked on the
concrete parse trees.  The compiler now has a couple of abstractions
that are well suited to particular phases of compilation.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.5 release schedule

[Jeremy Hylton]

It is critical, but I hadn't seen the bug report.  Feel free to assign
AST bugs to me and assign them a > 5 priority.

Jeremy

On 2/17/06, Armin Rigo <[EMAIL PROTECTED]> wrote:
> Hi,
>
> On Tue, Feb 14, 2006 at 09:24:57PM -0800, Neal Norwitz wrote:
> > http://www.python.org/peps/pep-0356.html
>
> There is at least one SF bug, namely "#1333982 Bugs of the new AST
> compiler", that in my humble opinion absolutely needs to be fixed before
> the release, even though I won't hide that I have no intention of fixing
> it myself.  Should I raise the issue here in python-dev, and see if we
> agree that it is critical?
>
> (Sorry if I should know about the procedure.  Does it then go in the
> PEP's Planned Features list?)
>
>
> A bientot,
>
> Armin
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.5 release schedule

[Jeremy Hylton]

Actually, it might be easier to assign separate bugs.  A number of the
old bugs appear to have been fixed.  It's hard to track individual
items within a bug report.

Jeremy

On 2/17/06, Jeremy Hylton <[EMAIL PROTECTED]> wrote:
> It is critical, but I hadn't seen the bug report.  Feel free to assign
> AST bugs to me and assign them a > 5 priority.
>
> Jeremy
>
> On 2/17/06, Armin Rigo <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > On Tue, Feb 14, 2006 at 09:24:57PM -0800, Neal Norwitz wrote:
> > > http://www.python.org/peps/pep-0356.html
> >
> > There is at least one SF bug, namely "#1333982 Bugs of the new AST
> > compiler", that in my humble opinion absolutely needs to be fixed before
> > the release, even though I won't hide that I have no intention of fixing
> > it myself.  Should I raise the issue here in python-dev, and see if we
> > agree that it is critical?
> >
> > (Sorry if I should know about the procedure.  Does it then go in the
> > PEP's Planned Features list?)
> >
> >
> > A bientot,
> >
> > Armin
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: 
> > http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
> >
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP for Better Control of Nested Lexical Scopes

[Jeremy Hylton]

Almann,

The lack of support for rebinding names in enclosing scopes is
certainly a wart.  I think option one is a better fit for Python,
because it more closely matches the existing naming semantics.  Namely
that assignment in a block creates a new name unless a global
statement indicates otherwise.  The revised rules would be that
assignment creates a new name unless a global or XXX statement
indicates otherwise.

The names of naming statements are quite hard to get right, I fear.  I
don't particularly like "use."  It's too generic.  (I don't
particularly like "scope" for option 2, either, for similar reasons. 
It doesn't indicate what kind of scope issue is being declared.)  The
most specifc thing I can think of is "free" to indicate that the
variable is free in the current scope.  It may be too specialized a
term to be familiar to most people.

I think free == global in the absence of other bindings.

Jeremy

On 2/20/06, Almann T. Goo <[EMAIL PROTECTED]> wrote:
> I am considering developing a PEP for enabling a mechanism to assign to free
> variables in a closure (nested function).  My rationale is that with the
> advent of PEP 227 , Python has proper nested lexical scopes, but can have
> undesirable behavior (especially with new developers) when a user makes
> wants to make an assignment to a free variable within a nested function.
> Furthermore, after seeing numerous kludges to "solve" the problem with a
> mutable object, like a list, as the free variable do not seem "Pythonic."  I
> have also seen mention that the use of classes can mitigate this, but that
> seems, IMHO, heavy handed in cases when an elegant solution using a closure
> would suffice and be more appropriate--especially when Python already has
> nested lexical scopes.
>
>  I propose two possible approaches to solve this issue:
>
>  1.  Adding a keyword such as "use" that would follow similar semantics as
> "global" does today.  A nested scope could declare names with this keyword
> to enable assignment to such names to change the closest parent's binding.
> The semantic would be to keep the behavior we experience today but tell the
> compiler/interpreter that a name declared with the "use" keyword would
> explicitly use an enclosing scope.  I personally like this approach the most
> since it would seem to be in keeping with the current way the language works
> and would probably be the most backwards compatible.  The semantics for how
> this interacts with the global scope would also need to be defined (should
> "use" be equivalent to a global when no name exists all parent scopes, etc.)
>
>
> def incgen( inc = 1 ) :
>a = 6
>def incrementer() :
>  use a
>  #use a, inc <-- list of names okay too
>  a += inc
>  return a
>return incrementer
>
>  Of course, this approach suffers from a downside that every nested scope
> that wanted to assign to a parent scope's name would need to have the "use"
> keyword for those names--but one could argue that this is in keeping with
> one of Python's philosophies that "Explicit is better than implicit" (PEP
> 20).  This approach also has to deal with a user declaring a name with "
> use" that is a named parameter--this would be a semantic error that could be
> handled like "global " does today with a SyntaxError.
>
>  2.  Adding a keyword such as "scope" that would behave similarly to
> JavaScript's " var" keyword.  A name could be declared with such a keyword
> optionally and all nested scopes would use the declaring scope's binding
> when accessing or assigning to a particular name.  This approach has similar
> benefits to my first approach, but is clearly more top-down than the first
> approach.  Subsequent "scope" declarations would create a new binding at the
> declaring scope for the declaring and child scopes to use.  This could
> potentially be a gotcha for users expecting the binding semantics in place
> today.  Also the scope keyword would have to be allowed to be used on
> parameters to allow such parameter names to be used in a similar fashion in
> a child scope.
>
>
> def incgen( inc = 1 ) :
>#scope inc <-- allow scope declaration for bound parameters (not a big
> fan of this)
>scope a = 6
>def incrementer() :
>  a += inc
>  return a
>return incrementer
>
>  This approach would be similar to languages like JavaScript that allow for
> explicit scope binding with the use of "var" or more static languages that
> allow re-declaring names at lower scopes.  I am less in favor of this,
> because I don't think it feels very "Pythonic".
>
>  As a point of reference, some languages such as Ruby will only bind a new
> name to a scope on assignment when an enclosing scope does not have the name
> bound.  I do believe the Python name binding semantics have issues (for
> which the "global" keyword was born), but I feel that the "fixing" the
> Python semantic to a more "Ruby-like" one adds as many problems as it solves
> since the "Ruby-like" one is just

Re: [Python-Dev] PEP for Better Control of Nested Lexical Scopes

[Jeremy Hylton]

I had to lookup top-post :-).

On 2/21/06, Bengt Richter <[EMAIL PROTECTED]> wrote:
> On Tue, 21 Feb 2006 08:02:08 -0500, "Jeremy Hylton" <[EMAIL PROTECTED]> wrote:
> >Jeremy
> Hey, only Guido is allowed to top-post. He said so ;-)

The Gmail UI makes it really easy to forget where the q

> But to the topic, it just occurred to me that any outer scopes could be given 
> names
> (including global namespace, but that would have the name global by default, 
> so
> global.x would essentially mean what globals()['x'] means now, except it would
> be a name error if x didn't pre-exist when accessed via 
> namespace_name.name_in_space notation.
>
>
> namespace g_alias  # g_alias.x becomes alternate spelling of global.x
> def outer():
> namespace mezzanine
> a = 123
> print a  # => 123
> print mezzanine.a  # => 123 (the name space name is visible and 
> functional locally)
> def inner():
> print mezzanine.a => 123
> mezznine.a =456
> inner()
> print a # = 456
> global.x = re-binds global x, name error if not preexisting.
>
> This would allow creating mezzanine like an attribute view of the slots in 
> that local namespace,
> as well as making namespace itself visible there, so the access to mezzanine 
> would look like a read access to
> an ordinary object named mezzanine that happened to have attribute slots 
> matching outer's local name space.
>
> Efficiency might make it desirable not to extend named namespaces with new 
> names, function locals being
> slotted in a fixed space tied into the frame (I think). But there are tricks 
> I guess.
> Anyway, I hadn't seen this idea before. Seems
>
> Regards,
> Bengt Richter
>
> >
> >On 2/20/06, Almann T. Goo <[EMAIL PROTECTED]> wrote:
> >> I am considering developing a PEP for enabling a mechanism to assign to 
> >> free
> >> variables in a closure (nested function).  My rationale is that with the
> >> advent of PEP 227 , Python has proper nested lexical scopes, but can have
> >> undesirable behavior (especially with new developers) when a user makes
> >> wants to make an assignment to a free variable within a nested function.
> >> Furthermore, after seeing numerous kludges to "solve" the problem with a
> >> mutable object, like a list, as the free variable do not seem "Pythonic."  
> >> I
> >> have also seen mention that the use of classes can mitigate this, but that
> >> seems, IMHO, heavy handed in cases when an elegant solution using a closure
> >> would suffice and be more appropriate--especially when Python already has
> >> nested lexical scopes.
> >>
> >>  I propose two possible approaches to solve this issue:
> >>
> >>  1.  Adding a keyword such as "use" that would follow similar semantics as
> >> "global" does today.  A nested scope could declare names with this keyword
> >> to enable assignment to such names to change the closest parent's binding.
> >> The semantic would be to keep the behavior we experience today but tell the
> >> compiler/interpreter that a name declared with the "use" keyword would
> >> explicitly use an enclosing scope.  I personally like this approach the 
> >> most
> >> since it would seem to be in keeping with the current way the language 
> >> works
> >> and would probably be the most backwards compatible.  The semantics for how
> >> this interacts with the global scope would also need to be defined (should
> >> "use" be equivalent to a global when no name exists all parent scopes, 
> >> etc.)
> >>
> >>
> >> def incgen( inc = 1 ) :
> >>a = 6
> >>def incrementer() :
> >>  use a
> >>  #use a, inc <-- list of names okay too
> >>  a += inc
> >>  return a
> >>return incrementer
> >>
> >>  Of course, this approach suffers from a downside that every nested scope
> >> that wanted to assign to a parent scope's name would need to have the "use"
> >> keyword for those names--but one could argue that this is in keeping with
> >> one of Python's philosophies that "Explicit is better than implicit" (PEP
> >> 20).  This approach also has to deal with a user declaring a name with "
> >> use" that is a named parameter--this would be a semantic error that could 
> >> be
> >> handled like "global 

Re: [Python-Dev] PEP for Better Control of Nested Lexical Scopes

[Jeremy Hylton]

On 2/21/06, Jeremy Hylton <[EMAIL PROTECTED]> wrote:
> I had to lookup top-post :-).
>
> On 2/21/06, Bengt Richter <[EMAIL PROTECTED]> wrote:
> > On Tue, 21 Feb 2006 08:02:08 -0500, "Jeremy Hylton" <[EMAIL PROTECTED]> 
> > wrote:
> > >Jeremy
> > Hey, only Guido is allowed to top-post. He said so ;-)
>
> The Gmail UI makes it really easy to forget where the q

Sorry about that.  Hit the send key by mistake.

The Gmail UI makes it really easy to forget where the quoted text is
in relation to your own text.

> > But to the topic, it just occurred to me that any outer scopes could be 
> > given names
> > (including global namespace, but that would have the name global by 
> > default, so
> > global.x would essentially mean what globals()['x'] means now, except it 
> > would
> > be a name error if x didn't pre-exist when accessed via 
> > namespace_name.name_in_space notation.

Isn't this suggestion that same as Greg Ewing's?

> > namespace g_alias  # g_alias.x becomes alternate spelling of global.x
> > def outer():
> > namespace mezzanine
> > a = 123
> > print a  # => 123
> > print mezzanine.a  # => 123 (the name space name is visible and 
> > functional locally)
> > def inner():
> > print mezzanine.a => 123
> > mezznine.a =456
> > inner()
> > print a # = 456
> > global.x = re-binds global x, name error if not preexisting.
> >
> > This would allow creating mezzanine like an attribute view of the slots in 
> > that local namespace,
> > as well as making namespace itself visible there, so the access to 
> > mezzanine would look like a read access to
> > an ordinary object named mezzanine that happened to have attribute slots 
> > matching outer's local name space.

I don't think using attribute access is particularly clear here.  It
introduces an entirely new concept, a first-class namespace, in order
to solve a small scoping problem.  It looks too much like attribute
access and not enough like accessing a variable.

Jeremy

> > Efficiency might make it desirable not to extend named namespaces with new 
> > names, function locals being
> > slotted in a fixed space tied into the frame (I think). But there are 
> > tricks I guess.
> > Anyway, I hadn't seen this idea before. Seems
> >
> > Regards,
> > Bengt Richter
> >
> > >
> > >On 2/20/06, Almann T. Goo <[EMAIL PROTECTED]> wrote:
> > >> I am considering developing a PEP for enabling a mechanism to assign to 
> > >> free
> > >> variables in a closure (nested function).  My rationale is that with the
> > >> advent of PEP 227 , Python has proper nested lexical scopes, but can have
> > >> undesirable behavior (especially with new developers) when a user makes
> > >> wants to make an assignment to a free variable within a nested function.
> > >> Furthermore, after seeing numerous kludges to "solve" the problem with a
> > >> mutable object, like a list, as the free variable do not seem 
> > >> "Pythonic."  I
> > >> have also seen mention that the use of classes can mitigate this, but 
> > >> that
> > >> seems, IMHO, heavy handed in cases when an elegant solution using a 
> > >> closure
> > >> would suffice and be more appropriate--especially when Python already has
> > >> nested lexical scopes.
> > >>
> > >>  I propose two possible approaches to solve this issue:
> > >>
> > >>  1.  Adding a keyword such as "use" that would follow similar semantics 
> > >> as
> > >> "global" does today.  A nested scope could declare names with this 
> > >> keyword
> > >> to enable assignment to such names to change the closest parent's 
> > >> binding.
> > >> The semantic would be to keep the behavior we experience today but tell 
> > >> the
> > >> compiler/interpreter that a name declared with the "use" keyword would
> > >> explicitly use an enclosing scope.  I personally like this approach the 
> > >> most
> > >> since it would seem to be in keeping with the current way the language 
> > >> works
> > >> and would probably be the most backwards compatible.  The semantics for 
> > >> how
> > >> this interacts with the global scope would also need to be defined 
> 

Re: [Python-Dev] PEP for Better Control of Nested Lexical Scopes

[Jeremy Hylton]

On 2/21/06, Terry Reedy <[EMAIL PROTECTED]> wrote:
>
> "Almann T. Goo" <[EMAIL PROTECTED]> wrote in message
> news:[EMAIL PROTECTED]
>
> > I certainly hope that an initiative like this doesn't get stymied by
> > the lack of a good name for such a keyword.  Maybe something like
> > "outer"?
>
> Adding a keyword has a cost that you have so far ignored.  Guido is
> rightfully very cautious about additions, especially for esthetic reasons.
>
> The issue of rebinding enclosed names was partly discussed in PEP 227.
> Sometime after the implementation of the PEP in 2.1, it was thoroughly
> discussed again (100+ posts?) in this forum.  There were perhaps 10
> different proposals, including, I believe, 'outer'.  Guido rejected them
> all as having costs greater than the benefits.  Perhaps you can find this
> discussion in the archives.  I remember it as a Jan-Feb discussion but
> might be wrong.

If I recall the discussion correctly, Guido said he was open to a
version of nested scopes that allowed rebinding.  Not sure that the
specifics of the previous discussion are necessary, but I recall being
surprised by the change in opinion since 2.1 :-).

Jeremy

>
> This thread so far seems like a rehash of parts of the earlier discussion.
> In the absence of indication from Guido that he is ready to reopen the
> issue, perhaps it would better go to comp.lang.python.  In and case,
> reconsideration is more likely to be stimulated by new experience with
> problems in real code than by repeats of 'orthogonality' desires and
> rejected changes.
>
> ---
>
> In another post, you rejected the use of class instances by opining:
>
> >Because I think that this is a workaround for a concept that the
> >language doesn't support elegantly with its lexically nested scopes.
>
> >IMO, you are emulating name rebinding in a closure by creating an
> >object to encapsulate the name you want to rebind
>
> Guido, on the other hand, views classes and instances as Python's method of
> doing what other (functional) languages do with closures.  From the PEP:
> "Given that this
> would encourage the use of local variables to hold state that is
> better stored in a class instance, it's not worth adding new
> syntax to make this possible (in Guido's opinion)."
> He reiterated this viewpoint in the post-PEP discussion mentioned above.  I
> think he would specificly reject the view that Python's alternative is a
> 'workaround' and 'emulation' of what you must consider to be the real
> thing.
>
> Terry Jan Reedy
>
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP for Better Control of Nested Lexical Scopes

[Jeremy Hylton]

On 2/22/06, Greg Ewing <[EMAIL PROTECTED]> wrote:
> Mark Russell wrote:
>
> > PEP 227 mentions using := as a rebinding operator, but rejects the
> > idea as it would encourage the use of closures.
>
> Well, anything that facilitates rebinding in outer scopes
> is going to encourage the use of closures, so I can't
> see that as being a reason to reject a particular means
> of rebinding. You either think such rebinding is a good
> idea or not -- and that seems to be a matter of highly
> individual taste.

At the time PEP 227 was written, nested scopes were contentious.  (I
recall one developer who said he'd be embarassed to tell his
co-workers he worked on Python if it had this feature :-).  Rebinding
was more contentious, so the feature was left out.  I don't think any
particular syntax or spelling for rebinding was favored more or less.

> On this particular idea, I tend to think it's too obscure
> as well. Python generally avoids attaching randomly-chosen
> semantics to punctuation, and I'd like to see it stay
> that way.

I agree.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP for Better Control of Nested Lexical Scopes

[Jeremy Hylton]

On 2/24/06, James Y Knight <[EMAIL PROTECTED]> wrote:
> On Feb 24, 2006, at 1:54 AM, Greg Ewing wrote:
> > Thomas Wouters wrote:
> >> On Thu, Feb 23, 2006 at 05:25:30PM +1300, Greg Ewing wrote:
> >>
> >>> As an aside, is there any chance that this could be
> >>> changed in 3.0? I.e. have the for-loop create a new
> >>> binding for the loop variable on each iteration.
> >>
> >> You can't do that without introducing a whole new scope
> >> for the body of the 'for' loop,
> >
> > There's no need for that. The new scope need only
> > include the loop variable -- everything else could
> > still refer to the function's main scope.
>
> No, that would be insane. You get the exact same problem, now even
> more confusing:
>
> l=[]
> for x in range(10):
>y = x
>l.append(lambda: (x, y))
>
> print l[0]()
>
> With your suggestion, that would print (0, 9).
>
> Unless python grows a distinction between creating a binding and
> assigning to one as most other languages have, this problem is here
> to stay.

The more practical complaint is that list comprehensions use the same
namespace as the block that contains them.  It's much easier to miss
an assignment to, say, i in a list comprehension than it is in a
separate statement in the body of a for loop.  Since list comps are
expressions, the only variable at issue is the index variable.  It
would be simple to fix by renaming, but I suspect we're stuck with the
current behavior for backwards compatibility reasons.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] quick status report

[Jeremy Hylton]

I made a few more minor revisions to the AST on the plane this
afternoon.  I'll check them in tomorrow when I get a chance to do a
full test run.

* Remove asdl_seq_APPEND.  All uses replaced with set
* Fix set_context() comments and check return value every where.
* Reimplement real arena for pyarena.c

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] iterator API in Py3.0

[Jeremy Hylton]

On 3/3/06, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> The double underscore convention is appropriate where the method is always
> invoked magically in normal code and not called directly.  The next() method 
> is
> differenct because it is a mixed case, sometimes called magically and 
> sometimes
> called directly.  In the latter case, the name is highly visible and therefore
> should not have double underscores.
>
> I suspect that those who feel differently are ones who usually avoid calling
> next() directly.  That's okay, but we shouldn't muck-up the naming for the 
> rest
> of us who often do have a need to use next().
>
> This is doubly important because we're now expanding the protocol to include
> send() and throw().  Adding underscores around them too will only make those
> methods look harder to use than they actually are.  Don't underestimate the
> psychological revulsion to calling code filled with piles of double 
> underscores.

I think it is a little odd that next is not spelled __next__, but I
appreciate the reasons given here in particular.  Every time I right
.next(), I'm happy that it doesn't have underscores.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] iterator API in Py3.0

[Jeremy Hylton]

On 3/3/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote:
> At 04:09 PM 3/3/2006 -0500, Jeremy Hylton wrote:
> >I think it is a little odd that next is not spelled __next__, but I
> >appreciate the reasons given here in particular.  Every time I right
> >.next(), I'm happy that it doesn't have underscores.
>
> But then next(ob) should make you even happier, because then there's no
> '.', either.  :)

I've got no problem with dots.  No shift key necessary :-).

On the subject of predilections and builtins, I find myself typing
.size() a lot lately.  Too much C++, I guess, but it seems so natural
as I type it.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Coverity Open Source Defect Scan of Python

[Jeremy Hylton]

On 3/6/06, Thomas Wouters <[EMAIL PROTECTED]> wrote:
>
>
>
> On 3/6/06, Barry Warsaw <[EMAIL PROTECTED]> wrote:
> > > Didn't we set up a "security swat team" some time ago?  If not, we
> > > should.  Regardless, since I have more free time these days, I'd like
> > > to be on it.
> >
> > Yep, it's called [EMAIL PROTECTED] (with a semi-secret backing mailing
> > list, which I'd be happy for you to join
!).  I definitely think that
> > group of folks at the least should review the results.
>
>
> Well, if we start volunteering here, I'll volunteer as well. (For either
> group.) Can't let Tim have all the fun!

I also sent mail to Ben volunteering.  I expect the scope of defects
recognized is larger than just security.  In particular, the compiler
has a large body of code that has never been released before.  It
would nice to catch a few of its bugs before a release :-).

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] conditional expressions - add parens?

[Jeremy Hylton]

On 3/6/06, Alex Martelli <[EMAIL PROTECTED]> wrote:
>
> On Mar 6, 2006, at 9:17 AM, Jim Jewett wrote:
> ...
> > I think that adding parentheses would help, by at least signalling
> > that the logic is longer than just the next (single) expression.
> >
> > level = (0 if "absolute_import" in self.futures else -1)
>
> +1 (just because I can't give it +3.1415926...!!!).  *Mandatory*
> parentheses make this form MUCH more readable.

Recent language features seem to be suffereing from excessive
parenthesisitis.  I worry that people will stop remembering which
expressions requirement parens in which context.  Perhaps the solution
is to require parens around all expressions, a simple consistent rule.
 If so, then adding parens around all statements is a fairly natural
extension, which solves a number of problems like how to make a richer
lambda.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bug Day?

[Jeremy Hylton]

On 3/8/06, Georg Brandl <[EMAIL PROTECTED]> wrote:
> I know, PyCon's just been, but not many bugs were closed and
> there really ought to be some issues resolved before 2.4.3 happens.
> The number of open bugs is again crawling to 900.
>
> I myself are looking at many bugs and patches over time, but with
> most of them I can't decide alone what to do. Writing here every
> time doesn't seem like the most sensible thing too.
>
> So, is there interest? Can I do anything to make it easier?

Would anyone be interested in a bug day on a weekday?  I don't have
time on weekends, but could probably find some time on a Wed or Fri
for bug fixing.  I'll probably do it regardless, but I'd have more fun
if a few others were online.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com