Re: [Python-Dev] Pickle implementation questions
Martin v. Löwis wrote: >> In developing a cPickle module for IronPython that's as compatible as >> possible with CPython, these questions have come up: > > [I wish you were allowed to read the source code of Python] on the other hand, it would be nice if someone actually used Bruce's questions and the clarifications to update the documentation; the ideas behind the internal pickle interfaces aren't exactly obvious, even if you have the source. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] 2.5 and beyond
I'm glad to see Anthony ratcheting down. At this point, we need to be fixing bugs and improving doc. Maybe Anthony and I should have a contest to see who can revert the most changes. :-) There are at least 6 bugs that really, really need to be fixed before release. Several of these are AST bugs. Jeremy knows about them and plans to fix them once he's back from vacation. Anyone else wanna help out? One is for a socket problem and another is for doc. The current list of serious bugs are in the PEP: http://www.python.org/dev/peps/pep-0356/ If there are any bugs you think should be considered show stoppers, mail them to the list and I will update the PEP. If you are a committer, just update the PEP yourself. We really need everyone to help. There were a lot of changes that didn't have tests and/or NEWS entries. I tried to reply to the checkin messages for those I noticed. I have tons of messages in my inbox where I don't know if the issue was addressed or not. Can everyone try to find the holes? And new ones keep popping up! Please let the author know they need to fix the problem. It's really tempting to just revert these changes... We also need to fix the test suite. This warning needs to be addressed! Lib/struct.py:63: DeprecationWarning: struct integer overflow masking is deprecated return o.pack(*args) Since we are in feature freeze, now seems like a good time to make a PEP for 2.6: http://www.python.org/dev/peps/pep-0361/ It's pretty empty right now. The plan is to make the schedule in a year from now. Start adding your new features to the PEP, not the code. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
Hi, On Mon, Jun 26, 2006 at 12:23:00PM -0700, Guido van Rossum wrote: > Feedback (also about misrepresentation of alternatives I don't favor) > is most welcome, either to me directly or as a followup to this post. So my 2 cents, particularly about when things are computed and ways to control that explicitly: there was a point in time where I could say that I liked Python because language design was not constrained by performance issues. Looks like it's getting a matter of the past, small step by small step. I'll have to get used to mentally filter out 'static' or whatever the keyword will be, liberally sprinkled in programs I read to make them slightly faster. Maybe I should, more constructively, propose to start a thread on the subject of: what would be required to achieve similar effects as the intended one at the implementation level, without strange early-computation semantics? I'm not talking about Psyco stuff here; there are way to do this with reasonably-simple refactorings of global variable accesses. I have experimented a couple of years ago with making them more direct (just like a lot of people did, about the "faster LOAD_GLOBAL" trend). I dropped this as it didn't make things much faster, but it had a nice side-effect: allowing call-backs for binding changes. This would be a good base on top of which to make transparent, recomputed-when-changed constant-folding of simple expressions. Building dicts for switch and keeping them up-to-date... Does it make sense for me to continue this discussion? A bientot, Armin. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Moving the ctypes repository to python.org
Martin v. Löwis schrieb: > Thomas Heller wrote: >> What I did was at a certain time develop in the 'branch_1_0' branch, leaving >> HEAD for experimental work. Later I decided that this was wrong, cvs >> removed all >> files in HEAD, and added them back from a branch_1_0 checkout. Maybe doing >> this was another bad idea, as the trunk in the converted SVN repository >> only lists _ctypes.c revisions corresponding to the CVS version numbers >> 1.307 up to the current CVS head 1.340. All the older versions from 1.1 up >> to >> 1.226.2.55 show up in the branch_1_0 branch that cvs2svn has created - >> although >> in CVS only the versions 1.226.0.2 up to 1.226.2.55 were ever in the >> branch_1_0 >> branch. Is that a bug in cvs2svnn? > > I doubt it. I'm pretty sure the subversion repository *does* contain all > the old files, in the old revisions. What happens if you do the > following on your converted subversion repository: > > 1. find out the oldest version of the files from svn log. Say this is >version 1000. > 2. Explicitly check out the trunk at version 950 (i.e. some plenty >revisions before your copied the files from the branch). > > I expect that this will give you the files just before you deleted > them; doing "svn log" on this sandbox will then give you all the old > log messages and versions. > > If that is what happens, here is why: "svn log" will trace a file > through all its revisions, and across "svn copy"s, back to when it > was added into the repository. At that point, "svn log" stops. > An earlier file with the same name which got removed is considered > as a different file, so "svn log" does not show its revisions. > > If you don't want that do happen, you could try to "outdate" (cvs -o) > the deletion and readdition in CVS, purging that piece of history. > I'm not entirely certain whether this should work. You mean 'cvs admin -o', right? Yes, that works. Here is what I did: - got a local copy of the cvs repository from SF with rsync. All the following was done on thos local copy. - I ran 'cvs log' over the whole repository, and noted the files that were removed in the HEAD and later readded from a branch. - I wrote a script that calls 'cvs admin -o' on them. - removed those directories that should not be added to Python SVN, these were the ctypes-java, misc and CVSROOT directories. - called 'cvs2svn --dump-only' to create a subversion dumpfile. The file size is around 100 MB. - Created a local SVN repository, called 'svnadmin load'. Now I can checkout from the local SVN repository, and the whole history on the files that I checked is present. The svn checkup has this structure, which is OK imo: ctypes branches tags trunk The remaining questions are: - Do I need special rights to call 'svnadmin load' to import this dumpfile into Python SVN, or are the normal commit rights sufficient? What exactly is the URL/PATH where it should be imported (some sandbox, I assume)? - What about the Python trunk? Should changes from the sandbox be merged into Modules/_ctypes and Lib/ctypes, or would it be better (or possible at all) to use the external mechanism? Thanks, Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
On Fri, 30 Jun 2006, Neal Norwitz wrote: > The current list of serious bugs are in the PEP: > > http://www.python.org/dev/peps/pep-0356/ Among them is this one: Incorrect LOAD/STORE_GLOBAL generation http://python.org/sf/1501934 The question is, what behaviour is preferable for this code: g = 1 def f(): g += 1 f() Should this raise an UnboundLocalError or should it increment g? (Or, in other words, should augmented assignment be considered a local binding like regular assignment, or not?) -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal to eliminate PySet_Fini
> > That was a purely altruistic proposal. I've already > discovered that sets are finalized and that some code that > works with dict emulating a set may not work with a set. It > will not make much difference for me if my proposal will be > implemented in 2.6 or even in 3.0, but the sooner it will > happen the fewer people will stumble on the same problem that > I did. I also feel that dummy allocated on the heap and the > free set list are complicating the code with no gain. > Can this not be resolved by carefully adjusting the order of finalization? If code can be bootstrapped it can be strootbapped. As a side note, is there a finalization order list for imported modules? Kristján ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] msvccompiler.py: some remarks
On 6/30/06, Jeroen Ruigrok van der Werven <[EMAIL PROTECTED]> wrote: > > Forget about Visual Studio 8 and .NET 2.0. It won't help here. > > I only have .NET 1.1 and 2.0 and Visual Studio 2005 (8) installed. Why > should I forget about it? Is Python compiled with much older compilers > and thus unable to work together in a nice way or? The standard Python binary uses the MSVC 7.1 CRT (msvcr71.dll). Visual Studio 2005 will not compile code which uses that CRT, so Python extensions built with that compiler are not compatible with Python built to use msvcr71.dll. The only compilers supported for building extensions compatible with the standard Python binary are gcc (mingw) and VS 2003 (MSVC 7.1) (including the free MS Toolkit compiler 2003 if you have it, but sadly MS have withdrawn it from distribution) - precisely because they have options to link with msvcr71.dll. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
[Ka-Ping Yee, on http://www.python.org/dev/peps/pep-0356/ ] > Among them is this one: > > Incorrect LOAD/STORE_GLOBAL generation > http://python.org/sf/1501934 > > The question is, what behaviour is preferable for this code: > > g = 1 > def f(): > g += 1 > > f() > > Should this raise an UnboundLocalError or should it increment g? > > (Or, in other words, should augmented assignment be considered > a local binding like regular assignment, or not?) Of course it should, since that's the way it _is_ treated in all released Pythons, and there was no intent to change the semantics (let alone a FutureWarning in 2.4 to alert users that the meaning was going to change in 2.5). The Reference Manual also makes no distinction between assignment statements and augmented assignment statements in any relevant respect. The change in behavior here in 2.5 is plainly a bug (although someone may want to argue "it should" be different in P3K). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
Ka-Ping Yee <[EMAIL PROTECTED]> writes: > On Fri, 30 Jun 2006, Neal Norwitz wrote: >> The current list of serious bugs are in the PEP: >> >> http://www.python.org/dev/peps/pep-0356/ > > Among them is this one: > > Incorrect LOAD/STORE_GLOBAL generation > http://python.org/sf/1501934 > > The question is, what behaviour is preferable for this code: > > g = 1 > def f(): > g += 1 > > f() > > Should this raise an UnboundLocalError or should it increment g? I didn't think there was any question: this change in behaviour from 2.4 is just an accidental change, a bug that should be fixed. If you want to elevate it to feature status, I think we've missed the freeze :) (and also, I oppose the idea). Cheers, mwh -- If I didn't have my part-time performance art income to help pay the bills, I could never afford to support my programming lifestyle.-- Jeff Bauer, 21 Apr 2000 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
Ping> The question is, what behaviour is preferable for this code: Ping> g = 1 Ping> def f(): Ping> g += 1 Ping> f() If you treat "g += 1" as "g = g + 1" then it should create a local variable with a value of 2. There being no global statement in f() it must not modify the global variable. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
On Fri, 30 Jun 2006 00:05:10 -0700, Neal Norwitz <[EMAIL PROTECTED]> wrote: >I'm glad to see Anthony ratcheting down. At this point, we need to be >fixing bugs and improving doc. Maybe Anthony and I should have a >contest to see who can revert the most changes. :-) > >There are at least 6 bugs that really, really need to be fixed before >release. Several of these are AST bugs. Jeremy knows about them and >plans to fix them once he's back from vacation. Anyone else wanna >help out? One is for a socket problem and another is for doc. The >current list of serious bugs are in the PEP: > > http://www.python.org/dev/peps/pep-0356/ > Please add #1494314 to the list. http://sourceforge.net/tracker/index.php?func=detail&aid=1494314&group_id=5470&atid=105470 Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Cleanup of test harness for Python
Hello all, According to the thread that includes http://mail.python.org/pipermail/python-dev/2006-June/065727.html there will be some effort in 2.6 to make the tests in Python more consistent. I would like to help with that effort, partly to sneak in some checks for CPython internal tests that should be excluded from Jython, but mainly to understand the future implementation of Python for which the tests provide the only real spec. Which of the current tests is closest to an "ideal" test, so I can use it as a model? Thanks, -Frank Wierzbicki ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
On Jun 30, 2006, at 3:05 AM, Neal Norwitz wrote: > If there are any bugs you think should be considered show stoppers, > mail them to the list and I will update the PEP. I just submitted http://python.org/sf/1515169 for the ImportWarning issue previously discussed here. IMO it's important. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] msvccompiler.py: some remarks
Jeroen Ruigrok van der Werven wrote: > On 6/29/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: >> We should remove/change this comment. It is utterly misleading. > > To a warning/error stating that you miss a compiler? Correct: that you miss VS 2003, or should request mingw. >> Forget about Visual Studio 8 and .NET 2.0. It won't help here. > > I only have .NET 1.1 and 2.0 and Visual Studio 2005 (8) installed. Why > should I forget about it? Is Python compiled with much older compilers > and thus unable to work together in a nice way or? "Much" is a relative thing, but yes. Python 2.3 and before is compiled with VC6, Python 2.4 and 2.5 are compiled with VS 2003. You cannot compile extensions with a different compiler version because the CRT versions will clash (msvcrt4 vs. msvcr71 vs. msvcr8) Google for details, this has been discussed both technically and politically many times before. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] sys.settrace() in Python 2.3 vs. 2.4
I've previously asked on python-list, but have recieved no responses or explanations. Maybe someone here with a better memory can help, and I apologize for asking a somewhat off-topic question about such an archaic version of Python. According to my reading of Python 2.3 docs, the call to goo() should exit with a KeyboardInterrupt... import sys def goo(): while 1: pass count = [100] def foo(frame, event, arg): count[0] -= 1 if not count[0]: raise KeyboardInterrupt return foo sys.settrace(foo) goo() In Python 2.3, the above call to goo() doesn't return. Adding print statements suggests that foo() is only called for each line executed in goo() once. It exits with a KeyboardInterrupt in 2.4, as expected, where foo() is called for essentially every operaition performed. Does anyone have an idea why this is the case? I've checked the release notes for both 2.3 and 2.4 and found no discussion of trace functions in them or in sourceforge (I could be entering the wrong search terms, of course). Any pointers as to why there is a difference would be appreciated. Thank you, - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
> The question is, what behaviour is preferable for this code: > > g = 1 > def f(): > g += 1 > > f() > > Should this raise an UnboundLocalError or should it increment g? I think it should increment (i.e. rebind) g, for the same reason that g = [1] def f(): g[0] += 1 f() rebinds g[0]. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
> I think it should increment (i.e. rebind) g, for the same reason that > > g = [1] > def f(): > g[0] += 1 > f() > > rebinds g[0]. I saw messages out of sequence and did not realize that this would be a change in behavior from 2.4. Sigh. I hope Py3000 has lexical scoping a la Scheme... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cleanup of test harness for Python
On 6/30/06, Frank Wierzbicki <[EMAIL PROTECTED]> wrote: Hello all,According to the thread that includeshttp://mail.python.org/pipermail/python-dev/2006-June/065727.htmlthere will be some effort in 2.6 to make the tests in Python moreconsistent. I would like to help with that effort, partly to sneak insome checks for CPython internal tests that should be excluded fromJython, but mainly to understand the future implementation of Python for which the tests provide the only real spec. Which of the currenttests is closest to an "ideal" test, so I can use it as a model?We don't have any labeled as "ideal". Either doctests or unittest tests are considered good form these days. Probably looking at the newer tests would be a good start. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pickle implementation questions
Thanks for your responses, Martin!
Martin v. Löwis wrote:
> Bruce Christensen wrote:
> > - Where are object.__reduce__ and object.__reduce_ex__ defined, and how
> > does copy_reg._reduce_ex fit into the picture?
>
> See
>
> http://docs.python.org/lib/node69.html
So just to be clear, is it something like this?
class object:
def __reduce__(self):
return copy_reg._reduce_ex(self, -1)
def __reduce_ex__(self, protocol):
return copy_reg._reduce_ex(self, protocol)
Does _reduce_ex's behavior actually change depending on the specified protocol
version? The only difference that I can see or think of is that an assert
causes it to fail if the protocol is >= 2.
> > - What does copy_reg.constructor() do?
>
> It does this:
>
> def constructor(object):
> if not callable(object):
> raise TypeError("constructors must be callable")
So it is part of the public interface? It's exported in __all__, but it appears
that it's undocumented.
Thanks,
--Bruce
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pickle implementation questions
Fredrik Lundh wrote: > on the other hand, it would be nice if someone actually used Bruce's questions > and the clarifications to update the documentation; the ideas behind the > internal pickle interfaces aren't exactly obvious, even if you have the > source. I've found a few other places where the docs are misleading at best, and nonexistent or simply wrong at worst. I've been able to figure out most things by reverse-engineering pickle's behavior, but that's often a slow process. If anyone is interested, I'd be happy to compile a list of places the docs could be improved. --Bruce ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pickle implementation questions
Please do help us improve the docs. Patches are the best (most likely to be applied the fastest), bug reports are welcome too. Especially when they contain your preferred wording in the text. n -- On 6/30/06, Bruce Christensen <[EMAIL PROTECTED]> wrote: > Fredrik Lundh wrote: > > on the other hand, it would be nice if someone actually used Bruce's > questions > > and the clarifications to update the documentation; the ideas behind > the > > internal pickle interfaces aren't exactly obvious, even if you have > the > > source. > > I've found a few other places where the docs are misleading at best, and > nonexistent or simply wrong at worst. I've been able to figure out most > things > by reverse-engineering pickle's behavior, but that's often a slow > process. > > If anyone is interested, I'd be happy to compile a list of places the > docs could > be improved. > > --Bruce > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] For sandboxing: alternative to crippling file()
Hi Brett, On Thu, Jun 29, 2006 at 11:48:36AM -0700, Brett Cannon wrote: > 1) Is removing 'file' from the builtins dict in PyInterpreterState (and > maybe some other things) going to be safe enough to sufficiently hide 'file' > confidently (short of someone being stupid in their C extension module and > exposing 'file' directly)? No. >>> object.__subclasses__() [..., ] Maybe this one won't work if __subclasses__ is forbidden, but in general I think there *will* be a way to find this object. A bientot, Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] For sandboxing: alternative to crippling file()
On 6/30/06, Armin Rigo <[EMAIL PROTECTED]> wrote: Hi Brett,On Thu, Jun 29, 2006 at 11:48:36AM -0700, Brett Cannon wrote:> 1) Is removing 'file' from the builtins dict in PyInterpreterState (and> maybe some other things) going to be safe enough to sufficiently hide 'file' > confidently (short of someone being stupid in their C extension module and> exposing 'file' directly)?No.>>> object.__subclasses__()[..., ]Maybe this one won't work if __subclasses__ is forbidden, but in general I think there *will* be a way to find this object.Yeah, that's been my (what I thought was paranoid) feeling. Glad I am not the only one who thinks that hiding file() is near impossible. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] how long to wait for expat to incorporate a fix to prevent a crasher?
Lib/test/crashers/xml_parsers.py is a crasher that involves expat (bug report at http://python.org/sf/1296433). What is at issue here is that there is a 'for' loop in expat where the status of the parser is not checked. Because of this, the loop continues on its merry way, which is a problem because pyexpat sets all handlers to 0 upon error and the 'for' loop executes a handler. =) We all know what happens if you try to execute memory location 0x0. Anyway, the fault is not on our end since expat should be checking the status of the parser before going around the loop again instead of blindly assuming that everything is fine after a characterDataHandler() call (especially since there is no error return code and there is a parser status flag for this exact reason). I have filed a bug report at http://sourceforge.net/support/tracker.php?aid=1515266 and attached a possible patch.The question is how long do we wait for the expat developers to patch and do a micro release? Do we just leave this possible crasher in and just rely entirely on the expat developers, or do we patch our copy and use that until they get around to doing their next version push? -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal to eliminate PySet_Fini
Kristján V. Jónsson ccpgames.com> writes: > Can this not be resolved by carefully adjusting the order of finalization? Absolutely. This is exactly what I did in my "interned" patch and this is what prompted my proposal. > If code can be bootstrapped it can be strootbapped. Agree. However, the code that does not need bootstaping is often simpler and less fragile for the same reason. > As a side note, is there a finalization order list for imported modules? I did not know that imported modules could be finalized. If they could, I would guess the proper order would ne the reverse on the order of initialization. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pickle implementation questions
[Bruce Christensen]
> So just to be clear, is it something like this?
I hope you've read PEP 307:
http://www.python.org/dev/peps/pep-0307/
That's where __reduce_ex__ was introduced (along with all the rest of
pickle protocol 2).
> class object:
> def __reduce__(self):
> return copy_reg._reduce_ex(self, -1)
>
> def __reduce_ex__(self, protocol):
> return copy_reg._reduce_ex(self, protocol)
The implementation is more like:
class object:
def __common_reduce__(self, proto=0):
if self.__class__.__reduce__ is not object.__reduce__:
# The class overrode __reduce__, so call the override.
# From PEP 307:
# The 'object' class implements both __reduce__ and
# __reduce_ex__; however, if a subclass overrides __reduce__
# but not __reduce_ex__, the __reduce_ex__ implementation
# detects this and calls __reduce__.
return self.__reduce__()
elif proto < 2:
return copy_reg._reduce_ex(self, proto)
else:
# about 130 lines of C code exploiting proto 2
__reduce__ = __reduce_ex__ = __common_reduce__
> Does _reduce_ex's behavior actually change depending on the specified protocol
> version? The only difference that I can see or think of is that an assert
> causes it to
> fail if the protocol is >= 2.
That's right. As above, the object reduce methods never call
copy_reg._reduce_ex() when proto >= 2.
Note that __reduce_ex__ doesn't exist for the _benefit_ of object: it
was introduced in protocol 2 for the benefit of user classes that want
to exploit protocol-specific pickle opcodes in their own __reduce__
methods. They couldn't do that using the old-time __reduce__ because
__reduce__ wasn't passed the protocol version.
copy_reg._reduce_ex exists only because Guido got fatally weary of
writing mountains of C code, so left what "should be" a rarely-taken
path coded in Python.
>>> - What does copy_reg.constructor() do?
>> It does this:
>>
>> def constructor(object):
>> if not callable(object):
>> raise TypeError("constructors must be callable")
> So it is part of the public interface? It's exported in __all__, but it
> appears
> that it's undocumented.
It's documented in the Library Reference Manual, in the `copy_reg` docs:
"""
constructor(object)
Declares object to be a valid constructor. If object is not callable
(and hence not valid as a constructor), raises TypeError.
"""
Unfortunately, while all the "safe for unpickling?" gimmicks (of which
this is one -- see PEP 307 again) were abandoned in Python 2.3, the
docs and code comments still haven't entirely caught up.
copy_reg.constructor() exists now only for backward compatibility (old
code may still call it, but it no longer has any real use).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] LOAD_CONST POP_TOP
Hi,
the following patch tries to fix the LOAD_CONST POP_TOP optimization
lost in 2.5 (bug #1333982).
An example for this is:
def f():
'a' # docstring
'b'
Georg
PS: Hmm. While looking, I see that 2.4 doesn't optimize away other constants
like
def g():
1
Index: Python/compile.c
===
--- Python/compile.c(Revision 47150)
+++ Python/compile.c(Arbeitskopie)
@@ -775,10 +775,16 @@
}
break;
+ case LOAD_CONST:
+ cumlc = lastlc + 1;
+ /* Skip over LOAD_CONST POP_TOP */
+ if (codestr[i+3] == POP_TOP) {
+ memset(codestr+i, NOP, 4);
+ cumlc = 0;
+ break;
+ }
/* Skip over LOAD_CONST trueconst
JUMP_IF_FALSE xx POP_TOP */
- case LOAD_CONST:
- cumlc = lastlc + 1;
j = GETARG(codestr, i);
if (codestr[i+3] != JUMP_IF_FALSE ||
codestr[i+6] != POP_TOP ||
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] how long to wait for expat to incorporate a fix to prevent a crasher?
On Friday 30 June 2006 14:19, Brett Cannon wrote: > The question is how long do we wait for the expat developers to patch and > do a micro release? Do we just leave this possible crasher in and just > rely entirely on the expat developers, or do we patch our copy and use > that until they get around to doing their next version push? Sigh. Too much to do all around. I'll try to take a look at this over the weekend. -Fred -- Fred L. Drake, Jr. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python memory model (low level)
I Have been thinking about software floating point, and there are some aspects of Python and decimal that puzzle me. Basically, they are things that are wanted for this sort of thing and seem to be done in very contorted ways, so I may have missed something. Firstly, can Python C code assume no COMPACTING garbage collector, or should it allow for things shifting under its feet? Secondly, is there any documentation on the constraints and necessary ritual when allocating chunks of raw data and/or types of variable size? Decimal avoids the latter. Thirdly, I can't find an efficient way for object-mangling code to access class data and/or have some raw data attached to a class (as distinct from an instance). Fourthly, can I assume that no instance of a class will remain active AFTER the class disappears? This would mean that it could use a pointer to class-level raw data. I can explain why all of those are the 'right' way to approach the problem, at an abstract level, but it is quite possible that Python does not support the abstract model of class implementation that I am thinking of. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python memory model (low level)
On Fri, Jun 30, 2006, Nick Maclaren wrote: > > I Have been thinking about software floating point, and there are > some aspects of Python and decimal that puzzle me. Basically, they > are things that are wanted for this sort of thing and seem to be > done in very contorted ways, so I may have missed something. > > Firstly, can Python C code assume no COMPACTING garbage collector, > or should it allow for things shifting under its feet? > > Secondly, is there any documentation on the constraints and necessary > ritual when allocating chunks of raw data and/or types of variable > size? Decimal avoids the latter. Without answering your specific questions, keep in mind that Python and Python-C code are very different things. The current Decimal implementation was designed to be *readable* and efficient *Python* code. For a look at what the Python-C implementation of Decimal might look closer to, take a look at the Python long implementation. -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "I saw `cout' being shifted "Hello world" times to the left and stopped right there." --Steve Gonedes ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python memory model (low level)
Aahz <[EMAIL PROTECTED]> wrote: > > Without answering your specific questions, keep in mind that Python and > Python-C code are very different things. The current Decimal > implementation was designed to be *readable* and efficient *Python* code. > For a look at what the Python-C implementation of Decimal might look > closer to, take a look at the Python long implementation. Er, perhaps I should have said explicitly that I was looking at the Decimal-in-C code and not the Python. Most of my questions don't make any sense at the Python level. But you have a good point. The long code will be both simpler and have had a LOT more work done on it - but it will address only the object of variable size issue, as it doesn't need class-level data in the same way as Decimal and I do. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
[EMAIL PROTECTED] wrote: > Ping> The question is, what behaviour is preferable for this code: > > Ping> g = 1 > Ping> def f(): > Ping> g += 1 > > Ping> f() > > If you treat "g += 1" as "g = g + 1" then it should create a local variable > with a value of 2. py> g = 1 py> def f(): ... g = g + 1 ... py> f() Traceback (most recent call last): File "", line 1, in ? File "", line 2, in f UnboundLocalError: local variable 'g' referenced before assignment Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python memory model (low level)
[Nick Maclaren] > I Have been thinking about software floating point, and there are > some aspects of Python and decimal that puzzle me. Basically, they > are things that are wanted for this sort of thing and seem to be > done in very contorted ways, so I may have missed something. > > Firstly, can Python C code assume no COMPACTING garbage collector, > or should it allow for things shifting under its feet? CPython never relocates objects. The address of a Python object is fixed from birth to death. > Secondly, is there any documentation on the constraints and necessary > ritual when allocating chunks of raw data and/or types of variable > size? Decimal avoids the latter. Note that because CPython never relocates objects, it's impossible to allocate "in one gulp" an object of a mutable type with a dynamically size-varying member. So, e.g., list objects have a fixed-size object header (which never moves), while the list "guts" are separately allocated; OTOH, tuples are immutable, so can (& do) allocate all the space they need in one gulp. See the Python/C API Reference Manual, esp. "Allocating Objects on the Heap", "Supporting Cyclic Garbage Collection", and "Memory Management". > Thirdly, I can't find an efficient way for object-mangling code to > access class data and/or have some raw data attached to a class (as > distinct from an instance). Don't know what "raw data" might mean here. Any Python object can be bound to any attribute of a class. In Python, e.g., class MyClass: mydata = ['xyz', 12] def method(self): MyClass.mydata.append(-1) # or, more inheritance-friendly self.__class__.mydata.append(-1) This is substantially more long-winded in C. > Fourthly, can I assume that no instance of a class will remain active > AFTER the class disappears? This would mean that it could use a > pointer to class-level raw data. It would be an error to free the memory for a class if the class is reachable. So long as an instance I of a class C is reachable, C is also reachable (at least via I.__class__) so C _must_ not disappear. C disappears only if it becomes unreachable, and one of the garbage collection mechanisms then gets around to freeing its memory. All gc mechanisms in CPython rely on accurate reference counts. > I can explain why all of those are the 'right' way to approach the > problem, at an abstract level, but it is quite possible that Python > does not support the abstract model of class implementation that I > am thinking of. Since we don't know what you're thinking of, you're on your own there ;-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
James Y Knight wrote: > On Jun 30, 2006, at 3:05 AM, Neal Norwitz wrote: >> If there are any bugs you think should be considered show stoppers, >> mail them to the list and I will update the PEP. > > I just submitted http://python.org/sf/1515169 for the ImportWarning > issue previously discussed here. IMO it's important. At the moment (i.e. without an acceptable alternative implementation) it's primarily a policy issue. There really isn't any bug here; (to speak with Microsoft's words): This behavior is by design. Only the release manager or the BDFL could revert the feature, and Guido already stated that the warning stays until Python 3, and probably even after that. I personally believe the only chance to get this changed now is a well-designed alternative implementation (although this is no promise that such an alternative would actually be accepted). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pickle implementation questions
Tim Peters wrote:
> I hope you've read PEP 307:
I have. Thanks to you and Guido for writing it! It's been a huge help.
> The implementation is more like:
[snip]
Thanks! That helps a lot. PEP 307 and the pickle module docs describe the end
result pretty well, but they don't always make it clear where things are
implemented. I'm trying to make sure that I'm getting the right interaction
between object.__reduce(_ex)__, pickle, and copy_reg..
One (hopefully) last question: is object.__reduce(_ex)__ really implemented in
object? The tracebacks below would indicate that pickle directly implements the
behavior that the specs say is implemented in object. However, that could be
because frames from C code don't show up in tracebacks. I'm not familiar enough
with CPython to know for sure.
>>> import copy_reg
>>> def bomb(*args, **kwargs):
... raise Exception('KABOOM! %r %r' % (args, kwargs))
...
>>> copy_reg._reduce_ex = bomb
>>> import pickle
>>> pickle.dumps(object())
Traceback (most recent call last):
File "", line 1, in ?
File "C:\Python24\lib\pickle.py", line 1386, in dumps
Pickler(file, protocol, bin).dump(obj)
File "C:\Python24\lib\pickle.py", line 231, in dump
self.save(obj)
File "C:\Python24\lib\pickle.py", line 313, in save
rv = reduce(self.proto)
File "", line 2, in bomb
Exception: KABOOM! (, 0) {}
>>> class NewObj(object):
... def __reduce__(self):
... raise Exception("reducing NewObj")
...
>>> import pickle
>>> pickle.dumps(NewObj())
Traceback (most recent call last):
File "", line 1, in ?
File "C:\Python24\lib\pickle.py", line 1386, in dumps
Pickler(file, protocol, bin).dump(obj)
File "C:\Python24\lib\pickle.py", line 231, in dump
self.save(obj)
File "C:\Python24\lib\pickle.py", line 313, in save
rv = reduce(self.proto)
File "", line 3, in __reduce__
Exception: reducing NewObj
> It's documented in the Library Reference Manual, in the `copy_reg` docs:
Oops. :)
Again, thanks for the help.
--Bruce
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python memory model (low level)
"Tim Peters" <[EMAIL PROTECTED]> wrote: [ Many useful answers ] Thanks very much! That helps. Here are a few points where we are at cross-purposes. I am talking about the C level. What I am thinking of is the standard method of implementing the complicated housekeeping of a class (e.g. inheritance) in Python, and the basic operations in C (for efficiency). The model that I would like to stick to is that the Python layer never knows about the actual object implementation, and the C never knows about the housekeeping. The housekeeping would include the class derivation, which would (inter alia) fix the size of a number. The C code would need to allocate some space to store various constants and workspace, shared between all instances of the derived class. This would be accessible from the object it returns. Each instance would be of a length specified by its derivation (i.e. like Decimal), but would be constant for all members of the class (i.e. not like long). So it would be most similar to tuple in that respect. Operations like addition would copy the pointer to the class data from the arguments, and ones like creation would need to be passed the appropriate class and whatever input data they need. I believe that, using the above approach, it would be possible to achieve good efficiency with very little C - certainly, it has worked in other languages. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Moving the ctypes repository to python.org
Thomas Heller wrote: > - Do I need special rights to call 'svnadmin load' to import this dumpfile > into Python SVN, or are the normal commit rights sufficient? It's called "svnadmin" for a reason :-) Neal Norwitz or myself will have to do that; we need to do it on the repository machine locally. I would likely take subversion write access offline for the time of the import, so that I can rollback the entire repository in case of an operator mistake. > What exactly is the URL/PATH where it should be imported (some sandbox, > I assume)? My view is that this is the "projects" repository; with ctypes being a project, it should go into the root directory (i.e. as a sibling to python, peps, distutils, stackless, ...). If you prefer to see it in sandbox, this could work as well. > - What about the Python trunk? Should changes from the sandbox be merged > into Modules/_ctypes and Lib/ctypes, or would it be better (or possible at > all) > to use the external mechanism? I would prefer to see two-way merges going on, at least until 2.5 is released (i.e. no changes to Modules/ctypes except for bug fixes). Using svn:external is a risky thing wrt. to branching/tagging: When we tag the Python tree, we want to tag the entire source tree. With svn:external, only the external link would be in the tag, i.e. later changes to the external link would modify old tags. This is undesirable. This problem could be solved with a versioned external link; this would mean that ctypes could not be edited directly, but that one would have to go through the original repository URL to perform modifications, and then update the external link. So I think I still would prefer two-way merges. There are tools to make the merges pretty mechanic. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal to eliminate PySet_Fini
Kristján V. Jónsson wrote: > As a side note, is there a finalization order list for imported modules? If they are Python modules, more or less, yes. Extension modules cannot currently be finalized (I plan to change that for Py3k). See PyImport_Cleanup for the precise algorithm used; there are patches floating around to make this rely more on the garbage collector. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ImportWarning flood
On Sun, 25 Jun 2006 17:51:17 -0700, Guido van Rossum <[EMAIL PROTECTED]> wrote: >On 6/24/06, Jean-Paul Calderone <[EMAIL PROTECTED]> wrote: >> >Actually, your application *was* pretty close to being broken a few >> >weeks ago, when Guido wanted to drop the requirement that a package >> >must contain an __init__ file. In that case, "import math" would have >> >imported the directory, and given you an empty package. >> >>But this change was *not* made, and afaict it is not going to be made. > >Correct. We'll stick with the warning. (At least until Py3k but most >likely also in Py3k.) > Even given that it emits completely spurious warnings for any package that happens to share a name with a directory in whatever the working path is (say, your home directory)? How about if someone grovels through import.c and figures out how to make the warning information only show up if the import actually fails? Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ImportWarning flood
On 6/30/06, Jean-Paul Calderone <[EMAIL PROTECTED]> wrote: > On Sun, 25 Jun 2006 17:51:17 -0700, Guido van Rossum <[EMAIL PROTECTED]> > wrote: > >On 6/24/06, Jean-Paul Calderone <[EMAIL PROTECTED]> wrote: > >> >Actually, your application *was* pretty close to being broken a few > >> >weeks ago, when Guido wanted to drop the requirement that a package > >> >must contain an __init__ file. In that case, "import math" would have > >> >imported the directory, and given you an empty package. > >> > >>But this change was *not* made, and afaict it is not going to be made. > > > >Correct. We'll stick with the warning. (At least until Py3k but most > >likely also in Py3k.) > > > > Even given that it emits completely spurious warnings for any package that > happens to share a name with a directory in whatever the working path is > (say, your home directory)? > > How about if someone grovels through import.c and figures out how to make > the warning information only show up if the import actually fails? That would work I think. But it's not easy. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sys.settrace() in Python 2.3 vs. 2.4
Josiah Carlson wrote: > Any pointers as to why there is a difference would be appreciated. This was fixed in r35540, r35541, r35542, r35543, by Nick Bastin and Armin Rigo, in response to #765624. Enough pointers :-? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] how long to wait for expat to incorporate a fix to prevent a crasher?
Brett Cannon wrote: > The question is how long do we wait for the expat developers to patch > and do a micro release? Do we just leave this possible crasher in and > just rely entirely on the expat developers, or do we patch our copy and > use that until they get around to doing their next version push? If you have a patch, you should commit it to our copy. Make sure you activate the test case, so that somebody incorporating the next Expat release doesn't mistakenly roll back your change. Of course, you might wait a few days to see whether Fred creates another release that we could incorporate without introducing new features. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Empty Subscript PEP on Wiki - keep or toss?
Noam Raphael posted an empty subscript PEP on the Python Wiki: http://wiki.python.org/moin/EmptySubscriptListPEP It's not linked to by any other pages on the wiki. Is there a reason it wasn't added to the peps repository? Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pickle implementation questions
Bruce Christensen wrote:
> Thanks! That helps a lot. PEP 307 and the pickle module docs describe the end
> result pretty well, but they don't always make it clear where things are
> implemented. I'm trying to make sure that I'm getting the right interaction
> between object.__reduce(_ex)__, pickle, and copy_reg..
You really should ignore the existance of copy_reg._reduce_ex. It's an
implementation detail - it could have been implemented just as well in
C directly, in which case you couldn't as easily replace it with a
different function. It might be implemented that way in the next
release, or the entire __reduce_ex__ implementation might be lifted
to Python some day.
> One (hopefully) last question: is object.__reduce(_ex)__ really implemented in
> object?
Sure.
> Traceback (most recent call last):
> File "", line 1, in ?
> File "C:\Python24\lib\pickle.py", line 1386, in dumps
> Pickler(file, protocol, bin).dump(obj)
> File "C:\Python24\lib\pickle.py", line 231, in dump
> self.save(obj)
> File "C:\Python24\lib\pickle.py", line 313, in save
> rv = reduce(self.proto)
> File "", line 2, in bomb
> Exception: KABOOM! (, 0) {}
You don't get a stack frame for C functions (normally, anyway):
there is no file/line number information available.
The reduce thing you are seeing really comes from
# Check for a __reduce_ex__ method, fall back to __reduce__
reduce = getattr(obj, "__reduce_ex__", None)
Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python memory model (low level)
Nick Maclaren <[EMAIL PROTECTED]> wrote: > "Tim Peters" <[EMAIL PROTECTED]> wrote: > > [ Many useful answers ] > > Thanks very much! That helps. Here are a few points where we are at > cross-purposes. [snip] > I believe that, using the above approach, it would be possible to > achieve good efficiency with very little C - certainly, it has worked > in other languages. If I understand you correctly (I apologize if I am not), you are talking about subclassing. Subclassing already has a mechanism and implementation in CPython, both in the Python-language and C-implementation levels. Further, I would expect that everything relating to Decimal's current functionality (contexts, etc.) will be implemented appropriately. If I were to offer any advice to you, it would be to relax for a few months and let the Google Summer of Code project complete. I have faith in Facundo's mentoring capability (in that I have faith in his original decimal implementation), and I expect that the C implementation of Decimal will satisfy the vast majority of your concerns about floating point math and Python. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Empty Subscript PEP on Wiki - keep or toss?
[EMAIL PROTECTED] wrote: > Noam Raphael posted an empty subscript PEP on the Python Wiki: > > http://wiki.python.org/moin/EmptySubscriptListPEP > > It's not linked to by any other pages on the wiki. Is there a reason it > wasn't added to the peps repository? The most likely reason is that he didn't submit the PEP to the PEP editors. The next likely reason is that the PEP editors did not have time to add it, yet. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Empty Subscript PEP on Wiki - keep or toss?
[EMAIL PROTECTED] wrote: > Noam Raphael posted an empty subscript PEP on the Python Wiki: > > http://wiki.python.org/moin/EmptySubscriptListPEP > > It's not linked to by any other pages on the wiki. Is there a reason it > wasn't added to the peps repository? Perhaps the author forgot to submit it to the PEP editor, or he decided to abandon it after the mostly negative discussion here. Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pickle implementation questions
[Tim Peters]
>> I hope you've read PEP 307:
[Bruce Christensen]
> I have. Thanks to you and Guido for writing it! It's been a huge help.
You're welcome -- although we were paid for that, so thanks aren't needed ;-)
>> The implementation is more like:
>> [snip]
> Thanks! That helps a lot. PEP 307 and the pickle module docs describe the end
> result pretty well, but they don't always make it clear where things are
> implemented.
Well, "where" and "how" are implementation details. Alas, those
aren't always clearly separated from the semantics (and since Guido &
I both like operational definitions, stuff we write is especially
prone to muddiness on such points). The layers of backward
compatibility for now out-of-favor gimmicks don't help either -- this
is akin to reading the Windows API docs, finding around six functions
that _sound_ relevant, and then painfully discovering none of them
actually do what you hope they do, one at a time :-)
> I'm trying to make sure that I'm getting the right interaction between
> object.__reduce(_ex)__, pickle, and copy_reg.
Alas, I'm sure I don't remember sufficient details anymore myself.
> One (hopefully) last question: is object.__reduce(_ex)__ really implemented in
> object?
Yes, although I think you're overlooking this bit of the "acts as if"
pseudo-implementation from my last note:
elif proto < 2:
return copy_reg._reduce_ex(self, proto)
That is, the `object` implementation left the proto < 2 cases coded in
Python. You won't get to the (hoped to be) common path:
else:
# about 130 lines of C code exploiting proto 2
unless you ask for proto 2.
> The tracebacks below would indicate that pickle directly implements the
> behavior that the specs say is implemented in object. However, that could be
> because frames from C code don't show up in tracebacks.
That's right, they don't, and the C `object` code calls back into
copy_reg in proto < 2 cases.
> I'm not familiar enough with CPython to know for sure.
>
> >>> import copy_reg
> >>> def bomb(*args, **kwargs):
> ... raise Exception('KABOOM! %r %r' % (args, kwargs))
> ...
> >>> copy_reg._reduce_ex = bomb
> >>> import pickle
> >>> pickle.dumps(object())
You're defaulting to protocol 0 there, so, as above, the `object`
implementation actually calls copy_reg._reduce_ex(self, 0) in this
case. Much the same if you do:
>>> pickle.dumps(object(), 1)
I think it's a misfeature of pickle that it defaults to the oldest
protocol instead of the newest, but not much to be done about that in
Python 2.
Do one of these instead and the traceback will go away:
>>> pickle.dumps(object(), 2)
>>> pickle.dumps(object(), -1)
>>> pickle.dumps(object(), pickle.HIGHEST_PROTOCOL)
> Traceback (most recent call last):
> File "", line 1, in ?
> File "C:\Python24\lib\pickle.py", line 1386, in dumps
> Pickler(file, protocol, bin).dump(obj)
> File "C:\Python24\lib\pickle.py", line 231, in dump
> self.save(obj)
> File "C:\Python24\lib\pickle.py", line 313, in save
> rv = reduce(self.proto)
> File "", line 2, in bomb
> Exception: KABOOM! (, 0) {}
It's _certainly_ an implementation accident that the `object` coding
happens to call back into `copy_reg` here. There was no intent that
users be able to monkey-patch copy_reg and replace _reduce_ex(). It
was left coded in Python purely as a cost/benefit tradeoff.
> >>> class NewObj(object):
> ... def __reduce__(self):
> ... raise Exception("reducing NewObj")
In this case, it doesn't matter at all how `object` implements
__reduce__ or __reduce_ex__, because you're explicitly saying that
NewObj has its own __reduce__ method, and that overrides `object`'s
implementation. IOW, you're getting exactly what you ask for in this
case, and regardless of pickle protocol specified:
> >>> import pickle
> >>> pickle.dumps(NewObj())
Ask for protocols 1 or 2 here, and you'll get the same traceback. It
would be a bug if you didn't :-)
> Traceback (most recent call last):
> File "", line 1, in ?
> File "C:\Python24\lib\pickle.py", line 1386, in dumps
> Pickler(file, protocol, bin).dump(obj)
> File "C:\Python24\lib\pickle.py", line 231, in dump
> self.save(obj)
> File "C:\Python24\lib\pickle.py", line 313, in save
> rv = reduce(self.proto)
> File "", line 3, in __reduce__
> Exception: reducing NewObj
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sys.settrace() in Python 2.3 vs. 2.4
"Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > > Josiah Carlson wrote: > > Any pointers as to why there is a difference would be appreciated. > > This was fixed in r35540, r35541, r35542, r35543, by Nick Bastin > and Armin Rigo, in response to #765624. Enough pointers :-? Yes, thank you Martin. I would guess it wasn't backported to the 2.3 branch due to a change in the maybe_call_line_trace() definition, which answers my question, but makes me sad (I wanted this functionality *pout*). I'll just have to gracefully degrade functionality for older Pythons. Thank you again Martin, - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python memory model (low level)
Tim Peters wrote: > Don't know what "raw data" might mean here. Any Python object can be > bound to any attribute of a class. In Python, e.g., > > class MyClass: > > mydata = ['xyz', 12] > > def method(self): > MyClass.mydata.append(-1) > # or, more inheritance-friendly > self.__class__.mydata.append(-1) > > This is substantially more long-winded in C. To get a starting point: PyDict_GetItemString(MyClass->tp_dict, "mydata") is the equivalent of self.__class__.mydata That way, the raw data would get exposed to the Python level. If you don't want this to happen, you could also revert the lookup: static PyObject *mydata; /* = PyDict_New() */ and then PyDict_GetItem(mydata, MyClass) If "raw" means "non-PyObject", you would have to wrap the raw data pointer with a CObject first. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] how long to wait for expat to incorporate a fix to prevent a crasher?
On 6/30/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: Brett Cannon wrote:> The question is how long do we wait for the expat developers to patch> and do a micro release? Do we just leave this possible crasher in and> just rely entirely on the expat developers, or do we patch our copy and > use that until they get around to doing their next version push?If you have a patch, you should commit it to our copy. Make sure youactivate the test case, so that somebody incorporating the next Expat release doesn't mistakenly roll back your change.OK, will do. Of course, you might wait a few days to see whether Fred creates anotherrelease that we could incorporate without introducing new features. Yeah, I am going to wait a little while.-Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
On Fri, 30 Jun 2006, Andrew Koenig wrote: > I saw messages out of sequence and did not realize that this would be a > change in behavior from 2.4. Sigh. Yes, this is not a good time to change it. > I hope Py3000 has lexical scoping a la Scheme... Me too -- that would be really nice. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] For sandboxing: alternative to crippling file()
On Fri, 30 Jun 2006, Brett Cannon wrote: > On 6/30/06, Armin Rigo <[EMAIL PROTECTED]> wrote: > > >>> object.__subclasses__() > > [..., ] > > > > Maybe this one won't work if __subclasses__ is forbidden, but in general > > I think there *will* be a way to find this object. > > Yeah, that's been my (what I thought was paranoid) feeling. Glad I am not > the only one who thinks that hiding file() is near impossible. If you want to do this right, it should be about *making* hiding possible. If you can't hide things, it will be hard to get very far. I realize that may be difficult for Python 2.x, but hiding is pretty essential for security. It would be really good to keep this in mind for the design of Python 3k. (It doesn't mean we can't have introspection, just that we need to agree on some discipline for how to do it.) -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] traceback regression
python.org/sf/1515343 fixes python.org/sf/1515163, which is a new-in-2.5 regression. On the one hand, the regression only affects >>> raise "string1", "string2" which is both obscure and deprecated. On the other hand, it is a regression, and it is something I bumped into while working with unittest. [Note that I probably won't be checking email again for a week, so I'm afraid I won't be very responsive to comments.] -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
On 6/30/06, Ka-Ping Yee <[EMAIL PROTECTED]> wrote: > On Fri, 30 Jun 2006, Andrew Koenig wrote: > > I saw messages out of sequence and did not realize that this would be a > > change in behavior from 2.4. Sigh. > > Yes, this is not a good time to change it. > > > I hope Py3000 has lexical scoping a la Scheme... > > Me too -- that would be really nice. That's not a very constructive proposal (especially since I don't know Scheme). Perhaps you could elaborate on what needs to change? -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] For sandboxing: alternative to crippling file()
On 6/30/06, Ka-Ping Yee <[EMAIL PROTECTED]> wrote: On Fri, 30 Jun 2006, Brett Cannon wrote:> On 6/30/06, Armin Rigo <[EMAIL PROTECTED]> wrote:> > >>> object.__subclasses__()> > [..., ] > >> > Maybe this one won't work if __subclasses__ is forbidden, but in general> > I think there *will* be a way to find this object.>> Yeah, that's been my (what I thought was paranoid) feeling. Glad I am not > the only one who thinks that hiding file() is near impossible.If you want to do this right, it should be about *making* hidingpossible. If you can't hide things, it will be hard to get very far. Well, this is only file() we are worrying about leaking out. Stuff from import are the worry here. I realize that may be difficult for Python 2.x, but hiding is prettyessential for security. It would be really good to keep this in mindfor the design of Python 3k. (It doesn't mean we can't have introspection, just that we need to agree on some discipline for how to do it.)That's fine; I personally have no issue with tweaking the security model for Py3K. But I am worrying about 2.x here.-Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Lexical scoping in Python 3k
On Fri, 30 Jun 2006, Guido van Rossum wrote: > On 6/30/06, Ka-Ping Yee <[EMAIL PROTECTED]> wrote: > > On Fri, 30 Jun 2006, Andrew Koenig wrote: > > > I hope Py3000 has lexical scoping a la Scheme... > > > > Me too -- that would be really nice. > > That's not a very constructive proposal (especially since I don't know > Scheme). Perhaps you could elaborate on what needs to change? Sorry! I should have been more clear. Right now, Python supports the existence of nested scopes: a = 3 def f(): a = 4 def g(): a = 5 print a # prints 5 g() print a # prints 4 f() print a # prints 3 The above example shows that there are three distinct scopes, and that each one has a distinct binding named 'a' -- assigning to one doesn't affect the others. a = 3 def f(): b = 4 def g(): c = 5 print a, b, c # i can see all three g() f() The above example shows that all of the scopes can be *read*. But in today's Python, not all of the scopes can be *written*. a = 3 def f(): b = 4 def g(): c = 5 a, b, c = 0, 1, 2 # changes local c, not outer a and b g() f() The code in g() can affect its own local, 'c', and it can affect the global variable 'a' if it declares 'global a', but no matter what you write in g(), it cannot assign to 'b' (or to any other intermediate scope). This is a strange limitation and it would be nice to remove it. The asymmetry comes from Python having one foot in the new paradigm of nested lexical scopes and one foot still in the older paradigm of only two scopes, local and global. Most other languages that support lexical scoping (including Scheme, JavaScript, Ruby, Perl, E, Java, Smalltalk) provide a uniform way to read and write to scopes at all levels. This is done by letting programmers specify the scope in which they want a variable bound (usually with a keyword like "var" in JavaScript, "my" in Perl, or "define" in E). So here are some thoughts on how Python might be adjusted to support this. I'm not saying these would be the only ways, but at least they're some ideas to start with. In JavaScript, the "var" keyword is required whenever you want to declare a local variable. Anything without "var" is assumed to be a global name. The cleanest and most consistent solution that comes to mind would be to adopt exactly this for Python. Without "var": a = 3 # global def f(): b = 4 # global def g(): c = 5 # global a, b, c = 0, 1, 2 # changes all three globals g() f() print a, b, c # prints 0, 1, 2 With "var": var a = 3 # global def f(): var b = 4 # local to f def g(): var c = 5 # local to g a, b, c = 0, 1, 2 # changes outer a, outer b, and c print c # prints 2 g() print b # prints 1 f() print a # prints 0 print b # no such name print c # no such name But that is a big change. Perhaps it would be too unpythonic to suddenly require declarations for all local variables. So an alternative would be to retain the default assumption that undeclared variables are local. Here's what we'd get: Without "var": a = 3 def f(): b = 4 def g(): c = 5 a, b, c = 0, 1, 2 # changes local c, not outer a and b g() f() With "var": var a = 3 def f(): var b = 4 def g(): var c = 5 a, b, c = 0, 1, 2 # changes outer a, outer b, and c g() f() Now i think this is a little bit weird, because the statement "var b = 4" in an outer scope changes the meaning of "b" in an inner scope. But it does have the virtue of retaining behaviour compatible with today's Python, while offering a way to get proper lexical scopes for those who want to use them. Thoughts? Other ideas? -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
> That's not a very constructive proposal (especially since I don't know > Scheme). Perhaps you could elaborate on what needs to change? The fundamental principle is that the binding of every name is determined during compilation, not during execution. This property does not quite apply to Python at present. For example: x = 42 def f(): y = x x = 123 return y f() This example fails with "local variable 'x' referenced before assignment" because the compiler sees that f contains an assignment to x, so it makes x a local variable, and then when you try to assign x to y during execution, it fails. This behavior is consistent with the notion of lexical scoping. However, if I write def g(): return x x = 42 g() the result is 42. With lexical scoping, I believe it should be undefined. The reason is that when the compiler encounters the definition of g, variable x is not yet bound, and there is nothing in the body of g that would bind it. Therefore, g requires a binding to exist at the time it is compiled; because no such binding exists, this example would be an error (at compile time) under lexical scoping. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
Guido van Rossum wrote: > On 6/30/06, Ka-Ping Yee <[EMAIL PROTECTED]> wrote: >> On Fri, 30 Jun 2006, Andrew Koenig wrote: >>> I hope Py3000 has lexical scoping a la Scheme... >> Me too -- that would be really nice. > > That's not a very constructive proposal (especially since I don't know > Scheme). Perhaps you could elaborate on what needs to change? I believe the essence of their request for lexical scope boils down to allowing rebinding. Such code like the following is legal in Scheme: def f(x): def incr(): x = x + 1 return x def decr(): x = x - 1 return x return (incr, decr) (incr, decr) = f(1) print incr() # 2 print incr() # 3 print decr() # 2 print decr() # 1 -- FWIW, the Scheme equivalent would be something like: (define f (lambda (x) (list (lambda () (set! x (+ x 1)) x) (lambda () (set! x (- x 1)) x (let ([fs (f 1)]) (let ([incr (car fs)] [decr (cadr fs)]) (display (incr)) (newline); 2 (display (incr)) (newline); 3 (display (decr)) (newline); 2 (display (decr)) (newline))) ; 1 As a more personal aside, I can't imagine where I would use this in any python program I have ever wrote. I actually never noticed that rebinding wasn't allowed until recently. -- Scott Dial [EMAIL PROTECTED] [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
[Andrew Koenig] >>> I saw messages out of sequence and did not realize that this would be a >>> change in behavior from 2.4. Sigh. [Ka-Ping Yee] >> Yes, this is not a good time to change it. >>> I hope Py3000 has lexical scoping a la Scheme... >> Me too -- that would be really nice. [Guido] > That's not a very constructive proposal (especially since I don't know > Scheme). Perhaps you could elaborate on what needs to change? It's effectively the opposite of Python <0.1 wink>: a name is local to a scope in Scheme if and only if a declaration says it is. For example, the "let" family of forms is often used for this purpose, and (let ((x 2) (y 3))# declares x and y as local to this `let`, and gives initial values (let ((x 7) (z (+ x y))) # x comes from outer `let` so is 2, and z is 2+3=5 (* z x)))# x comes from inner `let`, so this is 5*7=35 If you use `let*` instead of `let` in the inner one, z picks up the inner x=7, so that z is 7+3=10, and the result is 7*10 = 70 instead. The bindings in a `let` "happen" in an undefined order. In `let*`, a binding is visible "to its right" within the `let*`. Then there's `letrec`, which allows establishing mutually recursive bindings. While the `let` family is entirely about declaration, there are lots of other forms that mix in some declaration as part of their purpose (for example, the names in a lambda's argument list are local to the lambda's body), but they're all explicit in Scheme. I read "a la Scheme" here as "actually nothing like Scheme, except I want a non-tricky way to rebind a name from an enclosing scope within an enclosed scope". In Scheme, the scope a name x belongs to is found by searching enclosing scopes until you hit the first with an explicit "x belongs to me" declaration (OK, there's a hokey fallback to "top level" definitions too). Searching "textually up" always suffices (there's no form of delayed declaration -- a name must be declared before use). Scheme's assignment of assignment: (set! variable expression) has nothing to do with establishing the scope of `variable`. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
> That sounds like a bug, not a feature. It's frequently useful to have > forward references in function bodies to names that are not yet globally > bound, e.g. for classes, or mutually-recursive functions. The trouble is that you don't necessarily know in what scope they will be defined, so I think that forcing you to be explicit about it is useful. Can you show me an example of where you think it isn't? Incidentally, I think that lexical scoping would also deal with the problem that people often encounter in which they have to write things like "lambda x=x:" where one would think "lambda x:" would suffice. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
> I read "a la Scheme" here as "actually nothing like Scheme, except I > want a non-tricky way to rebind a name from an enclosing scope within > an enclosed scope". Almost. What I really want is for it to be possible to determine the binding of every name by inspecting the source text of the program. Right now, it is often possible to do so, but sometimes it isn't. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
At 07:04 PM 6/30/2006 -0400, Andrew Koenig wrote: >However, if I write > > def g(): > return x > x = 42 > g() > >the result is 42. With lexical scoping, I believe it should be undefined. > >The reason is that when the compiler encounters the definition of g, >variable x is not yet bound, and there is nothing in the body of g that >would bind it. Therefore, g requires a binding to exist at the time it is >compiled; because no such binding exists, this example would be an error (at >compile time) under lexical scoping. That sounds like a bug, not a feature. It's frequently useful to have forward references in function bodies to names that are not yet globally bound, e.g. for classes, or mutually-recursive functions. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
[Andrew Koenig] > ... > Incidentally, I think that lexical scoping would also deal with the problem > that people often encounter in which they have to write things like "lambda > x=x:" where one would think "lambda x:" would suffice. They _shouldn't_ encounter that at all anymore. For example, >>> def f(x): ... return lambda: x+1 >>> f(3)() 4 works fine in modern Pythons. Earlier Python's had a no-exceptions 3-scope implementation (local, global, builtin), and in those the "x" in the lambda body was "not local" (was either global or builtin, although the compiler couldn't tell which of those two it was). In _those_ Pythons people had to write "lambda x=x: x+1" instead, to suck the binding of the outer x into the lambda body, but if people are still doing that they're confused. Modern Pythons do have lexical scoping + global + builtin, although there's no way to spell "rebind a name local to an outer scope from within an inner scope". ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
On 6/30/06, Andrew Koenig <[EMAIL PROTECTED]> wrote: > > I read "a la Scheme" here as "actually nothing like Scheme, except I > > want a non-tricky way to rebind a name from an enclosing scope within > > an enclosed scope". > > Almost. What I really want is for it to be possible to determine the > binding of every name by inspecting the source text of the program. Right > now, it is often possible to do so, but sometimes it isn't. Then your example def f(): return x x = 42 print f() is entirely well-defined -- x is a global and the compiler in fact generates code that benefits from knowing that it's not a local. Python knows which locals there are; also which locals there are in surrounding function scopes. It *could* also know which globals and builtins there are, except the language currently allows dynamic rebinding of module-level variables so that they replace builtins. E.g. def f(): return len([]) print f() # prints 0 def len(x): return "booh" print f() # prints "booh" del len print f() # prints 0 again Worse, instead if explicitly overriding len in the module, it could have been an assignment to __main__.len in some other module. We've been thinking on how to deal with this for years, since nobody really likes it in all its freedom. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Lexical scoping in Python 3k
Ka-Ping Yee <[EMAIL PROTECTED]> wrote: [snip lexical scoping option] > Now i think this is a little bit weird, because the statement > "var b = 4" in an outer scope changes the meaning of "b" in an > inner scope. But it does have the virtue of retaining behaviour > compatible with today's Python, while offering a way to get proper > lexical scopes for those who want to use them. > > Thoughts? Other ideas? Using a keyword in an outer scope to state that a variable could be used in a nested scope is counter to the current method for accessing a parent scope with 'global'. Using 'var' as the equivalent of 'global', only for nested scopes, would be a more reasonable approach. However, I'm -1 on the feature now, for the same reasons I've been -1 on the feature for the last 2 times it has come up. In many of the cases where lexically nested scopes have been used to solve problems in Python, and programmers have run into a 'limitation' where not being able to modify a value in a parent scope has hindered them, the problem could have been better solved with another method that was more readable, more extensible, etc. What I asked before, and what I'd like to ask again, is if there are any _nontrivial uses_ of lexically nested scopes which are made cumbersome by our inability to write to parent scopes. If there aren't, then I'm going to again have to argue against new syntax, keywords, and their use. If there are, then we'll see how compelling such uses are. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
On Fri, 30 Jun 2006, Andrew Koenig wrote: > The fundamental principle is that the binding of every name is determined > during compilation, not during execution. This property does not quite > apply to Python at present. I think this property does apply. In your example: > def g(): > return x > x = 42 > g() > > the result is 42. It is already known at compile time that the "return x" in g() refers to an 'x' in the outer scope. 'x' cannot be a local variable to g() because there are no statements in g() that bind 'x'. Regardless of whether the binding itself exists yet, you (the reader) and the compiler can know which scope to look in for the binding at runtime. Have i understood your desired property correctly? -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
[Andrew Koenig] > Almost. What I really want is for it to be possible to determine the > binding of every name by inspecting the source text of the program. Right > now, it is often possible to do so, but sometimes it isn't. Local names are always determined at compile-time in Python. What you can't always determine is whether a non-local (to any enclosing scope) name will end up getting resolved from the module globals or from __builtin__. The runtime error in: def f(): y = x x = 1 f() doesn't occur because Python doesn't know "x" is local to "f" at compile-time (it does know that), it's because Python's compiler doesn't do any flow analysis to detect potential use-before-definition. Instead the runtime initalizes locals to a special "not bound yet" value that the LOAD_FAST (really "load local") opcode special-cases. Note that this is quite unlike Scheme, in which declaration must appear before use (ignoring fancy letrec cases), and declaration must also supply an initial binding (Scheme has no "unbound local" problem because there's no way to create an uninitialized local). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
Tim Peters <[EMAIL PROTECTED]> wrote: >> ... >> Incidentally, I think that lexical scoping would also deal with the >> problem >> that people often encounter in which they have to write things like >> "lambda >> x=x:" where one would think "lambda x:" would suffice. > > They _shouldn't_ encounter that at all anymore. For example, > def f(x): > ... return lambda: x+1 f(3)() > 4 > > works fine in modern Pythons. Yes but: >>> a = [] >>> for i in range(10): ... a.append(lambda: i) ... >>> print [x() for x in a] [9, 9, 9, 9, 9, 9, 9, 9, 9, 9] This subtle semantic of lambda is quite confusing, and still forces people to use the "i=i" trick. Giovanni Bajo ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
[Giovanni Bajo] > Yes but: > > >>> a = [] > >>> for i in range(10): > ... a.append(lambda: i) > ... > >>> print [x() for x in a] > [9, 9, 9, 9, 9, 9, 9, 9, 9, 9] > > This subtle semantic of lambda is quite confusing, and still forces people to > use the "i=i" trick. So stay away from excruciating abuses of lexical scoping you don't understand ;-) What do you _expect_ `i` to refer to? "Oh, it should guess that I didn't really mean to defer evaluation of the lambda body at all, but instead evaluate the lambda body at the time I define the lambda and then synthesize some other function that captures the specific outer bindings in effect at lambda-definition time" doesn't really cut it. Try spelling what you think you want here in Scheme. Before it works, you'll probably end up with some equally "atrocious" (let ((i i)) ...) gimmick to force capturing each binding for `i` as it flies by. Else Scheme will also use the outer binding for `i` in effect at the time the lambdas are _executed_. This isn't typical use for lambda, and I don't think it's what Andrew had in mind. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Lexical scoping in Python 3k
> With "var": > > var a = 3 > def f(): > var b = 4 > def g(): > var c = 5 > a, b, c = 0, 1, 2 # changes outer a, outer b, and c > g() > f() > > Now i think this is a little bit weird, because the statement > "var b = 4" in an outer scope changes the meaning of "b" in an > inner scope. But it does have the virtue of retaining behaviour > compatible with today's Python, while offering a way to get proper > lexical scopes for those who want to use them. > > Thoughts? Other ideas? Maybe an object, like self, for referring to enclosing scopes? a = 3 def f(): b = 4 def g(): c = 5 outer.outer.a, outer.b, c = 0, 1, 2 # changes outer a, outer b, and c g() f() Chaining the keyword looks a little weird, but it is not often that you have to refer to variables in the enclosing scope of the enclosing scope. I have often wanted something similar to that for global variables, instead of the global declaration: cache = None def init(): if not global.cache: global.cache = init_cache() -- mvh Björn ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
> >>> a = [] > >>> for i in range(10): > ... a.append(lambda: i) > ... > >>> print [x() for x in a] > [9, 9, 9, 9, 9, 9, 9, 9, 9, 9] Isn't this exactly what you'd expect? Maybe I've been writing Python for too long... :-). Bill ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
[Giovanni Bajo]
> Yes but:
>
a = []
for i in range(10):
> ... a.append(lambda: i)
> ...
print [x() for x in a]
> [9, 9, 9, 9, 9, 9, 9, 9, 9, 9]
>
> This subtle semantic of lambda is quite confusing, and still forces people to
> use the "i=i" trick.
[Tim Peters]
> So stay away from excruciating abuses of lexical scoping you don't
> understand What do you expect `i` to refer to? "Oh, it should
> guess that I didn't really mean to defer evaluation of the lambda body
> at all, but instead evaluate the lambda body at the time I define the
> lambda and then synthesize some other function that captures the
> specific outer bindings in effect at lambda-definition time" doesn't
> really cut it.
I think I understand what happens, I just don't know whether this can be
"fixed" or not. Unless you are saying that the above behaviour is not only a
complex side-effect the way things are, but the way things should be. Do you
agree that it would be ideal if the above code generated range(10) instead of
[9]*10, or you believe that the current behaviour is more sound (and if so,
why)?
As for actual implementing this change of semantic, the fact that `i` is a
local variable in the outer scope (assuming it's all within a function),
doesn't make it possible for Python to early-bound it, by realizing that, since
`i` is not an argument of the lambda, and it's a local of the outer scope? At
worse, couldn't Python do the "i=i" trick by itself when it sees that `i` is a
local in the outer scope? Right now I can't think off-hand of a case in which
this would break things.
[Tim Peters]
> This isn't typical use for lambda,
Yes, maybe it's not the most used idiom and Andrew wasn't referring to this,
but it happens quite often to me (where 'often' means 'many times' among my
rare usages of lambda).
For instance, in GUI code, it's common to do things like:
for b in self.buttons:
self.setEventCallback(b, "clicked", lambda: self.label.setText("I pressed
button %r" % b))
... which of course won't work, as written above.
Giovanni Bajo
wink.gif
Description: GIF image
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ImportWarning flood
It's up to the release manager now to decide whether the pitchforks at Google or the pitchforks in the larger Python community are sharper. ;-) --Guido (ducks) On 6/30/06, Shane Hathaway <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: > > On 6/30/06, Jean-Paul Calderone <[EMAIL PROTECTED]> wrote: > >> How about if someone grovels through import.c and figures out how to make > >> the warning information only show up if the import actually fails? > > > > That would work I think. But it's not easy. > > I just posted a patch intended to solve this. With my patch, > find_module() collects the warnings in a list and only generates > ImportWarning if no package is found. It works, but I have not done > extensive testing. > > I also discovered and tried to resolve what appear to be memory leaks > involving the "copy" variable. If the ImportWarning patch doesn't fly, > I'll be happy to post a different patch that only fixes the leaks. > > The patch is #1515361: > > https://sourceforge.net/tracker/index.php?func=detail&aid=1515361&group_id=5470&atid=305470 > > Shane > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] zlib module build failure on Mac OSX 10.4.7
Just upgraded my Mac to OSX 10.4.7 yesterday. svn up'd Python trunk, then "make clean ; configure ; make" and I see that building the zlib module fails: gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -I/Users/skip/src/python-svn/trunk/./Include -I/Users/skip/src/python-svn/trunk/./Mac/Include -I/Users/skip/local/include -I../Include -I. -I/usr/local/include -I/Users/skip/src/python-svn/trunk/Include -I/Users/skip/src/python-svn/trunk/build -c /Users/skip/src/python-svn/trunk/Modules/zlibmodule.c -o build/temp.macosx-10.3-ppc-2.5/Users/skip/src/python-svn/trunk/Modules/zlibmodule.o /Users/skip/src/python-svn/trunk/Modules/zlibmodule.c: In function 'PyZlib_uncopy': /Users/skip/src/python-svn/trunk/Modules/zlibmodule.c:724: warning: implicit declaration of function 'inflateCopy' gcc -bundle -undefined dynamic_lookup build/temp.macosx-10.3-ppc-2.5/Users/skip/src/python-svn/trunk/Modules/zlibmodule.o -L/Users/skip/local/lib -L/usr/local/lib -lz -o build/lib.macosx-10.3-ppc-2.5/zlib.so -Wl,-search_paths_first *** WARNING: renaming "zlib" since importing it failed: dlopen(build/lib.macosx-10.3-ppc-2.5/zlib.so, 2): Symbol not found: _inflateCopy Referenced from: build/lib.macosx-10.3-ppc-2.5/zlib.so Expected in: dynamic lookup Anybody else seen this? I checked the buildbot trunk osx 10.4. It seemed to have no trouble. And what's with the "10.3" bit in the directory names? Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] doc for new restricted execution design for Python
Mark Hammond wrote: > that helps "mozilla the platform" more than it helps "firebox the browser" ^^^ Firebox - the sandfoxed web browser! -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
"Giovanni Bajo" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > Yes but: > a = [] for i in range(10): > ... a.append(lambda: i) > ... print [x() for x in a] > [9, 9, 9, 9, 9, 9, 9, 9, 9, 9] > > This subtle semantic of lambda is quite confusing, and still forces > people to > use the "i=i" trick. The 'subtle sematic' had nothing to do with lambda but with Python functions. The above is exactly equivalent (except the different .funcname) to a = [] for i in range(10): def f(): return i a.append(f) del f That should be equally confusing (or not), and equally requires the 'i=i' trick (or not). As is, either function definitiion is a constant and the loop makes useless duplicates. Either form would have the same effect is hoisted out of the loop. Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] For sandboxing: alternative to crippling file()
Brett Cannon wrote: > 1) Is removing 'file' from the builtins dict in PyInterpreterState (and > maybe some other things) going to be safe enough to sufficiently hide > 'file' confidently (short of someone being stupid in their C extension > module and exposing 'file' directly)? > > 2) Changing open() to return C-implemented delegate objects for files > (and thus won't type check, but this is Python so I am not worried about > that too much) and delegate socket objects for IP and URL addresses. My suggestion is to change things so that the constructor of the file type doesn't open files (at least in restricted mode). Then it wouldn't matter if untrusted code had real file objects, as they couldn't use them to get access to any other files. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
"Giovanni Bajo" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > [Giovanni Bajo] >> Yes but: >> > a = [] > for i in range(10): >> ... a.append(lambda: i) >> ... > print [x() for x in a] >> [9, 9, 9, 9, 9, 9, 9, 9, 9, 9] >. Do you agree that it would be ideal if the above code > generated range(10) instead of [9]*10, No. You are trying to reify an optical illusion resulting from putting a constant function definition inside a loop. Making the meaning of 'def f(): return i' depend on the definition-time context by partially and variably evaluating the body would make code much harder to read and understand. Consider: if a: i=666 def f(): return i > At > worse, couldn't Python do the "i=i" trick by itself when it sees that `i` > is a > local in the outer scope? Right now I can't think off-hand of a case in > which > this would break things. It would make code more fragile. for i in range(666): print name[i] ... ... def total(num): return cost[item]*num Now someone decides first loop should have more expressive loop var name and changes the first line to for item in range(666): print name[item] and the meaning of total is completely changed. Adding such long-range coupling between language statements strikes me as a poor idea. Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Lexical scoping in Python 3k
Ka-Ping Yee wrote: > while offering a way to get proper > lexical scopes for those who want to use them. I don't disagree with anything you said, but I think it would be a good idea to avoid using phrases like "proper lexical scopes", which is likely to set people off on a tangent. The issue isn't lexicality, it's writeability. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] zlib module build failure on Mac OSX 10.4.7
Maybe do a make distclean. There was a problem where old versions of zlib (those without inflateCopy) weren't supported. They are now, but it's a configure check. That coupled with the upgrade and the 10.3 in the pathname, seems like it's just something didn't get cleaned up properly. You could always rm -rf build/ n -- On 6/30/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Just upgraded my Mac to OSX 10.4.7 yesterday. svn up'd Python trunk, then > "make clean ; configure ; make" and I see that building the zlib module > fails: > > gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd > -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. > -I/Users/skip/src/python-svn/trunk/./Include > -I/Users/skip/src/python-svn/trunk/./Mac/Include -I/Users/skip/local/include > -I../Include -I. -I/usr/local/include > -I/Users/skip/src/python-svn/trunk/Include > -I/Users/skip/src/python-svn/trunk/build -c > /Users/skip/src/python-svn/trunk/Modules/zlibmodule.c -o > build/temp.macosx-10.3-ppc-2.5/Users/skip/src/python-svn/trunk/Modules/zlibmodule.o > /Users/skip/src/python-svn/trunk/Modules/zlibmodule.c: In function > 'PyZlib_uncopy': > /Users/skip/src/python-svn/trunk/Modules/zlibmodule.c:724: warning: > implicit declaration of function 'inflateCopy' > gcc -bundle -undefined dynamic_lookup > build/temp.macosx-10.3-ppc-2.5/Users/skip/src/python-svn/trunk/Modules/zlibmodule.o > -L/Users/skip/local/lib -L/usr/local/lib -lz -o > build/lib.macosx-10.3-ppc-2.5/zlib.so -Wl,-search_paths_first > *** WARNING: renaming "zlib" since importing it failed: > dlopen(build/lib.macosx-10.3-ppc-2.5/zlib.so, 2): Symbol not found: > _inflateCopy > Referenced from: build/lib.macosx-10.3-ppc-2.5/zlib.so > Expected in: dynamic lookup > > Anybody else seen this? I checked the buildbot trunk osx 10.4. It seemed > to have no trouble. And what's with the "10.3" bit in the directory names? > > Skip > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
Andrew Koenig wrote: > Incidentally, I think that lexical scoping would also deal with the problem > that people often encounter in which they have to write things like "lambda > x=x:" where one would think "lambda x:" would suffice. This is another red herring. Python's problem here is not because its scoping isn't lexical (it is). It's because Scheme implicitly introduces new scopes in various places where the equivalent Python constructs don't. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
Tim Peters wrote: > Note that this is quite unlike Scheme, in which declaration must > appear before use (ignoring fancy letrec cases), I think that's overstating things a bit -- mutually recursive functions are quite easy to write in Scheme and don't look at all "fancy" (unless you object for some reason to using (define ...)). > and declaration must > also supply an initial binding (Scheme has no "unbound local" problem > because there's no way to create an uninitialized local). That much is true. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Lexical scoping in Python 3k
Josiah Carlson wrote: > What I asked before, and what I'd like to ask again, is if there are any > _nontrivial uses_ of lexically nested scopes which are made cumbersome > by our inability to write to parent scopes. The trouble with taking that position is that the very cases which would benefit are very *simple* ones, where it would be cumbersome to refactor it to use a class, or mutable object in the outer scope, etc. So you've effectively set up your acceptance criteria to be unmeetable. > If there aren't, then I'm > going to again have to argue against new syntax, keywords, and their use. There's one very simple way we could do this in Py3k without requiring any new syntax or keywords: just redefine the meaning of "global" to mean "not local". -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 and beyond
Giovanni Bajo wrote: a = [] for i in range(10): > > ... a.append(lambda: i) > ... > print [x() for x in a] > > [9, 9, 9, 9, 9, 9, 9, 9, 9, 9] > > This subtle semantic of lambda is quite confusing, and still forces people to > use the "i=i" trick. This has *nothing* to do with the semantics of lambda! It's because Python's for-loop doesn't put its control variable in a new scope, the way Scheme's equivalent construct does. *That's* what needs to be addressed to fix this problem. I've made a suggestion about that before, but Guido rejected it, so I won't repeat it here. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
