[Python-Dev] Document performance requirements?
On 7/21/06, Nick Coghlan <[EMAIL PROTECTED]> wrote: > However, I'm also struggling to think of a case other than list vs deque where > the choice of a builtin or standard library data structure would be dictated > by big-O() concerns. OK, but that doesn't mean the information is unimportant. +1 on making this something of a priority. People looking for this info should find it in the obvious place. Some are unobvious. (How fast is dict.__eq__ on average? Worst case?) -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Caching float(0.0)
On 9/29/06, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > (I just checked the program I'm working on, and my analysis tells me > that the most common floating point value in that program is 121.216, > which occurs 32 times. from what I can tell, 0.0 isn't used at all.) *bemused look* Fredrik, can you share the reason why this number occurs 32 times in this program? I don't mean to imply anything by that; it just sounds like it might be a fun story. :) Anyway, this kind of static analysis is probably more entertaining than relevant. For your enjoyment, the most-used float literals in python25\Lib, omitting test directories, are: 1e-006: 5 hits 4.0: 6 hits 0.05: 7 hits 6.0: 8 hits 0.5: 13 hits 2.0: 25 hits 0.0: 36 hits 1.0: 62 hits There are two hits each for -1.0 and -0.5. In my own Python code, I don't even have enough float literals to bother with. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 355 status
On 9/30/06, Giovanni Bajo <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: > > OK. Pronouncement: PEP 355 is dead. [...] > > It would be terrific if you gave us some clue about what is > wrong in PEP355, [...] Here are my guesses. I believe Guido rejected this PEP for a lot of reasons. By the way, what I'm about to do is known as "channeling Guido (badly)" and I'm pretty sure it annoys him. Sorry, Guido. Please don't treat the following as authoritative; I have never met Guido and obviously I cannot speak for him. - I don't think Guido ever saw much benefit from "path objects". That is, the Motivation was not compelling. I think the main motivation is to eliminate some clutter and add a handful of useful methods to the stdlib, so it's easy to see how this could be the case. - Guido just flat-out didn't like the looks of the PEP. Too much weirdness. (path.py contains more weirdness, including some stuff Guido particularly disliked, and I think it's fair to say that PEP355 suffered somewhat by association.) - Any proposal to add a Second Way To Do It has to meet a very high standard. PEP355 was too big to be considered an incremental change. Yet it didn't even attempt to fix all the perceived problems with the existing APIs. A more thorough job would have had a better chance. - Nobody liked the API design--too many methods. - Now we're hearing rumors of better ideas out there, which comes as a relief. I suspect any one of these could have scuttled the proposal. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyFAQ: thread-safe interpreter operations
Way back on 11/22/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Nick Coghlan schrieb: > > Martin v. Löwis wrote: > >> I personally consider it "good style" to rely on implementation details > >> of CPython; > > > > Is there a 'do not' missing somewhere in there? > > No - I really mean it. I can find nothing wrong with people relying on > reference counting to close files, for example. It's a property of > CPython, and not guaranteed in other Python implementations - yet it > works in a well-defined way in CPython. Code that relies on that feature > is not portable, but portability is only one goal in software > development, and may be irrelevant for some projects. It's not necessarily future-portable either. Having your software not randomly break over time is relevant for most nontrivial projects. > Similarly, it's fine when people rely on the C type "int" to have > 32-bits when used with gcc on x86 Linux. Relying on behavior that's implementation-defined in a particular way for a reason (like int being 32 bits on 32-bit hardware) is one thing. Relying on behavior that even the implementors might not be consciously aware of (or consider important to retain across versions) is another. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyFAQ: thread-safe interpreter operations
On 11/27/06, Aahz <[EMAIL PROTECTED]> wrote: > On Mon, Nov 27, 2006, Jason Orendorff wrote: > > Way back on 11/22/06, "Martin v. L?wis" <[EMAIL PROTECTED]> wrote: > >> [...] I can find nothing wrong with people relying on > >> reference counting to close files, for example. It's a property of > >> CPython, and not guaranteed in other Python implementations - yet it > >> works in a well-defined way in CPython. [...] > > > > [Feh.] > > We recently had this discussion at my day job. We ended up agreeing > that using close() was an encouraged but not required style, because to > really avoid breakage we'd have to go with a full-bore try/except style > for file handling, and that would require too many changes (especially > without upgrading to 2.5, and we're still using 2.2/2.3). Well, CPython's refcounting is something Python-dev is (understatement) very conscious of. I think I've even heard assurances that it won't change Any Time Soon. But this isn't the case for every CPython implementation detail. Remember what brought all this up. If it's obscure enough that Fredrik Lundh has to ask around, I wouldn't bet the ranch on it. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Encouraging developers
On 3/5/07, A.M. Kuchling <[EMAIL PROTECTED]> wrote: > Any ideas for fixing this problem? The current developer FAQ says: 2.4 How can I become a developer? There's only one way to become a developer, and that's through the School of Hard Knocks. http://mail.python.org/pipermail/python-dev/2002-September/028725.html That's a little glib. And maybe inaccurate. That message (by Raymond Hettinger and probably not originally intended to be the first thing developers-aspirant see) seems at odds with Martin's characterization, in this thread, of Raymond's own experience. I would submit a doc patch, but what's the use. ;) I should be explicit-- I greatly admire the python-dev community and its processes. I don't get the feeling much happens in private e-mail. Quite the opposite: it feels like important decisions are regularly made on python-dev. I don't think it's hard to contribute. I don't think the stdlib is a huge mess of brokenness. And I don't think the community is either. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [ 1669539 ] Change (fix!) os.path.isabs() semantics on Win32
On 3/7/07, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Terry Jones schrieb: > > I do think the behavior can be improved, and that it should be fixed, but > > at a place where other incompatible changes will also be being made, > > Indeed, 2.6 is such a place. Any feature release can contain > incompatible behavior, and any feature release did contain incompatible > behavior. Just look at the "porting to" sections of past whatsnew files. While we're at it, patch 1669539 makes a similar incompatible change to ntpath.isabs(). On Windows there are: - true relative paths, like Lib\ntpath.py - true absolute paths, like C:\Python25 and \\server\share - oddities, like C:ntpath.py and \Python25 isabs() is inconsistent about oddities: >>> ntpath.isabs(r'C:ntpath.py') False >>> ntpath.isabs(r'\Python25') True I don't think there's any logic behind this behavior. The current documentation for isabs() is: isabs(path) Return True if path is an absolute pathname (begins with a slash). The patch makes isabs(oddity) return False. I don't think existing code is a huge concern here. Google Code Search suggests that no one thinks about the oddities. Most existing code using isabs() has acceptable-but-slightly-odd behavior for oddities, and that kind of code would have different acceptable-but-slightly-odd behavior under the proposed change. And oddities are rare. The patch is incomplete (no docs) but ripe for a note of encouragement (or summary rejection) from a committer. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Patch 1644818: Allow importing built-in submodules
On 3/12/07, Miguel Lobo <[EMAIL PROTECTED]> wrote: > Anyway, I'm intrigued about this "review 5 other patches" procedure you > suggest. What exactly would be involved in such a review? Please note that > I hadn't touched CPython code before I wrote my patch and I haven't been > following CPython development closely. Hi Miguel, This is how we suck you in... ;) You don't have to be an expert to review patches. The following procedure would qualify you: 1. Find a patch that it appears no one has ever touched (0 comments, assigned to nobody, etc.) 2. Pretty much every patch should include a unit test and documentation. If something is missing from the patch you're looking at, post a comment that says "Incomplete, no docs/tests". 3. Repeat until you've commented on five patches. If you find such clerical work beneath you, you can go further--build Python from source, apply patches, and verify that they work. It's not hard (google "python developer faq"). But it's not required. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] minidom and DOM level 2
The lib ref claims that minidom supports DOM Level 1. Does anyone know what parts of Level 2 are not implemented? I wasn't able to find anything offhand. It seems to be more a matter of what's not documented, or what's not covered by the regression tests. So. I'd be happy to do some diffing between the implementation, documentation, tests, and the Recommendation, and submit patches for whatever needs it. If anyone thinks that's worthwhile. Anyone? -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] minidom and DOM level 2
On 3/23/07, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Jason Orendorff schrieb: > > The lib ref claims that minidom supports DOM Level 1. Does anyone > > know what parts of Level 2 are not implemented? I wasn't able to find > > anything offhand. > > I now looked at it closely, and the only thing missing from DOM Level > 2 Core (that I could find) is the EntityReference interface, and > Document::createEntityReference. I'm not sure what semantics goes with it. OK, I think this is worthwhile then. :) I'll read the spec and submit a patch. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Hindsight on Py_UNICODE_WIDE?
Scheme is adding Unicode support in an upcoming standard: (DRAFT) http://www.r6rs.org/document/lib-html/r6rs-lib-Z-H-3.html I have two questions for the python-dev team about Python's Unicode experiences. If it's convenient, please take a moment to reply. Thanks in advance. 1. In hindsight, what do you think about PEP 261, the Py_UNICODE_WIDE build option? On balance, has this been good, bad, or indifferent? What's good/bad about it? 2. The idea of multiple string representations has come up (that is, where all strings are Unicode, but in memory some are 8-bit, some 16-bit, and some 32-bit--each string uses the narrowest possible representation). This has been discussed here for Python 3000. My question is: Is this for real? How far along is it? How likely is it? Thanks, Jason ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] minidom and DOM level 2
On 4/7/07, Andrew Clover <[EMAIL PROTECTED]> wrote: > Jason Orendorff wrote: > > OK, I think this is worthwhile then. :) I'll read the spec and submit > > a patch. > > You're planning to implement EntityReference in minidom? That'll be fun! > :-) One of the nastier corners of DOM and XML in general. Mmm. So I'm finding. EntityReferences seem to force detailed knowledge of entity handling into the DOM implementation. expat doesn't expose the information in a particularly helpful way. In a word, blaargh. I'd be happy to set this aside and work on Level 1 compliance: > Incidentally minidom falls far short of passing even Level 1 Core for > more reasons than omission of EntityReference. I noted the main known > problems with it here: >http://pyxml.sourceforge.net/topics/compliance.html Very nice. Thanks for posting this. I don't suppose you'd be willing to update it for Python 2.5, would you? Martin, have you looked at this? Some of these might be hard to fix, given expat. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] minidom and DOM level 2
On 4/7/07, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > In any case, the *claim* certainly is that minidom supports > level 2 core. Any proof to the contrary indicates a bug; > patches are welcome. OK-- I'll work on this. I can fix the easy ones, anyway. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] minidom and DOM level 2
On 4/13/07, Andrew Clover <[EMAIL PROTECTED]> wrote:
> Jason Orendorff wrote:
> > I don't suppose you'd be willing to update it for Python 2.5, would you?
>
> Can do, but at this point I'm not aware of any work having been done on
> the issues listed there between the 2.3 and 2.5 releases.
I've been running the DOM test suite against trunk, using your test
harness. It's kind of alarming at first that over 100 tests fail. :)
But many of the failures involve entity references.
An even larger portion involve error cases: we accept things we should
check and reject. For example, doc.createElement('\t') should fail.
These are certainly bugs, and they're easy to fix. I'm working
through them.
> The danger is people may be used to the "wrong" minidom behaviours,
> given they have been static for so long and are in many cases central to
> how minidom works.
When I get to these, I'll post about it.
-j
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] minidom -> new-style classes?
I'm working on minidom's DOM Level 1 compliance, targeting Python 2.6. We have some bugs involving DOM property behavior. For example, setting the nodeValue attribute of an Element is supposed to have no effect. We don't implement this. The right way to implement these quirks is using new-style classes and properties. Right now minidom uses old-style classes and lots of hackery, and it's pretty broken. (Another example--there is an Attr._set_prefix method, but it is *not* called from __setattr__.) Surely nobody is subclassing these classes. You don't subclass DOM interfaces--the DOM doesn't work that way. So this change should be OK. Right? -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] minidom -> new-style classes?
On 4/17/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > Perhaps a rewrite could target 3.0 and 2.6 could use a backported > version of this *if* py3k compatibility mode is enabled? I'd love to > see at least the 3.0 version cleaned up. A lot of these bugs can be fixed without forking. I've been conservative so far. I looked at a diff this morning. Even pickled documents shouldn't break. If this patch lands, I can look at further cleanup after that. Thanks everyone, -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Patch reviews and request
OK, here's the patch I'd like to direct attention to: http://python.org/sf/1704134 [ 1704134 ] minidom Level 1 DOM compliance This is only the first step toward DOM Level 1 compliance. It fixes the stuff that's easy to fix. Here are the patch reviews. I put more detailed comments in the SF tracker. http://python.org/sf/1704547 [ 1704547 ] Use MoveFileEx() to implement os.rename() on windows -1. This changes the documented behavior of a commonly used function. http://python.org/sf/1678345 [ 1678345 ] A fix for the bug #1528074 [warning: quite slow] This can be rejected. http://python.org/sf/1673007 [ 1673007 ] urllib2 requests history + HEAD support urllib2.urlopen() object seems like the wrong place for history to be attached. These objects are pretty ephemeral, in my code anyway. Patch is extremely rough. -1. http://python.org/sf/1665292 [ 1665292 ] Datetime enhancements The patch here adds __int__ and __float__ to datetime.timedelta. I'm -1 on that. It also implements > < == comparison between timedelta objects and numbers, which is right out--they don't have compatible hash codes. http://python.org/sf/1652328 [ 1652328 ] stream writing support in wave.py This can be rejected. (Update: After my comments on this one, Neal closed it.) The following patches look good, but I didn't attempt to run them. I just read the source code. http://python.org/sf/1669481 [ 1669481 ] subprocess: Support close_fds on Win32 Looks good, and a definite +1. http://python.org/sf/1704621 [ 1704621 ] interpreter crash when multiplying large lists Yep, it crashes. Patch looks good. +1. http://python.org/sf/1692664 [ 1692664 ] warnings.py gets filename wrong for eval/exec Nice to have. +1. http://python.org/sf/1676135 [ 1676135 ] Remove trailing slash from --prefix Also nice to have. +0. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Wither PEP 335 (Overloadable Boolean Operators)?
On 5/18/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > While reviewing PEPs, I stumbled over PEP 335 ( Overloadable Boolean > Operators) by Greg Ewing. -1. "and" and "or" affect the flow of control. It's a matter of taste, but I feel the benefit is too small here to add another flow-control quirk. I like that part of the language to be simple. Anyway, if this *is* done, logically it should cover "(... if ... else ...)" as well. Same use cases. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Order of operations
On 8/29/07, Dirkjan Ochtman <[EMAIL PROTECTED]> wrote: > Alexandre Vassalotti wrote: > > C doesn't have an exponentiation operator. You use the pow() function, > > instead: > > Wouldn't it make more sense, then, to have unary +/- have higher > precedence than the ** operator, so that -3**2 == 9? No, that would have been really bad. Anyone who's had high school algebra expects -x**2 to be -(x**2) and not (-x)**2. I think the weirdness comes from parsing -a/b as (-a)/b rather than -(a/b). It should be the latter, if compatibility with math notation is more important than compatibility with C. Oh well. Maybe in Python 4. :) -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Removing the GIL (Me, not you!)
On 9/12/07, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Now we are getting into details: you do NOT have to lock > an object to modify its reference count. An atomic > increment/decrement operation is enough. One could measure the performance hit incurred by using atomic operations for refcounting by hacking a few macros -- right? Deferred reference counting (DRC for short) might help... http://www.memorymanagement.org/glossary/d.html#deferred.reference.counting I can explain a little more how this works if anyone's interested. DRC basically eliminates reference counting for locals--that is, pointers from the stack to an object. An object becomes refcounted only when some other object gets a pointer to it. The drawback is that destructors aren't called quite as promptly as in true refcounting. (They're still called in the right order, though--barring cycles, an object's destructor is called before its children's destructors.) What counts as "stack" is up to the implementation; typically it means "the C stack". This could be used to eliminate most refcounting in C code, although listobject.c would keep it. The amount of per-platform assembly code needed is surprisingly small (and won't change, once you've written it--the Tamarin JavaScript VM does this). You could go further and treat the Python f_locals and interpreter stack as "stack". I think this would eliminate all refcounting in the interpreter. Of course, it complicates matters that f_locals is actually an object visible from Python. Just a thought, not a demand, please don't flame me, -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Removing the GIL (Me, not you!)
On 9/13/07, Justin Tulloss <[EMAIL PROTECTED]> wrote: > 1. Use message passing and transactions. [...] > 2. Do it perl style. [...] > 3. Come up with an elegant way of handling multiple python processes. [...] > 4. Remove the GIL, use transactions for python objects, [...] The SpiderMonkey JavaScript engine takes a very different approach, described here: http://developer.mozilla.org/en/docs/SpiderMonkey_Internals:_Thread_Safety The SpiderMonkey C API threading model should sound familiar: C code can assume that simple operations, like dictionary lookups, are atomic and thread-safe. C code must explicitly JS_SuspendRequest() before doing blocking I/O or number-crunching (just like Py_BEGIN_ALLOW_THREADS). The main difference is that SpiderMonkey's "requests" are not mutually exclusive, the way the GIL is. SpiderMonkey does fine-grained locking for mutable objects to avoid race conditions. The clever bit is that SpiderMonkey's per-object locking does *not* require a context switch or even an atomic instruction, in the usual case where an object is *not* shared among threads. (Programs that embed SpiderMonkey therefore run faster if they manage to ensure that threads share relatively few mutable objects. JavaScript doesn't have modules.) Suppose Python went this route. There would still have to be a "stop-the-world" global lock, because the cycle collector won't work if other threads are going about changing pointers. (SpiderMonkey's GC does the same thing.) Retaining such a lock has another advantage: this change could be completely backward-compatible to extensions. Just use this global lock as the GIL when entering a non-thread-safe extension (all existing extensions would be considered non-thread-safe). This means non-thread-safe extensions would be hoggish (but not much worse than they are already!). Making an existing extension thread-safe would require some thought, but it wouldn't be terribly hard. In the simplest cases, the extension writer could just add a flag to the type saying "ok, I'm thread-safe". Refcounting is another major issue. SpiderMonkey uses GC instead. CPython would need to do atomic increfs/decrefs. (Deferred refcounting could mitigate the cost.) The main drawback (aside from the amount of work) is the patent. SpiderMonkey's license grants a worldwide, royalty-free license, but not under the Python license. I think this could be wrangled, if the technical approach looks worthwhile. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [poll] New name for __builtins__
On Nov 29, 2007 11:54 AM, Guido van Rossum <[EMAIL PROTECTED]> wrote: > But then I thought, what if we renamed the __builtin__ module instead > to builtins, and left __builtins__ alone? Hmm. __builtins__ is a magic hook, but __builtin__-the-module isn't the thing it hooks, exactly, not the way __import__ hooks import or __iter__ hooks iter(). Really the __builtin__ module *implements* the __builtins__ hook protocol. It would be cool to have a name for __builtin__ the module that suggests that. I suggest sys.builtins. The builtins module feels both central enough and magical enough to belong in sys. And a lot of other stuff in sys has the same "it's fun but slightly crazy to tweak this knob" vibe. And, for sandboxers, mysandbox.builtins seems like a nice parallel to sys.builtins, with "sys" serving the bonus role of suggesting "unrestricted access". -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __eq__ vs hash
On Fri, Apr 4, 2008 at 9:38 AM, Guido van Rossum <[EMAIL PROTECTED]> wrote: > What specific code breaks? Maybe we need to turn this into a warning > in order to be more backwards compatible? I looked at Mercurial. It doesn't use __hash__ at all. It uses __eq__ in two files, three total uses: http://hg.intevation.org/mercurial/crew/file/6c4e12682fb9/mercurial/commands.py http://hg.intevation.org/mercurial/crew/file/6c4e12682fb9/mercurial/context.py -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ',' precedence in documentation
What I really want is for the need to be less common. What if assert recognized certain commonly used expression types and actually generated appropriate error messages? >>> assert foo.answer == 42 AssertionError: expected foo.answer == 42; actual: 'a suffusion of yellow' Maybe that's too magical. :( Failing that, I wish the message could look sort of like a comment. assert cond || message Yes, I know that symbol is used for something else in other languages... -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] opcode dispatch optimization
On Wed, Dec 31, 2008 at 11:44 AM, Christian Heimes wrote: > The patch makes use of a GCC feature where labels can be used as values: > http://gcc.gnu.org/onlinedocs/gcc/Labels-as-Values.html . I didn't know > about the feature and got confused by the unary && operator. Right. SpiderMonkey (Mozilla's JavaScript interpreter) does this, and it was good for a similar win on platforms that use GCC. (It took me a while to figure out why it was so much faster, so I think this patch would be better with a few very specific comments!) SpiderMonkey calls this optimization "threaded code" too, but this isn't the standard meaning of that term. See: http://en.wikipedia.org/wiki/Threaded_code -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] bytes.from_hex() [Was: PEP 332 revival in coordination with pep 349?]
Instead of byte literals, how about a classmethod bytes.from_hex(), which works like this:
# two equivalent things
expected_md5_hash = bytes.from_hex('5c535024cac5199153e3834fe5c92e6a')
expected_md5_hash = bytes([92, 83, 80, 36, 202, 197, 25, 145, 83, 227, 131, 79, 229, 201, 46, 106])
It's just a nicety; the former fits my brain a little better. This would work fine both in 2.5 and in 3.0.
I thought about unicode.encode('hex'), but obviously it will continue
to return a str in 2.x, not bytes. Also the pseudo-encodings
('hex', 'rot13', 'zip', 'uu', etc.) generally scare me. And now
that bytes and text are going to be two very different types, they're
even weirder than before. Consider:
text.encode('utf-8') ==> bytes
text.encode('rot13') ==> text
bytes.encode('zip') ==> bytes
bytes.encode('uu') ==> text (?)
This state of affairs seems kind of crazy to me.
Actually users trying to figure out Unicode would probably be better served if bytes.encode() and text.decode() did not exist.
-j
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bytes.from_hex() [Was: PEP 332 revival in coordination with pep 349?]
On 2/15/06, Guido van Rossum <[EMAIL PROTECTED]> wrote: > Actually users trying to figure out Unicode would probably be better served> if bytes.encode() and text.decode() did not exist.[...]It would be better if the signature of text.encode() always returned a bytes object. But why deny the bytes object a decode() method if textobjects have an encode() method? I agree, text.encode() and bytes.decode() are both swell. It's the other two that bother me. I'd say there are two "symmetric" API flavors possible (t and b are text and bytes objects, respectively, where text is a string type,either str or unicode; enc is an encoding name):- b.decode(enc) -> t; t.encode(enc) -> b- b = bytes(t, enc); t = text(b, enc) I'm not sure why one flavor would be preferred over the other,although having both would probably be a mistake. I prefer constructor flavor; the word "bytes" feels more concrete than "encode". But I worry about constructors being too overloaded. >>> text(b, enc) # decode >>> text(mydict) # repr >>> text(b) # uh... decode with default encoding? -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Path PEP: some comments (equality)
On 2/20/06, Mark Mc Mahon <[EMAIL PROTECTED]> wrote:
It seems that the Path module as currently defined leaves equalitytesting up to the underlying string comparison. My guess is that thisis fine for Unix (maybe not even) but it is a bit lacking for Windows.
Should the path class implement an __eq__ method that might do some ofthe following things: - Get the absolute path of both self and the other path - normcase both - now see if they are equal
This has been suggested to me many times.Unfortunately, since Path is a subclass of string, this breaks stuff in weird ways.For example: 'x.py' == path('x.py') == path('X.PY') == 'X.PY', but '
x.py' != 'X.PY'And hashing needs to be consistent with __eq__: hash('x.py') == hash(path('X.PY')) == hash('X.PY') ???Granted these problems would only pop up in code where people are mixing Path and string objects. But they would cause really obscure bugs in practice, very difficult for a non-expert to figure out and fix. It's safer for Paths to behave just like strings.
-j
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pre-PEP: The "bytes" object
On 2/22/06, Neil Schemenauer <[EMAIL PROTECTED]> wrote: @classmethoddef fromhex(self, data):data = "" '', data)return bytes(binascii.unhexlify(data))If it's to be a classmethod, I guess that should be "return self( binascii.unhexlify(data))".-j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pre-PEP: The "bytes" object
Neil Schemenauer wrote:
> Ron Adam <[EMAIL PROTECTED]> wrote:
>> Why was it decided that the unicode encoding argument should be ignored
>> if the first argument is a string? Wouldn't an exception be better
>> rather than give the impression it does something when it doesn't?
>
>From the PEP:
>
> There is no sane meaning that the encoding can have in that
> case. str objects *are* byte arrays and they know nothing about
> the encoding of character data they contain. We need to assume
> that the programmer has provided str object that already uses
> the desired encoding.
>
> Raising an exception would be a valid option. However, passing the
> string through unchanged makes the transition from str to bytes
> easier.
Does it?
I am quite certain the bytes PEP is dead wrong on this. It should be changed.
Suppose I have code like this:
def faz(s):
return s.encode('utf-16be')
If I want to transition from str to bytes, how should I change this code?
def faz(s):
return bytes(s, 'utf-16be') # OOPS - subtle bug
This silently does the wrong thing when s is a str. If I hadn't read
the PEP, I would confidently assume that bytes(str, encoding) ==
bytes(unicode, encoding), modulo the default encoding. I'd be wrong.
But there's a really good reason to think this. Wherever a unicode
argument is expected in Python 2.x, you can pass a str and it'll be
silently decoded. This is an extremely strong convention. It's even
embedded in PyArg_ParseTuple(). I can't think of any exceptions to
the rule, offhand.
Is this special case special enough to break the rules? Arguable. I
suspect not. But even if so, allowing the breakage to pass silently
is surely a mistake. It should just refuse the temptation to guess,
and throw an exception--right?
Now you may be thinking: the str/unicode duality of text, and the
bytes/text duality of the "str" type, are *bad* things, and we're
trying to get rid of them. True. My view is, we'll be rid of them in
3.0 regardless. In the meantime, there is no point trying to pretend
that 2.0 "str" is bytes and not text. It just ain't so; you'll only
succeed in confusing people and causing bugs. (And in 3.0 you're
going to turn around and tell them "str" *is* text!)
Good APIs make simple, sensible, comprehensible promises. I like
these promises:
- bytes(arg) works like array.array('b', arg)
- bytes(arg1, arg2) works like bytes(arg1.encode(arg2))
I dislike these promises:
- bytes(s, [ignored]), where s is a str, works like array.array('b', s)
- bytes(u, [encoding]), where u is a unicode,
works like bytes(u.encode(encoding))
It seems more Pythonic to differentiate based on the number of
arguments, rather than the type.
-j
P.S. As someone who gets a bit agitated when the word "Pythonic" or
the Zen of Python is taken in vain, I'd like to know if anyone feels
I've done so here, so I can properly apologize. Thanks.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] bytes thoughts
1. Maybe there should be a more obvious way to spell "bytes([0])*N".
I went through "bytes([0]*N)" and "bytes('\0'*N)" before I realized
there was a memory-efficient way to do it.
1a. Likewise, slice-assignment nicely handles memmove(), but there's
no memset().
2. Having a plural noun as a type name is awkward. I wish we could
call it "buffer" (which, conveniently, also tells you that it's
mutable, even if you don't know the word "mutable" :-). Alas.
3. I wrote a toy BytesIO class to go with the toy bytes object:
http://wiki.python.org/moin/BytesIO
(I hope this isn't considered wiki abuse -- it seemed as worthy and
relevant as most of what's in there.)
-j
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] String initialization (was: The "i" string-prefix: I18n'ed strings)
A compiler hook on string initialization, eh?
I have a distantly related story--this isn't important, just another
random Python use case for the file. (The i"xyzzy" proposal wouldn't
help this case.)
In scons, your SConscripts (makefiles, essentially) are Python source
code. You typically have SConscripts throughout your source tree.
Any SConscript could have something like this:
sort_exe = Program('sort', ['main.c', 'timsort.c'])
The problem is dealing with relative filenames. The only sane way to
resolve "main.c" to an abspath is relative to the source file that
physically contains that string literal token.[1] But that's
impossible to determine at run time.
scons uses some cleverness to guess the directory. It's always right,
except when it's wrong. Maddening.
So, what does this have to do with string initialization hooks? If
scons could "decorate" string constants as part of SConscript
compilation/execution, this problem could actually be solved.
-j
[1] Well, this is my opinion, but it's the right one.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 355 (object-oriented paths)
Talin, everything you wrote is really compelling. If path.py weren't
so ridiculously useful to me, I would be completely convinced. :)
For example, I agree 100% with this:
> Another reason why I am a bit dubious about a class-based approach
> is that it tends to take anything that is related to a filepath and lump
> them into a single module.
...and this:
> one thing that irks me (and others) about the Path class in Java is
> that it makes no distinction between methods that are merely textual
> conversions, and methods which actually go out and touch the disk.
...until I remember that in practice, d.parent and d.files('*.txt') on
the same object; or f.ext and f.isfile(); are things I do all the time
without thinking. I think I can see why.
Separate modules only make sense for separate use cases. In
real-world code where you're "doing stuff with files and directories",
you're going to randomly need os.remove(), shutil.copyfile(),
os.path.isdir(), and/or glob.glob(). I have one big mental junk
drawer with all this stuff in it. The way the stdlib partitions them
does not fit my brain. I have trouble believing some other
theoretical partition would be much better, though I'd love to see
someone try.
Lastly-- Is nontrivial path manipulation really rare? Practically
every program I write "does stuff with files and directories".
Scripts often do little else; in larger programs, main() often does 5
or 50 lines of this kind of stuff, while the rest of the program is
mostly filesystem-unaware.
-j
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3102: Keyword-only arguments
On 4/30/06, Edward Loper <[EMAIL PROTECTED]> wrote (referring to keyword-only arguments): > I see two possible reasons: > >- A function's author believes that calls to the function will be > easier to read if certain parameters are passed by name, rather > than positionally; and they want to enforce that calling > convention on their users. This seems to me to go against the > "consenting adults" principle. > >- A function's author believes they might change the signature in the > future to accept new positional arguments, and they will want to put > them before the args that they declare keyword-only. > > Both of these motivations seem fairly weak. Certainly, neither seems to > warrant a significant change to function definition syntax. I disagree. I think the use cases are more significant than you suggest, and the proposed change less significant. Readability and future-compatibility are key factors in API design. How well a language supports them determines how sweet its libraries can be. Even relatively simple high-level functions often have lots of clearly inessential "options". When I design this kind of function, I often wish for keyword-only arguments. path.py's write_lines() is an example. In fact... it feels as though I've seen "keyword-only" arguments in a few places in the stdlib. Am I imagining this? Btw, I don't think the term "consenting adults" applies. To me, that refers to the agreeable state of affairs where you, the programmer about to do something dangerous, know it's dangerous and indicate your consent somehow in your source code, e.g. by typing an underscore. That underscore sends a warning. It tells you to think twice. It tells you the blame is all yours if this doesn't work. It makes consent explicit (both mentally and syntactically). I'm +1 on the use cases but -0 on the PEP. The proposed syntax isn't clear; I think I want a new 'explicit' keyword or something. (Like that'll happen. Pfft.) -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] total ordering.
On 5/11/06, Vladimir 'Yu' Stepanov <[EMAIL PROTECTED]> wrote: > If for Python-3000 similar it will be shown concerning types > str(), int(), complex() and so on, and the type of exceptions > will strongly vary, it will make problematic redefinition of > behavior of function of sorting. I don't see what you mean by "redefinition of behavior of function of sorting". Is this something a Python programmer might want to do? Can you give an example? On 5/16/06, Vladimir 'Yu' Stepanov <[EMAIL PROTECTED]> wrote: > It will be possible it conveniently to use as exception of > management by a stream, for indication of necessity to involve > `.__r(eq|ne|le|lt|ge|gt|cmp)__()' a method. This kind of a class > can carry out function, similarly to StopIteration for `.next()'. There are no .__r(eq|ne|le|lt|ge|gt|cmp)__() methods, for a logical reason which you might enjoy deducing yourself... > At present time similar function is carried out with exception > NotImplemented. This exception is generated in a number of > mathematical operations. For this reason I ask to consider an > opportunity of creation of a new class. Can you explain this? NotImplemented isn't an exception. (NotImplementedError is, but that's something quite different.) NotImplemented has exactly one purpose in Python, as far as I can tell. What mathematical operations do you mean? -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] total ordering.
Vladimir,
Your examples seem to indicate that you've misunderstood the change
that's proposed for Python 3000. Especially this:
On 5/17/06, Vladimir 'Yu' Stepanov <[EMAIL PROTECTED]> wrote:
> # BEGIN: Emulation python3000
> if type(a) is not type(b) and (
> not operator.isNumberType(a) or
> not operator.isNumberType(b)
> ):
> raise TypeError("python3000: not-comparable types",
> (a,b))
> # END: Emulation python3000
Python 3000 will not do anything like this. It'll try a.__cmp__(b),
and failing that b.__cmp__(a) (but imagine this using tp_ slots
instead of actual Python method calls), and if both return
NotImplemented, it'll throw a TypeError (rather than guess, which is
what it does now).
There's a lot of legacy oddness in PyObject_RichCompare() and its many
helper functions; presumably they'll delete some of that, but it won't
be anything you care about.
Comparison with None should also continue to work as it does now,
unless I missed something.
-j
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
On 5/26/06, Facundo Batista <[EMAIL PROTECTED]> wrote: > I think that we can do one of the following, when we found "-1 * (1, 2, 3)": > > - Treat -1 as 0 and return an empty tuple (actual behavior). > - Treat the negative as a reverser, so we get back (3, 2, 1). > - Raise an error. No, no, no. The important invariant is that n * seq is loop(seq)[:n*len(seq)] where loop(seq) is an endless loop of the elements of seq. So obviously, if n is negative, the result should be an infinite sequence that's == to loop(seq). -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] partition() (was: Remove str.find in 3.0?)
Concerning names for partition(), I immediately thought of break().
Unfortunately it's taken.
So, how about snap()?
head, sep, tail = line.snap(':')
-j
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding a conditional expression in Py3.0
On 9/20/05, Guido wrote: > On 9/20/05, Jason Orendorff <[EMAIL PROTECTED]> wrote: > > return (if q: q.popleft() else: None) > > return (if q then q.popleft() else None) > > return q ? q.popleft() : None > > > > Hmmm. Score one for ?:. > > Why? Just because it's shorter? Just a gut response to the look. The verbose forms strike me as cluttered in this particular case. In the multiline case, it doesn't look like clutter because the if/elif/else bits line up, which fits the way Python has already trained my brain. > (Oh, and a way to decide between colon or no colon: we're not using > colons in list comps and genexprs either.) (grin) Easily fixed: print "average weight:", avg(for c in chefs: c.weight) rdict = dict(for k, v in D.iteritems(): v, k) Honestly, I think I would prefer this syntax. Examples from real code, before and after: lines = [line for line in pr.block.body if line.logical_line.strip() != ''] lines = [for line in pr.block.body: if line.logical_line.strip() != '': line] row.values = \ [line[col.start:col.end].strip() for col in columns] row.values = \ [for col in columns: line[col.start:col.end].rstrip()] return [p for p in self.listdir(pattern) if p.isdir()] return [for p in self.listdir(pattern): if p.isdir(): p] -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 343 and __with__
I'm -1 on PEP 343. It seems ...complex. And even with all the complexity, I *still* won't be able to type with self.lock: ... which I submit is perfectly reasonable, clean, and clear. Instead I have to type with locking(self.lock): ... where locking() is apparently either a new builtin, a standard library function, or some 6-line contextmanager I have to write myself. So I have two suggestions. 1. I didn't find any suggestion of a __with__() method in the archives. So I feel I should suggest it. It would work just like __iter__(). class RLock: @contextmanager def __with__(self): self.acquire() try: yield finally: self.release() __with__() always returns a new context manager object. Just as with iterators, a context manager object has "cm.__with__() is cm". The 'with' statement would call __with__(), of course. Optionally, the type constructor could magically apply @contextmanager to __with__() if it's a generator, which is the usual case. It looks like it already does similar magic with __new__(). Perhaps this is too cute though. 2. More radical: Let's get rid of __enter__() and __exit__(). The only example in PEP 343 that uses them is Example 4, which exists only to show that "there's more than one way to do it". It all seems fishy to me. Why not get rid of them and use only __with__()? In this scenario, Python would expect __with__() to return a coroutine (not to say "iterator") that yields exactly once. Then the "@contextmanager" decorator wouldn't be needed on __with__(), and neither would any type constructor magic. The only drawback I see is that context manager methods implemented in C will work differently from those implemented in Python. Since C doesn't have coroutines, I imagine there would have to be enter() and exit() slots. Maybe this is a major design concern; I don't know. My apologies if this is redundant or unwelcome at this date. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 and __with__
Phillip J. Eby writes:
> You didn't offer any reasons why this would be useful and/or good.
It makes it dramatically easier to write Python classes that correctly
support 'with'. I don't see any simple way to do this under PEP 343;
the only sane thing to do is write a separate @contextmanager
generator, as all of the examples do.
Consider:
# decimal.py
class Context:
...
def __enter__(self):
???
def __exit__(self, t, v, tb):
???
DefaultContext = Context(...)
Kindly implement __enter__() and __exit__(). Make sure your
implementation is thread-safe (not easy, even though
decimal.getcontext/.setcontext are thread-safe!). Also make sure it
supports nested 'with DefaultContext:' blocks (I don't mean lexically
nested, of course; I mean nested at runtime.)
The answer requires thread-local storage and a separate stack of saved
context objects per thread. It seems a little ridiculous to me.
Whereas:
class Context:
...
def __with__(self):
old = decimal.getcontext()
decimal.setcontext(self)
try:
yield
finally:
decimal.setcontext(old)
As for the second proposal, I was thinking we'd have one mental model
for context managers (block template generators), rather than two
(generators vs. enter/exit methods). Enter/exit seemed superfluous,
given the examples in the PEP.
> [T]his multiplies the difficulty of implementing context managers in C.
Nonsense.
static PyObject *
lock_with()
{
return PyContextManager_FromCFunctions(self, lock_acquire,
lock_release);
}
There probably ought to be such an API even if my suggestion is in
fact garbage (as, admittedly, still seems the most likely thing).
Cheers,
-j
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 and __with__
The argument I am going to try to make is that Python coroutines need
a more usable API.
> Try to explain the semantics of the with statement without referring to the
> __enter__ and __exit__ methods, and then see if you still think they're
> superfluous ;)
>
> The @contextmanager generator decorator is just syntactic sugar [...]
> [T]he semantics of the with statement itself can
> only be explained in terms of the __enter__ and __exit__ methods.
That's not true. It can certainly use the coroutine API instead.
Now... as specified in PEP 342, the coroutine API can be used to
implement 'with', but it's ugly. I think this is a problem with the
coroutine API, not the idea of using coroutines per se. Actually I
think 'with' is a pretty tame use case for coroutines. Other Python
objects (dicts, lists, strings) have convenience methods that are
strictly redundant but make them much easier to use. Coroutines
should, too.
This:
with EXPR as VAR:
BLOCK
expands to this under PEP 342:
_cm = contextmanager(EXPR)
VAR = _cm.next()
try:
BLOCK
except:
try:
_cm.throw(*sys.exc_info())
except:
pass
raise
finally:
try:
_cm.next()
except StopIteration:
pass
except:
raise
else:
raise RuntimeError
Blah. But it could look like this:
_cm = (EXPR).__with__()
VAR = _cm.start()
try:
BLOCK
except:
_cm.throw(*excinfo)
else:
_cm.finish()
I think that looks quite nice.
Here is the proposed specification for start() and finish():
class coroutine: # pseudocode
...
def start(self):
""" Convenience method -- exactly like next(), but
assert that this coroutine hasn't already been started.
"""
if self.__started:
raise ValueError # or whatever
return self.next()
def finish(self):
""" Convenience method -- like next(), but expect the
coroutine to complete without yielding again.
"""
try:
self.next()
except (StopIteration, GeneratorExit):
pass
else:
raise RuntimeError("coroutine didn't finish")
Why is this good?
- Makes coroutines more usable for everyone, not just for
implementing 'with'.
- For example, if you want to feed values to a coroutine, call
start() first and then send() repeatedly. Quite sensible.
- Single mental model for 'with' (always uses a coroutine or
lookalike object).
- No need for "contextmanager" wrapper.
- Harder to implement a context manager object incorrectly
(it's quite easy to screw up with __begin__ and __end__).
-j
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 and __with__
Right after I sent the preceding message I got a funny feeling I'm
wasting everybody's time here. I apologize. Guido's original concern
about speedy C implementation for locks stands. I don't see a good
way around it.
By the way, my expansion of 'with' using coroutines (in previous
message) was incorrect. The corrected version is shorter; see below.
-j
This:
with EXPR as VAR:
BLOCK
would expand to this under PEP 342 and my proposal:
_cm = (EXPR).__with__()
VAR = _cm.next()
try:
BLOCK
except:
_cm.throw(*sys.exc_info())
finally:
try:
_cm.next()
except (StopIteration, GeneratorExit):
pass
else:
raise RuntimeError("coroutine didn't finish")
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __doc__ behavior in class definitions
Martin, These two cases generate different bytecode. def foo(): # foo.func_code.co_flags == 0x43 print x# LOAD_FAST 0 x = 3 class Foo: # .co_flags == 0x40 print x# LOAD_NAME 'x' x = 3 In functions, local variables are just numbered slots. (co_flags bits 1 and 2 indicate this.) The LOAD_FAST opcode is used. If the slot is empty, LOAD_FAST throws. In other code, the local variables are actually stored in a dictionary. LOAD_NAME is used. This does a locals dictionary lookup; failing that, it falls back on the globals dictionary; and failing that, it falls back on builtins. Why the discrepancy? Beats me. I would definitely implement what CPython does up to this point, if that's your question. Btw, functions that use 'exec' are in their own category way out there: def foo2(): # foo2.func_code.co_flags == 0x42 print x # LOAD_NAME 'x' exec "x=3" # don't ever do this, it screws everything up print x Pretty weird. Jython seems to implement this. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed changes to PEP 343
On 10/7/05, Fredrik Lundh <[EMAIL PROTECTED]> wrote:
> the whole concept might be perfectly fine on the "this construct corre-
> sponds to this code" level, but if you immediately end up with things that
> are not what they seem, and names that don't mean what the say, either
> the design or the description of it needs work.
>
> ("yes, I know you can use this class to manage the context, but it's not
> really a context manager, because it's that method that's a manager, not
> the class itself. yes, all the information that belongs to the context are
> managed by the class, but that doesn't make... oh, shut up and read the
> PEP")
Good points... Maybe it is the description that needs work.
Here is a description of iterators, to illustrate the parallels:
An object that has an __iter__ method is iterable. It can plug
into the Python 'for' statement. obj.__iter__() returns an
iterator. An iterator is a single-use, forward-only view of a
sequence. 'for' calls __iter__() and uses the resulting
iterator's next() method.
(This is just as complicated as PEP343+changes, but not as
mindboggling, because the terminology is better. Also because
we're used to iterators.)
Now contexts, per PEP 343 with Nick's proposed changes:
An object that has a __with__ method is a context. It can plug
into the Python 'with' statement. obj.__with__() returns a
context manager. A context manager is a single-use object that
manages a single visit into a context. 'with' calls __with__()
and uses the resulting context manager's __enter__() and
__exit__() methods.
A contextmanager is a function that returns a new context manager.
Okay, that last bit is weird. But note that PEP 343 has this oddness
even without the proposed changes. Perhaps either "context manager"
or contextmanager should be renamed, regardless of whether Nick's
changes are accepted.
With the changes, context managers will be (conceptually) single-use.
So maybe a different term might be appropriate. Perhaps "ticket".
"A ticket is a single-use object that manages a single visit into a
context."
-j
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Divorcing str and unicode (no more implicit conversions).
-1 on keeping the source encoding of string literals. Python should definitely decode them at compile time. -1 on decoding implicitly "as needed". This causes decoding to happen late, in unpredictable places. Decodes can fail; they should happen as early and as close to the data source as possible. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Jython and CPython
On 12/13/05, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: Fredrik Lundh wrote:> BTW, what's the policy wrt. Jython-specific modules in the standard library?I don't think there is enough precedence to have a policy. So far, theonly places that explicitly support Jython is the test suite, pickle, and platform (I wouldn't really count in site here). Actually there's some Jython-specific code in xml/sax/__init__.py. Two places, both questionable. One of them refers to sys.registry. The other appears to be a workaround for Jython not having 4-argument __import__. > If the portability problem can be solved by checking things into Jython > instead, I think I would prefer that. Yes, it can be solved that way: Jython could implement pyexpat. I don't know just how crazy that idea is; my impression is that it could be done, perhaps imperfectly, as a wrapper around SAX. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 8 updates/clarifications
Barry Warsaw wrote: > - If your class is intended to be subclassed, and you have attributes > that you do not want subclasses to use, consider naming them with > double leading underscores and no trailing underscores. This invokes > Python's name mangling algorithm, where the name of the class is > mangled into the attribute name. This helps avoid attribute name > collisions should subclasses inadvertently contain attributes with the > same name. > > Note 1: Note that only the simple class name is used in the mangled > name, so if a subclass chooses both the same class name and attribute > name, you can still get name collisions. > > Note 2: Name mangling can make certain uses, such as debugging, less > convenient. However the name mangling algorithm is well documented > and easy to perform manually. Hmm. How about just: "Put two leading underscores on an attribute's name to strongly discourage code outside the class from accessing it." -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ElementTree in stdlib
On 12/13/05, Walter Dörwald <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: > > I don't think that SAX is unpythonic, but it's pretty low-level and > > mostly of use to people writing higher-level XML parsers (my parsexml > > module uses it). > > Having to define classes that conform to a certain API and registering > instances of those classes as callbacks with the parser doesn't look > that pythonic to me. An iterator API seems much more pythonic. Strongly agree. This very morning I wrote a long tirade about how I wish Python had true coroutines, for the sole reason that I could wrap SAX in an iterator-based API. Eventually I decided it was SAX's fault for having such a crummy API, so I didn't post it. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ElementTree in stdlib
Guido van Rossum wrote: > On 12/13/05, Walter Dörwald <[EMAIL PROTECTED]> wrote: > > Having to define classes that conform to a certain API and registering > > instances of those classes as callbacks with the parser doesn't look > > that pythonic to me. An iterator API seems much more pythonic. > > Perhaps. Although the SAX API lets you leave a callback undefined if > you don't have a need to handle those events; that's a bit trickier to > do with an iterator. Well, suppose you want to dump the text of a document. for e in iterparse(filename): if e.isText(): out.write(e.data) Not tricky. > > Also the different callbacks have different signatures. True. With SAX I always have to look up the signatures. The iterator yields Node-like objects in document order. I don't have to remember signatures. But the biggest advantage of an iterator-based API would be: when you hit an element, you can easily pass control to a function that knows how to parse that particular element. parsePlay() can call parseAct(), which can call parseScene(). To do anything like that with SAX, you have to write a bunch of dispatch code. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Ph.D. dissertation ideas?
Brett, You could create a downloadable corpus of Python source code, and maybe a web site through which people can easily browse/search it, contribute to it, and maintain it. The point would be to support language designers, tool developers, and researchers. Several python-dev folks have their own corpuses; I think other people would be happy to use a free one if it were out there. Of course there's no need to limit it to Python... Creating a really *good* corpus is maybe not super-easy; I imagine there are myriad linguistics papers explaining the nuances. Hey, cross-discipline research--cool points! Once this exists, there's no shortage of research questions you can quickly and easily answer with it. What percentage of Python programs use functional programming techniques? How often are list comprehensions used? What do people use generators for? And if you do something web-based, you can certainly work XML in there somewhere. :) -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] str with base
It seems dumb to support *parsing* integers in weird bases, but not *formatting* them in weird bases. Not a big deal, but if you're going to give me a toy, at least give me the whole toy! The %b idea is a little disappointing in two ways. Even with %b, Python is still dumb by the above criterion. And, I suspect users that don't know about %b are unlikely to find it when they want it. I know I've never looked for it there. I think a method 5664400.to_base(13) sounds nice. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] str with base
On 1/18/06, Donovan Baarda <[EMAIL PROTECTED]> wrote: > I think supporting arbitrary bases for floats is way overkill and not > worth considering. If you mean actual base-3 floating-point arithmetic, I agree. That's outlandish. But if there were a stdlib function to format floats losslessly in hex or binary, Tim Peters would use it at least once every six weeks to illustrate the finer points of floating point arithmetic. <0.00390625 wink> +1.0 -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] basenumber redux
On 1/17/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> Alex Martelli wrote:
> > But this doesn't apply to the Python Standard Library, for example see
> > line 1348 of imaplib.py: "if isinstance(date_time, (int, float)):".
> [...]
> > Being able to change imaplib to use basenumber instead of (int, float)
> > won't make it SIMPLER, but it will surely make it BETTER -- why should
> > a long be rejected, or a Decimal, for that matter?
>
> Right. I think this function should read
>
> if isinstance(date_time, str) and \
> (date_time[0],date_time[-1]) == ('"','"'):
> return date_time# Assume in correct format
>
> if isinstance(date_time, (tuple, time.struct_time)):
> tt = date_time
> else:
> tt = time.localtime(date_time)
So... arbitrary number-like objects should work, but arbitrary
sequence-like objects should fail? Hmmm. Maybe that "if
isinstance()" line should say "if hasattr(date_time, '__getitem__'):".
Am I sure? No. The original author of imaplib apparently got it
wrong, and Martin got it wrong, and they're both smarter than me.
Really this is just further proof that type-checking is a royal pain
in Python. Or rather, it's not hard to cover the builtin and stdlib
types, but it's hard to support "duck typing" too. Are we going about
this the right way? Options:
1. Redesign the API so each parameter has a clearly defined set of
operations it must support, thus eliminating the need for
type-checking. Drawback: An annoying API might be worse than the
problem we're trying to solve.
2. Write a set of imprecise, general-purpose type-checking functions
(is_real_number(v), is_sequence(v), ...) and use those. (They are
imprecise because the requirements are vague and because it's not
really possible to pin them down.) Drawback: Error-prone, compounded
by deceptively clean appearance.
3. Write very specific custom type-checking code each time you need
it (the imaplib approach). Drawbacks: Error-prone (as we've seen),
precarious, tedious, unreadable.
4. Use the "better-to-ask-forgiveness-than-permission" idiom.
Drawback: Potential bad behavior on error, again potentially worse
than the original problem.
Yuck. Does anyone have the answer to this one? Or is the problem not
as bad as it looks?
-j
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 343 and __context__()
I just noticed that my name is in PEP 343 attached to the idea of the __context__() method, and I'm slightly queasy over it. The rationale was to help e.g. decimal.DecimalContext support 'with'. Maybe that's a bad idea. DecimalContext has a few problems. In code where it matters, every function you write has to worry about it. (That is, you can't just write __decimal_context__ = ... at the top of the file and be done with it, the way you can with, say, __metaclass__.) And DecimalContext doesn't fit in with generators. sys.stdout has similar problems. It feels like PEP 343 will make people want to follow this model. That is, we'll see more global behavior-controlling variables in the future. There are grizzlier fates; I just wondered if anyone had thought of this. Cheers, -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 and __context__()
On 1/20/06, Nick Coghlan <[EMAIL PROTECTED]> wrote: > Jason Orendorff wrote: > > DecimalContext has a few problems. In code where it matters, every > > function you write has to worry about it. (That is, you can't just > > write __decimal_context__ = ... at the top of the file and be done > > with it, the way you can with, say, __metaclass__.) > > No, you write "decimal.setcontext(...)" instead. You seem to be implying these are roughly equal in convenience; I disagree. Suppose I have banking.py, in which it's important to use a particular precision and rounding. Now I have to put context-munging code in every single function that banking.py exposes. And around every 'yield'. Even with 'with', that's a lot of extra lines of code. I'd much prefer to put a one-liner at the top of the file, if it were possible (...but I don't see how, yet). Again, none of this is likely to matter--unless you're interleaving banking and heavy scientific calculations, which I try to avoid. So, not a big deal. Thanks for the response. > > And > > DecimalContext doesn't fit in with generators. > > It does fit actually - you simply have to remember to restore the original > context around any invocations of yield. Feh! "Fit" is to "can be made to work with a bit of effort, just don't forget to follow the rules" as Python is to C++. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The path module PEP
Thanks for doing this. I'm not sure anyone that matters here is
actually keen on path, but I guess we'll see. A few comments:
On 1/24/06, BJörn Lindqvist <[EMAIL PROTECTED]> wrote:
> The following points summarizes the design:
>
> - Path extends from string, therefore all code which expects
> string pathnames need not be modified and no existing code will
> break.
Actually, I would prefer a Path that *didn't* subclass string, and a
new "%p" format-thingy in PyArg_ParseTuple(). %p would expect either
a Path object or a string. Stdlib C functions that take paths would
be changed from using %s or %u to %p. This would pretty much
eliminate the need for path objects to act like strings (except where
__cmp__, __hash__, and common sense dictate).
The only reason I didn't do this in path.py is that I don't have the
required write access to the Python source tree. ;) Subclassing
str/unicode seemed like the next best thing.
> [...]omitted:
> * Function for opening a path - better handled by the builtin
> open().
Aside: I added this to support a few people who liked the idea of
"openable objects", meaning anything that has .open(), analogous to
"writeable objects" being anything with .write(). I don't use it
personally.
Examples 1 and 2 have errors. In example 1, the "after" code should be:
d = path('/usr/home/guido/bin')
for f in d.files('*.py'):
f.chmod(0755)
In example 2, the "before" code is missing a line -- the call to
os.path.walk(). (Actually it should probably use os.walk(), which
looks much nicer.)
I suspect you'll be asked to change the PEP to remove __div__ for
starters, in which case I propose using the Path constructor as the
replacement for os.path.join(). In that case, Path.joinpath can be
dropped.
-j
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The path module PEP
On 1/25/06, Toby Dickenson <[EMAIL PROTECTED]> wrote:
> On Tuesday 24 January 2006 20:22, BJörn Lindqvist wrote:
> > #Replacing glob.glob
> > glob.glob("/lib/*.so")
> > ==>
> > Path("/lib").glob("*.so")
>
> This definition seems confusing because it splits the glob pattern string in
> two ('/lib', and '*.so'). [...]
Well, let's make this look more like real code:
#line 1
LIB_DIR = "/lib"
==>
LIB_DIR = Path("/lib")
#line 296
libs = glob.glob(os.path.join(LIB_DIR, "*.so"))
==>
libs = LIB_DIR.files("*.so")
Clearer? In d.files(pattern), d is simply the root directory for the
search. The same is true of all the searching methods: dirs(),
walkfiles(), walkdirs(), etc.
I actually never use path.glob(). For example, here files() is
actually more accurate, and the word "files" is surely clearer than
"glob". Given files(), dirs(), and listdir(), I have never found a
real use case for glob().
-j
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The path module PEP
On 1/24/06, Ian Bicking <[EMAIL PROTECTED]> wrote: > There's kind of a lot of methods in here, which is a little bothersome. > It also points towards the motivation for the class -- too many > options in too many places in the stdlib. But throwing them *all* in > one class consolidates but doesn't simplify, especially with duplicate > functionality. I agree. Let me explain why there's so much cruft in path.py. The design is heavily skewed toward people already very familiar with the existing stdlib equivalents, because that is the market for third-party modules. I think my users want to type p.methodname() for whatever method name they already know, and have it *just work*. A sloppy API you've already learned is easier to pick up than a clean API you've never seen before. Ergo, cruft. A stdlib Path should have different design goals. It should have less redundancy, fewer methods overall, and PEP-8-compliant names. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator
It's controversial that Path subclasses str. Some people think it's
flat-out wrong. Even Bjorn argues that it's a practicality-vs-purity
tradeoff. But a strong argument can be made that Path *should* be a
string subclass, practicality be damned. Proof follows.
I. Here's an example of the sort of thing you might say if you did
*not* think of paths as strings:
On 1/25/06, Stephen J. Turnbull <[EMAIL PROTECTED]> wrote:
> I think it's logical to expect that
> Path('home') / 'and/or'
> points to a file named "and/or" in directory "home", not to a file
> named "or" in directory "home/and".
This makes no sense whatsoever. Ergo, by reductio ad absurdum, paths
are strings.
II. And here is the sort of thing you'd say if you thought of paths
*solely* as strings:
> (2) Note that '/' is also the path separator used by URIs, which RFC
> 2396 gives different semantics from Unix. Most of my Python usage to
> date has been heavily web-oriented, and I'd have little use for /
> unless it follows RFC 2396.
The quandary is resolved by pointing out that URIs are not paths (in
the sense of os.path and generally this whole horrible thread). Thus
not all strings are paths.
Hence the paths are a proper subset of the strings. By the existence
of os.path, they have their own commonly-used operations. By
definition, then, Path is a subclass of string, QED.
Do I really buy all this? I dunno. To say "paths aren't strings" is
all very well, and in a very abstract sense I almost agree--but you
have to admit it sort of flies in the face of, you know, reality.
Filesystem paths are in fact strings on all operating systems I'm
aware of. And it's no accident or performance optimization. It's
good design.
-j
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator
On 1/28/06, Stephen J. Turnbull <[EMAIL PROTECTED]> wrote: > Please note that my point was entirely different from trying to decide > whether to subclass strings. Noted -- sorry I took you out of context there; that was careless. > Jason> Filesystem paths are in fact strings on all operating > Jason> systems I'm aware of. > > I have no idea what you could mean by that. The data structure used > to represent a filesystem on all OS filesystems I've used is a graph > of directories and files. A filesystem object is located by > traversing a path in that graph. You seem to think that because I said "operating systems", I'm talking about kernel algorithms and such. I'm not. By "on all operating systems" I really mean systems, not kernels: system APIs, standard tools, documentation, the conventions everyone follows--that sort of thing. Userspace. Thought experiment: How are filesystem paths used? Well, programs pass them into system calls like open() and chmod(). Programs use them to communicate with other programs. Users pass them to programs. Compare this to how you'd answer the question "How are integers used?": I think paths are used more for communication, less for computation. Their utility for communication is tightly bound to their string-nature. Essentially all APIs involving filesystem paths treat them as strings. It's not just that they take string parameters. The way they're designed, they encourage users to think of paths as strings, not graph-paths. Java's stdlib is the only API that even comes close to distinguishing paths from strings. The .NET class library doesn't bother. Many many people much smarter than I have thought about creating non-string-oriented filesystem APIs. Somehow it hasn't caught on. Essentially all users expect to see a filesystem path as a string of characters in the conventional format. Display it any other way (say, as a sequence of edge-names) and you risk baffling them. My position is (a) the convention that paths are strings really does exist, embedded in the design and culture of the dominant operating systems--in fact it's overwhelming, and I'm surprised anyone can miss it; (b) there might be a reason for it, even aside from momentum. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
