[Python-Dev] Operator module deprecations

2009-01-24 Thread Raymond Hettinger
I would like to deprecate some outdated functions in the operator module. The isSequenceType(), isMappingType(), and isNumberType() functions never worked reliably and now their intended purpose has been largely fulfilled by ABCs. The isCallable() function has long been deprecated and I think it

Re: [Python-Dev] Additional behaviour for itertools.combinations

2009-01-24 Thread Raymond Hettinger
Raymond Hettinger wrote: Since I expect students to be among the users for the comb/perm functions, there is some merit to keeping the API as simple as possible. Besides, it is not hard to use the existing tool as a primitive to get to the one you want: def mycombinations(iterable, r_seq

Re: [Python-Dev] Operator module deprecations

2009-01-25 Thread Raymond Hettinger
module deprecations +1 indeedy. On Sat, Jan 24, 2009 at 5:22 PM, Nick Coghlan wrote: Brett Cannon wrote: On Sat, Jan 24, 2009 at 14:46, Raymond Hettinger wrote: I would like to deprecate some outdated functions in the operator module. The isSequenceType(), isMappingType(), and isNumberType

[Python-Dev] Python 3.0.1

2009-01-27 Thread Raymond Hettinger
With the extensive changes in the works, Python 3.0.1 is shaping-up to be a complete rerelease of 3.0 with API changes and major usability fixes. It will fully supplant the original 3.0 release which was hobbled by poor IO performance. I propose to make the new release more attractive by backpo

Re: [Python-Dev] Python 3.0.1

2009-01-27 Thread Raymond Hettinger
From: "Guido van Rossum" In that case, I recommend just releasing it as 3.1. I had always anticipated a 3.1 release much sooner than the typical release schedule. That is great idea. It's a strong cue that there is a somewhat major break with 3.0 (removed functions, API fixes, huge performan

Re: [Python-Dev] Python 3.0.1

2009-01-27 Thread Raymond Hettinger
From: ""Martin v. Löwis"" Releasing 3.1 6 months after 3.0 sounds reasonable; I don't think it should be released earlier (else 3.0 looks fairly ridiculous). I think it should be released earlier and completely supplant 3.0 before more third-party developers spend time migrating code. We neede

[Python-Dev] pprint(iterator)

2009-01-27 Thread Raymond Hettinger
It is becoming the norm in 3.x for functions to return iterators, generators, or views whereever possible. I had a thought that pprint() ought to be taught to print iterators: pprint(enumerate(seq)) pprint(map(somefunc, somedata)) pprint(permutations(elements)) pprint(mydict.items()

Re: [Python-Dev] pprint(iterator)

2009-01-27 Thread Raymond Hettinger
[Guido van Rossum] My only thought is that whatever you do, target Python 3.1, not 3.0.1. Of course. Do you have any thoughts on the most useful display format? What do you want to see from pprint(mydict.items())? Raymond ___ Python-Dev mailing

Re: [Python-Dev] Python 3.0.1

2009-01-27 Thread Raymond Hettinger
[Benjamin Peterson] It seems like we are arguing over the version number of basically the same thing. I would like to see 3.0.1 released in early February for nearly the reasons you name. However, it seems to me that there are two kinds of issues: those like __cmp__ removal and some silly IO bug

Re: [Python-Dev] Python 3.0.1 (io-in-c)

2009-01-27 Thread Raymond Hettinger
[Antoine Pitrou] Now here are some performance figures. Text I/O is done in utf-8 with universal newlines enabled: That's a substantial boost. How does it compare to Py2.x equivalents? Raymond ___ Python-Dev mailing list Python-Dev@python.org http

Re: [Python-Dev] Python 3.0.1

2009-01-27 Thread Raymond Hettinger
[Martin] I would fear that than 3.1 gets the same fate as 3.0. In May, we will all think "what piece of junk was that 3.1 release, let's put it to history", and replace it with 3.2. By then, users will wonder if there is ever a 3.x release that is any good. I thought the gist of Guido's idea w

Re: [Python-Dev] Python 3.0.1

2009-01-27 Thread Raymond Hettinger
If something gets left in 3.0.1 and then ripped-out in 3.1, I think we're doing more harm than good. Very little code has been ported to 3.0 so far. One there is a base, all changes become more difficult. In the interests of our users, I vote for sooner than later. Also, 3.0 is a special case

Re: [Python-Dev] Python 3.0.1

2009-01-27 Thread Raymond Hettinger
[Matthew Wilkes] I didn't know 3.0 is considered a broken release, but teething troubles are to be expected. Knowing this, I would be reluctant to use 3.0.1, it sounds like too small a change. Not to worry. Many of the major language features are stable and many of the rough edges are qui

Re: [Python-Dev] Python 3.0.1 (io-in-c)

2009-01-28 Thread Raymond Hettinger
[Scott David Daniels] Comparison of three cases (including performance rations): MB/S MB/SMB/S in C in py3k in 2.7 C/3k 2.7/3k ** Text append ** 10M write 1e6 units at a time261.00 218.000 1540.000 1

Re: [Python-Dev] Python 3.0.1 (io-in-c)

2009-01-28 Thread Raymond Hettinger
[Adam Olsen] It'd also help if the file repr gave the encoding: +1 from me too. That will be a big help. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.o

Re: [Python-Dev] Python 3.0.1

2009-01-29 Thread Raymond Hettinger
From: "Guido van Rossum" On the one hand I understand that those folks want a stable target. On the other hand I think they would prefer to find out sooner rather than later they're using stuff they shouldn't be using any more. It's a delicate balance for sure, and I certainly don't want to open

[Python-Dev] Broken Test -- test_distutils

2009-01-29 Thread Raymond Hettinger
In the past couple of days, test_distutils started failing. It looks like a pure python error and may have been introduced by guilherme.polo's checkins: File "c:\py27\lib\distutils\tests\test_sdist.py", line 119, in test_make_distr ibution spawn('tar --help') File "c:\py27\lib\distutils\s

Re: [Python-Dev] Broken Test -- test_distutils

2009-01-29 Thread Raymond Hettinger
[Tarek Ziadé] That's me. I'll fix this problem right now. Thanks. I appreciate it. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options

Re: [Python-Dev] Python 3.0.1

2009-01-29 Thread Raymond Hettinger
[Aahz] At the same time, I think each individual change that doesn't clearly fall into the PEP6 process of being a bugfix needs to be vetted beyond what's permitted for not-yet-released versions. To get the ball rolling, I have a candidate for discussion. Very late in the 3.0 process (after f

Re: [Python-Dev] Python 3.0.1

2009-01-29 Thread Raymond Hettinger
On the one hand, it is an API change or new feature because people can (if they choose) access the dbm directly. OTOH, it is basically a performance fix for shelves whose API won't change at all. The part that is visible and incompatible is that 3.0.1 shelves won't be readable by 3.0.0. Tha

Re: [Python-Dev] Python 3.0.1

2009-01-29 Thread Raymond Hettinger
A couple additional thoughts FWIW: * whichdb() selects from multiple file formats, so 3.0.1 would still be able to read 3.0.0 files. It is the 2.x shelves that won't be readable at all under any scenario. * If you're thinking that shelves have very few users and that 3.0.0 has had few ado

Re: [Python-Dev] pprint(iterator)

2009-01-29 Thread Raymond Hettinger
Along the lines of what others have said: pprint() cannot consume an unknown iterator. Perhaps so. It's nice to have printing be free of side-effects (other than the actual printing). I've been working with 3.0 daily for several months (on a book project) and mostly think it's great. But

Re: [Python-Dev] Python 3.0.1

2009-01-29 Thread Raymond Hettinger
[Guido van Rossum] Sorry, not convinced. No worries. Py3.1 is not far off. Just so I'm clear. Are you thinking that 3.0.x will never have fast shelves, or are you thinking 3.0.2 or 3.0.3 after some external deployment and battle-testing for the module? Raymond ___

Re: [Python-Dev] C API for appending to arrays

2009-02-02 Thread Raymond Hettinger
[Hrvoje Niksic] The one thing missing from the array module is the ability to directly access array values from C. Please put a feature request on the bug tracker. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/

Re: [Python-Dev] Partial function application 'from the right'

2009-02-03 Thread Raymond Hettinger
I still haven't seen any real code presented that would benefit from partial.skip or partial_right. # some Articles have timestamp attributes and some don't stamp = partial_right(getattr, 'timestamp', 0) lastupdate = max(map(stamp, articles)) # some beautiful soup nodes have a name attribute an

[Python-Dev] Warnings

2009-02-05 Thread Raymond Hettinger
import os os.tmpnam() RuntimeWarning: tmpnam is a potential security risk to your program Are these runtime warnings necessary? Suppressing these warnings is a pita for one-off uses of os.tmpnam() or os.tempnam(). I would hate for this sort of thing to propagate throughout the standard libr

Re: [Python-Dev] cpython (2.7): note Ellipsis syntax

2011-07-30 Thread Raymond Hettinger
On Jul 30, 2011, at 11:28 PM, Georg Brandl wrote: > > (Also, there must have been some reason to make "..." available everywhere > for Python 3.) > It's really nice for stub functions: def foo(x): ... Raymond ___ Python-Dev mailing list Pytho

Re: [Python-Dev] GIL removal question

2011-08-10 Thread Raymond Hettinger
On Aug 10, 2011, at 4:15 AM, Nick Coghlan wrote: >> After implementing the aforementioned step 5, you will find that the >> performance of everything, including the threaded code, will be quite a bit >> worse. Frankly, this is probably the most significant obstacle to have any >> kind of GIL-

Re: [Python-Dev] Moving forward with the concurrent package

2011-08-10 Thread Raymond Hettinger
On Aug 10, 2011, at 1:36 PM, Antoine Pitrou wrote: > Le Wed, 10 Aug 2011 14:54:33 -0500, > Benjamin Peterson a écrit : >> 2011/8/10 Brian Curtin : >>> Now that we have concurrent.futures, is there any plan for >>> multiprocessing to follow suit? PEP 3148 mentions a hope to add or move >>> things

Re: [Python-Dev] [Python-checkins] cpython (2.7): Fix closes Issue12722 - link heapq source in the text format in the

2011-08-10 Thread Raymond Hettinger
On Aug 10, 2011, at 1:58 PM, Ezio Melotti wrote: > we would have to update all the links manually to link to h.p.o instead of > s.p.o. sed is your friend. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/list

Re: [Python-Dev] cpython: Add Py_RETURN_NOTIMPLEMENTED macro. Fixes #12724.

2011-08-15 Thread Raymond Hettinger
On Aug 15, 2011, at 5:35 AM, Nick Coghlan wrote: > On Mon, Aug 15, 2011 at 10:17 PM, Antoine Pitrou wrote: >> AFAICT, often with True and False: >> >>x = (some condition) ? Py_True : Py_False; >>Py_INCREF(x); >>return x; > > And that's an idiom that works better with a Py_RETURN ma

Re: [Python-Dev] PEP 393 Summer of Code Project

2011-08-26 Thread Raymond Hettinger
On Aug 26, 2011, at 8:51 PM, Terry Reedy wrote: > > > On 8/26/2011 8:42 PM, Guido van Rossum wrote: >> On Fri, Aug 26, 2011 at 3:57 PM, Terry Reedy wrote: > >>> My impression is that a UFT-16 implementation, to be properly called such, >>> must do len and [] in terms of code points, which is

Re: [Python-Dev] Maintenance burden of str.swapcase

2011-09-06 Thread Raymond Hettinger
On Sep 6, 2011, at 1:36 PM, Martin v. Löwis wrote: > I think this is what people underestimate. I can't name > applications either - but that doesn't mean they don't exist. Google code search is pretty good indicator that this method has near zero uptake. If it dies, I don't think anyone will

Re: [Python-Dev] range objects in 3.x

2011-09-27 Thread Raymond Hettinger
On Sep 27, 2011, at 11:24 AM, Ethan Furman wrote: > Alexander Belopolsky wrote: >> On Tue, Sep 27, 2011 at 2:23 AM, Greg Ewing >> wrote: >> .. >>> And I don't like "linspace" either. Something more self >>> explanatory such as "subdivide" or "interpolate" might >>> be better. >> "Grid" would be

Re: [Python-Dev] range objects in 3.x

2011-09-27 Thread Raymond Hettinger
On Sep 27, 2011, at 3:22 PM, Ethan Furman wrote: > Well, actually, I'd be using it with dates. ;) FWIW, an approach using itertools is pretty general but even it doesn't work for dates :-) >>> from itertools import count, takewhile >>> from decimal import Decimal >>> from fractions import Fra

Re: [Python-Dev] counterintuitive behavior (bug?) in Counter with +=

2011-10-07 Thread Raymond Hettinger
On Oct 3, 2011, at 6:12 AM, Lars Buitinck wrote: > After some digging, I found out that Counter [2] does not > have __iadd__ and += copies the entire left-hand side in __add__! This seems like a reasonable change for Py3.3. > I also figured out that I should use the update method instead, whi

Re: [Python-Dev] Code cleanups in stable branches?

2011-11-02 Thread Raymond Hettinger
On Nov 1, 2011, at 1:31 PM, Barry Warsaw wrote: > On Oct 31, 2011, at 06:23 PM, Éric Araujo wrote: > >> I thought that patches that clean up code but don’t fix actual bugs were >> not done in stable branches. Has this changed? > > I hope not. Sure, if they fix actual bugs, that's fine, but as

Re: [Python-Dev] None as slice params to list.index() and tuple.index()

2011-11-06 Thread Raymond Hettinger
On Nov 6, 2011, at 12:49 AM, Petri Lehtinen wrote: > Currently, find(), rfind(), index(), rindex(), count(), startswith() > and endswith() of str, bytes and bytearray accept None. Should > list.index() and tuple.index() accept it, too? The string methods accept None as a historical artifact of b

Re: [Python-Dev] PyDict_Get/SetItem and dict subclasses

2011-11-06 Thread Raymond Hettinger
On Nov 6, 2011, at 1:26 PM, Martin v. Löwis wrote: > Am 06.11.2011 17:39, schrieb Antoine Pitrou: >> Le 05/11/2011 17:34, Éric Araujo a écrit : >>> Hi Victor, >>> PyDict_GetItem() and PyDict_SetItem() don't call __getitem__ and __setitem__ for dict subclasses. Is there a reason fo

Re: [Python-Dev] Python 3.4 Release Manager

2011-11-22 Thread Raymond Hettinger
On Nov 22, 2011, at 7:50 PM, Larry Hastings wrote: > > > I've volunteered to be the Release Manager for Python 3.4. Awesome. Thanks for stepping up. > The FLUFL has already given it his Sloppy Wet Kiss Of Approval, E! > and we talked to Georg and he was for it too. There's no forma

Re: [Python-Dev] Merging 3.2 to 3.3 is messy because "Misc/NEWS"

2011-11-25 Thread Raymond Hettinger
On Nov 25, 2011, at 6:18 PM, Jesus Cea wrote: > On 12/11/11 16:56, Éric Araujo wrote: >> Ezio and I chatted a bit about his on IRC and he may try to write >> a Python parser for Misc/NEWS in order to write a fully automated >> merge tool. > > Anything new in this front? :-) To me, it would make

Re: [Python-Dev] Deprecation policy

2011-11-28 Thread Raymond Hettinger
On Oct 24, 2011, at 5:58 AM, Ezio Melotti wrote: > Hi, > our current deprecation policy is not so well defined (see e.g. [0]), and it > seems to me that it's something like: > 1) deprecate something and add a DeprecationWarning; > 2) forget about it after a while; > 3) wait a few versions unt

[Python-Dev] Warnings

2011-11-30 Thread Raymond Hettinger
When updating the documentation, please don't go overboard with warnings. The docs need to be worded affirmatively -- say what a tool does and show how to use it correctly. See http://docs.python.org/documenting/style.html#affirmative-tone The docs for the subprocess module currently have SEVEN w

Re: [Python-Dev] Warnings

2011-12-06 Thread Raymond Hettinger
On Dec 6, 2011, at 7:23 PM, Cameron Simpson wrote: > This assures that files are flushed [...] > > It does not. It _ensures_ that files are flushed. The doco style "affirmative > tone" _assures_. The coding practice _ensures_! > > Pedanticly, > -- > Cameron Simpson I can assure you that I'v

Re: [Python-Dev] str.format implementation

2011-12-13 Thread Raymond Hettinger
On Dec 12, 2011, at 7:56 PM, Ben Wolfson wrote: > I personally would prefer (1) to (2) or (3), and (3) to (2), had I my > druthers, but it doesn't matter a *whole* lot to me; I'd prefer any of > them to nothing (or to changing the docs to reflect the current batty > behavior). +1 on changing the

Re: [Python-Dev] Hash collision security issue (now public)

2011-12-28 Thread Raymond Hettinger
FWIW, Uncle Timmy considers the non-randomized hashes to be a virtue. It is believed that they give us better-than-random results for commonly encountered datasets. A change to randomized hashes would have a negative performance impact on those cases. Also, randomizing the hash wreaks havoc on do

Re: [Python-Dev] PEP 7 clarification request: braces

2012-01-02 Thread Raymond Hettinger
On Jan 1, 2012, at 8:44 PM, Nick Coghlan wrote: > I've been having an occasional argument with Benjamin regarding braces > in 4-line if statements: > > if (cond) >statement; > else >statement; > > vs. > > if (cond) { >statement; > } else { >statement; > } Really? Do we

Re: [Python-Dev] PEP 7 clarification request: braces

2012-01-02 Thread Raymond Hettinger
On Jan 2, 2012, at 12:31 PM, Benjamin Peterson wrote: > I might add that assuming you have braces, PEP 7 would want you to format it > as > > if (cond) { >statement; > } > else { >more_stuff; > } > Running ``grep -B1 else Objects/*c`` shows that we've happily lived with a mixture o

Re: [Python-Dev] PEP 7 clarification request: braces

2012-01-02 Thread Raymond Hettinger
On Jan 2, 2012, at 2:09 PM, Tim Delaney wrote: > I'd also point out that if you're expecting braces, not having them can make > the code less readable. If a programmer's mind explodes when they look at the simple and beautiful examples in K&R's The C Programming Language, then they've got pro

Re: [Python-Dev] PEP 7 clarification request: braces

2012-01-02 Thread Raymond Hettinger
On Jan 2, 2012, at 4:27 PM, Nick Coghlan wrote: > With my perception of the status quo corrected, I can stop worrying > about preserving a non-existent consistency. +1 QOTD Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python

Re: [Python-Dev] PEP for allowing 'raise NewException from None'

2012-01-27 Thread Raymond Hettinger
On Jan 26, 2012, at 7:19 PM, Ethan Furman wrote: > One of the open issues from PEP 3134 is suppressing context: currently there > is no way to do it. This PEP proposes one. Thanks for proposing fixes to this issue. It is an annoying problem. Raymond __

Re: [Python-Dev] threading.Semaphore()'s counter can become negative for non-ints

2012-01-31 Thread Raymond Hettinger
On Jan 29, 2012, at 6:11 PM, John O'Connor wrote: > On Sat, Jan 28, 2012 at 3:07 PM, Benjamin Peterson > wrote: >> But why would you want to pass a float? It seems like API abuse to me. >> > > Agreed. Anything else seems meaningless. I concur. This is very much a non-problem. There is no ne

Re: [Python-Dev] Add a frozendict builtin type

2012-02-29 Thread Raymond Hettinger
On Feb 27, 2012, at 10:53 AM, Victor Stinner wrote: > A frozendict type is a common request from users and there are various > implementations. ISTM, this request is never from someone who has a use case. Instead, it almost always comes from "completers", people who see that we have a frozenset

Re: [Python-Dev] Add a frozendict builtin type

2012-02-29 Thread Raymond Hettinger
On Feb 29, 2012, at 1:08 PM, Paul Moore wrote: > As it stands, I don't find the PEP compelling. The hardening use case > might be significant but Victor needs to spell it out if it's to make > a difference. If his sandboxing project needs it, the type need not be public. It can join dictproxy an

Re: [Python-Dev] Add a frozendict builtin type

2012-02-29 Thread Raymond Hettinger
On Feb 29, 2012, at 3:52 PM, Victor Stinner wrote: > I don't know if hardening Python is a compelling argument to add a new > builtin type. It isn't. Builtins are for general purpose use. It is not something most people should use; however, if it is a builtin, people will be drawn to frozend

Re: [Python-Dev] Add a frozendict builtin type

2012-02-29 Thread Raymond Hettinger
On Feb 29, 2012, at 4:23 PM, Victor Stinner wrote: > One of my colleagues implemented recently its own frozendict class > (which the "frozendict" name ;-) I write new collection classes all the time. That doesn't mean they warrant inclusion in the library or builtins. There is a use case for Lis

Re: [Python-Dev] cpython: PEP 417: Adding unittest.mock

2012-03-16 Thread Raymond Hettinger
On Mar 16, 2012, at 4:54 AM, Nick Coghlan wrote: > Don't forgot you also have the option of splitting out a separate > HOWTO tutorial section, leaving the main docs as a pure API reference. > (I personally find that style easier to use than the ones which try to > address both needs in the main m

Re: [Python-Dev] Playing with a new theme for the docs

2012-03-20 Thread Raymond Hettinger
On Mar 20, 2012, at 5:37 PM, Antoine Pitrou wrote: > Georg Brandl wrote: >> Hi all, >> >> recently I've grown a bit tired of seeing our default Sphinx theme, >> especially as so many other projects use it. I decided to play around >> with something "clean" this time, and this is the result: >>

Re: [Python-Dev] this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed)

2012-04-07 Thread Raymond Hettinger
On Apr 7, 2012, at 3:08 AM, Paul Moore wrote: > Use cases: > > Display the current time to a human (e.g. display a calendar or draw a > wall clock): use system clock, i.e. time.time() or > datetime.datetime.now(). > Event scheduler, timeout: time.monotonic(). > Benchmark, profiling: time.clock()

Re: [Python-Dev] this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed)

2012-04-07 Thread Raymond Hettinger
Just to clarify my previous post. It seems clear that benchmarking and timeout logic would benefit from a clock that cannot be adjusted by NTP. I'm unclear on whether time.sleep() will be based on the same clock so that timeouts and sleeps are on the same basis. For scheduling logic (such as t

Re: [Python-Dev] PEP 418 glossary

2012-04-11 Thread Raymond Hettinger
On Apr 11, 2012, at 2:49 AM, Jim Jewett wrote: > I believe PEP 418 (or at least the discussion) would benefit greatly > from a glossary to encourage people to use the same definitions. This sort of information is a good candidate for the HOW-TO section of the docs. Raymond

[Python-Dev] Suggested addition to PEP 8 for context managers

2012-04-15 Thread Raymond Hettinger
We should publish some advice on creating content managers. Context managers are a general purpose tool but have a primary use case of creating and releasing resources. This creates an expectation that that is what the context managers are doing unless they explicitly say otherwise. For example

Re: [Python-Dev] Suggested addition to PEP 8 for context managers

2012-04-18 Thread Raymond Hettinger
On Apr 18, 2012, at 1:38 PM, Guido van Rossum wrote: > > Let's change this to something more reasonable, e.g. > > """ > If operators with different priorities are used, consider adding > whitespace around the operators with the lowest priority(ies). This is > very much to taste, however, never

Re: [Python-Dev] docs.python.org pointing to Python 3 by default?

2012-05-20 Thread Raymond Hettinger
On May 18, 2012, at 11:24 AM, Barry Warsaw wrote: > At what point should we cut over docs.python.org to point to the Python 3 > documentation by default? Wouldn't this be an easy bit to flip in order to > promote Python 3 more better? My experience teaching and consulting suggests that this wou

Re: [Python-Dev] dictnotes.txt out of date?

2012-06-13 Thread Raymond Hettinger
On Jun 13, 2012, at 10:35 AM, Eli Bendersky wrote: > Did you mean to send this to the list, Raymond? Yes. I wanted to find-out whether someone approved changing all the dict tunable parameters. I thought those weren't supposed to have changed. PEP 412 notes that the existing parameters were

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-13 Thread Raymond Hettinger
On Jun 13, 2012, at 2:37 PM, Mark Shannon wrote: > I think that for combined tables a growth factor of x2 is best, > but I don't have any hard evidence to back that up. I believe that change should be reverted. You've undone work that was based on extensive testing and timings of many python

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-14 Thread Raymond Hettinger
On Jun 14, 2012, at 4:32 AM, Kristján Valur Jónsson wrote: > I would like to two or more compile time > settings to choose from: Memory optimal, speed optimal, (and mix). A compile time option would be nice. The default should be what we've had though. The new settings cause a lot more collis

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Raymond Hettinger
On Jun 18, 2012, at 12:35 PM, Antoine Pitrou wrote: > You are right. I was thinking 50 nanoseconds (which for a - relatively > high-end - 3GHz CPU puts us at 150 cycles). The last guidance I read from Intel said that a cache miss was roughly as expensive as a floating-point divide. When a dicti

Re: [Python-Dev] [Python-ideas] itertools.chunks(iterable, size, fill=None)

2012-07-01 Thread Raymond Hettinger
On Jun 30, 2012, at 10:44 PM, Stefan Behnel wrote: >> >> Another addition could be a new subsection on grouping (chunking) that >> would discuss post-processing of grouper (as discussed above), as well as >> other recipes, including ones specific to strings and sequences. It would >> essentially

Re: [Python-Dev] [Python-ideas] itertools.chunks(iterable, size, fill=None)

2012-07-02 Thread Raymond Hettinger
On Jul 1, 2012, at 5:01 AM, Stefan Behnel wrote: > To address the main problem of users not finding what they need, what about > simply extending the docstring of the grouper() Here's a small change to the docstring: http://hg.python.org/cpython/rev/d32f21d87363 FWIW, if you're interested i

Re: [Python-Dev] Do we need __length_hint__ at all? (Was PEP 0424: A method for exposing a length hint)

2012-07-16 Thread Raymond Hettinger
On Jul 16, 2012, at 1:37 AM, Mark Shannon wrote: > To quote from PEP 424: > >> Rationale >> = >> Being able to pre-allocate lists based on the expected size, as estimated by >> ``__length_hint__``, can be a significant optimization. CPython has been >> observed to run some code faster t

Re: [Python-Dev] PEP 0424: A method for exposing a length hint

2012-07-16 Thread Raymond Hettinger
On Jul 15, 2012, at 7:22 AM, Antoine Pitrou wrote: > On Mon, 16 Jul 2012 00:08:41 +1000 > Nick Coghlan wrote: >> Right, I agree on the value in being able to return something to say "this >> cannot be converted to a concrete container". > > Who would be able to return that, apart from trivial c

Re: [Python-Dev] PEP 0424: A method for exposing a length hint

2012-07-16 Thread Raymond Hettinger
On Jul 16, 2012, at 12:36 AM, Stefan Behnel wrote: > I'd like to more visibly repeat my suggestion to make this a slot method > "tp_length_hint()" in extension types that returns a Py_ssize_t. That is merely an implementation detail, but it would be a nice improvement. Raymond ___

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-20 Thread Raymond Hettinger
On Jul 18, 2012, at 3:30 AM, Nick Coghlan wrote: > - Eugene Toder's patch to add an AST optimisation step to the compiler > chain (http://bugs.python.org/issue11549) (I've asked Eugene about > this patch more recently and his current thought is that subsequent > improvements to the peephole optim

Re: [Python-Dev] PEP 0424: A method for exposing a length hint

2012-08-02 Thread Raymond Hettinger
On Aug 1, 2012, at 1:46 AM, Mark Shannon wrote: > > ''' > Being able to pre-allocate lists based on the expected size, as estimated by > __length_hint__, > can be a significant optimization. > PyPy has been observed to run some code slower than CPython, purely because > this optimization is ab

[Python-Dev] What's New in Python 3.3

2012-08-08 Thread Raymond Hettinger
Hello all, I'll soon be starting the edits of Whatsnew for 3.3. When I did this for 3.2, it took over 150 hours of work to research all the changes. This time there are many more changes, so my previous process won't work (reviewing every "new in 3.3" entry in the docs, every entry in the vol

[Python-Dev] Installation on Macs

2012-08-14 Thread Raymond Hettinger
On Mountain Lion, the default security settings only allow installation of applications downloaded from the Mac App Stored and "identified developers". We need to either become an "identified developer" or include some instructions on how to change the security settings (System Preference -- Gen

Re: [Python-Dev] issues found by Coverity

2012-09-12 Thread Raymond Hettinger
On Sep 11, 2012, at 6:32 AM, Christian Heimes wrote: > > maybe you have noticed a bunch of commits I made the last couple of > days. I noticed! Thank you for all the work to improve quality. Raymond___ Python-Dev mailing list Python-Dev@python.org

[Python-Dev] More compact dictionaries with faster iteration

2012-12-09 Thread Raymond Hettinger
The current memory layout for dictionaries is unnecessarily inefficient. It has a sparse table of 24-byte entries containing the hash value, key pointer, and value pointer. Instead, the 24-byte entries should be stored in a dense table referenced by a sparse table of indices. For example, the di

Re: [Python-Dev] More compact dictionaries with faster iteration

2012-12-09 Thread Raymond Hettinger
On Dec 9, 2012, at 10:03 PM, MRAB wrote: > What happens when a key is deleted? Will that leave an empty slot in > the entry array? Yes. See the __delitem__() method in the pure python implemention at http://code.activestate.com/recipes/578375 > If so, will the array be compacted if the propor

Re: [Python-Dev] More compact dictionaries with faster iteration

2012-12-10 Thread Raymond Hettinger
On Dec 10, 2012, at 2:48 AM, Christian Heimes wrote: > On the other hand every lookup and collision checks needs an additional > multiplication, addition and pointer dereferencing: > > entry = entries_baseaddr + sizeof(PyDictKeyEntry) * idx Currently, the dict implementation allows alternati

Re: [Python-Dev] More compact dictionaries with faster iteration

2012-12-10 Thread Raymond Hettinger
On Dec 10, 2012, at 4:06 AM, Mark Shannon wrote: >> Instead, the 24-byte entries should be stored in a >> dense table referenced by a sparse table of indices. > > What minimum size and resizing factor do you propose for the entries array? There are many ways to do this. I don't know which is

Re: [Python-Dev] More compact dictionaries with faster iteration

2012-12-10 Thread Raymond Hettinger
On Dec 10, 2012, at 1:38 PM, PJ Eby wrote: > Without numbers it's hard to say for certain, but the advantage to > keeping ordered dictionaries a distinct type is that the standard > dictionary type can then get that extra bit of speed in exchange for > dropping the ordering requirement. I expec

Re: [Python-Dev] More compact dictionaries with faster iteration

2012-12-10 Thread Raymond Hettinger
On Dec 10, 2012, at 7:04 PM, Mark Shannon wrote: >> Another approach is to pre-allocate the two-thirds maximum >> (This is simple and fast but gives the smallest space savings.) > > What do you mean by maximum? A dict with an index table size of 8 gets resized when it is two-thirds full, so th

Re: [Python-Dev] More compact dictionaries with faster iteration

2012-12-10 Thread Raymond Hettinger
On Dec 10, 2012, at 4:02 AM, Serhiy Storchaka wrote: > On 10.12.12 05:30, Raymond Hettinger wrote: >> On Dec 9, 2012, at 10:03 PM, MRAB > <mailto:pyt...@mrabarnett.plus.com>> wrote: >>> What happens when a key is deleted? Will that leave an empty slot in >>

Re: [Python-Dev] Mercurial workflow question...

2012-12-16 Thread Raymond Hettinger
On Dec 13, 2012, at 7:00 PM, Chris Jerdonek wrote: > On Thu, Dec 13, 2012 at 6:48 PM, R. David Murray > wrote: >> On Thu, 13 Dec 2012 20:21:24 -0500, Trent Nelson wrote: >>>- Use a completely separate clone to house all the intermediate >>> commits, then generate a diff once

Re: [Python-Dev] More compact dictionaries with faster iteration

2012-12-18 Thread Raymond Hettinger
On Dec 11, 2012, at 1:13 AM, Antoine Pitrou wrote: >> >> On Dec 10, 2012, at 2:48 AM, Christian Heimes >> wrote: >> >>> On the other hand every lookup and collision checks needs an >>> additional multiplication, addition and pointer dereferencing: >>> >>> entry = entries_baseaddr + sizeof(Py

Re: [Python-Dev] More compact dictionaries with faster iteration

2013-01-06 Thread Raymond Hettinger
On Jan 3, 2013, at 2:22 AM, Maciej Fijalkowski wrote: > Hello everyone. > > Thanks raymond for writing down a pure python version ;-) Thanks for running with it. > > I did an initial port to RPython for experiments. The results (on > large dicts only) are inconclusive - it's either a bit fas

Re: [Python-Dev] More compact dictionaries with faster iteration

2013-01-06 Thread Raymond Hettinger
On Jan 6, 2013, at 1:40 PM, Kristján Valur Jónsson wrote: > The memory part is also why I am interested in this approach. > Another thing has been bothering me. This is the fact that with the default > implementation, the smll table is only ever populated up to a certain > percentage, I cant

RE: [Python-Dev] adding key argument to min and max

2004-12-01 Thread Raymond Hettinger
for mostcommon() to a module of favorite utilities: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/347615 Alternatively, the recipe for a bag class is a more flexible and still reasonably efficient: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/259174 Raymond Hettinge

RE: [Python-Dev] adding key argument to min and max

2004-12-01 Thread Raymond Hettinger
> I don't want to put words into your mouth, so is this a vote against a > key= argument for min and max? Right. I don't think there is any need. > If nsmallest/nlargest get key= arguments, this would definitely cover > the same cases. Right. > If a key= argument gets vetoed for min and ma

RE: [Python-Dev] adding key argument to min and max

2004-12-01 Thread Raymond Hettinger
> -Original Message- > From: [EMAIL PROTECTED] [mailto:python-dev- > [EMAIL PROTECTED] On Behalf Of Steven Bethard > Sent: Wednesday, December 01, 2004 4:04 PM > To: [EMAIL PROTECTED] > Subject: [Python-Dev] adding key argument to min and max > > This is my first post to Python dev, so I

[Python-Dev] [Python-checkins] python/dist/src/Doc/lib liblogging.tex, 1.33, 1.34

2004-12-02 Thread Raymond Hettinger
Reminder: The head is now for Py2.5. Please also do a checkin for release24-maint. Raymond ___ Python-Dev mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/

[Python-Dev] Py2.5: Deprecated Cookie classes and interface

2004-12-04 Thread Raymond Hettinger
Any objections to removing Cookie.Cookie, Cookie.SmartCookie, and Cookie.SerialCookie?     Raymond ___ Python-Dev mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mail

[Python-Dev] Deprecated xmllib module

2004-12-04 Thread Raymond Hettinger
Hmph. The xmllib module has been deprecated since Py2.0 but is not listed in PEP 4. Question: Do we have to keep it for another two years because of that omission? It seems somewhat silly to keep an obsolete, supplanted module that doesn’t full support XML 1.0. Raymond __

RE: [Python-Dev] Deprecated xmllib module

2004-12-05 Thread Raymond Hettinger
> As for PEP 4: I don't know whether it needs to be listed there. It > appears that the PEP is largely unmaintained (I, personally, do not > really maintain it). So one option would be to just stop using PEP 4 > for recording deprecations, since we now have the warnings module. +1 Raymond

RE: [Python-Dev] Deprecated xmllib module

2004-12-05 Thread Raymond Hettinger
> > It seems somewhat silly to keep an obsolete, supplanted module that > > doesn’t full support XML 1.0. > > I mostly agree with Fredrik. What good does removal of xmllib do? A few thoughts: * Deprecated modules just get moved to the lib-old directory. If someone has ancient code relying on t

RE: [Python-Dev] proposal+patch: sys.gettickeraccumulation()

2004-12-05 Thread Raymond Hettinger
minate a small step or two from the eval-loop. Those efforts should not be discarded lightly. -1 on adding it directly. -0 on adding it as a #ifdeffed compile option (with the default being to exclude it). Raymond Hettinger ___ Python-Dev mailing list [EM

[Python-Dev] pystone rant [was] proposal+patch: sys.gettickeraccumulation()

2004-12-05 Thread Raymond Hettinger
(Old message) > > =) Parrotbench and PyBench would be enough for me to sign off on any > > performance hit. There is also pystone. pystone is good a predicting the relative performance of python apps on difference hardware/software environments. As a tool for evaluating proposed language change

RE: [Python-Dev] Deprecated xmllib module

2004-12-06 Thread Raymond Hettinger
> > > I think it would be better to *remove* the xmllib documentation. > > > Existing code which needs the module will continue to work even > > > without documentation, and new code is unlikely to be written for > > > a module that has no documentation, and where importing the module > > > gives a

<    5   6   7   8   9   10   11   12   13   14   >