Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

[Raymond Hettinger]

[John Arbash Meine]

So 'for x in s: break' is about 2x faster than next(iter(s)) and 3x
faster than (iter(s).next()).


The first version uses direct calls for the for-loop opcodes.
The other two have to do functions/method look-up and
make a pure python function call (neither is very fast).

FWIW, a "choose" method had been previously proposed,
discussed and rejected.

I don't find the optimization issue to be very interesting in
the case of retrieving an arbitrary element.  This isn't
the kind of thing that typically appears in an inner-loop;
afterall, if you've retrieved an arbitrary element without
removing it, then successive calls to "choose" could
potentially retrieve the exact same element again and again.


Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

[Stephen J. Turnbull]

"Martin v. Löwis" writes:

 > > res.get() would be a fairly obvious way to do it. Enough that I would
 > > probably never have gone searching for any of the other answers. Though
 > > personally, I think I would call it "set.peek()", but the specific name
 > > doesn't really matter to me.
 > 
 > Somebody proposed to call it .any(); this I like best (except that one
 > might expect that any takes a predicate as the argument).

Why not allow that?

def any(self, predicate=lambda x: True, default=None)
for a in self:
if predicate(a):
break
else:
return default
return a


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

[Martin v. Löwis]

> Why not allow that?
> 
> def any(self, predicate=lambda x: True, default=None)
> for a in self:
> if predicate(a):
> break
> else:
> return default
> return a

I'm +0 (given that I'm still skeptical about the need to have something
like this). Also setting aside the moratorium here, which may disallow
that kind of change for a foreseeable feature in any form.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

[Willi Richert]

Hi,

I totally agree regarding the efficiency. Code that relies on a fast "non-
removing .pop()" probably has other worse bottlenecks that should be targetted 
first.

This would, however, relief every programmer who needs this the first time in 
his Python experience, to research how this could be most reasonably done. And 
it is more of a style question. I find the "for x in some_set: break" rather 
ugly.

wr

Am Montag, 26. Oktober 2009 08:29:36 schrieben Sie:
> I don't find the optimization issue to be very interesting in
> the case of retrieving an arbitrary element.  This isn't
> the kind of thing that typically appears in an inner-loop;
> afterall, if you've retrieved an arbitrary element without
> removing it, then successive calls to "choose" could
> potentially retrieve the exact same element again and again.
> 
> 
> Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Possible language summit topic: buildbots

[Martin v. Löwis]

David Bolen wrote:
> MRAB  writes:
> 
>> Couldn't you write a script to check the status periodically?
> 
> Sure, I suppose scraping the web status page would work.  If it
> happened frequently I'd probably be forced to do something like that,
> but it's relatively low frequency (though I guess it does have a big
> impact in terms of availability) makes it hard to dedicate time to
> that compared to my "real world" work :-)

In addition, if it was happening frequently, we would rather investigate
the problem and fix it, than working around.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Possible language summit topic: buildbots

[Martin v. Löwis]

>>> This sounds like something that should be reported
>>> upstream. Particularly if you know how to reproduce it.  Has it been?
>>
>> No, largely because I can't reproduce it at all.  It's happened maybe
>> 4-5 times in the past 2 years or so.  All that I see is that my end
>> looks good yet the master end seems not to be dispatching jobs (it
>> never shows an explicit disconnect for my slave though).

It's not really reproducible. I think it sometimes happens when I
restart the master; sometimes, some clients fail to reconnect
(properly).

> It's easy to set up if not, the BuildSlave initializer accepts a list of
> email addresses that will be notified when that slave goes offline,
> notify_on_missing:
> 
> http://buildbot.net/apidocs/buildbot.buildslave.AbstractBuildSlave-
> class.html#__init__

I tried that out a couple of weeks ago, but never received any email.
I didn't have the time to look into this further since.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reworking the GIL

[Antoine Pitrou]

Terry Reedy  udel.edu> writes:
> 
> I am curious as to whether the entire mechanism is or can be turned off 
> when not needed -- when there are not threads (other than the main, 
> starting thread)?

It is an implicit feature: when no thread is waiting on the GIL, the GIL-holding
thread isn't notified and doesn't try to release it at all (in the eval loop,
that is; GIL-releasing C extensions still release it).

Note that "no thread is waiting on the GIL" can mean one of two things:
- either there is only one Python thread
- or the other Python threads are doing things with the GIL released (zlib/bz2
compression, waiting on I/O, sleep()ing, etc.)

So, yes, it automatically "turns itself off".

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Clean up Python/thread_*.h ?

[Ulrich Eckhardt]

On Saturday 24 October 2009, Christian Heimes wrote:
> Antoine Pitrou wrote:
> > * The unused file: thread_wince.h
> >
> > Windows CE has actually been using thread_nt.h since January 2009 (see
> > http://bugs.python.org/issue4893 ). thread_wince.h is still there, but
> > it's unused and grep brings no instance of it being mentioned anywhere in
> > the source tree.
>
> What about older versions of Windows CE? Maybe they still require the
> wince thread header?

Windows CE support in recent 2.x (and probably 3.x) is broken due to heavy 
bitrot. I can't imagine any CE platform this code would compile against. 
Further, I don't see many people using CE < 4, with CE 6 being current, in 
fact they are _very_ rare judging from Usenet activities.

For current CE versions, porting the NT-threads code is the easier way out.

+1 for removal of thread_wince.{h,c}

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reworking the GIL

[Andrew MacIntyre]

Brett Cannon wrote:
It's up to Andrew to get the support in. While I have faith he will, 
this is why we have been scaling back the support for alternative OSs 
for a while and will continue to do so. I suspect the day Andrew stops 
keeping up will be the day we push to have OS/2 be externally maintained.


Notwithstanding my desire to keep OS/2 supported in the Python tree,
keeping up has been more difficult of late:
- OS/2 is unquestionably a "legacy" environment, with system APIs
different in flavour and semantics from the current mainstream (though
surprisingly capable in many ways despite its age).
- The EMX runtime my OS/2 port currently relies on to abstract the 
system API to a Posix-ish API is itself a legacy package, essentially

unmaintained for some years :-(  This has been a source of increasing
pain as Python has moved with the mainstream... with regard to Unicode
support and threads in conjunction with multi-processing, in particular.

Real Life hasn't been favourably disposed either...

I have refrained from applying the extensive patches required to make
the port feature complete for 2.6 and later while I investigate an
alternate Posix emulating runtime (derived from FreeBSD's C library,
and which is used by Mozilla on OS/2), which would allow me to dispense
with most of these patches.  But it has an issue or two of its own...

The cost in effort has been compounded by effectively having to try and
maintain two ports - 2.x and 3.x.  And the 3.x port has suffered more
as its demands are higher.

So while I asked to keep the OS/2 thread support alive, if a decision 
were to be taken to remove OS/2 support from the Python 3.x sources I

could live with that.  A completed migration to Mercurial might well
make future port maintenance easier for me.

Regards,
Andrew.

--
-
Andrew I MacIntyre "These thoughts are mine alone..."
E-mail: andy...@bullseye.apana.org.au  (pref) | Snail: PO Box 370
   andy...@pcug.org.au (alt) |Belconnen ACT 2616
Web:http://www.andymac.org/   |Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reworking the GIL

[Sturla Molden]

Antoine Pitrou skrev:

- priority requests, which is an option for a thread requesting the GIL
to be scheduled as soon as possible, and forcibly (rather than any other
threads). 
So Python threads become preemptive rather than cooperative? That would 
be great. :-)


time.sleep should generate a priority request to re-acquire the GIL; and 
so should all other blocking standard library functions with a time-out.



S.M.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reworking the GIL

[Kristján Valur Jónsson]

> -Original Message-
> From: python-dev-bounces+kristjan=ccpgames@python.org
> [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf
> Of Sturla Molden
> time.sleep should generate a priority request to re-acquire the GIL;
> and
> so should all other blocking standard library functions with a time-
> out.

I don't agree.  You have to be very careful with priority.  time.sleep() does 
not promise to wake up in any timely manner, and neither do the timeout 
functions.  Rather, the timeout is a way to prevent infinite wait.

In my experience (from stackless python) using priority wakeup for IO can 
result in very erratic scheduling when there is much IO going on, every IO 
trumping another.  You should stick to round robin except for very special and 
carefully analysed cases.
K
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reworking the GIL

[sstein...@gmail.com]

On Oct 26, 2009, at 10:09 AM, Kristján Valur Jónsson wrote:





-Original Message-
From: python-dev-bounces+kristjan=ccpgames@python.org
[mailto:python-dev-bounces+kristjan=ccpgames@python.org] On  
Behalf

Of Sturla Molden
time.sleep should generate a priority request to re-acquire the GIL;
and
so should all other blocking standard library functions with a time-
out.


I don't agree.  You have to be very careful with priority.   
time.sleep() does not promise to wake up in any timely manner, and  
neither do the timeout functions.  Rather, the timeout is a way to  
prevent infinite wait.


In my experience (from stackless python) using priority wakeup for  
IO can result in very erratic scheduling when there is much IO going  
on, every IO trumping another.  You should stick to round robin  
except for very special and carefully analysed cases.


All the IO tasks can also go in their own round robin so that CPU time  
is correctly shared among all waiting IO tasks.


IOW, to make sure that all IO tasks get a fair share *in relation to  
all other IO tasks*.


Tasks can be put into the IO round robin when they "pull the IO alarm"  
so to speak, so there's no need to decide before-hand which task goes  
in which round robin pool.


I'm not familiar with this particular code in Python, but I've used  
this in other systems for years to make sure that IO tasks don't  
starve the rest of the system and that the most "insistent" IO task  
doesn't starve all the others.


S

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

[Alexander Belopolsky]

On Sun, Oct 25, 2009 at 1:48 AM, Nick Coghlan  wrote:
> Terry Reedy wrote:
>>>  In this model, a
>>> get() method, or even a __getitem__ with s[k] is k, is only natural.
>>
>> No, if key and value were the same thing, the get method would be
>> nonesense, as Guido said. But since they are not, since the implict key
>> is an abstract class, retrieving the representative, perhaps canonical
>> object given *any* member *is* natural. Being able to do so is also a
>> standard practice in mathematics.
>
> To formalise this invariant to some degree: given a set "s" and two
> items "k1" and "k2", such that "k1 in s" and "k2 in s" and "k1 == k2",
> there is a single item "k" in s such that "k1 == k" and "k2 == k".
>
> If __getitem__ were to be provided by sets, then the last part of that
> invariant could be written "s[k1] is s[k2]".
>

Thanks, Nick and Terry for a much more precise formulation of the
point that I was trying to present.  When I wrote s[k] is k, I had in
mind for k stored is the set or among the keys of an equivalent dict.
Formally alltrue(s[k] is k for k in s).  Nick's invariant is of course
a better expression of the same idea.

I believe interning is a worthwhile use case for Python to have "one
obvious way to do it."   Martin suggests that such a way already
exists and it involves storing interned objects as both keys and
values in a dictionary.  While this may have been obvious before sets
became available, I agree with Steven that in post 2.4 python users
are likely to look at set first and will only turn to dictionary after
discovering that the functionality that they are looking for is not
available in set.

Even if you know to use a dictionary, there are still some non-obvious
tricks to learn about.  First, you need to learn about setdefault, a
much criticized method that led to a new class being introduced to the
standard library:

http://mail.python.org/pipermail/python-dev/2003-February/033321.html
http://docs.python.org/library/collections.html?highlight=defaultdict#collections.defaultdict

Second, there is no obvious way to pre-seed the dictionary, i.e.,
given a list l of keys,

d = {}
for k in l:
   d[k] = k

When I was looking for a dict method to accomplish that, I discovered
dict.fromkeys, which of course was not the right method.  I still
don't know if a better solution exists than calling setdefault(k, k)
in a loop.

As experience with defaultdict has shown, it may be more appropriate
to introduce a specialized and properly named class in collections
rather than overloading set with new methods, but such approach would
lead to steep learning curve.  Collections.defaultdict, could be
cross-referenced from dict.setdefault, but a hypothetical
collections.interningset would most likely remain undiscovered for the
majority of users.

Here is an alternative idea on how storing interned objects in a set
can be supported.  Currently set.add method returns None and had no
effect when set already has an object equal to the one being added.  I
propose to consider changing that behavior to make set.add return the
added object or the set member that is equal to the object being
added.  It is unlikely that many programs rely on the return value
being None (with doctests being a probable exception), so adding this
feature is unlikely to cause much grief.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reworking the GIL

[Sturla Molden]

Antoine Pitrou skrev:

- priority requests, which is an option for a thread requesting the GIL
to be scheduled as soon as possible, and forcibly (rather than any other
threads). T

Should a priority request for the GIL take a priority number?

- If two threads make a priority requests for the GIL, the one with the 
higher priority should get the GIL first. 

- If a thread with a low priority make a priority request for the GIL, 
it should not be allowed to "preempt" (take the GIL away from) a 
higher-priority thread, in which case the priority request would be 
ignored. 

Related issue: Should Python threads have priorities? They are after all 
real OS threads.


S.M.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reworking the GIL

[Antoine Pitrou]

Sturla Molden  molden.no> writes:
> 
> Antoine Pitrou skrev:
> > - priority requests, which is an option for a thread requesting the GIL
> > to be scheduled as soon as possible, and forcibly (rather than any other
> > threads). T
> Should a priority request for the GIL take a priority number?

Er, I prefer to keep things simple. If you have lots of I/O you should probably
use an event loop rather than separate threads.

> Related issue: Should Python threads have priorities? They are after all 
> real OS threads.

Well, precisely they are OS threads, and the OS already assigns them (static or
dynamic) priorities. No need to replicate this.

(to answer another notion expressed in another message, there's no "round-robin"
scheduling either)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

[Terry Reedy]

Alexander Belopolsky wrote:


Here is an alternative idea on how storing interned objects in a set
can be supported.  Currently set.add method returns None and had no
effect when set already has an object equal to the one being added.  I
propose to consider changing that behavior to make set.add return the
added object or the set member that is equal to the object being
added.  It is unlikely that many programs rely on the return value
being None (with doctests being a probable exception), so adding this
feature is unlikely to cause much grief.


I had exactly the same idea, but did not post because it violates the 
general rule that mutators return None. On the other hand, the returned 
value here would not be the mutated collection, so no chaining is 
possible. And 'add' is clearly intended to change something.


On the other hand, frozensets do not have an add method.

Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reworking the GIL

[Daniel Stutzbach]

On Mon, Oct 26, 2009 at 10:58 AM, Antoine Pitrou wrote:

> Er, I prefer to keep things simple. If you have lots of I/O you should
> probably
> use an event loop rather than separate threads.
>

On Windows, sometimes using a single-threaded event loop is sometimes
impossible.  WaitForMultipleObjects(), which is the Windows equivalent to
select() or poll(), can handle a maximum of only 64 objects.

Do we really need priority requests at all?  They seem counter to your
desire for simplicity and allowing the operating system's scheduler to do
its work.

That said, if a thread's time budget is merely paused during I/O rather than
reset, then a thread making frequent (but short) I/O requests cannot starve
the system.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

[Guido van Rossum]

All this talk of equivalence classes makes me dizzy. :-)

- If sets were to grow an API to non-destructively access the object
stored in the set for a particular key, then dicts should have such a
method too.

- Ditto for an API to non-destructively get an arbitrary element.

- I'm far from convinced that we urgently need either API. But I'm
also not convinced it's unneeded.

- I still wish we could go back in time and unify sets and dicts, if
only to find out how that experiment would turn out.

-- 
--Guido van Rossum

PS. My elbow needs a couple more weeks of rest. Limiting myself to
ultra-short emails.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] One obvious way to do interning [Was: Retrieve an arbitrary element from a set without removing it]

[Alexander Belopolsky]

Changing the subject to reflect branched discussion and forwarding to
python-ideas where it probably belongs.

On Mon, Oct 26, 2009 at 12:02 PM, Terry Reedy  wrote:
> Alexander Belopolsky wrote:
>
>> Here is an alternative idea on how storing interned objects in a set
>> can be supported.  Currently set.add method returns None and has no
>> effect when set already has an object equal to the one being added.  I
>> propose to consider changing that behavior to make set.add return the
>> added object or the set member that is equal to the object being
>> added.  It is unlikely that many programs rely on the return value
>> being None (with doctests being a probable exception), so adding this
>> feature is unlikely to cause much grief.
>
> I had exactly the same idea, but did not post because it violates the
> general rule that mutators return None.

Is there such a rule?  What about set/dict pop?

> On the other hand, the returned
> value here would not be the mutated collection, so no chaining is possible.

I assume you refer to chaining as in s.add(1).add(2) which would be
enabled if s.add(..) returned s.  My proposal would enable a different
type of "chaining" which I would find useful, but ready to hear
objections:

v = compute_value()
s.add(v)
# use v

can, with my proposal, be rewritten as v = s.add(compute_value()) with
an added benefit that v that is used is the "canonical" value.

> And 'add' is clearly intended to change something.
>
Is this an argument for or against the proposal?

> On the other hand, frozensets do not have an add method.

However, PySet_Add "works with frozenset instances (like
PyTuple_SetItem() it can be used to fill-in the values of brand new
frozensets before they are exposed to other code). "


I will experiment with changing  PySet_Add to see how much it would
break and whether it will be helpful in implementing set-based
interning of python strings.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reworking the GIL

[Antoine Pitrou]

Daniel Stutzbach  stutzbachenterprises.com> writes:
> 
> Do we really need priority requests at all?  They seem counter to your
> desire for simplicity and allowing the operating system's scheduler to do
> its work.

No, they can be disabled (removed) if we prefer. With priority requests
disabled, latency results becomes less excellent but still quite good.

Running ccbench on a dual core machine gives the following latency results,
first with then without priority requets.

--- Latency --- (with prio requests)

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 2 ms.)
CPU threads=2: 0 ms. (std dev: 2 ms.)
CPU threads=3: 0 ms. (std dev: 2 ms.)
CPU threads=4: 0 ms. (std dev: 2 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 3 ms. (std dev: 2 ms.)
CPU threads=2: 3 ms. (std dev: 2 ms.)
CPU threads=3: 3 ms. (std dev: 2 ms.)
CPU threads=4: 4 ms. (std dev: 3 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 2 ms.)
CPU threads=1: 0 ms. (std dev: 2 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 2 ms.)
CPU threads=4: 0 ms. (std dev: 1 ms.)

--- Latency --- (without prio requests)

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 2 ms.)
CPU threads=1: 5 ms. (std dev: 0 ms.)
CPU threads=2: 3 ms. (std dev: 3 ms.)
CPU threads=3: 9 ms. (std dev: 7 ms.)
CPU threads=4: 22 ms. (std dev: 23 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 1 ms.)
CPU threads=1: 8 ms. (std dev: 2 ms.)
CPU threads=2: 5 ms. (std dev: 4 ms.)
CPU threads=3: 21 ms. (std dev: 32 ms.)
CPU threads=4: 19 ms. (std dev: 26 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 1 ms.)
CPU threads=1: 0 ms. (std dev: 2 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

[Chris Bergstresser]

On Mon, Oct 26, 2009 at 11:38 AM, Guido van Rossum  wrote:
> - If sets were to grow an API to non-destructively access the object
> stored in the set for a particular key, then dicts should have such a
> method too.
>
> - Ditto for an API to non-destructively get an arbitrary element.
>
> - I'm far from convinced that we urgently need either API. But I'm
> also not convinced it's unneeded.

   These clearly aren't urgently needed, but I do think they're needed
and useful.  For those who want a use-case for getting an arbitrary
element from a set, I've run into the need several times over the last
year, and each time I'm a little surprised I had the need and a little
surprised there wasn't an good way of going about it.
   In the most recent example, I was writing some UI code.  I had a
set of all the open window references so I could clean them up at the
end of the program.  I needed to call some API function that required
a window reference as the first argument, but it returned a global
value common to all the window references.

   I like the proposed set.get() method, personally.  list.get(index)
gets the item at that index, dict.get(key) gets the item associated
with that key, set.get() gets an item, but doesn't place any
guarantees on which item is returned.  Makes sense to me.  I also like
the idea there aren't any guarantees about which item is returned--it
allows subclasses to implement different behavior (so one might always
return the last item placed in the set, another could always return a
random item, another could implement some round-robin behavior, and
all would fulfill the contract of the set class).
   The existing methods aren't great for accomplishing this, mainly
because they're obfuscatory.  "iter(s).next()" is probably clearest,
and at least will throw a StopIteration exception if the set is empty.
 "for x in s: break" is just confusing, easy to accidentally confuse
with "for x in s: pass", and causes an unrevealing NameError if the
set is empty.  Add in all the other idioms for accomplishing the same
thing ("x, = s", etc.) and I think there's a good argument for adding
the method to sets, if only to provide a single obvious way of doing
it--and throwing a single, appropriate exception if the set is empty.

-- Chris
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] One obvious way to do interning [Was: Retrieve an arbitrary element from a set without removing it]

[Terry Reedy]

Alexander Belopolsky wrote:

Changing the subject to reflect branched discussion and forwarding to
python-ideas where it probably belongs.

On Mon, Oct 26, 2009 at 12:02 PM, Terry Reedy  wrote:

Alexander Belopolsky wrote:


Here is an alternative idea on how storing interned objects in a set
can be supported.  Currently set.add method returns None and has no
effect when set already has an object equal to the one being added.  I
propose to consider changing that behavior to make set.add return the
added object or the set member that is equal to the object being
added.  It is unlikely that many programs rely on the return value
being None (with doctests being a probable exception), so adding this
feature is unlikely to cause much grief.

I had exactly the same idea, but did not post because it violates the
general rule that mutators return None.


Is there such a rule?  What about set/dict pop?


The rule perhaps should be restated as 'Collection mutators return None 
or possible an item but not the collection.'



On the other hand, the returned
value here would not be the mutated collection, so no chaining is possible.




And 'add' is clearly intended to change something.


Is this an argument for or against the proposal?


Mildly for.


On the other hand, frozensets do not have an add method.


However, PySet_Add "works with frozenset instances (like
PyTuple_SetItem() it can be used to fill-in the values of brand new
frozensets before they are exposed to other code). "


I will experiment with changing  PySet_Add to see how much it would
break and whether it will be helpful in implementing set-based
interning of python strings.


Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it

[Raymond Hettinger]

Chris Bergstresser]
  I like the proposed set.get() method, personally. 


Sets have been implemented in many languages, but  I've only 
seen one that implemented this functionality  (the "arb" function 
in SETL).  For the most part, it seems that this is an uncommon need.


Also consider that there is value to keeping the set-api as
simple as possible.  Right now, anyone who was exposed
to the basics of sets in school can understand the set-api
with a near zero learning curve.  Some of that would be
lost if you add methods that make identity/equality distinctions
or that fetch the same arbitrary value over and over again.

Besides, it is trivial to write a short function that encapsulates
this behavior if it is part of your personal style of expression.




  The existing methods aren't great for accomplishing this, mainly
because they're obfuscatory.  "iter(s).next()" is probably clearest,
and at least will throw a StopIteration exception if the set is empty.
"for x in s: break" is just confusing ...


A short comment would do wonders.


Raymond


P.S.  It is weird that this thread is gaining traction at the same
time as the moratorium thread.  Go figure :-)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it

[Guido van Rossum]

On Mon, Oct 26, 2009 at 12:23 PM, Raymond Hettinger  wrote:
> P.S.  It is weird that this thread is gaining traction at the same
> time as the moratorium thread.  Go figure :-)

I'm beginning to think maybe adding a method to a built-in object type
*should* fall under the moratorium.

-- 
--Guido van Rossum

PS. My elbow needs a couple more weeks of rest. Limiting myself to
ultra-short emails.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Set methods for mapping views

[Raymond Hettinger]

[GvR]

I still wish we could go back in time and unify sets and dicts, if
only to find out how that experiment would turn out.


I'm curious about the outcome of another experiment along those lines.
Is anyone seeing uptake for the set methods on mapping views in Py3.x?

I haven't seen any code using it, nor any bug reports or documentation
requests, nor any code in the ASPN cookbook, nor mention of it 
on the newsgroup or python-help.


Has anyone here seen any hints about how this is faring in the wild?


Raymond

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reworking the GIL

[Collin Winter]

On Sun, Oct 25, 2009 at 1:22 PM, Antoine Pitrou  wrote:
> Having other people test it would be fine. Even better if you have an
> actual multi-threaded py3k application. But ccbench results for other
> OSes would be nice too :-)

My results for an 2.4 GHz Intel Core 2 Duo MacBook Pro (OS X 10.5.8):

Control (py3k @ r75723)

--- Throughput ---

Pi calculation (Python)

threads=1: 633 iterations/s.
threads=2: 468 ( 74 %)
threads=3: 443 ( 70 %)
threads=4: 442 ( 69 %)

regular expression (C)

threads=1: 281 iterations/s.
threads=2: 282 ( 100 %)
threads=3: 282 ( 100 %)
threads=4: 282 ( 100 %)

bz2 compression (C)

threads=1: 379 iterations/s.
threads=2: 735 ( 193 %)
threads=3: 733 ( 193 %)
threads=4: 724 ( 190 %)

--- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 1 ms. (std dev: 1 ms.)
CPU threads=2: 1 ms. (std dev: 2 ms.)
CPU threads=3: 3 ms. (std dev: 6 ms.)
CPU threads=4: 2 ms. (std dev: 3 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 975 ms. (std dev: 577 ms.)
CPU threads=2: 1035 ms. (std dev: 571 ms.)
CPU threads=3: 1098 ms. (std dev: 556 ms.)
CPU threads=4: 1195 ms. (std dev: 557 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 2 ms.)
CPU threads=2: 4 ms. (std dev: 5 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 1 ms. (std dev: 4 ms.)



Experiment (newgil branch @ r75723)

--- Throughput ---

Pi calculation (Python)

threads=1: 651 iterations/s.
threads=2: 643 ( 98 %)
threads=3: 637 ( 97 %)
threads=4: 625 ( 95 %)

regular expression (C)

threads=1: 298 iterations/s.
threads=2: 296 ( 99 %)
threads=3: 288 ( 96 %)
threads=4: 287 ( 96 %)

bz2 compression (C)

threads=1: 378 iterations/s.
threads=2: 720 ( 190 %)
threads=3: 724 ( 191 %)
threads=4: 718 ( 189 %)

--- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 1 ms.)
CPU threads=2: 0 ms. (std dev: 1 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 1 ms. (std dev: 5 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 1 ms. (std dev: 0 ms.)
CPU threads=2: 2 ms. (std dev: 1 ms.)
CPU threads=3: 2 ms. (std dev: 2 ms.)
CPU threads=4: 2 ms. (std dev: 1 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 2 ms. (std dev: 3 ms.)
CPU threads=3: 0 ms. (std dev: 1 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)


I also ran this through Unladen Swallow's threading microbenchmark,
which is a straight copy of what David Beazley was experimenting with
(simply iterating over 100 ints in pure Python) [1].
"iterative_count" is doing the loops one after the other,
"threaded_count" is doing the loops in parallel using threads.

The results below are benchmarking py3k as the control, newgil as the
experiment. When it says "x% faster", that is a measure of newgil's
performance over py3k's.

With two threads:

iterative_count:
Min: 0.336573 -> 0.387782: 13.21% slower  # I've run this
configuration multiple times and gotten the same slowdown.
Avg: 0.338473 -> 0.418559: 19.13% slower
Significant (t=-38.434785, a=0.95)

threaded_count:
Min: 0.529859 -> 0.397134: 33.42% faster
Avg: 0.581786 -> 0.429933: 35.32% faster
Significant (t=70.100445, a=0.95)


With four threads:

iterative_count:
Min: 0.766617 -> 0.734354: 4.39% faster
Avg: 0.771954 -> 0.751374: 2.74% faster
Significant (t=22.164103, a=0.95)
Stddev: 0.00262 -> 0.00891: 70.53% larger

threaded_count:
Min: 1.175750 -> 0.829181: 41.80% faster
Avg: 1.224157 -> 0.867506: 41.11% faster
Significant (t=161.715477, a=0.95)
Stddev: 0.01900 -> 0.01120: 69.65% smaller


With eight threads:

iterative_count:
Min: 1.527794 -> 1.447421: 5.55% faster
Avg: 1.536911 -> 1.479940: 3.85% faster
Significant (t=35.559595, a=0.95)
Stddev: 0.00394 -> 0.01553: 74.61% larger

threaded_count:
Min: 2.424553 -> 1.677180: 44.56% faster
Avg: 2.484922 -> 1.723093: 44.21% faster
Significant (t=184.766131, a=0.95)
Stddev: 0.02874 -> 0.02956: 2.78% larger


I'd be interested in multithreaded benchmarks with less-homogenous workloads.

Collin Winter

[1] - 
http://code.google.com/p/unladen-swallow/source/browse/tests/performance/bm_threading.py
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it

[Jesse Noller]

On Mon, Oct 26, 2009 at 3:56 PM, Guido van Rossum  wrote:
> On Mon, Oct 26, 2009 at 12:23 PM, Raymond Hettinger  wrote:
>> P.S.  It is weird that this thread is gaining traction at the same
>> time as the moratorium thread.  Go figure :-)
>
> I'm beginning to think maybe adding a method to a built-in object type
> *should* fall under the moratorium.

So far, fiddling with the PEP, I'm on the fence - adding a method to a
built-in object type is sort of a grey area (at least in my mind). It
depends on if doing so significantly alters the language/syntax.

jesse
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Set methods for mapping views

[geremy condra]

On Mon, Oct 26, 2009 at 3:56 PM, Raymond Hettinger  wrote:
> [GvR]
>>
>> I still wish we could go back in time and unify sets and dicts, if
>> only to find out how that experiment would turn out.
>
> I'm curious about the outcome of another experiment along those lines.
> Is anyone seeing uptake for the set methods on mapping views in Py3.x?
>
> I haven't seen any code using it, nor any bug reports or documentation
> requests, nor any code in the ASPN cookbook, nor mention of it on the
> newsgroup or python-help.
>
> Has anyone here seen any hints about how this is faring in the wild?
>
>
> Raymond
>

Next version of Graphine will use them heavily for graph
merges and set operations on graphs.

Geremy Condra
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it

[geremy condra]

On Mon, Oct 26, 2009 at 3:56 PM, Guido van Rossum  wrote:
> On Mon, Oct 26, 2009 at 12:23 PM, Raymond Hettinger  wrote:
>> P.S.  It is weird that this thread is gaining traction at the same
>> time as the moratorium thread.  Go figure :-)
>
> I'm beginning to think maybe adding a method to a built-in object type
> *should* fall under the moratorium.
>
> --
> --Guido van Rossum

Seems like any changes requiring support from uninvolved
developers should fall under the moratorium. If the proposal
is to add a set.get() function to the set API, then this clearly
falls under that heading.

Geremy Condra
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it

[Antoine Pitrou]

Jesse Noller  gmail.com> writes:
> 
> So far, fiddling with the PEP, I'm on the fence - adding a method to a
> built-in object type is sort of a grey area (at least in my mind). It
> depends on if doing so significantly alters the language/syntax.

We have recently added things like float.fromhex() which IMHO shouldn't be
blocked by the moratorium (*), although they technically add a method to a 
built-in.

(*) it is a minor new feature aimed at catching up with some established
standard for an exact, non-ambiguous string representation of floats

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Set methods for mapping views

[Raymond Hettinger]

I'm curious about the outcome of another experiment along those lines.
Is anyone seeing uptake for the set methods on mapping views in Py3.x?



Next version of Graphine will use them heavily for graph
merges and set operations on graphs.


That's great!  Thanks for the data point.
I look forward to reading the code.


Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it

[Mark Dickinson]

On Mon, Oct 26, 2009 at 7:56 PM, Guido van Rossum  wrote:
> I'm beginning to think maybe adding a method to a built-in object type
> *should* fall under the moratorium.

Another possible test case for this decision is the int.from_bytes and
int.to_bytes methods, proposed a while ago, and implemented
by Alexandre Vassalotti.  See:

http://bugs.python.org/issue1023290

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it

[Guido van Rossum]

On Mon, Oct 26, 2009 at 1:19 PM, Antoine Pitrou  wrote:
> Jesse Noller  gmail.com> writes:
>>
>> So far, fiddling with the PEP, I'm on the fence - adding a method to a
>> built-in object type is sort of a grey area (at least in my mind). It
>> depends on if doing so significantly alters the language/syntax.
>
> We have recently added things like float.fromhex() which IMHO shouldn't be
> blocked by the moratorium (*), although they technically add a method to a
> built-in.
>
> (*) it is a minor new feature aimed at catching up with some established
> standard for an exact, non-ambiguous string representation of floats

Okay, so it remains a gray area.

-- 
--Guido van Rossum

PS. My elbow needs a couple more weeks of rest. Limiting myself to
ultra-short emails.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it

[Willi Richert]

For those of you who want to tinker with it, I posted the patch against the 
current trunk at http://bugs.python.org/issue7212

Have fun,
wr

Am Montag, 26. Oktober 2009 21:32:32 schrieb Guido van Rossum:
> On Mon, Oct 26, 2009 at 1:19 PM, Antoine Pitrou  wrote:
> > Jesse Noller  gmail.com> writes:
> >> So far, fiddling with the PEP, I'm on the fence - adding a method to a
> >> built-in object type is sort of a grey area (at least in my mind). It
> >> depends on if doing so significantly alters the language/syntax.
> >
> > We have recently added things like float.fromhex() which IMHO shouldn't
> > be blocked by the moratorium (*), although they technically add a method
> > to a built-in.
> >
> > (*) it is a minor new feature aimed at catching up with some established
> > standard for an exact, non-ambiguous string representation of floats
> 
> Okay, so it remains a gray area.
> 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reworking the GIL

[Antoine Pitrou]

Collin Winter  gmail.com> writes:
> 
> My results for an 2.4 GHz Intel Core 2 Duo MacBook Pro (OS X 10.5.8):

Thanks!


[the Dave Beazley benchmark]
> The results below are benchmarking py3k as the control, newgil as the
> experiment. When it says "x% faster", that is a measure of newgil's
> performance over py3k's.
> 
> With two threads:
> 
> iterative_count:
> Min: 0.336573 -> 0.387782: 13.21% slower  # I've run this
> configuration multiple times and gotten the same slowdown.
> Avg: 0.338473 -> 0.418559: 19.13% slower

Those numbers are not very in line with the other "iterative_count" results.
Since iterative_count just runs the loop N times in a row, results should be
proportional to the number N ("number of threads").

Besides, there's no reason for single-threaded performance to be degraded since
the fast path of the eval loop actually got a bit streamlined (there is no
volatile ticker to decrement).

> I'd be interested in multithreaded benchmarks with less-homogenous workloads.

So would I.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reworking the GIL

[Collin Winter]

On Mon, Oct 26, 2009 at 2:43 PM, Antoine Pitrou  wrote:
> Collin Winter  gmail.com> writes:
> [the Dave Beazley benchmark]
>> The results below are benchmarking py3k as the control, newgil as the
>> experiment. When it says "x% faster", that is a measure of newgil's
>> performance over py3k's.
>>
>> With two threads:
>>
>> iterative_count:
>> Min: 0.336573 -> 0.387782: 13.21% slower  # I've run this
>> configuration multiple times and gotten the same slowdown.
>> Avg: 0.338473 -> 0.418559: 19.13% slower
>
> Those numbers are not very in line with the other "iterative_count" results.
> Since iterative_count just runs the loop N times in a row, results should be
> proportional to the number N ("number of threads").
>
> Besides, there's no reason for single-threaded performance to be degraded 
> since
> the fast path of the eval loop actually got a bit streamlined (there is no
> volatile ticker to decrement).

I agree those numbers are out of line with the others and make no
sense. I've run it with two threads several times and the results are
consistent on this machine. I'm digging into it a bit more.

Collin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] One obvious way to do interning [Was: Retrieve an arbitrary element from a set without removing it]

[Nick Coghlan]

Terry Reedy wrote:
> Alexander Belopolsky wrote:
>> Terry Reedy wrote:
>>> I had exactly the same idea, but did not post because it violates the
>>> general rule that mutators return None.
>>
>> Is there such a rule?  What about set/dict pop?
> 
> The rule perhaps should be restated as 'Collection mutators return None
> or possible an item but not the collection.'

And to clarify the rationale for that guideline: it is to make it clear
that the mutator is changing the container in place and *not* creating a
new container object.

myset.pop() # No new container, returns popped object
mylist.sort()   # No new container, returns None
sorted(mylist)  # New container, so return it
mystr.lower()   # Creates new string, so return it

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Set methods for mapping views

[Nick Coghlan]

Raymond Hettinger wrote:
> [GvR]
>> I still wish we could go back in time and unify sets and dicts, if
>> only to find out how that experiment would turn out.
> 
> I'm curious about the outcome of another experiment along those lines.
> Is anyone seeing uptake for the set methods on mapping views in Py3.x?
> 
> I haven't seen any code using it, nor any bug reports or documentation
> requests, nor any code in the ASPN cookbook, nor mention of it on the
> newsgroup or python-help.
> 
> Has anyone here seen any hints about how this is faring in the wild?

If anyone is looking for further explanation as to why Guido's
moratorium on core language changes is a good idea, allowing a chance
for answers to questions like Raymond's above a chance to evolve
naturally is what I see as the most important rationale.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reworking the GIL

[exarkun]

On 04:18 pm, dan...@stutzbachenterprises.com wrote:
On Mon, Oct 26, 2009 at 10:58 AM, Antoine Pitrou 
wrote:

Er, I prefer to keep things simple. If you have lots of I/O you should
probably
use an event loop rather than separate threads.


On Windows, sometimes using a single-threaded event loop is sometimes
impossible.  WaitForMultipleObjects(), which is the Windows equivalent 
to

select() or poll(), can handle a maximum of only 64 objects.


This is only partially accurate.  For one thing, WaitForMultipleObjects 
calls are nestable.  For another thing, Windows also has I/O completion 
ports which are not limited to 64 event sources.  The situation is 
actually better than on a lot of POSIXes.

Do we really need priority requests at all?  They seem counter to your
desire for simplicity and allowing the operating system's scheduler to 
do

its work.


Despite what I said above, however, I would also take a default position 
against adding any kind of more advanced scheduling system here.  It 
would, perhaps, make sense to expose the APIs for controlling the 
platform scheduler, though.


Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] RELEASED Python 2.6.4

[Barry Warsaw]

On behalf of the Python community, I'm happy to announce the  
availability of Python 2.6.4.  This is the latest production-ready  
version in the Python 2.6 series.


We had a little trouble with the Python 2.6.3 release; a number of  
unfortunate regressions were introduced.  I take responsibility for  
rushing it out, but the good news is that Python 2.6.4 fixes the known  
regressions in 2.6.3.  We've had a lengthy release candidate cycle  
this time, and are confident that 2.6.4 is a solid release.  We highly  
recommend you upgrade to Python 2.6.4.


Please see the NEWS file for all the gory details.

http://www.python.org/download/releases/2.6.4/NEWS.txt

Source tarballs and the Windows installers can be downloaded from the  
Python 2.6.4 page.  The Mac OS X disk image will be uploaded soon.


   http://www.python.org/download/releases/2.6.4/

For more information on Python 2.6 in general, please see

   http://docs.python.org/whatsnew/2.6.html

Please report bugs for any Python version in the Python tracker.

   http://bugs.python.org

Enjoy,
-Barry

Barry Warsaw
ba...@python.org
Python 2.6 Release Manager
(on behalf of the entire python-dev team)



PGP.sig
Description: This is a digitally signed message part
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Set methods for mapping views

[Steven D'Aprano]

On Tue, 27 Oct 2009 08:54:52 am Nick Coghlan wrote:
> Raymond Hettinger wrote:
> > [GvR]
> >
> >> I still wish we could go back in time and unify sets and dicts, if
> >> only to find out how that experiment would turn out.
> >
> > I'm curious about the outcome of another experiment along those
> > lines. Is anyone seeing uptake for the set methods on mapping views
> > in Py3.x?
> >
> > I haven't seen any code using it, nor any bug reports or
> > documentation requests, nor any code in the ASPN cookbook, nor
> > mention of it on the newsgroup or python-help.
> >
> > Has anyone here seen any hints about how this is faring in the
> > wild?
>
> If anyone is looking for further explanation as to why Guido's
> moratorium on core language changes is a good idea, allowing a chance
> for answers to questions like Raymond's above a chance to evolve
> naturally is what I see as the most important rationale.

I don't understand that rationale. Let's take a concrete example. The 
new `yield from` syntax was accepted but now will be delayed by the 
moratorium. How would the addition of `yield from` delay or prevent 
people using set methods on mapping views?

If a proposed feature directly competes with a feature in 3.x, then it 
might delay usage of the 3.x feature -- but if that were the case, the 
proposal would almost certainly be rejected on the basis that 3.x 
already has a feature to do that very thing.


-- 
Steven D'Aprano
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Set methods for mapping views

[Antoine Pitrou]

Steven D'Aprano  pearwood.info> writes:
> 
> I don't understand that rationale. Let's take a concrete example. The 
> new `yield from` syntax was accepted

Was it?


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Set methods for mapping views

[Guido van Rossum]

On Mon, Oct 26, 2009 at 4:06 PM, Antoine Pitrou  wrote:
> Steven D'Aprano  pearwood.info> writes:
>>
>> I don't understand that rationale. Let's take a concrete example. The
>> new `yield from` syntax was accepted
>
> Was it?

No.

-- 
--Guido van Rossum

PS. My elbow needs a couple more weeks of rest. Limiting myself to
ultra-short emails.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Set methods for mapping views

[Steven D'Aprano]

On Tue, 27 Oct 2009 10:09:14 am Guido van Rossum wrote:
> On Mon, Oct 26, 2009 at 4:06 PM, Antoine Pitrou  
wrote:
> > Steven D'Aprano  pearwood.info> writes:
> >> I don't understand that rationale. Let's take a concrete example.
> >> The new `yield from` syntax was accepted
> >
> > Was it?
>
> No.

I thought it had been. My mistake. Serves me right for not checking the 
PEP first. But my point still stands.



-- 
Steven D'Aprano
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reworking the GIL

[sstein...@gmail.com]

On Oct 26, 2009, at 6:45 PM, exar...@twistedmatrix.com wrote:
Despite what I said above, however, I would also take a default  
position against adding any kind of more advanced scheduling system  
here.  It would, perhaps, make sense to expose the APIs for  
controlling the platform scheduler, though.


I would also like to have an exposed API and optional profiling  
(optional as in conditional compilation, not as in some sort of  
profiling 'flag' that slows down non-profiling runs) so that you can  
see what's going on well enough to use the API to tune a particular  
platform for a particular workload.


S


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reworking the GIL

[Cameron Simpson]

On 26Oct2009 22:45, exar...@twistedmatrix.com  wrote:
| On 04:18 pm, dan...@stutzbachenterprises.com wrote:
| >On Mon, Oct 26, 2009 at 10:58 AM, Antoine Pitrou write:
| >Do we really need priority requests at all?  They seem counter to your
| >desire for simplicity and allowing the operating system's
| >scheduler to do
| >its work.
| 
| Despite what I said above, however, I would also take a default
| position against adding any kind of more advanced scheduling system
| here.  It would, perhaps, make sense to expose the APIs for
| controlling the platform scheduler, though.

+1 to both sentences from me.
-- 
Cameron Simpson  DoD#743
http://www.cskk.ezoshosting.com/cs/

Plague, Famine, Pestilence, and C++ stalk the land. We're doomed! Doomed!
- Simon E Spero
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Set methods for mapping views

[Brett Cannon]

On Mon, Oct 26, 2009 at 15:59, Steven D'Aprano  wrote:

> On Tue, 27 Oct 2009 08:54:52 am Nick Coghlan wrote:
> > Raymond Hettinger wrote:
> > > [GvR]
> > >
> > >> I still wish we could go back in time and unify sets and dicts, if
> > >> only to find out how that experiment would turn out.
> > >
> > > I'm curious about the outcome of another experiment along those
> > > lines. Is anyone seeing uptake for the set methods on mapping views
> > > in Py3.x?
> > >
> > > I haven't seen any code using it, nor any bug reports or
> > > documentation requests, nor any code in the ASPN cookbook, nor
> > > mention of it on the newsgroup or python-help.
> > >
> > > Has anyone here seen any hints about how this is faring in the
> > > wild?
> >
> > If anyone is looking for further explanation as to why Guido's
> > moratorium on core language changes is a good idea, allowing a chance
> > for answers to questions like Raymond's above a chance to evolve
> > naturally is what I see as the most important rationale.
>
> I don't understand that rationale. Let's take a concrete example. The
> new `yield from` syntax was accepted but now will be delayed by the
> moratorium. How would the addition of `yield from` delay or prevent
> people using set methods on mapping views?
>
>
It doesn't, but the point is we have already added several things to the
language in Python 3 that have gone mostly unused from the community thus
far. We do not need to continue to pile on the new features when we already
have a stack that we need to see if they pan out.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com