Re: [Python-Dev] Fixing the GIL (with a BFS scheduler)

2010-05-18 Thread Antoine Pitrou
On Tue, 18 May 2010 08:45:41 +0200 Lennart Regebro wrote: > >> > >> Are you referring to the "New GIL"? > > > > Yes. > > At has been shown, it also in certain cases will race with the OS > scheduler, so this is not already fixed, although apparently improved, > if I understand correctly. "Race"

Re: [Python-Dev] Fixing the GIL (with a BFS scheduler)

2010-05-18 Thread Nick Coghlan
Nir Aides wrote: > I would like to restart this thread with 2 notes from the lively discussion: > > a) Issue 7946 (and this discussion?) concerns Python 3.2 > b) The GIL problems are not specific to OSX. The old and new GIL > misbehave on GNU/Linux and Windows too. I think Antoine and Bill went o

Re: [Python-Dev] Fixing the GIL (with a BFS scheduler)

2010-05-18 Thread Stefan Behnel
Bill Janssen, 17.05.2010 23:09: Most folks don't use "threads" Seems like a somewhat reasonable assumption to me. they use a higher-level abstraction like the nltk library. I have my doubts that this applies to "most folks" - likely not even to most of those who use threads. Stefan ___

Re: [Python-Dev] Fixing the GIL (with a BFS scheduler)

2010-05-18 Thread Lennart Regebro
On Tue, May 18, 2010 at 12:53, Antoine Pitrou wrote: > "Race" is a strange term here and I'm not sure what you mean. > The issue found out by Dave Beazley can't be reasonably described by > this word, I think. OK, maybe "race" is the wrong word. But that doesn't mean the issue doesn't exist. > P

Re: [Python-Dev] Fixing the GIL (with a BFS scheduler)

2010-05-18 Thread Antoine Pitrou
Le mardi 18 mai 2010 à 14:16 +0200, Lennart Regebro a écrit : > > Please read and understand the issue report mentioned by Nir before > > trying to make statements based on rumours heard here and there. > > Oh, so Dave Beazleys reports is a rumour now. Your and other people's grandiloquent inter

Re: [Python-Dev] Fixing the GIL (with a BFS scheduler)

2010-05-18 Thread Lennart Regebro
On Tue, May 18, 2010 at 14:52, Antoine Pitrou wrote: > > Le mardi 18 mai 2010 à 14:16 +0200, Lennart Regebro a écrit : >> > Please read and understand the issue report mentioned by Nir before >> > trying to make statements based on rumours heard here and there. >> >> Oh, so Dave Beazleys reports i

Re: [Python-Dev] Fixing the GIL (with a BFS scheduler)

2010-05-18 Thread Martin v. Löwis
Lennart Regebro wrote: > On Tue, May 18, 2010 at 14:52, Antoine Pitrou wrote: >> Le mardi 18 mai 2010 à 14:16 +0200, Lennart Regebro a écrit : Please read and understand the issue report mentioned by Nir before trying to make statements based on rumours heard here and there. >>> Oh, so D

Re: [Python-Dev] Fixing the GIL (with a BFS scheduler)

2010-05-18 Thread Dj Gilcrease
On Tue, May 18, 2010 at 3:43 PM, "Martin v. Löwis" wrote: > So please join us in considering the issue fixed unless you can provide > a really world example that demonstrates the contrary. The server software I maintain (openrpg) experiences this issue with when I tried porting the server code to

Re: [Python-Dev] Fixing the GIL (with a BFS scheduler)

2010-05-18 Thread Mike Klaas
On Sun, May 16, 2010 at 1:07 PM, Nir Aides wrote: > Relevant Python issue: http://bugs.python.org/issue7946 Is there any chance Antoine's "gilinter" patch from that issue might be applied to python 2.7? I have been experiencing rare long delays in simple io operations in multithreaded python ap

Re: [Python-Dev] Fixing the GIL (with a BFS scheduler)

2010-05-18 Thread Antoine Pitrou
On Tue, 18 May 2010 14:39:43 -0700 Mike Klaas wrote: > On Sun, May 16, 2010 at 1:07 PM, Nir Aides wrote: > > > Relevant Python issue: http://bugs.python.org/issue7946 > > Is there any chance Antoine's "gilinter" patch from that issue might > be applied to python 2.7? I have been experiencing r

[Python-Dev] Python versions for Ubuntu 10.10 (Maverick Meerkat)

2010-05-18 Thread Barry Warsaw
I just wanted to let the python-dev community know about some tracks we had at the recently concluded Ubuntu Developer Summit in Brussels. Among the several Python-related discussions, we talked about what versions of Python will be supported and default in the next version of Ubuntu (10.10, code

[Python-Dev] Incorrect length of collections.Counter objects / Multiplicity function

2010-05-18 Thread Gustavo Narea
Hello, everyone. I've checked the new collections.Counter class and I think I've found a bug: > >>> from collections import Counter > >>> c1 = Counter([1, 2, 1, 3, 2]) > >>> c2 = Counter([1, 1, 2, 2, 3]) > >>> c3 = Counter([1, 1, 2, 3]) > >>> c1 == c2 and c3 not in (c1, c2) > True > >>> # Perfect

[Python-Dev] Unordered tuples/lists

2010-05-18 Thread Gustavo Narea
Hello, everybody. I've been searching for a data structure like a tuple/list *but* unordered -- like a set, but duplicated elements shouldn't be removed. I have not even found a recipe, so I'd like to write an implementation and contribute it to the "collections" module in the standard library.

[Python-Dev] Summing up

2010-05-18 Thread Antoine Pitrou
On Tue, 18 May 2010 21:43:30 +0200 "Martin v. Löwis" wrote: > > I can understand why Antoine is being offended: it's his implementation > that you attacked. You literally said "At has been shown, it also in > certain cases will race with the OS scheduler, so this is not already > fixed", claiming

Re: [Python-Dev] Unordered tuples/lists

2010-05-18 Thread Guido van Rossum
This is typically called a "bag". Maybe searching for that will help you find a recipe? On Tue, May 18, 2010 at 3:13 PM, Gustavo Narea wrote: > Hello, everybody. > > I've been searching for a data structure like a tuple/list *but* unordered -- > like a set, but duplicated elements shouldn't be re

Re: [Python-Dev] Unordered tuples/lists

2010-05-18 Thread Benjamin Peterson
2010/5/18 Guido van Rossum : > This is typically called a "bag". Maybe searching for that will help > you find a recipe? Yes, and we have one in Python 2.7+ called collections.Counter. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python

Re: [Python-Dev] Unordered tuples/lists

2010-05-18 Thread Steven D'Aprano
On Wed, 19 May 2010 08:13:42 am Gustavo Narea wrote: > Hello, everybody. > > I've been searching for a data structure like a tuple/list *but* > unordered -- like a set, but duplicated elements shouldn't be > removed. I have not even found a recipe, so I'd like to write an > implementation and contr

Re: [Python-Dev] Unordered tuples/lists

2010-05-18 Thread Oleg Broytman
On Tue, May 18, 2010 at 11:13:42PM +0100, Gustavo Narea wrote: > To sum up, it would behave like a tuple or a list, except when it's compared > with another object: They would be equivalent if they're both unordered > tuples/lists, and have the same elements. There can be mutable and immutable >

Re: [Python-Dev] Unordered tuples/lists

2010-05-18 Thread Ben Finney
Gustavo Narea writes: > I've been searching for a data structure like a tuple/list *but* > unordered -- like a set, but duplicated elements shouldn't be removed. By that description, you're looking for the “Bag” pattern. […] > A multiset is not exactly what I need: I still need to use the > ele

Re: [Python-Dev] Fixing the GIL (with a BFS scheduler)

2010-05-18 Thread Mike Klaas
On Tue, May 18, 2010 at 2:50 PM, Antoine Pitrou wrote: > There's no chance for this since the patch relies on the new GIL. > (that's unless there's a rush to backport the new GIL in 2.7, of course) Thanks I missed that detail. > I think your "rare long delays" might be related to the old GIL's

Re: [Python-Dev] Summing up

2010-05-18 Thread David Beazley
Antoine, This is a pretty good summary that mirrors my thoughts on the GIL matter as well. In the big picture, I do think it's desirable for Python to address the multicore performance issue--namely to not have the performance needlessly thrashed in that environment. The original new GIL ad

Re: [Python-Dev] Fixing the GIL (with a BFS scheduler)

2010-05-18 Thread Antoine Pitrou
On Tue, 18 May 2010 17:26:44 -0700 Mike Klaas wrote: > > > I think your "rare long delays" might be related to the old GIL's own > > problems, though. How long are they? > > Typically between 20 and 60s. You mean milliseconds I suppose? If it's the case, then you may simply be witnessing garbag

Re: [Python-Dev] Summing up

2010-05-18 Thread Nick Coghlan
On 19/05/10 10:35, David Beazley wrote: Antoine, This is a pretty good summary that mirrors my thoughts on the GIL matter as well. In the big picture, I do think it's desirable for Python to address the multicore performance issue--namely to not have the performance needlessly thrashed in that

Re: [Python-Dev] Fixing the GIL (with a BFS scheduler)

2010-05-18 Thread Martin v. Löwis
> I think the new GIL should be given a year or so in the wild before > you start trying to optimize theoretical issues you may run into. If > in a year people come back and have some examples of where a proper > scheduler would help improve speed on multi-core systems even more, > then we can addr