Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-25 Thread Scott David Daniels
John Arbash Meinel wrote: res = heads(node1, node2) if len(res) == 1: # What is the 'obvious' way to get the node out? I posit that there *isn't* an obvious way to get the single item out of a 1-entry frozenset. for x in res: break list(res)[0] set(res).pop() iter(res).next() [x for x in res]

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-25 Thread Martin v. Löwis
> We were using sets to track the tips of a graph, and to compare whether > one node was an ancestor of another one. We were caching that answer > into frozensets, since that made them immutable. If > > res = heads(node1, node2) > if len(res) == 1: > # What is the 'obvious' way to get the node o

Re: [Python-Dev] Reworking the GIL

2009-10-25 Thread Terry Reedy
Antoine Pitrou wrote: Hello there, The last couple of days I've been working on an experimental rewrite of the GIL. Since the work has been turning out rather successful (or, at least, not totally useless and crashing!) I thought I'd announce it here. I am curious as to whether the entire mech

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-25 Thread geremy condra
> Martin v. Löwis wrote: >>> Hmm, perhaps when using sets as work queues? >> >> A number of comments: >> >> - it's somewhat confusing to use a set as a *queue*, given >>   that it won't provide FIFO semantics. >> - there are more appropriate and direct container structures >>   available, including

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-25 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin v. Löwis wrote: >> Hmm, perhaps when using sets as work queues? > > A number of comments: > > - it's somewhat confusing to use a set as a *queue*, given > that it won't provide FIFO semantics. > - there are more appropriate and direct contai

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread exarkun
On 01:28 am, db3l@gmail.com wrote: exar...@twistedmatrix.com writes: This sounds like something that should be reported upstream. Particularly if you know how to reproduce it. Has it been? No, largely because I can't reproduce it at all. It's happened maybe 4-5 times in the past 2 years

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread David Bolen
MRAB writes: > Couldn't you write a script to check the status periodically? Sure, I suppose scraping the web status page would work. If it happened frequently I'd probably be forced to do something like that, but it's relatively low frequency (though I guess it does have a big impact in terms

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread David Bolen
exar...@twistedmatrix.com writes: > This sounds like something that should be reported > upstream. Particularly if you know how to reproduce it. Has it been? No, largely because I can't reproduce it at all. It's happened maybe 4-5 times in the past 2 years or so. All that I see is that my end

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread exarkun
On 25 Oct, 09:36 pm, db3l@gmail.com wrote: I think the other issue most likely to cause a perceived "downtime" with the Windows build slave that I've had a handful of cases over the past two years where the build slave appears to be operating properly, but the master seems to just queue up j

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread MRAB
David Bolen wrote: [snip] I think the other issue most likely to cause a perceived "downtime" with the Windows build slave that I've had a handful of cases over the past two years where the build slave appears to be operating properly, but the master seems to just queue up jobs as if it were down

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-25 Thread Steven D'Aprano
On Mon, 26 Oct 2009 05:43:52 am Martin v. Löwis wrote: > What Alexander wants is that the set type also becomes useful for > storing canonical representations. I find that inappropriate, since > dicts already provide the one obvious way to do it. Only because dicts came first in Python rather than

Re: [Python-Dev] Reworking the GIL

2009-10-25 Thread Brett Cannon
> > [SNIP - a lot of detail on what sounds like a good design] > > Now what remains to be done? > > Having other people test it would be fine. Even better if you have an > actual multi-threaded py3k application. But ccbench results for other > OSes would be nice too :-) > (I get good results under

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Antoine Pitrou
ssteinerX gmail.com gmail.com> writes: > On Oct 25, 2009, at 5:43 PM, Martin v. Löwis wrote: > > > Only turning on the slave occasionally makes it useless. > > For certain use cases; not mine. Let's say that for the use case we are talking here (this is python-dev), Martin's statement holds tr

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread sstein...@gmail.com
On Oct 25, 2009, at 5:43 PM, Martin v. Löwis wrote: Only turning on the slave occasionally makes it useless. For certain use cases; not mine. S ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Un

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Martin v. Löwis
> It's not so much that I *require* the slave to shut down, more that > I'm not sure how well I'll be able to ensure that it's up all the > time, and I'm trying to understand the implications of that. My basic > impression is that it's not really going to work, unfortunately. There is a significa

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread David Bolen
"Martin v. Löwis" writes: > The remaining issue is the popups; if a process still has a popup, > you can't even terminate it properly. There are two kinds of popups: > system-generated ones, and CRT-generated ones. For the CRT ones, we > once had a way to turn them off, but I'm not sure whether t

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Glyph Lefkowitz
On Oct 25, 2009, at 3:06 PM, Martin v. Löwis wrote: (*) it may help if Buildbot would create a Win32 job object, and then use TerminateJobObject. Contributions are welcome. Some work has already been done on this, but it needs help. At the root it's a Twisted issue: http://twistedmatr

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Paul Moore
2009/10/25 "Martin v. Löwis" : >> I've been trying to get some feedback about firing up buildbots on Cloud >> Servers for a while now and haven't had much luck.  I'd love to find a >> way of having buildbots come to life, report to the mother ship, do the >> build, then go away 'till next time they

[Python-Dev] Buildbot alternate renderings

2009-10-25 Thread Martin v. Löwis
Triggered by the recent discussion, I looked at the changes in status display that the buildbot people have implemented recently. I added a few links into alternate renderings; see http://www.python.org/dev/buildbot/ Regards, Martin ___ Python-Dev maili

Re: [Python-Dev] [TIP] Possible language summit topic: buildbots

2009-10-25 Thread Martin v. Löwis
> Brett actually wants web hooks so pony-build will ping an App Engine web > app when there is more data, ala PubSubHubbub. Or hell, just have > pony-build have an Atom feed with updates and simply use PuSH. In other > words I want to be told when there is an update, not have to poll to > find out.

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Martin v. Löwis
> The most *exciting* part of pony-build, apart from the always-riveting > spectacle of "titus rediscovering problems that buildbot solved 5 years ago", > is the loose coupling of recording server to the build slaves and build > reporters. My plan is to enable a simple and lightweight XML-RPC and/

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread sstein...@gmail.com
On Oct 25, 2009, at 3:35 PM, Martin v. Löwis wrote: I don't need to know that it works on every checkin For us, that is a fairly important requirement, though. Reports get more and more useless if they aren't instantaneous. Sometimes, people check something in just to see how the build slaves

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Martin v. Löwis
> I don't need to know that it works on every checkin For us, that is a fairly important requirement, though. Reports get more and more useless if they aren't instantaneous. Sometimes, people check something in just to see how the build slaves react. Regards, Martin __

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Martin v. Löwis
> This is supported in recent versions of BuildBot with a special kind of > slave: > > http://djmitche.github.com/buildbot/docs/0.7.11/#On_002dDemand- > _0028_0022Latent_0022_0029-Buildslaves Interesting. Coming back to "PSF may spend money", let me say this: If somebody would volunteer to set u

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Martin v. Löwis
> it shouldn't be difficult to cobble together a build script that spins up a > buildslave on EC2 and runs the tests there; I wrote something similar a > few years ago for an infrequently connected home machine. Ok - so it would be the master running this script? Sounds reasonable to me. As for E

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Martin v. Löwis
> I think that money can help in two ways in this case. > > First, there are now a multitude of cloud hosting providers which will > operate a slave machine for you. BuildBot has even begun to support > this deployment use-case by allowing you to start up and shut down vms > on demand to save on

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Antoine Pitrou
Martin v. Löwis v.loewis.de> writes: > > If it's offline too often, I'm skeptical that it would be useful. If > you report breakage after a day, then it will be difficult to attribute > this to a specific commit. It is most useful to have continuous > integration if error reports are instantaneou

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread sstein...@gmail.com
On Oct 25, 2009, at 2:32 PM, Martin v. Löwis wrote: I've been trying to get some feedback about firing up buildbots on Cloud Servers for a while now and haven't had much luck. I'd love to find a way of having buildbots come to life, report to the mother ship, do the build, then go away '

[Python-Dev] Reworking the GIL

2009-10-25 Thread Antoine Pitrou
Hello there, The last couple of days I've been working on an experimental rewrite of the GIL. Since the work has been turning out rather successful (or, at least, not totally useless and crashing!) I thought I'd announce it here. First I want to stress this is not about removing the GIL. There s

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Martin v. Löwis
> OK, sounds useful. If I'm offline for a while, do multiple builds get > queued, or only the "last" one? IIRC, it will only build the last one, then with a huge blame list. > If the former, I can imagine coming > back to a pretty huge load if the slave breaks while I'm on holiday > :-( If it's

Re: [Python-Dev] [TIP] Possible language summit topic: buildbots

2009-10-25 Thread Brett Cannon
On Sun, Oct 25, 2009 at 07:13, wrote: > On 12:48 pm, c...@msu.edu wrote: > >> >> [snip] >> >> >> The most *exciting* part of pony-build, apart from the always-riveting >> spectacle of "titus rediscovering problems that buildbot solved 5 years >> ago", >> is the loose coupling of recording server

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Martin v. Löwis
> As I have no specific experience maintaining any of the CPython build > slaves, I can't speak to any maintenance issues which these slaves have > encountered. I would expect that they are as minimal as the issues I > have encountered maintaining slaves for other projects, but perhaps this > is w

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Martin v. Löwis
Antoine Pitrou wrote: >> For a), I think we can solve this only by redundancy, i.e. create more >> build slaves, hoping that a sufficient number would be up at any point >> in time. > > We are already doing this, aren't we? > http://www.python.org/dev/buildbot/3.x/ > > It doesn't seem to work ver

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-25 Thread Martin v. Löwis
> Hmm, perhaps when using sets as work queues? A number of comments: - it's somewhat confusing to use a set as a *queue*, given that it won't provide FIFO semantics. - there are more appropriate and direct container structures available, including a dedicated Queue module (even though this

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread exarkun
On 06:32 pm, mar...@v.loewis.de wrote: I've been trying to get some feedback about firing up buildbots on Cloud Servers for a while now and haven't had much luck. I'd love to find a way of having buildbots come to life, report to the mother ship, do the build, then go away 'till next time

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-25 Thread Martin v. Löwis
> If you actually care about that aspect of the semantics for a particular > application, it would be far clearer to use a full dict() and live with > the additional memory usage. While I can see how saving that > pointer-per-entry might make sense in some circumstances, for most I > would see it a

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-25 Thread Martin v. Löwis
> Given Alexander's premise, I agree with your response. But his premise > is wrong. Python's current builtin set class maps abstract equivalence > classes to representative members. And this *is* useful. Mapping > arbitrary members of such classes to representative members, sometimes > called 'can

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread C. Titus Brown
On Sun, Oct 25, 2009 at 07:32:52PM +0100, "Martin v. L?wis" wrote: > > I've been trying to get some feedback about firing up buildbots on Cloud > > Servers for a while now and haven't had much luck. I'd love to find a > > way of having buildbots come to life, report to the mother ship, do the > >

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Martin v. Löwis
> I've been trying to get some feedback about firing up buildbots on Cloud > Servers for a while now and haven't had much luck. I'd love to find a > way of having buildbots come to life, report to the mother ship, do the > build, then go away 'till next time they're required. I'm not quite sure w

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-25 Thread Florian Weimer
* James Y. Knight: > On Oct 25, 2009, at 2:50 AM, Terry Reedy wrote: > >> Alex Martelli wrote: >>> Next(s) would seem good... >> >> That does not work. It has to be next(iter(s)), and that has been >> tried and eliminated because it is significantly slower. > > But who cares about the speed of get

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Paul Moore
2009/10/25 : > If you run a build slave and it's offline when a build is requested, the > build will be queued and run when the slave comes back online.  So if the > CPython developers want to work this way (I wouldn't), then we don't need > pony-build; BuildBot will do just fine. OK, sounds usef

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread exarkun
On 05:47 pm, p.f.mo...@gmail.com wrote: 2009/10/25 : Perhaps this is a significant portion of the problem.  Maintaining a build slave is remarkably simple and easy.  I maintain about half a dozen slaves and spend at most a few minutes a month operating them. Actually setting one up in the fir

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Paul Moore
2009/10/25 : > Perhaps this is a significant portion of the problem.  Maintaining a build > slave is remarkably simple and easy.  I maintain about half a dozen slaves > and spend at most a few minutes a month operating them. Actually setting one > up in the first place might take a bit longer, sin

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-25 Thread James Y Knight
On Oct 25, 2009, at 2:50 AM, Terry Reedy wrote: Alex Martelli wrote: Next(s) would seem good... That does not work. It has to be next(iter(s)), and that has been tried and eliminated because it is significantly slower. But who cares about the speed of getting an arbitrary element from a

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread sstein...@gmail.com
On Oct 25, 2009, at 10:05 AM, exar...@twistedmatrix.com wrote: First, there are now a multitude of cloud hosting providers which will operate a slave machine for you. BuildBot has even begun to support this deployment use-case by allowing you to start up and shut down vms on demand to save

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Antoine Pitrou
twistedmatrix.com> writes: > > To me, that raises the question of why people aren't more concerned with > the status of the build system. Shouldn't developers care if the code > they're writing works or not? The fact that we ask questions and publicly express worries should hint that we /are/

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread sstein...@gmail.com
On Oct 25, 2009, at 9:50 AM, exar...@twistedmatrix.com wrote: Actually setting one up in the first place might take a bit longer, since it involves installing the necessary software and making sure everything's set up right, but the actual slave configuration itself is one command: buildb

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread sstein...@gmail.com
On Oct 25, 2009, at 5:47 AM, Martin v. Löwis wrote: These are actually two issues: a) where do we get buildbot hardware and operators? I've been trying to get some feedback about firing up buildbots on Cloud Servers for a while now and haven't had much luck. I'd love to find a way of hav

Re: [Python-Dev] [TIP] Possible language summit topic: buildbots

2009-10-25 Thread exarkun
On 12:48 pm, c...@msu.edu wrote: [snip] The most *exciting* part of pony-build, apart from the always-riveting spectacle of "titus rediscovering problems that buildbot solved 5 years ago", is the loose coupling of recording server to the build slaves and build reporters. My plan is to enable

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Jesse Noller
On Sun, Oct 25, 2009 at 8:48 AM, C. Titus Brown wrote: > [ x-posting to testing-in-python; please redirect followups to one list or > the other! ] > > Hi Mark, > > a few bits of information... > > --- > > I have a set of VM machines running some "core" build archs -- Linux, Mac OS > X, > Win XP,

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread exarkun
On 09:47 am, mar...@v.loewis.de wrote: Mark Dickinson wrote: Would it be worth spending some time discussing the buildbot situation at the PyCon 2010 language summit? In the past, I've found the buildbots to be an incredibly valuable resource; especially when working with aspects of Python or

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread exarkun
On 12:16 pm, solip...@pitrou.net wrote: For a), I think we can solve this only by redundancy, i.e. create more build slaves, hoping that a sufficient number would be up at any point in time. We are already doing this, aren't we? http://www.python.org/dev/buildbot/3.x/ It doesn't seem to work

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread C. Titus Brown
On Sun, Oct 25, 2009 at 08:54:46AM +, Mark Dickinson wrote: > Would it be worth spending some time discussing the buildbot situation > at the PyCon 2010 language summit? In the past, I've found the > buildbots to be an incredibly valuable resource; especially when > working with aspects of Py

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Antoine Pitrou
> For a), I think we can solve this only by redundancy, i.e. create more > build slaves, hoping that a sufficient number would be up at any point > in time. We are already doing this, aren't we? http://www.python.org/dev/buildbot/3.x/ It doesn't seem to work very well, it's a bit like a Danaides

Re: [Python-Dev] Clean up Python/thread_*.h ?

2009-10-25 Thread Andrew MacIntyre
Martin v. Löwis wrote: Making a decision and removing the files considered unnecessary would clarify what platforms still are/should be supported. I think any such removal should go through the PEP 11 process. Put a #error into the files for 3.2, and a removal notice into the PEP, then remove t

Re: [Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Martin v. Löwis
Mark Dickinson wrote: > Would it be worth spending some time discussing the buildbot situation > at the PyCon 2010 language summit? In the past, I've found the > buildbots to be an incredibly valuable resource; especially when > working with aspects of Python or C that tend to vary significantly

[Python-Dev] Possible language summit topic: buildbots

2009-10-25 Thread Mark Dickinson
Would it be worth spending some time discussing the buildbot situation at the PyCon 2010 language summit? In the past, I've found the buildbots to be an incredibly valuable resource; especially when working with aspects of Python or C that tend to vary significantly from platform to platform (for