Re: [Python-Dev] s/hotshot/lsprof

2005-11-22 Thread Armin Rigo
Hi Brett, On Mon, Nov 21, 2005 at 11:35:37PM -0800, Brett Cannon wrote: > (any chance we can replace profile with a wrapper to lsprof > without much issue?) Yes. In fact I am thinking about adding lsprof under the module name 'cProfile', to keep true to the (IMHO) good tradition of pickle/cPickl

Re: [Python-Dev] s/hotshot/lsprof

2005-11-22 Thread Floris Bruynooghe
On Mon, Nov 21, 2005 at 12:14:26PM +0100, Armin Rigo wrote: > Hi Brett, hi Floris, > > On Sat, Nov 19, 2005 at 04:12:28PM -0800, Brett Cannon wrote: > > Just for everyone's FYI while we are talking about profilers, Floris > > Bruynooghe (who I am cc'ing on this so he can contribute to the > > con

Re: [Python-Dev] s/hotshot/lsprof

2005-11-22 Thread Floris Bruynooghe
Hello On Mon, Nov 21, 2005 at 12:14:30PM +0100, Armin Rigo wrote: > On Sun, Nov 20, 2005 at 08:55:49PM -0500, Tim Peters wrote: > > We should note that hotshot didn't intend to reduce total time > > overhead. What it's aiming at here is to be less disruptive (than > > profile.py) to the code bein

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Brett Cannon
On 11/21/05, Armin Rigo <[EMAIL PROTECTED]> wrote: > Hi Martin, > > On Mon, Nov 21, 2005 at 10:29:55PM +0100, "Martin v. L?wis" wrote: > > > I see no incremental way of fixing some of the downsides of hotshot, > > > like its huge log file size and loading time. > > > > I haven't looked into the det

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Armin Rigo
Hi Martin, On Mon, Nov 21, 2005 at 10:29:55PM +0100, "Martin v. L?wis" wrote: > > I see no incremental way of fixing some of the downsides of hotshot, > > like its huge log file size and loading time. > > I haven't looked into the details myself, but it appears that some > google-summer-of-code c

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread skip
Brett> My question is whether anyone is willing to maintain it in the Brett> stdlib? My answer is: I'm not sure it matters at this point. There are so many profiling possibilities, it doesn't seem like we yet know which options are the best. There is some tacit crowning of "best of bree

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Nick Coghlan
Jim Jewett wrote: > Jeremy Hylton jeremy at alum.mit.edu >> Should lsprof be added to the standard distribution? >> Should hotshot be removed from the standard distribution? > >> These two aren't at all related, unless you believe that two is the >> maximum number of profiles allowed per Pytho

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Martin v. Löwis
Brett Cannon wrote: > But this worry, in my mind, is alleviated since I believe both Michael > and Armin are willing to maintain the code. With them both willing to > make sure it stays working (which is a pretty damn good commitment > since we have two core developers willing to keep this going a

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Martin v. Löwis
Armin Rigo wrote: > I see no incremental way of fixing some of the downsides of hotshot, > like its huge log file size and loading time. I haven't looked into the details myself, but it appears that some google-summer-of-code contributor has found some way of fixing it. > I doubt people often fin

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Armin Rigo
Hi Floris, On Mon, Nov 21, 2005 at 04:45:03PM +, Floris Bruynooghe wrote: > Afaik I did test recursive calls etc. It seems to show up in any test case I try, e.g. import hprofile def wait(m): if m > 0: wait(m-1) def f(n): wait(n) if n > 1:

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Armin Rigo
Hi Floris, On Mon, Nov 21, 2005 at 04:41:04PM +, Floris Bruynooghe wrote: > > Now Brett's > > student, Floris, extended hotshot to allow custom timers. This is > > essential, because it enables testing. The timing parts of hotshot were > > not tested previously. > > Don't be too enthousiast

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Brett Cannon
On 11/21/05, Jeremy Hylton <[EMAIL PROTECTED]> wrote: > Here's another attempt to disentagle some issues: > - Should lsprof be added to the standard distribution? > - Should hotshot be removed from the standard distribution? > > These two aren't at all related, unless you believe that two is the >

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Brett Cannon
On 11/21/05, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > Jeremy> Here's another attempt to disentagle some issues: > Jeremy> - Should lsprof be added to the standard distribution? > Jeremy> - Should hotshot be removed from the standard distribution? > > Adding another log to the fi

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread skip
Jeremy> Here's another attempt to disentagle some issues: Jeremy> - Should lsprof be added to the standard distribution? Jeremy> - Should hotshot be removed from the standard distribution? Adding another log to the fire, what about statprof, a sampling profiler, which Neil Schemenauer

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Jeremy Hylton
Here's another attempt to disentagle some issues: - Should lsprof be added to the standard distribution? - Should hotshot be removed from the standard distribution? These two aren't at all related, unless you believe that two is the maximum number of profiles allowed per Python distribution. I've

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Brett Cannon
On 11/21/05, Armin Rigo <[EMAIL PROTECTED]> wrote: > Hi Martin, > > On Mon, Nov 21, 2005 at 08:12:53AM +0100, "Martin v. L?wis" wrote: > > If bugs are in the heavily-used parts of the library, like regular > > expressions, it doesn't matter much if the original author goes > > away for some period

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Armin Rigo
Hi Barry, On Mon, Nov 21, 2005 at 11:40:37AM -0500, Barry Warsaw wrote: > Hi Armin. Actually it was SF #900092 that I was referring to. Ah, we're talking about different things then. The patch in SF #900092 is not related to hotshot, it's just ceval.c not producing enough events to allow a prec

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Barry Warsaw
On Mon, 2005-11-21 at 16:09 +0100, Armin Rigo wrote: > It might be me, but I find it a bit odd that you didn't do anything with > this fix. Hi Armin. Actually it was SF #900092 that I was referring to. We fixed this bug and those patches were applied to CVS (pre-svn conversion) for both 2.4.2

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Armin Rigo
Hi Barry, On Mon, Nov 21, 2005 at 09:25:05AM -0500, Barry Warsaw wrote: > hotshot used to produce incorrect data because it couldn't track exits > from functions due to exception propagation. We fixed that a while back It might be me, but I find it a bit odd that you didn't do anything with this

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Barry Warsaw
On Mon, 2005-11-21 at 12:14 +0100, Armin Rigo wrote: > Still, people generally agree that profile.py, while taking a longer > time overall, gives more meaningful results than hotshot. Now Brett's > student, Floris, extended hotshot to allow custom timers. This is > essential, because it enables

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Armin Rigo
Hi Martin, On Mon, Nov 21, 2005 at 08:12:53AM +0100, "Martin v. L?wis" wrote: > If bugs are in the heavily-used parts of the library, like regular > expressions, it doesn't matter much if the original author goes > away for some period of time - other contributors will fix the bugs > that they car

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Armin Rigo
Hi Tim, On Sun, Nov 20, 2005 at 08:55:49PM -0500, Tim Peters wrote: > We should note that hotshot didn't intend to reduce total time > overhead. What it's aiming at here is to be less disruptive (than > profile.py) to the code being profiled _while_ that code is running. > hotshot tries to stic

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Armin Rigo
Hi Brett, hi Floris, On Sat, Nov 19, 2005 at 04:12:28PM -0800, Brett Cannon wrote: > Just for everyone's FYI while we are talking about profilers, Floris > Bruynooghe (who I am cc'ing on this so he can contribute to the > conversation), for Google's Summer of Code, wrote a replacement for > 'prof

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Martin v. Löwis
Tim Peters wrote: > Center for Alcohol & Drug Treatment Besides Jamie Zawinski's definition, Google also told me it stands for Computer Aided Drafting Technology where "to draft" turns out to have two different meanings :-) Regards, Martin __

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Martin v. Löwis
Fredrik Lundh wrote: > is this some intrinsic property of profilers? if the existing tool has > problems, why not improve the tool itself? do we really need CADT- > based development in the standard library? It is, IMO, intrinsic to parts of the library that aren't used much. If bugs are in the

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Neil Schemenauer
Tim Peters <[EMAIL PROTECTED]> wrote: > We should note that hotshot didn't intend to reduce total time > overhead. What it's aiming at here is to be less disruptive (than > profile.py) to the code being profiled _while_ that code is running. A statistical profiler (e.g. http://wingolog.org/archi

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Fred L. Drake, Jr.
On Sunday 20 November 2005 21:02, Tim Peters wrote: > Since I didn't know what that meant, Google helpfully told me: > > Center for Alcohol & Drug Treatment On Sunday 20 November 2005 22:04, Steve Holden wrote: > I suspect you may already know that Fredrik referred to >  >        Cascade

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread A.M. Kuchling
On Sun, Nov 20, 2005 at 11:33:42PM +0100, Fredrik Lundh wrote: > do we really need CADT-based development in the standard library? I didn't recognize the acronym, but Google told me CADT = "Cascade of Attention-Deficit Teenagers"; see http://www.jwz.org/doc/cadt.html for a rant. --amk __

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Steve Holden
Tim Peters wrote: > [Martin v. Löwis] > >>>I'm really concerned that the same fate will happen to any new >>>profiling library: anybody but the original author will hate it, >>>write his own, and then suggest to replace the existing one. > > > [Fredrik Lundh] > >>is this some intrinsic property

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Tim Peters
[Martin v. Löwis] >> I'm really concerned that the same fate will happen to any new >> profiling library: anybody but the original author will hate it, >> write his own, and then suggest to replace the existing one. [Fredrik Lundh] > is this some intrinsic property of profilers? if the existing t

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Tim Peters
[Armin Rigo] ... > ... > 'hotshot', new from 2.2, is quite faster (reportedly, only 30% added > overhead). The log file is then loaded and turned into an instance of > the same 'pstats.Stats'. This loading takes ages. The reason is that > the log file only records events, and loading is done by

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread skip
Fredrik> (on the other hand, I'm not sure we need a profiler as part of Fredrik> the standard library either, but that's me...) Painful though hotshot can be at times, I occasionally find it extremely useful to zoom in on trouble spots. I haven't used profile in awhile and haven't tried

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Fredrik Lundh
Martin v. Löwis wrote: > The same could be said about hotshot, which was originally contributed > by Fred Drake, and hacked by Tim Peters, yourself, and others. Yet, now > people want to remove it again. > > I'm really concerned that the same fate will happen to any new > profiling library: anybod

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Martin v. Löwis
Michael Hudson wrote: >>As stated, it certainly is a bad idea. > > > This is a bit extreme... Yes, my apologies :-( >>To make it a good idea, there should also be some commitment to >>maintain this library for a number of years. So who would be >>maintaining it, and what are their plans for doi

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Michael Hudson
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > Armin Rigo wrote: >> If anyone feels like this is a bad idea, please speak up. > > As stated, it certainly is a bad idea. This is a bit extreme... > To make it a good idea, there should also be some commitment to > maintain this library for a numbe

Re: [Python-Dev] s/hotshot/lsprof

2005-11-19 Thread Brett Cannon
Just for everyone's FYI while we are talking about profilers, Floris Bruynooghe (who I am cc'ing on this so he can contribute to the conversation), for Google's Summer of Code, wrote a replacement for 'profile' that uses Hotshot directly. Thanks to his direct use of Hotshot and rewrite of pstats

Re: [Python-Dev] s/hotshot/lsprof

2005-11-19 Thread Martin v. Löwis
Armin Rigo wrote: > If anyone feels like this is a bad idea, please speak up. As stated, it certainly is a bad idea. To make it a good idea, there should also be some commitment to maintain this library for a number of years. So who would be maintaining it, and what are their plans for doing so?

Re: [Python-Dev] s/hotshot/lsprof

2005-11-19 Thread Aahz
On Sat, Nov 19, 2005, Armin Rigo wrote: > > If anyone feels like this is a bad idea, please speak up. This sounds like a good idea, and your presentation already looks almost like a PEP. How about going ahead and making it a formal PEP, which will make it easier to push through the dev process? -