[Armin Rigo] ... > ... > 'hotshot', new from 2.2, is quite faster (reportedly, only 30% added > overhead). The log file is then loaded and turned into an instance of > the same 'pstats.Stats'. This loading takes ages. The reason is that > the log file only records events, and loading is done by instantiating a > 'profile.Profile' and sending it all the events. In other words, it > takes exactly as long as the time it spared in the first place!
We should note that hotshot didn't intend to reduce total time overhead. What it's aiming at here is to be less disruptive (than profile.py) to the code being profiled _while_ that code is running. On modern boxes, any kind of profiling gimmick has the unfortunate side effect of _changing_ the runtime behavior of the code being profiled, at least by polluting I and D caches with droppings from the profiling code itself (or, in the case of profile.py, possibly overwhelming I and top-level D caches -- and distorting non-profiling runtime so badly that, e.g., networked apps may end up taking entirely different code paths). hotshot tries to stick with tiny little C functions that pack away a tiny amount of data each time, and avoid memory alloc/dealloc, to try to minimize this disruption. It looked like it was making real progress on this at one time ;-) > Moreover, for some reasons, the results given by hotshot seem sometimes > quite wrong. (I don't understand why, but I've seen it myself, and it's > been reported by various people, e.g. [2].) 'hotshot' doesn't know > about C calls, but it can log line events, although this information is > lost(!) in the final conversion to a 'pstats.Stats'. Ya, hotshot isn't finished. It had corporate support for its initial development, but lost that, and became an orphan then. That's the eventual fate of most profilers, alas. They're fiddly, difficult, and always wrong in some respect. Because of this, the existence of an eager maintainer without a real life is more important than the code ;-). _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
