mark florisson, 28.01.2012 20:07: > On 28 January 2012 18:38, Stefan Behnel wrote: >> Stefan Behnel, 27.01.2012 09:02: >>> any exception *propagation* is >>> still substantially slower than necessary, and that's a general issue. >> >> Here's a general take on a code object cache for exception propagation. >> >> https://github.com/scoder/cython/commit/ad18e0208 >> >> When I raise an exception in test code that propagates through a Python >> call hierarchy of four functions before being caught, the cache gives me >> something like a 2x speedup in total. Not bad. When I do the same for cdef >> functions, it's more like 4-5x. >> >> The main idea is to cache the objects in a reallocable C array and bisect >> into it based on the C code "__LINE__" of the exception, which should be >> unique enough for a given module. >> >> It's a global cache that doesn't limit the lifetime of code objects (well, >> up to the lifetime of the module, obviously). I don't know if that's a >> problem because the number of code objects is only bounded by the number of >> exception origination points in the C source code, which is usually quite >> large. However, only a tiny fraction of those will ever raise or propagate >> an exception in practice, so the real number of cached code objects will be >> substantially smaller. >> >> Maybe thorough test suites with lots of failure testing would notice a >> difference in memory consumption, even though a single code objects isn't >> all that large either... >> >> What do you think? > > Nice. I have a question, couldn't you save the frame object instead of > the code object?
Technically, yes. However, eventually, I'd like to make the CodeObject constant for the whole function and let CPython calculate the Python code source line based on the frame's "f_lasti" field when the line number is actually accessed. For now, I wouldn't mind cashing the whole frame until the above optimisation gets implemented. > I do think PyCodeObject is rather large, on my 64 bit machine it is > 120 bytes, not accounting for any of the objects it holds (not saying > that's a problem, just pointing it out). Hmm, ok. That's not really cheap, especially given the amount of redundancy in the content. Maybe we could intern the strings after creating them. > Would it help if we would pass in the position information string > object constant, to avoid the PyString_Format? That optimization would > only save 14% though. PyString_FromFormat() is impressively expensive. So, yes, as Vitja figured out, that would help. But I actually like the feature. > But additionally, the function name could be a > string object constant, which could be shared by all exceptions in one > function, avoiding another PyString_FromString. Yes, and in many cases, certainly for all Python functions, it's already there anyway. I originally rejected that idea because more string constants add to the module initialisation time (performance analysis pending here). It may still be worth doing at least for Python functions. Stefan _______________________________________________ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
