Re: [Numpy-discussion] tracing numpy data allocation with python callbacks

2012-05-21 Thread Thouis (Ray) Jones
On Mon, May 21, 2012 at 7:32 PM, Thouis (Ray) Jones wrote: > I submitted a PR for this functionality after cleaning it up a bit: > https://github.com/numpy/numpy/pull/284 I meant to ask here (and Travis reminded me in the PR): Currently, the trace_data_allocations() function (the only one added

Re: [Numpy-discussion] tracing numpy data allocation with python callbacks

2012-05-21 Thread Thouis (Ray) Jones
I submitted a PR for this functionality after cleaning it up a bit: https://github.com/numpy/numpy/pull/284 I've attached an example that produces HTML for a a sortable table tracking allocations while running through numpy.test(). Ray Jones sorttable.js Description: JavaScript source track_a

Re: [Numpy-discussion] tracing numpy data allocation with python callbacks

2012-05-17 Thread Thouis (Ray) Jones
On Thu, May 17, 2012 at 9:52 PM, Nathaniel Smith wrote: > I'd be tempted to just see if I could get by with massif or another > "real" heap profiler -- unfortunately the ones I know are C oriented, > but might still be useful... I got some very useful information from Fabien's technique, which le

Re: [Numpy-discussion] tracing numpy data allocation with python callbacks

2012-05-17 Thread Nathaniel Smith
On Thu, May 17, 2012 at 7:50 PM, Stéfan van der Walt wrote: > On Wed, May 16, 2012 at 12:34 PM, Thouis Jones wrote: >> I wondered, however, if there were a better way to accomplish the same >> goal, preferably in pure python. > > Fabien recently posted this; not sure if it addresses your use case

Re: [Numpy-discussion] tracing numpy data allocation with python callbacks

2012-05-17 Thread Stéfan van der Walt
On Wed, May 16, 2012 at 12:34 PM, Thouis Jones wrote: > I wondered, however, if there were a better way to accomplish the same > goal, preferably in pure python. Fabien recently posted this; not sure if it addresses your use case: http://fseoane.net/blog/2012/line-by-line-report-of-memory-usage/

[Numpy-discussion] tracing numpy data allocation with python callbacks

2012-05-16 Thread Thouis Jones
I recently had need of tracing numpy data allocation/deallocation. I was unable to find a simple way to do so, and so ended up putting the code below into ndarraytypes.h to allow me to trace allocations. A key part is that this jumps back into python, so I can inspect the stack and find out where