Re: [Numpy-discussion] Python memory management issues using, Linux. Maybe Numpy related.

2011-05-22 Thread Jeffrey Spencer
There are a few places where we (improperly) directly call malloc() instead of PyMem_Malloc(), so yes, you should rebuild numpy against TCMalloc in addition to the Python interpreter. How do I build against the -ltcmalloc library? Just not sure how to input compile time arguments with the build

Re: [Numpy-discussion] Python memory management issues using, Linux. Maybe Numpy, related.

2011-05-22 Thread Bruce Southey
On Sun, May 22, 2011 at 1:17 PM, Jeffrey Spencer wrote: > Further isolated the error and it seems to come in from the sum function in > numpy. I will probe further later but the code below causes the memory usage > to go up and up. If you change sum below to some other function there are no > issu

Re: [Numpy-discussion] Python memory management issues using, Linux. Maybe Numpy, related.

2011-05-22 Thread Eric Firing
On 05/22/2011 08:17 AM, Jeffrey Spencer wrote: > from numpy import arange, sum > > for x in range(1000): > inhibVal = sum(arange(15)) Memory usage stays constant with Ubuntu 11.04, 64-bit, using the numpy 1.5.1 package from ubuntu, and using 1.6.1.dev-a265004. efiring@manini:~$ un

Re: [Numpy-discussion] Python memory management issues using, Linux. Maybe Numpy, related.

2011-05-22 Thread Jeffrey Spencer
Further isolated the error and it seems to come in from the sum function in numpy. I will probe further later but the code below causes the memory usage to go up and up. If you change sum below to some other function there are no issues. from numpy import arange, sum for x in range(1

Re: [Numpy-discussion] AttributeError in numpy.distutils

2011-05-22 Thread Branimir Sesar
On 05/22/2011 04:17 AM, Ralf Gommers wrote: > > > On Thu, May 19, 2011 at 8:28 PM, Branimir Sesar > mailto:bse...@astro.caltech.edu>> wrote: > > Dear Numpy users, > > I've encountered an AttributeError in numpy.distutils > >File > > "/home/bsesar/usr/pydebug/lib/python2.7/site-

Re: [Numpy-discussion] Python memory management issues using Linux. Maybe Numpy related.

2011-05-22 Thread Robert Kern
On Sat, May 21, 2011 at 23:27, Jeffrey Spencer wrote: > I have had issues with Linux (Ubuntu 10.04 x86_64) not releasing memory > back to the OS. I have code here the exhibits the issue: > > http://stackoverflow.com/questions/5975255/memory-allocated-to-python-in-the-os-is-never-released-back-in-l

Re: [Numpy-discussion] Python memory management issues using, Linux. Maybe Numpy, related.

2011-05-22 Thread Jeffrey Spencer
Message: 3 Date: Sun, 22 May 2011 09:32:10 -0500 From: Bruce Southey Subject: Re: [Numpy-discussion] Python memory management issues using Linux. Maybe Numpy, related. To: Discussion of Numerical Python Message-ID: Content-Type: text/plain; charset=ISO-8859-1 On Su

Re: [Numpy-discussion] Python memory management issues using Linux. Maybe Numpy, related.

2011-05-22 Thread Jeffrey Spencer
Much reduced code exhibiting the issue. This shouldn't use much memory but just keeps going up and up because the copied window I assume gets left around and not taken up by the garbage collector. On 22/05/11 23:42, Ralf Gommers wrote: On Sun, May 22, 2011 at 2:54 PM, Jeffrey Spencer mailto

Re: [Numpy-discussion] datarray -- a couple of questions

2011-05-22 Thread Wes McKinney
On Sun, May 22, 2011 at 10:49 AM, Ben Walsh wrote: > Hi > > I've been looking at various labelled-array packages, including Pandas > (https://github.com/wesm/pandas) and datarray > (https://github.com/fperez/datarray). > > I was quite interested in the design discussion for datarray, and I had > a

[Numpy-discussion] datarray -- a couple of questions

2011-05-22 Thread Ben Walsh
Hi I've been looking at various labelled-array packages, including Pandas (https://github.com/wesm/pandas) and datarray (https://github.com/fperez/datarray). I was quite interested in the design discussion for datarray, and I had a couple of questions: 1) Is this mailing list the correct plac

Re: [Numpy-discussion] Python memory management issues using Linux. Maybe Numpy, related.

2011-05-22 Thread Bruce Southey
On Sun, May 22, 2011 at 8:42 AM, Ralf Gommers wrote: > > > On Sun, May 22, 2011 at 2:54 PM, Jeffrey Spencer > wrote: >> >> Sorry. I attached here the class and script to run it. It is just a >> minimalistic example of what I'm really doing that exhibits the same >> behavior. All the script does i

Re: [Numpy-discussion] Python memory management issues using Linux. Maybe Numpy, related.

2011-05-22 Thread Ralf Gommers
On Sun, May 22, 2011 at 2:54 PM, Jeffrey Spencer wrote: > Sorry. I attached here the class and script to run it. It is just a > minimalistic example of what I'm really doing that exhibits the same > behavior. All the script does is run at.createSpecific after instantiating > the class. I have seen

Re: [Numpy-discussion] Python memory management issues using Linux. Maybe Numpy, related.

2011-05-22 Thread Jeffrey Spencer
Sorry. I attached here the class and script to run it. It is just a minimalistic example of what I'm really doing that exhibits the same behavior. All the script does is run at.createSpecific after instantiating the class. I have seen this exhibited on many other cases but it runs fine in Windo

Re: [Numpy-discussion] finding longest contiguous array slices

2011-05-22 Thread Tom Krauss
On Sat, May 21, 2011 at 5:55 AM, Ben Elliston wrote: > I have a 3-d array of booleans (5 x 5 x 1000, say) and would like to > return a 5 x 5 of array slices containing the longest contiguous > sequence of True values along the z axis. What do you mean by "5 x 5 of array slices containing the long

Re: [Numpy-discussion] AttributeError in numpy.distutils

2011-05-22 Thread Ralf Gommers
On Thu, May 19, 2011 at 8:28 PM, Branimir Sesar wrote: > Dear Numpy users, > > I've encountered an AttributeError in numpy.distutils > > File > > "/home/bsesar/usr/pydebug/lib/python2.7/site-packages/numpy/distutils/command/build_src.py", > line 646, in swig_sources > extension.swig_opts.rem

Re: [Numpy-discussion] Python memory management issues using Linux. Maybe Numpy related.

2011-05-22 Thread Ralf Gommers
On Sun, May 22, 2011 at 6:27 AM, Jeffrey Spencer wrote: > I have had issues with Linux (Ubuntu 10.04 x86_64) not releasing memory > back to the OS. I have code here the exhibits the issue: > > > http://stackoverflow.com/questions/5975255/memory-allocated-to-python-in-the-os-is-never-released-back-