Re: [Numpy-discussion] Summation of large float32/float64 arrays

2010-05-23 Thread Alan G Isaac
On 5/23/2010 9:33 PM, Christoph Gohlke wrote: > 2 GB is the memory limit for 32-bit processes running in > user-mode under 64-bit Windows, unless the executable was specifically > built with 'IMAGE_FILE_LARGE_ADDRESS_AWARE' set. > > See

Re: [Numpy-discussion] Summation of large float32/float64 arrays

2010-05-23 Thread Christoph Gohlke
On 5/23/2010 6:04 PM, Alan G Isaac wrote: > On 5/21/2010 8:34 PM, Christoph Gohlke wrote: >>the 32-bit Python >> interpreter can only use 2 GB of your memory > > Why? > > >>> 2**32/1e9 > 4.294967296003 > Because 2 GB is the memory limit for 32-bit processes running i

Re: [Numpy-discussion] Summation of large float32/float64 arrays

2010-05-23 Thread Alan G Isaac
On 5/21/2010 8:34 PM, Christoph Gohlke wrote: > the 32-bit Python > interpreter can only use 2 GB of your memory Why? >>> 2**32/1e9 4.294967296003 Thanks, Alan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http:/

Re: [Numpy-discussion] Summation of large float32/float64 arrays

2010-05-21 Thread Christoph Gohlke
On 5/21/2010 2:30 PM, Alan G Isaac wrote: > On 5/21/2010 4:13 PM, Matthew Turk wrote: >> a1 = numpy.random.random((512,512,512)).astype("float32") > > > This consistently gives me a "MemoryError". > I believe I have plenty of physical memory. > (That should require about 1.3G during creation, rig

Re: [Numpy-discussion] Summation of large float32/float64 arrays

2010-05-21 Thread Alan G Isaac
On 5/21/2010 4:13 PM, Matthew Turk wrote: > a1 = numpy.random.random((512,512,512)).astype("float32") This consistently gives me a "MemoryError". I believe I have plenty of physical memory. (That should require about 1.3G during creation, right? I have 8G.) It seems I'm hitting some kind of 1G me

Re: [Numpy-discussion] Summation of large float32/float64 arrays

2010-05-21 Thread Matthew Turk
Hi Robert, > It's not quite an overflow. > > In [1]: from numpy import * > > In [2]: x = float32(16777216.0) > > In [3]: x + float32(0.9) > Out[3]: 16777216.0 > > You are accumulating your result in a float32. With the a.sum() > approach, you eventually hit a level where the next number to add is

Re: [Numpy-discussion] Summation of large float32/float64 arrays

2010-05-21 Thread Robert Kern
On Fri, May 21, 2010 at 15:13, Matthew Turk wrote: > Hi all, > > I have a possibly naive question.  I don't really understand this > particular set of output: > > In [1]: import numpy > > In [2]: a1 = numpy.random.random((512,512,512)).astype("float32") > > In [3]: a1.sum(axis=0).sum(axis=0).sum(a

[Numpy-discussion] Summation of large float32/float64 arrays

2010-05-21 Thread Matthew Turk
Hi all, I have a possibly naive question. I don't really understand this particular set of output: In [1]: import numpy In [2]: a1 = numpy.random.random((512,512,512)).astype("float32") In [3]: a1.sum(axis=0).sum(axis=0).sum(axis=0) Out[3]: 67110312.0 In [4]: a1.sum() Out[4]: 16777216.0 I re