Re: [Numpy-discussion] Huge arrays

2009-09-08 Thread David Cournapeau
On Wed, Sep 9, 2009 at 2:10 PM, Sebastian Haase wrote: > Hi, > you can probably use PyTables for this. Even though it's meant to > save/load data to/from disk (in HDF5 format) as far as I understand, > it can be used to make your task solvable - even on a 32bit system !! > It's free (pytables.org)

Re: [Numpy-discussion] Huge arrays

2009-09-08 Thread Sebastian Haase
Hi, you can probably use PyTables for this. Even though it's meant to save/load data to/from disk (in HDF5 format) as far as I understand, it can be used to make your task solvable - even on a 32bit system !! It's free (pytables.org) -- so maybe you can try it out and tell me if I'm right Or s

Re: [Numpy-discussion] Huge arrays

2009-09-08 Thread Sturla Molden
Daniel Platz skrev: > data1 = numpy.zeros((256,200),dtype=int16) > data2 = numpy.zeros((256,200),dtype=int16) > > This works for the first array data1. However, it returns with a > memory error for array data2. I have read somewhere that there is a > 2GB limit for numpy arrays on a 32 bit m

Re: [Numpy-discussion] Huge arrays

2009-09-08 Thread Charles R Harris
On Tue, Sep 8, 2009 at 7:30 PM, Daniel Platz < mail.to.daniel.pl...@googlemail.com> wrote: > Hi, > > I have a numpy newbie question. I want to store a huge amount of data > in an array. This data come from a measurement setup and I want to > write them to disk later since there is nearly no time

Re: [Numpy-discussion] dtype and dtype.char

2009-09-08 Thread Charles سمير Doutriaux
Ok I finally got it I was going at it backward... Instead of checking for NPY_INT64 and trying to figure out which letter it is different on each platform) I needed to check for NPY_LONGLONG /NPY_LONG/ NPY_INT, etc.. i.e I need to check for the numpy types that have an associated unique

Re: [Numpy-discussion] question about future support for python-3

2009-09-08 Thread Fernando Perez
On Tue, Sep 8, 2009 at 5:08 PM, David Cournapeau wrote: >  - it remains to be seen whether we can do the py3k support in the > same source tree as the one use for python >= 2.4. Having two source > trees would make the effort even much bigger, well over the current > developers capacity IMHO. I k

Re: [Numpy-discussion] question about future support for python-3

2009-09-08 Thread David Cournapeau
On Wed, Sep 9, 2009 at 9:37 AM, Darren Dale wrote: > Hi David, >> I already gave my own opinion on py3k, which can be summarized as: >>  - it is a huge effort, and no core numpy/scipy developer has >> expressed the urge to transition to py3k, since py3k does not bring >> much for scientific comput

Re: [Numpy-discussion] Huge arrays

2009-09-08 Thread David Cournapeau
On Wed, Sep 9, 2009 at 9:30 AM, Daniel Platz wrote: > Hi, > > I have a numpy newbie question. I want to store a huge amount of data > in  an array. This data come from a measurement setup and I want to > write them to disk later since there is nearly no time for this during > the measurement. To pu

[Numpy-discussion] Huge arrays

2009-09-08 Thread Daniel Platz
Hi, I have a numpy newbie question. I want to store a huge amount of data in an array. This data come from a measurement setup and I want to write them to disk later since there is nearly no time for this during the measurement. To put some numbers up: I have 2*256*200 int16 numbers which I w

Re: [Numpy-discussion] question about future support for python-3

2009-09-08 Thread Darren Dale
Hi David, On Tue, Sep 8, 2009 at 8:08 PM, David Cournapeau wrote: > On Wed, Sep 9, 2009 at 4:21 AM, Darren Dale wrote: >> I'm not a core numpy developer and don't want to step on anybody's >> toes here. But I was wondering if anyone had considered approaching >> the Python Software Foundation abou

Re: [Numpy-discussion] question about future support for python-3

2009-09-08 Thread David Cournapeau
On Wed, Sep 9, 2009 at 4:21 AM, Darren Dale wrote: > I'm not a core numpy developer and don't want to step on anybody's > toes here. But I was wondering if anyone had considered approaching > the Python Software Foundation about support to help get numpy working > with python-3? I already gave my

Re: [Numpy-discussion] question about future support for python-3

2009-09-08 Thread Charles R Harris
On Tue, Sep 8, 2009 at 5:57 PM, Christian Heimes wrote: > Darren Dale wrote: > > I'm not a core numpy developer and don't want to step on anybody's > > toes here. But I was wondering if anyone had considered approaching > > the Python Software Foundation about support to help get numpy working >

Re: [Numpy-discussion] question about future support for python-3

2009-09-08 Thread Christian Heimes
Darren Dale wrote: > I'm not a core numpy developer and don't want to step on anybody's > toes here. But I was wondering if anyone had considered approaching > the Python Software Foundation about support to help get numpy working > with python-3? What kind of support are you talking about? Develo

Re: [Numpy-discussion] Fwd: GPU Numpy

2009-09-08 Thread Christopher Barker
George Dahl wrote: > Sturla Molden molden.no> writes: >> Teraflops peak performance of modern GPUs is impressive. But NumPy >> cannot easily benefit from that. > I know that for my work, I can get around an order of a 50-fold speedup over > numpy using a python wrapper for a simple GPU matrix c

Re: [Numpy-discussion] question about future support for python-3

2009-09-08 Thread Darren Dale
Hi David, On Tue, Sep 8, 2009 at 3:56 PM, David Warde-Farley wrote: > Hey Darren, > > On 8-Sep-09, at 3:21 PM, Darren Dale wrote: > >> I'm not a core numpy developer and don't want to step on anybody's >> toes here. But I was wondering if anyone had considered approaching >> the Python Software Fo

Re: [Numpy-discussion] dtype and dtype.char

2009-09-08 Thread Charles سمير Doutriaux
Hi Robert, Ok we have a section of code that used to be like that: char t; switch(type) { case NPY_CHAR: t = 'c'; break; etc... I now replaced with char t; switch(type) { case NPY_CHAR: t = NPY_CHARLTR; break; But I'm still stuck with numpy.uint64 NPY_UINT6

Re: [Numpy-discussion] dtype and dtype.char

2009-09-08 Thread Robert Kern
2009/9/8 Charles سمير Doutriaux : > Hi, > > I'm testing our code on 64bit vs 32bit > > I just realized that the dtype.car is platform dependent. > > I guess it's normal > > her emy little test: > for t in > [numpy > .byte > ,numpy > .short > ,numpy > .int > ,numpy > .int32 > ,numpy > .float > ,nump

[Numpy-discussion] dtype and dtype.char

2009-09-08 Thread Charles سمير Doutriaux
Hi, I'm testing our code on 64bit vs 32bit I just realized that the dtype.car is platform dependent. I guess it's normal her emy little test: for t in [numpy .byte ,numpy .short ,numpy .int ,numpy .int32 ,numpy .float ,numpy .float32 ,numpy .double,numpy.ubyte,numpy.ushort,numpy.

Re: [Numpy-discussion] question about future support for python-3

2009-09-08 Thread David Warde-Farley
Hey Darren, On 8-Sep-09, at 3:21 PM, Darren Dale wrote: > I'm not a core numpy developer and don't want to step on anybody's > toes here. But I was wondering if anyone had considered approaching > the Python Software Foundation about support to help get numpy working > with python-3? It's a grea

Re: [Numpy-discussion] Fwd: GPU Numpy

2009-09-08 Thread George Dahl
Sturla Molden molden.no> writes: > > Erik Tollerud skrev: > >> NumPy arrays on the GPU memory is an easy task. But then I would have to > >> write the computation in OpenCL's dialect of C99? > > This is true to some extent, but also probably difficult to do given > > the fact that paralellizabl

[Numpy-discussion] question about future support for python-3

2009-09-08 Thread Darren Dale
I'm not a core numpy developer and don't want to step on anybody's toes here. But I was wondering if anyone had considered approaching the Python Software Foundation about support to help get numpy working with python-3? Thanks, Darren ___ NumPy-Discussi

Re: [Numpy-discussion] creating mesh data from xyz data

2009-09-08 Thread Neil Martinsen-Burrell
On 2009-09-08 10:38 , Christopher Barker wrote: > Giuseppe Aprea wrote: >> I have some files with data stored in columns: >> >> x1 y1 z1 >> x2 y2 z2 >> x3 y3 z3 >> x4 y4 z4 >> x5 y5 z5 >> I usually load data using 3 lists: x, y and z; I wonder if there is >>

Re: [Numpy-discussion] numpy/scipy/matplotlib + 10.6 + Apple python 2.6.1

2009-09-08 Thread Christopher Barker
David Cournapeau wrote: > I think it is best to avoid touching anything in /System. Yes, it is. > The better > solution is to install things locally, at least if you don't need to > share with several users one install. And if you do, you can put it in: /Library/Frameworks (/Library is kind Ap

Re: [Numpy-discussion] Behavior from a change in dtype?

2009-09-08 Thread Christopher Barker
Skipper Seabold wrote: > Hmm, okay, well I came across this in trying to create a recarray like > data2 below, so I guess I should just combine the two questions. key to understanding this is to understand what is going on under the hood in numpy. Travis O. gave a nice intro in an Enthought webca

Re: [Numpy-discussion] creating mesh data from xyz data

2009-09-08 Thread Christopher Barker
Giuseppe Aprea wrote: > I have some files with data stored in columns: > > x1 y1 z1 > x2 y2 z2 > x3 y3 z3 > x4 y4 z4 > x5 y5 z5 > I usually load data using 3 lists: x, y and z; I wonder if there is > any function which is able to take these 3 lists and retur

[Numpy-discussion] creating mesh data from xyz data

2009-09-08 Thread Giuseppe Aprea
Hi list, I have some files with data stored in columns: x1 y1 z1 x2 y2 z2 x3 y3 z3 x4 y4 z4 x5 y5 z5 ... and I need to make a contour plot of this data using matplotlib. The problem is that contour plot functions usually handle a different kind of inpu

Re: [Numpy-discussion] greppable file of all numpy functions ?

2009-09-08 Thread denis bzowy
denis bzowy t-online.de> writes: > > Does anyone have a program to generate a file with one line per Numpy function > / class / method, for local grepping ? Sorry I wasn't clear: I want just all defs, one per long line, like this: ... PyQt4.QtCore.QObject.findChildren(type type, QRegExp regExp)