[Numpy-discussion] ndarray from column data

2009-07-01 Thread Elaine Angelino
Hi there -- Is there a fast way to make a numpy ndarray from column data? For example, suppose I want to make an ndarray with 2 rows and 3 columns of different data types based on the following column data: C0 = [1,2] C1 = ['a','b'] C2 = [3.3,4.4] I could create an empty ndarray and fill the co

Re: [Numpy-discussion] Numpy complex types, packing and C99

2009-07-01 Thread David Cournapeau
On Thu, Jul 2, 2009 at 5:34 AM, Pauli Virtanen wrote: > Don't know, probably we can? yes, they are functions (the builtins are __real__ and __imag__) > > I think that for scipy.special this would be useful. Of course, > the operator semantics for complex numbers can't be used there, > but still

Re: [Numpy-discussion] Numpy complex types, packing and C99

2009-07-01 Thread David Cournapeau
On Thu, Jul 2, 2009 at 5:25 AM, Charles R Harris wrote: > > > On Wed, Jul 1, 2009 at 1:57 PM, Pauli Virtanen wrote: >> >> On 2009-07-01, Charles R Harris wrote: >> > On Wed, Jul 1, 2009 at 1:59 AM, David Cournapeau >> > wrote: >> >>    I would like to add an explicit configuration test to check

Re: [Numpy-discussion] Numpy complex types, packing and C99

2009-07-01 Thread Pauli Virtanen
On 2009-07-01, Charles R Harris wrote: [clip] > OK. I don't see a problem then. As David says the ufuncs > already work that way. Hmm... do we want to > implement creal{l,,f} and cimag{l,,f} also? They might come useful. > They are built in functions in gcc, can we detect that? Don't know, p

Re: [Numpy-discussion] Numpy complex types, packing and C99

2009-07-01 Thread Charles R Harris
On Wed, Jul 1, 2009 at 1:57 PM, Pauli Virtanen wrote: > On 2009-07-01, Charles R Harris wrote: > > On Wed, Jul 1, 2009 at 1:59 AM, David Cournapeau < > da...@ar.media.kyoto-u.ac.jp> wrote: > >>I would like to add an explicit configuration test to check that our > >> complex type is compatibl

Re: [Numpy-discussion] Numpy complex types, packing and C99

2009-07-01 Thread Charles R Harris
On Wed, Jul 1, 2009 at 1:57 PM, Pauli Virtanen wrote: > On 2009-07-01, Charles R Harris wrote: > > On Wed, Jul 1, 2009 at 1:59 AM, David Cournapeau < > da...@ar.media.kyoto-u.ac.jp> wrote: > >>I would like to add an explicit configuration test to check that our > >> complex type is compatibl

Re: [Numpy-discussion] Numpy complex types, packing and C99

2009-07-01 Thread Pauli Virtanen
On 2009-07-01, Charles R Harris wrote: > On Wed, Jul 1, 2009 at 1:59 AM, David Cournapeau > wrote: >>I would like to add an explicit configuration test to check that our >> complex type is compatible with the C99 complex type (when available). >> Is this ok? Seems OK to me. >>As curren

Re: [Numpy-discussion] Numpy complex types, packing and C99

2009-07-01 Thread Charles R Harris
On Wed, Jul 1, 2009 at 1:59 AM, David Cournapeau < da...@ar.media.kyoto-u.ac.jp> wrote: > Hi, > >I would like to add an explicit configuration test to check that our > complex type is compatible with the C99 complex type (when available). > Is this ok ? > >As currently defined, complex c t

Re: [Numpy-discussion] make html

2009-07-01 Thread Pauli Virtanen
On 2009-07-01, Nils Wagner wrote: > I am using > numpy.__version__ > '1.4.0.dev7094' > > make html yields > > /home/nwagner/svn/numpy/doc/source/reference/generated/numpy.trunc.rst:: > WARNING: document isn't included in any toctree > done > preparing documents... done > Exception occurred:

[Numpy-discussion] make html

2009-07-01 Thread Nils Wagner
Hi all, I am using >>> numpy.__version__ '1.4.0.dev7094' make html yields /home/nwagner/svn/numpy/doc/source/reference/generated/numpy.trunc.rst:: WARNING: document isn't included in any toctree done preparing documents... done Exception occurred: 2%] reference/generalized_ufuncs ures

Re: [Numpy-discussion] np.memmap and memory usage

2009-07-01 Thread Francesc Alted
A Wednesday 01 July 2009 15:04:08 Francesc Alted escrigué: > However, you can still speed-up out-of-core computations by using the > recently introduced tables.Expr class (PyTables 2.2b1, see [2]), which uses > a combination of the Numexpr [3] and PyTables advanced computing > capabilities: > >

Re: [Numpy-discussion] np.memmap and memory usage

2009-07-01 Thread David Cournapeau
On Wed, Jul 1, 2009 at 6:14 PM, Pauli Virtanen wrote: > Wed, 01 Jul 2009 10:17:51 +0200, Emmanuelle Gouillart kirjoitti: >>       I'm using numpy.memmap to open big 3-D arrays of Xray tomography >> data. After I have created a new array using memmap, I modify the >> contrast of every Z-slice (along

Re: [Numpy-discussion] np.memmap and memory usage

2009-07-01 Thread Emmanuelle Gouillart
Hi Francesc, many thanks for this very detailed and informative answer! This list is really great :D. I'm going to install pytables at once and I will try your scripts with my data. As your computations were made with files of the same size as mine, hopefully it should run

Re: [Numpy-discussion] np.memmap and memory usage

2009-07-01 Thread Emmanuelle Gouillart
Hi Pauli, thank you for your answer! I was indeed measuring the memory used with top, which is not the best tool for understanding what really happens. I monitored "free" during the execution of my program and indeed, the used numbers on the "+/-buffers/cache" line stays roughly c

Re: [Numpy-discussion] np.memmap and memory usage

2009-07-01 Thread Francesc Alted
A Wednesday 01 July 2009 10:17:51 Emmanuelle Gouillart escrigué: > Hello, > > I'm using numpy.memmap to open big 3-D arrays of Xray tomography > data. After I have created a new array using memmap, I modify the > contrast of every Z-slice (along the first dimension) inside a for loop, >

[Numpy-discussion] Multi thread loading data

2009-07-01 Thread Mag Gam
Is it possible to use loadtxt in a mult thread way? Basically, I want to process a very large CSV file (100+ million records) and instead of loading thousand elements into a buffer process and then load another 1 thousand elements and process and so on... I was wondering if there is a technique wh

Re: [Numpy-discussion] np.memmap and memory usage

2009-07-01 Thread Pauli Virtanen
Wed, 01 Jul 2009 10:17:51 +0200, Emmanuelle Gouillart kirjoitti: > I'm using numpy.memmap to open big 3-D arrays of Xray tomography > data. After I have created a new array using memmap, I modify the > contrast of every Z-slice (along the first dimension) inside a for loop, > for a better vis

Re: [Numpy-discussion] permutation symbol

2009-07-01 Thread Nils Wagner
On Tue, 30 Jun 2009 13:51:15 -0600 Charles R Harris wrote: > On Tue, Jun 30, 2009 at 12:26 PM, Nils Wagner > wrote: > >> On Tue, 30 Jun 2009 11:10:39 -0600 >> Charles R Harris wrote: >> > On Tue, Jun 30, 2009 at 10:56 AM, Charles R Harris < >> > charlesr.har...@gmail.com> wrote: >> > >> >> >

[Numpy-discussion] Numpy complex types, packing and C99

2009-07-01 Thread David Cournapeau
Hi, I would like to add an explicit configuration test to check that our complex type is compatible with the C99 complex type (when available). Is this ok ? As currently defined, complex c types (npy_cfloat, etc...) are not defined in a way such as they are binary compatible with the C99

[Numpy-discussion] np.memmap and memory usage

2009-07-01 Thread Emmanuelle Gouillart
Hello, I'm using numpy.memmap to open big 3-D arrays of Xray tomography data. After I have created a new array using memmap, I modify the contrast of every Z-slice (along the first dimension) inside a for loop, for a better visualization of the data. Although I call memmap.flush af

Re: [Numpy-discussion] Tutorial topics for SciPy'09 Conference

2009-07-01 Thread Fernando Perez
Hi, On Mon, Jun 1, 2009 at 10:20 PM, Fernando Perez wrote: > The time for the Scipy'09 conference is rapidly approaching, and we > would like to both announce the plan for tutorials and solicit > feedback from everyone on topics of interest. rather than rehash much here, where it's not easy to pa