Re: [Numpy-discussion] numpy.fft.irfftn fails apparently unexpectedly

2012-02-06 Thread Torgil Svensson
irfftn is an optimization for real input and does not take complex input. You have to use numpy.fft.ifftn instead: import numpy a_shape = (63, 4, 98) a = numpy.complex128(numpy.random.rand(*a_shape)+\ > ... 1j*numpy.random.rand(*a_shape)) axes = [0, 2] numpy.

Re: [Numpy-discussion] Counting the Colors of RGB-Image

2012-01-21 Thread Torgil Svensson
unique has an option to get indexes out which you can use in combination with sort to get the actual counts out. tab0 = zeros( 256*256*256 , dtype=int) col=ravel(((im0[...,0].astype('u4')*256+im0[...,1])*256)+im0[...,2]) col,idx=unique(sort(col),True) idx=hstack([idx,[2500*2500]]) tab0[col]=idx[1:

Re: [Numpy-discussion] fft help

2011-12-29 Thread Torgil Svensson
mph, I used both fftn and fft2, they both produce the same result. Is > there a restriction on the dimension of the input? power of 2 or some such? > > On 12/29/2011 07:21 AM, Torgil Svensson wrote: >> This is because fft computes one-dimensional transforms (on each row). &

Re: [Numpy-discussion] fft help

2011-12-29 Thread Torgil Svensson
This is because fft computes one-dimensional transforms (on each row). Try fft2 instead. //Torgil fft(a, n=None, axis=-1) Compute the one-dimensional discrete Fourier Transform. fft2(a, s=None, axes=(-2, -1)) Compute the 2-dimensional discrete Fourier Transform fftn(a, s=None, axes=Non

Re: [Numpy-discussion] Testing the python buffer protocol (bf_getbuffer / tp_as_buffer)

2011-12-16 Thread Torgil Svensson
What happens if you use y=numpy.frombuffer(x) ? //Torgil On Sat, Dec 17, 2011 at 1:41 AM, Soeren Sonnenburg wrote: > Hi, > > I've implemented the buffer protocol > (http://www.python.org/dev/peps/pep-3118/) for some matrix class and > when I manually call PyObject_GetBuffer on that object I se

Re: [Numpy-discussion] Array min from argmin along an axis?

2011-12-16 Thread Torgil Svensson
> |6> y[np.argmin(y, axis=0), np.arange(y.shape[1])] > array([0, 0, 0, 0, 0]) Can xrange in this case save me from creating a temporary array here or doesn't it matter? |6> y[np.argmin(y, axis=0), xrange(y.shape[1])] array([0, 0, 0, 0, 0]) //Torgil On Tue, Dec 13, 2011 at 11:27 PM, Robert Kern

Re: [Numpy-discussion] Can't mix np.newaxis with boolean indexing

2011-08-21 Thread Torgil Svensson
Since the result is one-dimensional after using boolean indexing you can always do: a[b][:, np.newaxis] array([[2], [3], [4]]) a[b][np.newaxis, :] array([[2, 3, 4]]) //Torgil On Sat, Aug 20, 2011 at 10:17 PM, Benjamin Root wrote: > On Sat, Aug 20, 2011 at 2:47 AM, Olivier Ver

Re: [Numpy-discussion] Efficient way to load a 1Gb file?

2011-08-14 Thread Torgil Svensson
Try the fromiter function, that will allow you to pass an iterator which can read the file line by line and not preload the whole file. file_iterator = iter(open('filename.txt') line_parser = lambda x: map(float,x.split('\t')) a=np.fromiter(itertools.imap(line_parser,file_iterator),dtype=float) Y

Re: [Numpy-discussion] convert csv file into recarray without pre-specifying dtypes and variable names

2007-07-19 Thread Torgil Svensson
Hi again, On 7/19/07, Torgil Svensson <[EMAIL PROTECTED]> wrote: If memory really is an issue, you have the nice "load_spec" version and can always convert the files once by iterating over the file twice like the attached script does. I discovered that my script was broken

Re: [Numpy-discussion] convert csv file into recarray without pre-specifying dtypes and variable names

2007-07-19 Thread Torgil Svensson
that is the same thing. I am not very familiar with zip, izip, map etc. just yet :) Thanks for the tip! 4. I called the function generated using exec, iter(). I need that function to transform the data using the types provided by the user. Best, Vincent On 7/18/07 7:57 PM, "Torgil S

Re: [Numpy-discussion] convert csv file into recarray without pre-specifying dtypes and variable names

2007-07-18 Thread Torgil Svensson
Nice, I haven't gone through all details. That's a nice new "missing" feature, maybe all instances where we can't find a conversion should be "nan". A few comments: 1. The "load_search" functions contains all memory/performance overhead that we wanted to avoid with the fromiter function. Does thi

Re: [Numpy-discussion] convert csv file into recarray without pre-specifying dtypes and variable names

2007-07-09 Thread Torgil Svensson
Elegant solution. Very readable and takes care of row0 nicely. I want to point out that this is much more efficient than my version for random/late string representation changes throughout the conversion but it suffers from 2*n memory footprint and large block copying if the string rep changes arr

Re: [Numpy-discussion] convert csv file into recarray without pre-specifying dtypes and variable names

2007-07-08 Thread Torgil Svensson
f you have missing values or 'mixed > types'. > > > > Vincent, > > Do you need to auto detect the column types? Things get a lot simpler if > you have some known schema for each file; then you can simply pass that to > some reader function. It's also mo

Re: [Numpy-discussion] convert csv file into recarray without pre-specifying dtypes and variable names

2007-07-08 Thread Torgil Svensson
nore the option of an int (i.e., everything is a float, date, or string) then your script is about twice as fast as mine!! Question: If you do ignore the int's initially, once the rec array is in memory, would there be a quick way to check if the floats could pass as int's? This may

Re: [Numpy-discussion] convert csv file into recarray without pre-specifying dtypes and variable names

2007-07-08 Thread Torgil Svensson
may seem like a backwards approach but it might be 'safer' if > you really want to preserve the int's. > > Thanks again! > > Vincent > > > On 7/8/07 5:52 AM, "Torgil Svensson" <[EMAIL PROTECTED]> wrote: > > > Given that both your script and

Re: [Numpy-discussion] convert csv file into recarray without pre-specifying dtypes and variable names

2007-07-08 Thread Torgil Svensson
On 7/8/07, Timothy Hochberg <[EMAIL PROTECTED]> wrote: On 7/8/07, Torgil Svensson <[EMAIL PROTECTED]> wrote: > Given that both your script and the mlab version preloads the whole > file before calling numpy constructor I'm curious how that compares in > speed to using

Re: [Numpy-discussion] convert csv file into recarray without pre-specifying dtypes and variable names

2007-07-08 Thread Torgil Svensson
Given that both your script and the mlab version preloads the whole file before calling numpy constructor I'm curious how that compares in speed to using numpy's fromiter function on your data. Using fromiter should improve on memory usage (~50% ?). The drawback is for string columns where we don

Re: [Numpy-discussion] annoying numpy string to float conversion behaviour

2007-06-26 Thread Torgil Svensson
On 6/26/07, Andrew Straw <[EMAIL PROTECTED]> wrote: > Torgil Svensson wrote: > > > This seems to indicate that float('nan') works on some platforms but > > str(nan) isn't. Is this true on Linux? Could anyone confirm this? What > > about float('

Re: [Numpy-discussion] annoying numpy string to float conversion behaviour

2007-06-26 Thread Torgil Svensson
On 6/26/07, Andrew Straw <[EMAIL PROTECTED]> wrote: > But, as Python is moving away from the libc for file IO in > Python 3K, perhaps string representation of floats would be considered, > too. (In fact for all I know, perhaps it has already been considered.) > Maybe you should email the python-3k-

Re: [Numpy-discussion] annoying numpy string to float conversion behaviour

2007-06-25 Thread Torgil Svensson
On 6/25/07, Torgil Svensson <[EMAIL PROTECTED]> wrote: > On 6/25/07, Robert Kern <[EMAIL PROTECTED]> wrote: > > NaNs and infs are IEEE-754 concepts. Python does run on non-IEEE-754 > > platforms, > > and I don't think that python-dev will want to entirely e

Re: [Numpy-discussion] annoying numpy string to float conversion behaviour

2007-06-25 Thread Torgil Svensson
On 6/25/07, Warren Focke <[EMAIL PROTECTED]> wrote: > False. No NaN should ever compare equal to anything, even itself. But if > the system is 754-compliant, it won't. > > "isnan(float(str(nan))) == True" would be nice, though. > Good point. Does this also hold true for the quiet nan's? //Torg

Re: [Numpy-discussion] annoying numpy string to float conversion behaviour

2007-06-25 Thread Torgil Svensson
On 6/25/07, Robert Kern <[EMAIL PROTECTED]> wrote: > NaNs and infs are IEEE-754 concepts. Python does run on non-IEEE-754 > platforms, > and I don't think that python-dev will want to entirely exclude them. You will > have to do *something* about those platforms. Possibly, they just won't > suppo

Re: [Numpy-discussion] annoying numpy string to float conversion behaviour

2007-06-25 Thread Torgil Svensson
On 6/22/07, Robert Kern <[EMAIL PROTECTED]> wrote: > Making float types parse/emit standard string > representations for NaNs and infs could probably go in if you were to provide > an > implementation and work out all of the bugs and cross-platform issues. The float types already emit string-repr

Re: [Numpy-discussion] annoying numpy string to float conversion behaviour

2007-06-21 Thread Torgil Svensson
On 6/21/07, David M. Cooke <[EMAIL PROTECTED]> wrote: > On Jun 20, 2007, at 04:35 , Torgil Svensson wrote: > > > Hi > > > > Is there a reason for numpy.float not to convert it's own string > > representation correctly? > > numpy.float is the Python f

Re: [Numpy-discussion] annoying numpy string to float conversion behaviour

2007-06-21 Thread Torgil Svensson
ctly the > same problem), numpy is not responsible for this, Python is. Python uses the > C standard library and in C by MS, NaN and Inf can be displayed, but not > read from a string, so this is the behaviour displayed here. > Wait for Python 3k... > > Matthieu > > 2007/

[Numpy-discussion] annoying numpy string to float conversion behaviour

2007-06-20 Thread Torgil Svensson
Hi Is there a reason for numpy.float not to convert it's own string representation correctly? Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32>>> import numpy >>> numpy.__version__ '1.0.3' >>> numpy.float("1.0") 1.0 >>> numpy.nan -1.#IND >>> numpy.float("-1.#I

Re: [Numpy-discussion] First Numerical Python code...comments?

2007-03-04 Thread Torgil Svensson
# Fill in SK with proper values for i in range(numel): RL = xord[i+1] - xord[i] RK = tens/RL SK[i][0] = SK[i][0] + RK SK[i][1] = SK[i][1] - RK IP1 = i+1 SK[IP1][0] = SK[IP1][0] + RK To get advantage of numpy you'll need to calculate with arrays instead of individual numbers: RL=xord

Re: [Numpy-discussion] Buffer PEP in NumPy SVN

2007-03-03 Thread Torgil Svensson
Impressive work and an very intresting approach. I have yet to figure out pros/cons or the variety of options this will give when writing applications/modules. Big plus for giving ctypes a way out of their crappy struct definitions. A few quick questions: Is the concept here that items always a

Re: [Numpy-discussion] concatenate and different numbers of dimensions

2007-02-12 Thread Torgil Svensson
As the error-message say, they must match number of dimensions, try: concatenate((tempLevel, temp[newaxis,...]), axis=0) On 2/13/07, Evan Mason <[EMAIL PROTECTED]> wrote: > Hi, I am trying to use concatenate to join together a 40x50x45 array and a > 50x45 array. The shape of the resulting array s

Re: [Numpy-discussion] Latest Array-Interface PEP

2007-01-12 Thread Torgil Svensson
On 1/12/07, Timothy Hochberg <[EMAIL PROTECTED]> wrote: > > > On 1/12/07, Torgil Svensson <[EMAIL PROTECTED]> wrote: > > On 1/12/07, Timothy Hochberg <[EMAIL PROTECTED]> wrote: > > [CHOP] > > > > > the core. I'm sure it's less effic

Re: [Numpy-discussion] Latest Array-Interface PEP

2007-01-12 Thread Torgil Svensson
On 1/12/07, Timothy Hochberg <[EMAIL PROTECTED]> wrote: > Things that act like arrays, but have different storage methods. This > details of this still seem pretty vague, but to the extent that I can figure > them out, it doesn't seem useful or necessary to tie this into the rest of > the array in

Re: [Numpy-discussion] Latest Array-Interface PEP

2007-01-12 Thread Torgil Svensson
On 1/12/07, Neal Becker <[EMAIL PROTECTED]> wrote: > How about: > > 1. A memory concept, of which buffer is an example. > 2. A view concept. > 3. A variety of common concrete types composing 1+2. Is there any difference between a view and an array that shares memory with another array? For acces

Re: [Numpy-discussion] Latest Array-Interface PEP

2007-01-12 Thread Torgil Svensson
On 1/12/07, Travis Oliphant <[EMAIL PROTECTED]> wrote: > This is true. So at what level do we propose the API. Single-item > access for sure, but what about > > array_interface->get_block_from_slice() ? > > Such a thing would be very useful for all kinds of large data-sets, from > images, and v

Re: [Numpy-discussion] Latest Array-Interface PEP

2007-01-11 Thread Torgil Svensson
On 1/11/07, Charles R Harris <[EMAIL PROTECTED]> wrote: > On 1/11/07, Torgil Svensson <[EMAIL PROTECTED]> wrote: > > Sure. I'm not objecting the memory model, what I mean is that data > > access between modules has a wider scope than just a memory model. > >

Re: [Numpy-discussion] Latest Array-Interface PEP

2007-01-11 Thread Torgil Svensson
On 1/11/07, Travis Oliphant <[EMAIL PROTECTED]> wrote: > Torgil Svensson wrote: >> Example1: We have a very large amount of data with a compressed >> internal representation >> >> Example2: We might want to generate data "on the fly" as it's needed

Re: [Numpy-discussion] Latest Array-Interface PEP

2007-01-10 Thread Torgil Svensson
On 1/4/07, Travis Oliphant <[EMAIL PROTECTED]> wrote: > > still be used to describe a complicated block of memory to another user. Thinking of the scope "seamless data exchange between modules" my concern with this PEP is that is might be too much focused on "block of memory" rather than "access t

Re: [Numpy-discussion] array with object

2006-11-29 Thread Torgil Svensson
It works if you replace the strings '2','3','5','6' with numbers instead. This case got a better error-message: >>> array(['2'],dtype=float32) Traceback (most recent call last): File "", line 1, in ? TypeError: a float is required //Torgil On 11/29/06, Lionel Roubeyrie <[EMAIL PROTECTED]> wr

Re: [Numpy-discussion] numpy: azip ?

2006-11-28 Thread Torgil Svensson
or if you want to stack array with more dimensions concatenate((a[...,newaxis],b[...,newaxis]),axis=-1) On 11/28/06, Robert Kern <[EMAIL PROTECTED]> wrote: > Robert wrote: > > I often need a function to zip 2 (1D) arrays together to a 2D array - > > similar as python's zip() does. > > Found no