Re: [Numpy-discussion] min() of array containing NaN

2008-08-13 Thread Joe Harrington
> I'm doing nothing. Someone else must volunteer. Fair enough. Would the code be accepted if contributed? > There is a > reasonable design rule that if you have a boolean argument which you > expect to only be passed literal Trues and Falses, you should instead > just have two different function

Re: [Numpy-discussion] reading *big* inhomogenous text matrices *fast*?

2008-08-13 Thread Sebastien Binet
Hi, > > Raymond Hettinger had a good talk at PyCon this year about the details > > of the Python containers. Here are the slides from the EuroPython > > version (I assume). > > > > http://www.pycon.it/static/pycon2/slides/containers.ppt > > Thanks! Looks like the only caveat is that the whole t

Re: [Numpy-discussion] reading *big* inhomogenous text matrices *fast*?

2008-08-13 Thread Daniel Lenski
On Wed, 13 Aug 2008 22:11:07 -0400, Zachary Pincus wrote: > Try profiling the code just to make sure that it is the list append > that's slow, and not something else happening on that line, e.g.. >From what you and others have pointed out, I'm pretty sure I must have been doing something else wro

Re: [Numpy-discussion] reading *big* inhomogenous text matrices *fast*?

2008-08-13 Thread Daniel Lenski
On Wed, 13 Aug 2008 21:42:51 -0500, Robert Kern wrote: > Here is the appropriate snippet in Objects/listobject.c: > > /* This over-allocates proportional to the list size, making > room > * for additional growth. The over-allocation is mild, but is * > enough to

Re: [Numpy-discussion] reading *big* inhomogenous text matrices *fast*?

2008-08-13 Thread Robert Kern
On Wed, Aug 13, 2008 at 21:07, Daniel Lenski <[EMAIL PROTECTED]> wrote: > On Wed, 13 Aug 2008 20:55:02 -0500, robert.kern wrote: >>> This is similar to what I tried originally! Unfortunately, repeatedly >>> appending to a list seems to be very slow... I guess Python keeps >>> reallocating and copy

Re: [Numpy-discussion] reading *big* inhomogenous text matrices *fast*?

2008-08-13 Thread Zachary Pincus
> This is similar to what I tried originally! Unfortunately, repeatedly > appending to a list seems to be very slow... I guess Python keeps > reallocating and copying the list as it grows. (It would be nice to > be > able to tune the increments by which the list size increases.) Robert's right

Re: [Numpy-discussion] reading *big* inhomogenous text matrices *fast*?

2008-08-13 Thread Daniel Lenski
On Wed, 13 Aug 2008 20:55:02 -0500, robert.kern wrote: >> This is similar to what I tried originally! Unfortunately, repeatedly >> appending to a list seems to be very slow... I guess Python keeps >> reallocating and copying the list as it grows. (It would be nice to be >> able to tune the increm

Re: [Numpy-discussion] non-linear array manipulation

2008-08-13 Thread Daniel Lenski
On Tue, 12 Aug 2008 10:37:51 -0400, Gong, Shawn (Contractor) wrote: > The following array manipulation takes long time because I can't find > ways to do in row/column, and have to do cell by cell. Would you check > to see if there is a nicer/faster way for this non-linear operation? > > for i in

Re: [Numpy-discussion] reading *big* inhomogenous text matrices *fast*?

2008-08-13 Thread robert . kern
On 2008-08-13, Daniel Lenski <[EMAIL PROTECTED]> wrote: > On Wed, 13 Aug 2008 16:57:32 -0400, Zachary Pincus wrote: >> Your approach generates numerous large temporary arrays and lists. If >> the files are large, the slowdown could be because all that memory >> allocation is causing some VM thrashi

Re: [Numpy-discussion] reading *big* inhomogenous text matrices *fast*?

2008-08-13 Thread Daniel Lenski
On Wed, 13 Aug 2008 16:57:32 -0400, Zachary Pincus wrote: > Your approach generates numerous large temporary arrays and lists. If > the files are large, the slowdown could be because all that memory > allocation is causing some VM thrashing. I've run into that at times > parsing large text files.

Re: [Numpy-discussion] test errors with numpy-1.1.0

2008-08-13 Thread David Cournapeau
On Wed, Aug 13, 2008 at 7:11 PM, Mathew Yeates <[EMAIL PROTECTED]> wrote: > On an AMD x86_64 with ATLAS installed I am getting errors like > ValueError: On entry to DLASD0 parameter number 9 had an illegal value > ValueError: On entry to ILAENV parameter number 2 had an illegal value Which platfor

[Numpy-discussion] test errors with numpy-1.1.0

2008-08-13 Thread Mathew Yeates
On an AMD x86_64 with ATLAS installed I am getting errors like ValueError: On entry to DLASD0 parameter number 9 had an illegal value ValueError: On entry to ILAENV parameter number 2 had an illegal value Anybody seen this before? Mathew ___ Numpy-di

Re: [Numpy-discussion] Mac OSX 4-way universal

2008-08-13 Thread robert . kern
On 2008-08-13, David Cournapeau <[EMAIL PROTECTED]> wrote: > On Wed, Aug 13, 2008 at 4:20 PM, Robert Kern <[EMAIL PROTECTED]> wrote: >> >> Hmm. Odd. I can't find the string "Can't install when cross-compiling" >> anywhere in the numpy or Python sources. Can you try again with the >> environment var

Re: [Numpy-discussion] Mac OSX 4-way universal

2008-08-13 Thread Chris Kees
You're right. I think it's coming from here: Lib/distutils/command/install.py:573 def run (self): # Obviously have to build before we can install if not self.skip_build: self.run_command('build') # If we built for any other platform, we can't install.

Re: [Numpy-discussion] Mac OSX 4-way universal

2008-08-13 Thread David Cournapeau
On Wed, Aug 13, 2008 at 4:20 PM, Robert Kern <[EMAIL PROTECTED]> wrote: > > Hmm. Odd. I can't find the string "Can't install when cross-compiling" > anywhere in the numpy or Python sources. Can you try again with the > environment variable DISTUTILS_DEBUG=1 set? You can find it in python svn: the

Re: [Numpy-discussion] min() of array containing NaN

2008-08-13 Thread Kevin Jacobs <[EMAIL PROTECTED]>
On Wed, Aug 13, 2008 at 4:01 PM, Robert Kern <[EMAIL PROTECTED]> wrote: > On Wed, Aug 13, 2008 at 14:37, Joe Harrington <[EMAIL PROTECTED]> wrote: > >>On Tue, Aug 12, 2008 at 19:28, Charles R Harris > >><[EMAIL PROTECTED]> wrote: > >>> > >>> > >>> On Tue, Aug 12, 2008 at 5:13 PM, Andrew Dalke < >

Re: [Numpy-discussion] Mac OSX 4-way universal

2008-08-13 Thread Robert Kern
On Wed, Aug 13, 2008 at 15:26, Chris Kees <[EMAIL PROTECTED]> wrote: > Here's a bit more. I tried sending the redirected output as attachments but > it got held up by the list server. I beg your pardon, then. You can send me the full logs in private email. > building extension "numpy.lib._compile

Re: [Numpy-discussion] C99 math funcs and windows

2008-08-13 Thread David Cournapeau
On Wed, Aug 13, 2008 at 3:20 PM, Matthieu Brucher <[EMAIL PROTECTED]> wrote: >> Do some people have a strong opinion on that ? Disabling sounds like >> the way to go for 1.2.0, but at some point, we will have to deal with >> this C99 problem on windows if people add more C99 code in it. > > Besides

Re: [Numpy-discussion] reading *big* inhomogenous text matrices *fast*?

2008-08-13 Thread Zachary Pincus
Hi Dan, Your approach generates numerous large temporary arrays and lists. If the files are large, the slowdown could be because all that memory allocation is causing some VM thrashing. I've run into that at times parsing large text files. Perhaps better would be to iterate through the file

Re: [Numpy-discussion] C99 math funcs and windows

2008-08-13 Thread Pauli Virtanen
Hi, Wed, 13 Aug 2008 13:06:47 -0500, David Cournapeau wrote: > Many tests (14) are failing on windows because of the recent changes in > some maths functions (C99 conformance). I believe the problem is caused > by mingw compiler (which is stuck at 3.4.5), but I still have to check > that. To solve

Re: [Numpy-discussion] Mac OSX 4-way universal

2008-08-13 Thread Chris Kees
Here's a bit more. I tried sending the redirected output as attachments but it got held up by the list server. building extension "numpy.lib._compiled_base" sources building extension "numpy.numarray._capi" sources building extension "numpy.fft.fftpack_lite" sources building extension "numpy.linal

Re: [Numpy-discussion] C99 math funcs and windows

2008-08-13 Thread Matthieu Brucher
> Do some people have a strong opinion on that ? Disabling sounds like > the way to go for 1.2.0, but at some point, we will have to deal with > this C99 problem on windows if people add more C99 code in it. Besides, not every one has a C99 compliant compiler on Windows... Is mingw compatible with

Re: [Numpy-discussion] min() of array containing NaN

2008-08-13 Thread Andrew Dalke
Robert Kern wrote: > Or we could implement the inner loop of the minimum ufunc to return > NaN if there is a NaN. Currently it just compares the two values > (which causes the unpredictable results since having a NaN on either > side of the < is always False). I would be amenable to that provided >

Re: [Numpy-discussion] Mac OSX 4-way universal

2008-08-13 Thread Robert Kern
On Wed, Aug 13, 2008 at 08:00, Chris Kees <[EMAIL PROTECTED]> wrote: > %python setup.py build > %sudo python setup.py install > ...snipped > adding 'build/scripts.macosx-10.5-universal-2.6/f2py64-32' to scripts > error: Can't install when cross-compiling Please don't snip. I need more context. -

[Numpy-discussion] reading *big* inhomogenous text matrices *fast*?

2008-08-13 Thread Dan Lenski
Hi all, I'm using NumPy to read and process data from ASCII UCD files. This is a file format for describing unstructured finite-element meshes. Most of the file consists of rectangular, numerical text matrices, easily and efficiently read with loadtxt(). But there is one particularly nasty se

Re: [Numpy-discussion] min() of array containing NaN

2008-08-13 Thread Robert Kern
On Wed, Aug 13, 2008 at 14:37, Joe Harrington <[EMAIL PROTECTED]> wrote: >>On Tue, Aug 12, 2008 at 19:28, Charles R Harris >><[EMAIL PROTECTED]> wrote: >>> >>> >>> On Tue, Aug 12, 2008 at 5:13 PM, Andrew Dalke <[EMAIL PROTECTED]> >>> wrote: On Aug 12, 2008, at 9:54 AM, Anne Archibald wrot

Re: [Numpy-discussion] min() of array containing NaN

2008-08-13 Thread Joe Harrington
>On Tue, Aug 12, 2008 at 19:28, Charles R Harris ><[EMAIL PROTECTED]> wrote: >> >> >> On Tue, Aug 12, 2008 at 5:13 PM, Andrew Dalke <[EMAIL PROTECTED]> >> wrote: >>> >>> On Aug 12, 2008, at 9:54 AM, Anne Archibald wrote: >>> > Er, is this actually a bug? I would instead consider the fact that >>> >

[Numpy-discussion] C99 on windows

2008-08-13 Thread David Cournapeau
Hi, The current trunk has 14 failures on windows (with mingw). 12 of them are related to C99 (see ticket 869). Can the people involved in recent changes to complex functions take a look at it ? I think this is high priority for 1.2.0 thanks, David ___

[Numpy-discussion] C99 math funcs and windows

2008-08-13 Thread David Cournapeau
Hi, Many tests (14) are failing on windows because of the recent changes in some maths functions (C99 conformance). I believe the problem is caused by mingw compiler (which is stuck at 3.4.5), but I still have to check that. To solve this, I see three solutions: - disabling the C99 tests on windo

Re: [Numpy-discussion] ravel() in ma/core.py

2008-08-13 Thread Eric Firing
Christoph T. Weidemann wrote: > Hi! > > I'm working with a subclass of ndarray and ran into an issue that > seems to be caused by a line in numpy/ma/core.py > The offending line is no. 1837 in version 1.1.0 or 2053 in the latest > SVN version (revision 5635): > r = ndarray.ravel(self._data).view(t

Re: [Numpy-discussion] min() of array containing NaN

2008-08-13 Thread Christopher Barker
Robert Kern wrote: > Or we could implement the inner loop of the minimum ufunc to return > NaN if there is a NaN. Currently it just compares the two values > (which causes the unpredictable results since having a NaN on either > side of the < is always False). I would be amenable to that provided >

Re: [Numpy-discussion] min() of array containing NaN

2008-08-13 Thread Alok Singhal
On 12/08/08: 18:31, Charles R Harris wrote: >OnTue, Aug 12, 2008 at 6:28 PM, Charles R Harris ><[EMAIL PROTECTED]> wrote: >I suppose you could use >min(a,b) = (abs(a - b) + a + b)/2 >which would have that effect. > >Hmm, that is for the max, min would

Re: [Numpy-discussion] unique1d returning indices

2008-08-13 Thread Stéfan van der Walt
2008/8/13 Robert Cimrman <[EMAIL PROTECTED]>: >> Yeah, that's why I think not many people used the extra return anyway. >> I will do as you say unless somebody steps in. > > ... but not before August 25, as I am about to leave on holidays and > have not managed to do it yet. I do not want to mess w

Re: [Numpy-discussion] SSEPlus + Framewave

2008-08-13 Thread David Cournapeau
On Wed, Aug 13, 2008 at 8:27 AM, Holger Rapp <[EMAIL PROTECTED]> wrote: >> >> You have to detect the presence of the library. If there are no such >> framework, you have to compile the module again (and for scipy under >> Windows, it is not simple). So developing a good plugin framework will >> hel

Re: [Numpy-discussion] unique1d returning indices

2008-08-13 Thread Robert Cimrman
Robert Cimrman wrote: > Stéfan van der Walt wrote: >> 2008/8/11 Robert Cimrman <[EMAIL PROTECTED]>: > Note also that the order of outputs has changed (previously unique1d() > returned (i, b) for return_index=True). Does this not constitute an API change? >>> It does. Are there many use

Re: [Numpy-discussion] ravel() in ma/core.py

2008-08-13 Thread Pierre GM
Christoph, Sorry to hear you're running into pbs w/ numpy.ma. Could you send me a stripped down version of your class so that I could perform tests and check whether calling np.ravel would work ? You could also write your own subclass of MaskedArray + your subclass and implement your own ravel.

Re: [Numpy-discussion] SSEPlus + Framewave

2008-08-13 Thread David Cournapeau
On Wed, Aug 13, 2008 at 3:28 AM, Holger Rapp <[EMAIL PROTECTED]> wrote: > What do you mean by compiling incompatible? It is my understanding > that (for example) Framewave (but also IPP) come in different flavors > (32bit, 64bit) which of course must be compiled in at compile time. > But which CPU

Re: [Numpy-discussion] SSEPlus + Framewave

2008-08-13 Thread Holger Rapp
> >> Or do it get you completly wrong? Is your intention to make a plugin >> architecture in the sense of: copy some directory with libs and >> config >> in your site-packages and then your multiplications are much >> faster? I >> would consider such a framework a bit overengineered, since spee

Re: [Numpy-discussion] Mac OSX 4-way universal

2008-08-13 Thread Chris Kees
%python setup.py build %sudo python setup.py install ...snipped adding 'build/scripts.macosx-10.5-universal-2.6/f2py64-32' to scripts error: Can't install when cross-compiling On 8/13/08 12:40 AM, "Robert Kern" <[EMAIL PROTECTED]> wrote: > On Tue, Aug 12, 2008 at 23:27, Kees, Christopher E > <[

Re: [Numpy-discussion] SSEPlus + Framewave

2008-08-13 Thread Matthieu Brucher
> What do you mean by compiling incompatible? It is my understanding > that (for example) Framewave (but also IPP) come in different flavors > (32bit, 64bit) which of course must be compiled in at compile time. > But which CPU is available and which features it delivers is indeed > done at runtime

Re: [Numpy-discussion] SSEPlus + Framewave

2008-08-13 Thread Holger Rapp
Hello David, > The problem is not so much the build part, but the clear separation I > was talking about. My experience with ATLAS convinced me the only way > to make sse work reliably is to detect the CPU arch at runtime; > compiling binaries incompatible on different arch is just not scalable >

Re: [Numpy-discussion] has f2py changed from 1.0.4 to 1.1.1 ????

2008-08-13 Thread mark
Robert, Charles - I am running Python 2.5.2 on Windows XP. I use the Enthought distribution, which comes with numpy 1.0.4. f2py works fine using mingw32 and gfortran. My command line is: python c:\python25\scripts\f2py.py -c -m besselaes --compiler=mingw32 --fcompiler=gnu95 besselaes.f90 Next I

Re: [Numpy-discussion] has f2py changed from 1.0.4 to 1.1.1 ????

2008-08-13 Thread mark
Charles and Robert - I am running Python 2.5.2 on Windows XP. I use the Enthought distribution, which comes with numpy 1.0.4. f2py works fine using mingw32 and gfortran. My command line is: python c:\python25\scripts\f2py.py -c -m besselaes --compiler=mingw32 --fcompiler=gnu95 besselaes.f90 Nex