Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Anne Archibald
2008/9/19 Eric Firing <[EMAIL PROTECTED]>: > Pierre GM wrote: > >>> It seems to me that there are pragmatic reasons >>> why people work with NaNs for missing values, >>> that perhaps shd not be dismissed so quickly. >>> But maybe I am overlooking a simple solution. >> >> nansomething solutions tend

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Robert Kern
On Sat, Sep 20, 2008 at 01:15, Charles R Harris <[EMAIL PROTECTED]> wrote: > I would be happy to implement nan sorts if someone can provide me with a > portable and easy way to detect nans for single, double, and long double > floats. And not have it fail if the architecture doesn't support nans.

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Charles R Harris
On Fri, Sep 19, 2008 at 11:41 PM, David Cournapeau < [EMAIL PROTECTED]> wrote: > Anne Archibald wrote: > > > > I, on the other hand, was making specifically that suggestion: users > > should not use nans to indicate missing values. Users should use > > masked arrays to indicate missing values. > >

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread David Cournapeau
Anne Archibald wrote: > > I, on the other hand, was making specifically that suggestion: users > should not use nans to indicate missing values. Users should use > masked arrays to indicate missing values. I agree it is the nicest solution in theory, but I think it is impractical (as mentioned by

[Numpy-discussion] NEW GENERATED DLL ERROR FOUND WITHIN f2PY.py

2008-09-19 Thread Blubaugh, David A.
To All, I have now been able to generate a .pyd file from a FORTRAN file that I am trying to interface with python. I was able to execute this with an additional insight into how f2py operates. It seems as though the documentation requires an upgrade, since there appears to be missing info

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Anne Archibald
2008/9/19 David Cournapeau <[EMAIL PROTECTED]>: > I guess my formulation was poor: I never use NaN as missing values > because I never use missing values, which is why I wanted the opinion of > people who use NaN in a different manner (because I don't have a good > idea on how those people would l

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread David Cournapeau
Robert Kern wrote: > On Fri, Sep 19, 2008 at 22:25, David Cournapeau > <[EMAIL PROTECTED]> wrote: > > > How, exactly? ndarray.min() is the where the implementation is. > Ah, I keep forgetting those are implemented in the array object, sorry for that. Now I understand Stefan point. Do I under

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread David Cournapeau
Alan G Isaac wrote: > On 9/19/2008 4:35 AM David Cournapeau apparently wrote: >> I never use NaN as missing value > > What do you use? > > Recently I needed to fill a 2d array with values > from computations that could "go wrong". > I created an array of NaN and then replaced > the elements where t

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Robert Kern
On Fri, Sep 19, 2008 at 22:25, David Cournapeau <[EMAIL PROTECTED]> wrote: > Stéfan van der Walt wrote: >> >> Why shouldn't we have "nanmin"-like behaviour for the C min itself? >> > > Ah, I was not arguing we should not do it in C, but rather we did not > have to do in C. The current behavior for

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread David Cournapeau
Stéfan van der Walt wrote: > > Why shouldn't we have "nanmin"-like behaviour for the C min itself? > Ah, I was not arguing we should not do it in C, but rather we did not have to do in C. The current behavior for nan with functions relying on ordering is broken; if someone prefer fixing it in C

Re: [Numpy-discussion] np.nan and ``is``

2008-09-19 Thread Andrew Dalke
On Sep 19, 2008, at 10:04 PM, Christian Heimes wrote: > Andrew Dalke wrote: >> There are a few things that Python-the-language guarantees are >> singleton >> objects which can be compared correctly with "is". > The empty tuple () and all interned strings are also guaranteed to be > singletons.

Re: [Numpy-discussion] newb question

2008-09-19 Thread Pierre GM
On Friday 19 September 2008 20:47:12 paul taney wrote: > Hi, > > What am I doing wrong here? The reshape doesn"t take. help(reshape) a.reshape(shape, order='C') Returns an array containing the data of a, but with a new shape. Refer to `numpy.reshape` for full documentation. You see tha

Re: [Numpy-discussion] newb question

2008-09-19 Thread Eric Firing
paul taney wrote: > Hi, > > What am I doing wrong here? The reshape doesn"t take. Reshape does not act in place, it returns either a new view or a copy. To reshape in place, you can assign to the shape attribute: In [13]:a = np.arange(10) In [14]:a.shape = (2,5) In [15]:a Out[15]: array([[0,

[Numpy-discussion] newb question

2008-09-19 Thread paul taney
Hi, What am I doing wrong here? The reshape doesn"t take. % cat test1.py import numpy as np a = np.uint8([39, 39, 231, 239, 39, 231, 39, 39, 231, 39, 39, 231, 239, 39, 231, 39, 39, 231, 39, 39, 231, 239, 39, 231, 39, 39, 231, 39, 39, 231, 239, 3

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Pierre GM
On Friday 19 September 2008 17:25:53 Alan G Isaac wrote: > On 9/19/2008 4:54 PM Pierre GM apparently wrote: > > Another way is > > ma.array(np.empty(yourshape,yourdtype), mask=True) > > which should work with earlier versions. > > Seems like ``mask`` would be a natural > keyword for ``ma.empty``?

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Alan G Isaac
On 9/19/2008 4:54 PM Pierre GM apparently wrote: > Another way is > ma.array(np.empty(yourshape,yourdtype), mask=True) > which should work with earlier versions. Seems like ``mask`` would be a natural keyword for ``ma.empty``? Thanks, Alan Isaac ___ N

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Pierre GM
On Friday 19 September 2008 16:35:23 Alan G Isaac wrote: > On 9/19/2008 4:54 AM Pierre GM apparently wrote: > > I know. I was more dreading the time when MaskedArrays would have to be > > ported to C. In a way, that would probably simplify a few issues. OTOH, I > > don't really see it happening any

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Pierre GM
On Friday 19 September 2008 16:28:34 Alan G Isaac wrote: > On 9/19/2008 11:46 AM Pierre GM apparently wrote: > a.mask=True > This is great, but is apparently > new behavior as of NumPy 1.2? I'm not sure, sorry. Another way is ma.array(np.empty(yourshape,yourdtype), mask=True) which should w

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Alan G Isaac
On 9/19/2008 4:54 AM Pierre GM apparently wrote: > I know. I was more dreading the time when MaskedArrays would have to be > ported > to C. In a way, that would probably simplify a few issues. OTOH, I don't > really see it happening any time soon. Is this possibly a GSoC sized project? Alan Isa

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Alan G Isaac
On 9/19/2008 11:46 AM Pierre GM apparently wrote: a.mask=True This is great, but is apparently new behavior as of NumPy 1.2? Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-d

Re: [Numpy-discussion] np.nan and ``is``

2008-09-19 Thread Christian Heimes
Andrew Dalke wrote: > There are a few things that Python-the-language guarantees are singleton > objects which can be compared correctly with "is". Those are: > >True, False, None The empty tuple () and all interned strings are also guaranteed to be singletons. String interning is used to o

Re: [Numpy-discussion] Understanding mgrid

2008-09-19 Thread Robert Kern
On Fri, Sep 19, 2008 at 14:13, Brad Malone <[EMAIL PROTECTED]> wrote: > Thanks for the response Robert. > > So, at least in this case, the results of mgrid (or indices) only provides > information about the spacing of the grid and not on the absolute value of > the point coordinates? No, they give

Re: [Numpy-discussion] Generating random samples without repeats

2008-09-19 Thread Anne Archibald
2008/9/19 Paul Moore <[EMAIL PROTECTED]>: > Anne Archibald gmail.com> writes: > >> This was discussed on one of the mailing lists several months ago. It >> turns out that there is no simple way to efficiently choose without >> replacement in numpy/scipy. > > That reassures me that I'm not missing

Re: [Numpy-discussion] Understanding mgrid

2008-09-19 Thread Brad Malone
Thanks for the response Robert. So, at least in this case, the results of mgrid (or indices) only provides information about the spacing of the grid and not on the absolute value of the point coordinates? In your example, is there a way to see within your x[ix], y[iy], and z[iz] matrices the same

Re: [Numpy-discussion] profiling line by line

2008-09-19 Thread Robert Kern
On Fri, Sep 19, 2008 at 07:00, Robert Cimrman <[EMAIL PROTECTED]> wrote: > Robert Kern wrote: >> Ah, found it. T_LONGLONG is a #define from structmember.h which is >> used to describe the types of attributes. Apparently, this was not >> added until Python 2.5. That particular member didn't actually

Re: [Numpy-discussion] np.nan and ``is``

2008-09-19 Thread Andrew Dalke
On Sep 19, 2008, at 7:52 PM, Christopher Barker wrote: > I don't know the interning rules, but I do know that you should never > count on them, then may not be consistent between implementations, or > even different runs. There are a few things that Python-the-language guarantees are singleton obj

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Alan G Isaac
On 9/19/2008 1:58 PM Robert Kern apparently wrote: > there are no objects inside non-object arrays. There is > nothing with identity inside the arrays to compare against. Got it. Thanks. Alan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Pierre GM
On Friday 19 September 2008 14:01:13 Eric Firing wrote: > Pierre GM wrote: > 2) convenient interfacing with extension code in C or C++. > > The latter is a factor in the present use of nan in matplotlib; using > nan for missing values in an array passed into extension code saves > having to pass a

Re: [Numpy-discussion] Understanding mgrid

2008-09-19 Thread Robert Kern
On Fri, Sep 19, 2008 at 12:59, Brad Malone <[EMAIL PROTECTED]> wrote: > Hi, I was wondering if someone could englighten me on what the geometrical > significance of numpy.mgrid is. I can play around with it and see trends in > the sizes and number of arrays, but why does it give the output that it

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Eric Firing
Pierre GM wrote: >> It seems to me that there are pragmatic reasons >> why people work with NaNs for missing values, >> that perhaps shd not be dismissed so quickly. >> But maybe I am overlooking a simple solution. > > nansomething solutions tend to be considerably faster, that might be one > re

[Numpy-discussion] Understanding mgrid

2008-09-19 Thread Brad Malone
Hi, I was wondering if someone could englighten me on what the geometrical significance of numpy.mgrid is. I can play around with it and see trends in the sizes and number of arrays, but why does it give the output that it does? Looking at the example shown below, why does it return a matrix and it

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Robert Kern
On Fri, Sep 19, 2008 at 11:34, Alan G Isaac <[EMAIL PROTECTED]> wrote: > On 9/19/2008 12:02 PM Peter Saffrey apparently wrote: >> >>> a = array([1,2,nan]) >> >>> nan in a >> False > > Huh. I'm inclined to call this a bug, > since normal Python behavior is that > ``in`` should check for identity::

Re: [Numpy-discussion] np.nan and ``is``

2008-09-19 Thread Christopher Barker
Alan G Isaac wrote: > Might someone explain this to me? > > >>> x = [1.,np.nan] > >>> np.nan in x > True > >>> np.nan in np.array(x) > False > >>> np.nan in np.array(x).tolist() > False > >>> np.nan is float(np.nan) > True not quite -- but I do know th

Re: [Numpy-discussion] np.nan and ``is``

2008-09-19 Thread Alan G Isaac
> On Fri, Sep 19, 2008 at 1:59 PM, Alan G Isaac <[EMAIL PROTECTED]> wrote: >> Might someone explain this to me? >> >> >>> x = [1.,np.nan] >> >>> np.nan in x >> True >> >>> np.nan in np.array(x) >> False >> >>> np.nan in np.array(x).tolist() >> False >> >>>

Re: [Numpy-discussion] np.nan and ``is``

2008-09-19 Thread Lisandro Dalcin
You, know, float are inmutable objects, and then 'float(f)' just returns a new reference to 'f' is 'f' is (exactly) of type 'float' In [1]: f = 1.234 In [2]: f is float(f) Out[2]: True I do not remember right now the implementations of comparisons in core Python, but I believe the 'in' operator i

[Numpy-discussion] np.nan and ``is``

2008-09-19 Thread Alan G Isaac
Might someone explain this to me? >>> x = [1.,np.nan] >>> np.nan in x True >>> np.nan in np.array(x) False >>> np.nan in np.array(x).tolist() False >>> np.nan is float(np.nan) True Thank you, Alan Isaac ___

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Alan G Isaac
On 9/19/2008 12:02 PM Peter Saffrey apparently wrote: > >>> a = array([1,2,nan]) > >>> nan in a > False Huh. I'm inclined to call this a bug, since normal Python behavior is that ``in`` should check for identity:: >>> xl = [1.,np.nan] >>> np.nan in xl True Alan

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Charles R Harris
On Fri, Sep 19, 2008 at 1:11 AM, David Cournapeau < [EMAIL PROTECTED]> wrote: > Anne Archibald wrote: > > > > Well, for example, you might ask that all the non-nan elements be in > > order, even if you don't specify where the nan goes. > > > Ah, there are two problems, then: >- sort >- how

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Alan G Isaac
On 9/19/2008 11:46 AM Pierre GM apparently wrote: > No, but you may do the opposite: just start with an array completely masked, > and unmasked it as you need: Very useful example. I did not understand this possibility. Alan ___ Numpy-discussion maili

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Alan G Isaac
On 9/19/2008 11:46 AM Pierre GM apparently wrote: > You can't compare NaNs to anything. How do you know this np.miss is a masked > value, when np.sqrt(-1.) is NaN ? I thought you could use ``is``. E.g., >>> np.nan == np.nan False >>> np.nan is np.nan True Alan

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Pierre GM
On Friday 19 September 2008 12:02:08 Peter Saffrey wrote: > Alan G Isaac american.edu> writes: > > Recently I needed to fill a 2d array with values > > from computations that could "go wrong". > Should I take the earlier advice and switch to masked arrays? > > Peter Yes. As you've noticed, you c

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Peter Saffrey
Alan G Isaac american.edu> writes: > Recently I needed to fill a 2d array with values > from computations that could "go wrong". > I created an array of NaN and then replaced > the elements where the computation produced > a useful value. I then applied ``nanmax``, > to get the maximum of the us

[Numpy-discussion] New patch for new mgrid / ogrid functionality

2008-09-19 Thread David M. Kaplan
Hi all, Attached is a newer version of my patch that adds new mgrid / ogrid functionality for working with arrays in addition to slices. In fact, I have attached two versions of the patch: index_tricks.patch, that is just the last version of the patch I sent, and index_tricks.new.patch, that has

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Pierre GM
On Friday 19 September 2008 11:36:17 Alan G Isaac wrote: > On 9/19/2008 11:09 AM Stefan Van der Walt apparently wrote: > > Masked arrays. Using NaN's for missing values is dangerous. You may > > do some operation, which generates invalid results, and then you have > > a mixed bag of missing and i

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Alan G Isaac
On 9/19/2008 11:09 AM Stefan Van der Walt apparently wrote: > Masked arrays. Using NaN's for missing values is dangerous. You may > do some operation, which generates invalid results, and then you have > a mixed bag of missing and invalid values. That rather evades my full question, I think?

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Stefan Van der Walt
On 19 Sep 2008, at 16:07 , Alan G Isaac wrote: > On 9/19/2008 4:35 AM David Cournapeau apparently wrote: >> I never use NaN as missing value > > What do you use? Masked arrays. Using NaN's for missing values is dangerous. You may do some operation, which generates invalid results, and then you

Re: [Numpy-discussion] Generating random samples without repeats

2008-09-19 Thread Paul Moore
Rick White stsci.edu> writes: > It seems like numpy.random.permutation is pretty suboptimal in its > speed. Here's a Python 1-liner that does the same thing (I think) > but is a lot faster: > > a = 1+numpy.random.rand(M).argsort()[0:N-1] > > This still has the the problem that it generates

Re: [Numpy-discussion] which one is best?

2008-09-19 Thread David M. Kaplan
Hi Arnar, Your two commands below aren't doing the same thing - one is doing a[i]*b[i] and the other is doing a[i]*b[j] for all i and j. As the second is harder, it takes longer. Cheers, David On Fri, 2008-09-19 at 09:08 -0500, [EMAIL PROTECTED] wrote: > I think > [x*y for x in a for y in b] >

Re: [Numpy-discussion] which one is best?

2008-09-19 Thread Alan G Isaac
Given your restrictions I'd probably pick [ x * y for x,y in izip(a,b) ] (note the change to ``izip``). But maybe you do not really need lists at all... You can also do c = (array(a)*array(b)).tolist() which is nice and explicit but builds 3 unnecessary arrays. Alan Isaac ___

Re: [Numpy-discussion] which one is best?

2008-09-19 Thread Arnar Flatberg
On Fri, Sep 19, 2008 at 4:09 PM, lorenzo <[EMAIL PROTECTED]> wrote: > > > On Fri, Sep 19, 2008 at 2:50 PM, Arnar Flatberg <[EMAIL PROTECTED]>wrote: > >> >> >> I think >> [x*y for x in a for y in b] >> feels pythonic, however it has a surprisingly lousy performance. >> >> > This returns a len(x)*le

Re: [Numpy-discussion] which one is best?

2008-09-19 Thread lorenzo
On Fri, Sep 19, 2008 at 2:50 PM, Arnar Flatberg <[EMAIL PROTECTED]>wrote: > > > I think > [x*y for x in a for y in b] > feels pythonic, however it has a surprisingly lousy performance. > > This returns a len(x)*len(y) long list, which is not what you want. This two methods seem equivalent: In [1

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Alan G Isaac
On 9/19/2008 4:35 AM David Cournapeau apparently wrote: > I never use NaN as missing value What do you use? Recently I needed to fill a 2d array with values from computations that could "go wrong". I created an array of NaN and then replaced the elements where the computation produced a useful va

Re: [Numpy-discussion] which one is best?

2008-09-19 Thread Arnar Flatberg
On Fri, Sep 19, 2008 at 3:09 PM, Stéfan van der Walt <[EMAIL PROTECTED]>wrote: > 2008/9/19 mark <[EMAIL PROTECTED]>: > > I need to multiply items in a list and need a list back. Which one of > > the four options is best (I thought in Python there was only one way > > to do something???) > > With t

[Numpy-discussion] Numpy 1.1.0 Silent Install

2008-09-19 Thread Didier CROUTZ
Hello, I have to deploy Numpy 1.1.0 on a set of XP PRO SP3 computers (with Pytho 2.5). I'd like to know if I can silently install this product by the way of a DOS command. Thanks for your response, Regards, D. Croutz. ___ Numpy-discussion mailing lis

Re: [Numpy-discussion] which one is best?

2008-09-19 Thread Stéfan van der Walt
2008/9/19 mark <[EMAIL PROTECTED]>: > I need to multiply items in a list and need a list back. Which one of > the four options is best (I thought in Python there was only one way > to do something???) With the emphasis on "preferably" and "obvious" :) "There should be one-- and preferably only on

Re: [Numpy-discussion] Generating random samples without repeats

2008-09-19 Thread Rick White
Paul Moore yahoo.co.uk> writes: > Robert Kern gmail.com> writes: >> On Thu, Sep 18, 2008 at 16:55, Paul Moore >> yahoo.co.uk> wrote: >>> I want to generate a series of random samples, to do simulations >>> based >>> on them. Essentially, I want to be able to produce a SAMPLESIZE * N >>> mat

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Stéfan van der Walt
2008/9/19 David Cournapeau <[EMAIL PROTECTED]>: > But cannot this be fixed at the python level of the max function ? I Why shouldn't we have "nanmin"-like behaviour for the C min itself? I'd rather have a specialised function to deal with the rare kinds of datasets where NaNs are guaranteed never

[Numpy-discussion] which one is best?

2008-09-19 Thread mark
I need to multiply items in a list and need a list back. Which one of the four options is best (I thought in Python there was only one way to do something???) >>> a = [1,2,3,4] >>> b = [10,20,30,40] >>> c = [] # method 1 >>> for i in range(len(a)): c.append(a[i]*b[i]) >>> c [10, 40, 90,

Re: [Numpy-discussion] profiling line by line

2008-09-19 Thread Robert Cimrman
Robert Kern wrote: > Ah, found it. T_LONGLONG is a #define from structmember.h which is > used to describe the types of attributes. Apparently, this was not > added until Python 2.5. That particular member didn't actually need to > be long long, so I've fixed that. Great, I will try it after it ap

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread David Cournapeau
Peter Saffrey wrote: > > I've posted my test code below, which gives me the results: > > $ ./arrayspeed3.py > list build time: 0.01 > list median time: 0.01 > array nanmedian time: 0.36 > > I must have done something wrong to hobble nanmedian in this way... I'm quite > new to numpy, so feel free to

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread David Cournapeau
Peter Saffrey wrote: > Pierre GM gmail.com> writes: > >> I think there were some changes on the C side of numpy between 1.0 and 1.1, >> you may have to recompile scipy and matplotlib from sources. What versions >> are you using for those 2 packages ? >> > > $ dpkg -l | grep scipy > ii python-sc

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Peter Saffrey
Pierre GM gmail.com> writes: > I think there were some changes on the C side of numpy between 1.0 and 1.1, > you may have to recompile scipy and matplotlib from sources. What versions > are you using for those 2 packages ? > $ dpkg -l | grep scipy ii python-scipy

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Peter Saffrey
David Cournapeau ar.media.kyoto-u.ac.jp> writes: > It may be that nanmedian is slow. But I would sincerly be surprised if > it were slower than python list, except for some pathological cases, or > maybe a bug in nanmedian. What do your data look like ? (size, number of > nan, etc...) > I've po

Re: [Numpy-discussion] Generating random samples without repeats

2008-09-19 Thread Paul Moore
Anne Archibald gmail.com> writes: > This was discussed on one of the mailing lists several months ago. It > turns out that there is no simple way to efficiently choose without > replacement in numpy/scipy. That reassures me that I'm not missing something obvious! I'm pretty new with numpy (I've

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread David Cournapeau
Peter Saffrey wrote: > > I rejoiced when I saw this answer, because it looks like a function I can just > drop in and it works. Unfortunately, nanmedian seems to be quite a bit slower > than just using lists (ignoring nan values from my experiments) and a > home-brew > implementation of median. I

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Pierre GM
On Friday 19 September 2008 05:51:55 Peter Saffrey wrote: > I would like to try the masked array approach, but the Ubuntu packages for > scipy and matplotlib depend on numpy. Does anybody know whether I can > naively do "sudo python setup.py install" on a more modern numpy without > disturbing sci

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Peter Saffrey
David Cournapeau ar.media.kyoto-u.ac.jp> writes: > You can use nanmean (from scipy.stats): > I rejoiced when I saw this answer, because it looks like a function I can just drop in and it works. Unfortunately, nanmedian seems to be quite a bit slower than just using lists (ignoring nan values fr

Re: [Numpy-discussion] Generating random samples without repeats

2008-09-19 Thread Anne Archibald
2008/9/19 Paul Moore <[EMAIL PROTECTED]>: > Robert Kern gmail.com> writes: >> On Thu, Sep 18, 2008 at 16:55, Paul Moore yahoo.co.uk> wrote: >> > I want to generate a series of random samples, to do simulations based >> > on them. Essentially, I want to be able to produce a SAMPLESIZE * N >> > mat

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread David Cournapeau
Stéfan van der Walt wrote: > > So am I. In all my use cases, NaNs indicate trouble. Yes, so I would like to have the opinion of people with other usage than ours. > > Because we have x.max() silently ignoring NaNs, which causes a lot of > head-scratching, swearing and failed experiments. But ca

Re: [Numpy-discussion] profiling line by line

2008-09-19 Thread Ondrej Certik
On Fri, Sep 19, 2008 at 10:37 AM, Robert Cimrman <[EMAIL PROTECTED]> wrote: > Ondrej Certik wrote: >> On Thu, Sep 18, 2008 at 4:12 PM, Ryan May <[EMAIL PROTECTED]> wrote: >>> Ondrej Certik wrote: On Thu, Sep 18, 2008 at 1:01 PM, Robert Cimrman <[EMAIL PROTECTED]> wrote: >> It requires Cyth

Re: [Numpy-discussion] profiling line by line

2008-09-19 Thread Robert Kern
On Wed, Sep 17, 2008 at 18:29, Robert Kern <[EMAIL PROTECTED]> wrote: > On Wed, Sep 17, 2008 at 18:09, Ondrej Certik <[EMAIL PROTECTED]> wrote: >> This is what I am getting: >> >> $ ./kernprof.py -l pystone.py >> Wrote profile results to pystone.py.lprof >> $ ./view_line_prof.py pystone.py.lprof >

Re: [Numpy-discussion] profiling line by line

2008-09-19 Thread Robert Kern
On Fri, Sep 19, 2008 at 03:33, Robert Cimrman <[EMAIL PROTECTED]> wrote: > I have Python 2.4.4 > > in "pyconfig.h" > > #define HAVE_LONG_LONG 1 > > in "pyport.h": > > #ifdef HAVE_LONG_LONG > #ifndef PY_LONG_LONG > #define PY_LONG_LONG long long > #endif > #endif /* HAVE_LONG_LONG */ > > so it seem

Re: [Numpy-discussion] Generating random samples without repeats

2008-09-19 Thread Pierre GM
On Friday 19 September 2008 05:08:20 Paul Moore wrote: > Robert Kern gmail.com> writes: > > On Thu, Sep 18, 2008 at 16:55, Paul Moore yahoo.co.uk> wrote: > > > I want to generate a series of random samples, to do simulations based > > > on them. Essentially, I want to be able to produce a SAMPLE

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Stéfan van der Walt
2008/9/19 David Cournapeau <[EMAIL PROTECTED]>: > Stéfan van der Walt wrote: >> >> I agree completely. > > Me too, but I am extremely biased toward nan is always bogus by my own > usage of numpy/scipy (I never use NaN as missing value, and nan is > always caused by divide by 0 and co). So am I. I

Re: [Numpy-discussion] Generating random samples without repeats

2008-09-19 Thread Paul Moore
Robert Kern gmail.com> writes: > On Thu, Sep 18, 2008 at 16:55, Paul Moore yahoo.co.uk> wrote: > > I want to generate a series of random samples, to do simulations based > > on them. Essentially, I want to be able to produce a SAMPLESIZE * N > > matrix, where each row of N values consists of eith

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Pierre GM
On Friday 19 September 2008 04:31:38 David Cournapeau wrote: > Pierre GM wrote: > > That said, numpy.nanmin, numpy.nansum... don't come with the heavy > > machinery of numpy.ma, and are therefore faster. > > I'm really going to have to learn C. > > FWIW, nanmean/nanmean/etc... are written in python

Re: [Numpy-discussion] A bug in loadtxt and how to convert a string array (hex data) to decimal?

2008-09-19 Thread Stéfan van der Walt
2008/9/18 Ryan May <[EMAIL PROTECTED]>: > It's because of how numpy handles strings arrays (which I admit I don't > understand very well.) Basically, it's converting the numbers properly, > but truncating them to 3 characters. Try this, which just forces it to > expand to strings 4 characters wid

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread David Cournapeau
Stéfan van der Walt wrote: > > I agree completely. Me too, but I am extremely biased toward nan is always bogus by my own usage of numpy/scipy (I never use NaN as missing value, and nan is always caused by divide by 0 and co). I like that sort raise an exception by default with NaN: it breaks the

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread David Cournapeau
Pierre GM wrote: > That said, numpy.nanmin, numpy.nansum... don't come with the heavy machinery > of numpy.ma, and are therefore faster. > I'm really going to have to learn C. > FWIW, nanmean/nanmean/etc... are written in python, cheers, David ___

Re: [Numpy-discussion] Suggestion for recarray.view

2008-09-19 Thread Pierre GM
On Friday 19 September 2008 04:13:39 Stéfan van der Walt wrote: > +1 and another +1 to your karma for requesting peer review. Let me > know if you need me to whip up a couple of tests for verifying the > different usage cases. That'd be lovely. I'm a bit swamped with tricky issues in mrecords an

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Pierre GM
On Friday 19 September 2008 04:10:24 Anne Archibald wrote: > (is there a convenience > function that makes a masked array with a mask everywhere the data is > nan?). numpy.ma.fix_invalid, that masks your Nans and Infs and sets the underlying data to some filling value. That way, you don't carry

Re: [Numpy-discussion] profiling line by line

2008-09-19 Thread Robert Cimrman
Ondrej Certik wrote: > On Thu, Sep 18, 2008 at 4:12 PM, Ryan May <[EMAIL PROTECTED]> wrote: >> Ondrej Certik wrote: >>> On Thu, Sep 18, 2008 at 1:01 PM, Robert Cimrman <[EMAIL PROTECTED]> wrote: > It requires Cython and a C compiler to build. I'm still debating > myself about the desired wo

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Stéfan van der Walt
2008/9/19 Anne Archibald <[EMAIL PROTECTED]>: > I think the numpy attitude to nans should be that they are unexpected > bogus values that signify that something went wrong with the > calculation somewhere. They can be left in place for most operations, > but any operation that depends on the value

Re: [Numpy-discussion] profiling line by line

2008-09-19 Thread Robert Cimrman
Robert Kern wrote: > On Thu, Sep 18, 2008 at 06:01, Robert Cimrman <[EMAIL PROTECTED]> wrote: >> Hi Robert, >> >> Robert Kern wrote: >>> On Mon, Sep 15, 2008 at 11:13, Arnar Flatberg <[EMAIL PROTECTED]> wrote: That would make me an extremely happy user, I've been looking for this for year

Re: [Numpy-discussion] Suggestion for recarray.view

2008-09-19 Thread Stéfan van der Walt
2008/9/19 Travis E. Oliphant <[EMAIL PROTECTED]>: >> #--- >> def view(self, dtype=None, type=None): >> if dtype is None: >> return ndarray.view(self, type) >> elif type is None: >> try: >> if issubclass(dtype, ndarray): >>

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Anne Archibald
2008/9/19 Pierre GM <[EMAIL PROTECTED]>: > On Friday 19 September 2008 03:11:05 David Cournapeau wrote: > >> Hm, I am always puzzled when I think about nan handling :) It always >> seem there is not good answer. > > Which is why we have masked arrays, of course ;) I think the numpy attitude to nan

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread Pierre GM
On Friday 19 September 2008 03:11:05 David Cournapeau wrote: > Hm, I am always puzzled when I think about nan handling :) It always > seem there is not good answer. Which is why we have masked arrays, of course ;) ___ Numpy-discussion mailing list Numpy

Re: [Numpy-discussion] Medians that ignore values

2008-09-19 Thread David Cournapeau
Anne Archibald wrote: > > Well, for example, you might ask that all the non-nan elements be in > order, even if you don't specify where the nan goes. Ah, there are two problems, then: - sort - how median use sort. For sort, I don't know how sort speed would be influenced by treating nan.