Re: [Numpy-discussion] invalid correlation coefficient from np.ma.corrcoef

2013-09-26 Thread josef . pktd
On Thu, Sep 26, 2013 at 6:42 PM, Nathaniel Smith wrote: > On 26 Sep 2013 21:59, "Faraz Mirzaei" wrote: >> >> Thanks Josef and Nathaniel for your responses. >> >> In the application that I have, I don't use the correlation coefficient >> matrix as a whole (so I don't care if it is PSD or not). I s

Re: [Numpy-discussion] invalid correlation coefficient from np.ma.corrcoef

2013-09-26 Thread Nathaniel Smith
On 26 Sep 2013 21:59, "Faraz Mirzaei" wrote: > > Thanks Josef and Nathaniel for your responses. > > In the application that I have, I don't use the correlation coefficient matrix as a whole (so I don't care if it is PSD or not). I simply read the off-diagonal elements for pair-wise correlation coe

Re: [Numpy-discussion] invalid correlation coefficient from np.ma.corrcoef

2013-09-26 Thread Faraz Mirzaei
Thanks Josef and Nathaniel for your responses. In the application that I have, I don't use the correlation coefficient matrix as a whole (so I don't care if it is PSD or not). I simply read the off-diagonal elements for pair-wise correlation coefficients. I use the pairwise correlation coefficient

Re: [Numpy-discussion] At.: use less RAM memory and increase execution speed

2013-09-26 Thread Michael Sarahan
xrange should be more memory efficient than range: http://stackoverflow.com/questions/135041/should-you-always-favor-xrange-over-range Replacing arrays with lists is probably a bad idea for a lot of reasons. You'll lose nice vectorization of simple operations, and all of numpy's other benefits.

Re: [Numpy-discussion] At.: use less RAM memory and increase execution speed

2013-09-26 Thread Robert Kern
On Thu, Sep 26, 2013 at 7:19 PM, Josè Luis Mietta < joseluismie...@yahoo.com.ar> wrote: > > Hi experts! > > I wanna use less RAM memory in my Monte Carlo simulations. In my algorithm I use numpy arrays and xrange() function. > I hear that I can reduce RAM used in my lagorithm if I do the next: > >

[Numpy-discussion] At.: use less RAM memory and increase execution speed

2013-09-26 Thread Josè Luis Mietta
Hi experts! I wanna use less RAM memory in my Monte Carlo simulations. In my algorithm I use numpy arrays and xrange() function. I hear that I can reduce RAM used in my lagorithm if I do the next: 1) replace xrange() for range(). 2) replace numpya arrays for python lists 3) use reset() funct

Re: [Numpy-discussion] invalid correlation coefficient from np.ma.corrcoef

2013-09-26 Thread Nathaniel Smith
On 26 Sep 2013 17:32, wrote: > > On Thu, Sep 26, 2013 at 7:35 AM, Nathaniel Smith wrote: > > By textbook I mean, users expect corrcoef to use this formula, which > > is printed in every textbook: > > https://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient#For_a_sample > > The

Re: [Numpy-discussion] invalid correlation coefficient from np.ma.corrcoef

2013-09-26 Thread josef . pktd
On Thu, Sep 26, 2013 at 7:35 AM, Nathaniel Smith wrote: > On Thu, Sep 26, 2013 at 11:51 AM, wrote: >> On Thu, Sep 26, 2013 at 4:21 AM, Nathaniel Smith wrote: >>> If you want a proper self-consistent correlation/covariance matrix, then >>> pairwise deletion just makes no sense period, I don't se

Re: [Numpy-discussion] invalid correlation coefficient from np.ma.corrcoef

2013-09-26 Thread Nathaniel Smith
On Thu, Sep 26, 2013 at 11:51 AM, wrote: > On Thu, Sep 26, 2013 at 4:21 AM, Nathaniel Smith wrote: >> If you want a proper self-consistent correlation/covariance matrix, then >> pairwise deletion just makes no sense period, I don't see how postprocessing >> can help. > > clipping to [-1, 1] and

Re: [Numpy-discussion] invalid correlation coefficient from np.ma.corrcoef

2013-09-26 Thread josef . pktd
On Thu, Sep 26, 2013 at 6:51 AM, wrote: > On Thu, Sep 26, 2013 at 4:21 AM, Nathaniel Smith wrote: >> If you want a proper self-consistent correlation/covariance matrix, then >> pairwise deletion just makes no sense period, I don't see how postprocessing >> can help. > > clipping to [-1, 1] and f

Re: [Numpy-discussion] invalid correlation coefficient from np.ma.corrcoef

2013-09-26 Thread josef . pktd
On Thu, Sep 26, 2013 at 4:21 AM, Nathaniel Smith wrote: > If you want a proper self-consistent correlation/covariance matrix, then > pairwise deletion just makes no sense period, I don't see how postprocessing > can help. clipping to [-1, 1] and finding the nearest positive semi-definite matrix.

Re: [Numpy-discussion] invalid correlation coefficient from np.ma.corrcoef

2013-09-26 Thread Nathaniel Smith
If you want a proper self-consistent correlation/covariance matrix, then pairwise deletion just makes no sense period, I don't see how postprocessing can help. If you want a matrix of correlations, then pairwise deletion makes sense. It's an interesting point that arguably the current ma.corrcoef

Re: [Numpy-discussion] Valid algorithm for generating a 3D Wiener Process?

2013-09-26 Thread Daπid
On 25 September 2013 19:41, David Goldsmith wrote: > "the 'angles' that describe the position undergo a random walk [actually, > it would seem that they don't, since they too fail the > varying-as-white-noise test], so the particle tends to move in the same > direction over short intervals"--is t