Re: [Numpy-discussion] Numpy array performance issue

2010-02-25 Thread Robert Kern
On Thu, Feb 25, 2010 at 10:20, Bruno Santos wrote: > This is the same example we discuss yesterday. I think I can help you this time, but when we ask for complete code, we mean complete, self-contained code that we can run immediately, not a fragment of code that needs variables to be initialized

Re: [Numpy-discussion] Numpy array performance issue

2010-02-25 Thread Bruno Santos
This is the same example we discuss yesterday. The working code is this one: lsPhasedValues = [aLoci[i] for i in xrange(length) if i%21==0 and aLoci[i]>0] I was able to get the same result after a while: aAux =aLoci[index_nSize] lsPhasedValues = numpy.unique1d(aAux[numpy.where(aAux>0)[0]]) I could

Re: [Numpy-discussion] Numpy array performance issue

2010-02-25 Thread Robert Kern
On Thu, Feb 25, 2010 at 07:51, Bruno Santos wrote: > I just realized that the line lsPhasedValues = > numpy.unique1d(aLoci[numpy.where(aLoci[index_nSize]>0)]) does not work > properly. > How can I get the unique values of an array based on their indexes? I don't know what that sentence means. Ple

Re: [Numpy-discussion] Numpy array performance issue

2010-02-25 Thread Bruno Santos
I just realized that the line lsPhasedValues = numpy.unique1d(aLoci[numpy.where(aLoci[index_nSize]>0)]) does not work properly. How can I get the unique values of an array based on their indexes? 2010/2/25 Bruno Santos > After implementation all the possibilities we discuss yesterday mi fastest

Re: [Numpy-discussion] Numpy array performance issue

2010-02-25 Thread Bruno Santos
After implementation all the possibilities we discuss yesterday mi fastest version is this one: index_nSize=numpy.arange(0,length,nSize) lsPhasedValues = numpy.unique1d(aLoci[numpy.where(aLoci[index_nSize]>0)]) ... bigaLoci = (aLoci>=r) k = (aLoci>=r).sum() This is taking around 0.12s for my tes

Re: [Numpy-discussion] Numpy array performance issue

2010-02-24 Thread Robert Kern
On Wed, Feb 24, 2010 at 12:38, Bruno Santos wrote: > This is probably me just being stupid. But what is the reason for this peace > of code not to be working: > index_nSize=numpy.arange(0,length,nSize) > lsPhasedValues = set([aLoci[i] for i in xrange(length) if (i%nSize==0 and > aLoci[i]>0)]) > ls

Re: [Numpy-discussion] Numpy array performance issue

2010-02-24 Thread Bruno Santos
This is probably me just being stupid. But what is the reason for this peace of code not to be working: index_nSize=numpy.arange(0,length,nSize) lsPhasedValues = set([aLoci[i] for i in xrange(length) if (i%nSize==0 and aLoci[i]>0)]) lsPhasedValues1 = numpy.where(aLoci[index_nSize]>0) print aLoci[in

Re: [Numpy-discussion] Numpy array performance issue

2010-02-24 Thread Bruno Santos
2010/2/24 Chris Colbert > In [4]: %timeit a = np.random.randint(0, 20, 100) > 10 loops, best of 3: 4.32 us per loop > > In [5]: %timeit (a>=10).sum() > 10 loops, best of 3: 7.32 us per loop > > In [8]: %timeit np.where(a>=10) > 10 loops, best of 3: 5.36 us per loop > > > am i missing

Re: [Numpy-discussion] Numpy array performance issue

2010-02-24 Thread Robert Kern
On Wed, Feb 24, 2010 at 11:50, Bruno Santos wrote: > In both versions your lsPhasedValues contains the number of positions in the > array that match a certain criteria. What I need in that step is the unique > values and not their positions. Oops! lsPhasedValues = np.unique1d(aLoci[j_nSize_mask

Re: [Numpy-discussion] Numpy array performance issue

2010-02-24 Thread Chris Colbert
In [4]: %timeit a = np.random.randint(0, 20, 100) 10 loops, best of 3: 4.32 us per loop In [5]: %timeit (a>=10).sum() 10 loops, best of 3: 7.32 us per loop In [8]: %timeit np.where(a>=10) 10 loops, best of 3: 5.36 us per loop am i missing something? On Wed, Feb 24, 2010 at 12:50 PM

Re: [Numpy-discussion] Numpy array performance issue

2010-02-24 Thread Bruno Santos
In both versions your lsPhasedValues contains the number of positions in the array that match a certain criteria. What I need in that step is the unique values and not their positions. 2010/2/24 Robert Kern > On Wed, Feb 24, 2010 at 11:19, Bruno Santos wrote: > > It seems that the python 2.6.4

Re: [Numpy-discussion] Numpy array performance issue

2010-02-24 Thread Robert Kern
On Wed, Feb 24, 2010 at 11:19, Bruno Santos wrote: > It seems that the python 2.6.4 has a more efficient implementation of the > lists. It runs faster on this version and slower on 2.5.4 on the same > machine with debian. A lot faster in fact. > I was trying to change my headche for the last coupl

Re: [Numpy-discussion] Numpy array performance issue

2010-02-24 Thread Bruno Santos
It seems that the python 2.6.4 has a more efficient implementation of the lists. It runs faster on this version and slower on 2.5.4 on the same machine with debian. A lot faster in fact. I was trying to change my headche for the last couple of weeks. But you migth give me a lot more optimizations

Re: [Numpy-discussion] Numpy array performance issue

2010-02-24 Thread Robert Kern
On Wed, Feb 24, 2010 at 10:40, Bruno Santos wrote: > Funny. Which version of python are you using? Python 2.5.4 on OS X. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an

Re: [Numpy-discussion] Numpy array performance issue

2010-02-24 Thread Bruno Santos
Funny. Which version of python are you using? My python is still better for small lists. But you are rigth it gets better with size, here how the same code performs on mine computer: In [1]: N = 100 In [2]: import numpy as np In [3]: A = np.random.randint(0, 21, N) ...: In [4]: L = A.tolist

Re: [Numpy-discussion] Numpy array performance issue

2010-02-24 Thread Robert Kern
On Wed, Feb 24, 2010 at 10:21, Bruno Santos wrote: >> The idiomatic way of doing this for numpy arrays would be: >> >> def test2(arrx): >>    return (arrx >= 10).sum() >> >  Even this versions takes more time to run than my original python version > with arrays. Works fine for me, and gets bette

Re: [Numpy-discussion] Numpy array performance issue

2010-02-24 Thread Bruno Santos
> > > > The idiomatic way of doing this for numpy arrays would be: > > def test2(arrx): >return (arrx >= 10).sum() > > Even this versions takes more time to run than my original python version > with arrays. >>> def test3(listx): ... return (listx>=10).sum() >>> t = timeit.Timer("test3(l

Re: [Numpy-discussion] Numpy array performance issue

2010-02-24 Thread Robert Kern
On Wed, Feb 24, 2010 at 09:55, Bruno Santos wrote: > Hello everyone, > I am using numpy arrays whenever I demand performance from my > algorithms. Nevertheless, I am having a performance issue at the moment > mainly because I am iterating several times over numpy arrays. Fot that > reason I decide

[Numpy-discussion] Numpy array performance issue

2010-02-24 Thread Bruno Santos
Hello everyone, I am using numpy arrays whenever I demand performance from my algorithms. Nevertheless, I am having a performance issue at the moment mainly because I am iterating several times over numpy arrays. Fot that reason I decided to use timeit to see the performance of different versions