[Numpy-discussion] Functions vs Methods

[Jaidev Deshpande]

Hi

It is said that function calls are expensive. Does that mean one must
use available methods instead?

Eg.  x is a NumPy array, and I need its transpose

Should I use

>>> x.T

or
>>> numpy.transpose(T)   ?

Does it matter which one I'm using ? If not, under what conditions
does it become important to think about this distinction?

Thanks
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] GSOC

[Jaidev Deshpande]

Hi!

> Along with test coverage, have any of you considered any systematic
> monitoring of NumPy performance?

I'm mildly obsessed with performance and benchmarking of NumPy. I used
to use a lot of MATLAB until a year back and I tend to compare Python
performance with it all the time. I generally don't feel happy until
I'm convinced that I've extracted the last bit of speed out of my
Python code.

I think the generalization of this idea is more or less equivalent to
performance benchmarking. Of course, I know there's a lot more than
'MATLAB vs Python' to it. I'd be more than happy to be involved. GSoC
or otherwise.

Where do I start?

Thanks
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] GSOC

[Jaidev Deshpande]

Hi Chris

> Documentation is specificsly excluded from GSoC (at least it was a
> couple years ago when I last was involved)

Documentation wasn't excluded last year from GSoC, there were quite a
few projects that required a lot of documentation.
But yes, there was no "documentation only" project.

Anyhow, it seems reasonable that testing alone can't be a project.
What about benchmarking and the related statistics? Does that qualify
as a worthwhile project (again, GSoC or otherwise)?

Thanks
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Working with MATLAB

[Jaidev Deshpande]

Dear List,

I frequently work with MATLAB and it is necessary for me many a times
to adapt MATLAB codes for NumPy arrays.

While for most practical purposes it works fine, I think there might
be a lot of 'under the hood' things that I might be missing when I
make the translations from MATLAB to Python.

Are there any 'best practices' for working on this transition?

Thanks
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Working with MATLAB

[Jaidev Deshpande]

Please ignore my question. I found what I needed on the scipy website.

I asked the question in haste.

I'm sorry.

Thanks
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Is there a pure numpy recipe for this?

[Jaidev Deshpande]

On Thu, Mar 27, 2014 at 1:18 AM, Slaunger  wrote:

> I am working on solving a recent recreational mathematical problem on
> Project Euler   . I have a solution, which works
> fine for small N up to 10^5 but it takes too long to compute for the actual
> problem, where N is of the order 2*10^7. The problem is nested loops, and I
> am hoping to avoid one level in the loop by using clever numpy magic (as I
> have done often before). However, there is one step, which I cannot figure
> out how to do using numpy operations alone, and I am hoping for some help
>
> The subproblem is that I have in principle k = 1, ..., N sets of boolean
> arrays
> f_k and g_k each of length N.
>
> For each k I need to find the number of elements i where both f_k[i] and
> g_k[i] are True and sum that up over all N values of k.
>
> A problem of the order 4*10^14 if I just do it brute force. This takes way
> too long (there is a one minute rule).
>
> However, after a lot of thinking and by using some properties of the f_k
> and
> g_k I have managed to construct using only pure numpy function and only a
> single loop over k, arrays
>
> f_k_changes_at
> g_k_changes_at
>
> which contain the indices i at which the functions change it boolean value
> from True to False or False to True.
>
> It so happens that the number of changes is only a small fraction of N, the
> fraction decreases with larger N, so the size of these changes_at arrays
> contains perhaps only 1000 elements instead of 1000 for each k, a
> significant reduction of complexity.
>
> Now, my problem is to figure out for how many values of i both f_k and g_k
> are True given the changes_at arrays.
>
> As this may be a little hard to understand here is a specific example of
> how
> these arrays can look like for k = 2 and N = 150
>
> f_2_changes_at = [  2   3  39  41  58  59  65  66  93 102 145]
> g_2_changes_at = [  2  94 101 146 149]
>
> with the boundary condition that f_2[0] = g_2[0] = False
>
> Which expands to
> if_2g_2   f_2 and g_2
> 0 F   F  F
> 1 F   F  F <-
> 2 T  T  T <-
> 3 F  T   F
> 4 F  T   F
> ...
> 38   F  T   F <-
> 39   T  T   T
> 40   T  T   T <-
> 41   F  T   F
> 42   F  T   F
> ...
> 57   F  T   F <-
> 58   T  T   T <-
> 59   F  T   F
> 60   F  T   F
> ...
> 64   F  T   F <-
> 65   T  T   T <-
> 66   F  T   F
> ...
> 92   F  T   F <-
> 93   T  T   T <-
> 94   T  F   F
> ...
> 100  T F   F <-
> 101  T T   T <-
> 102  F T   F
> ...
> 144  F T   F <-
> 145  T T   T <-
> 146  T F   F
> 147  T F   F
> 148  T F   F <-
> 149  T T   T <-
>
> With the sum of elements fulfilling the condition being (see arrows)
>
> (2 - 1) + (40 - 38) + (58 - 57) + (65 - 64) + (93 - 92) + (101 - 100) +
> (145
> - 144) + (149 - 148) =
> 1 + 2 + 1 + 1 + 1 + 1 + 1 + 1 = 9
>
> So, is there a numpy recipe for doing the equivalent process without
> expanding it into the full arrays?
>
> I have tried looping over each element in the changes_at arrays and build
> up
> the sums, but that is too inefficient as I then have an inner for loop
> containing conditional branching code
>
> Thanks in advance, Slaunger
>
>
>
> --
> View this message in context:
> http://numpy-discussion.10968.n7.nabble.com/Is-there-a-pure-numpy-recipe-for-this-tp37077.html
> Sent from the Numpy-discussion mailing list archive at Nabble.com.
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>

Can you provide a link to the problem itself?

-- 
JD
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] What does -0.0 means?

[Jaidev Deshpande]

Hi Arnaldo,
On 20 Jan 2016 21:56, "Arnaldo Russo"  wrote:
>
> Hi all,
>
> After fitting a curve, the output returns the initial value as -0.0. Why
not a simple 0.0?
> What is this difference?
> Sorry for the stupid question, but why exist a negative zero?

Haha, that used to stump me too, before I found out that there's such a
thing as a signed zero. The IEEE 754 standard accommodates it.

You can read more at https://en.m.wikipedia.org/wiki/Signed_zero

>
> Cheers,
> Arnaldo.
> ---
> Arnaldo D'Amaral Pereira Granja Russo
> Instituto Ambiental Boto Flipper
> institutobotoflipper .org
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Memory error with numpy.loadtxt()

[Jaidev Deshpande]

Hi

Is it possible to load a text file 664 MB large with integer values and
about 98% sparse? numpy.loadtxt() shows a memory error.

If it's not possible, what alternatives could I have?

The usable RAM on my machine running Windows 7 is 3.24 GB.

Thanks.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Largest possible numpy array

[Jaidev Deshpande]

Hi

How can I know the size of the largest possible 2-D array in numpy, given a
specific 'dtype' and my system memory?

How can one play around with this? Would it help to note that the array
might be say *m* megabytes on disk, say, *n* % sparse?

Also, is there some good literature about the largest possible 2-D arrays on
different computing platforms? Google isn't helping much.

Thanks
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion