Re: [Numpy-discussion] is there an efficient way to get a random set of subsets/combinations?

2012-02-20 Thread Val Kalatsky
Hi Slava, Since your k is only 10, here is a quickie: import numpy as np arr = np.arange(n) for i in range(k): np.random.shuffle(arr) print np.sort(arr[:p]) If your ever get non-unique entries in a set of k=10 for your n and p, consider yourself lucky:) Val On Mon, Feb 20, 2012 at 10:35

Re: [Numpy-discussion] is there an efficient way to get a random set of subsets/combinations?

2012-02-20 Thread Christopher Jordan-Squire
If you're using numpy 2.0 (the development branch), the function numpy.random.choice might do what you're looking for. -Chris On Mon, Feb 20, 2012 at 8:35 PM, Yaroslav Halchenko wrote: > Hi to all Numeric  Python experts, > > could not think of a mailing list with better fit to my question which

[Numpy-discussion] is there an efficient way to get a random set of subsets/combinations?

2012-02-20 Thread Yaroslav Halchenko
Hi to all Numeric Python experts, could not think of a mailing list with better fit to my question which might have an obvious answer: straightforward (naive) Python code to answer my question would be something like import random, itertools n,p,k=100,50,10 # don't try to run with this numbers

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Travis Oliphant
Interesting you bring this up. I actually have a working prototype of using Python to emit LLVM. I will be showing it at the HPC tutorial that I am giving at PyCon.I will be making this available after PyCon to a wider audience as open source. It uses llvm-py (modified to work with LL

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Robert Kern
On Mon, Feb 20, 2012 at 19:55, Paul Anton Letnes wrote: > > On 20. feb. 2012, at 16:29, Sturla Molden wrote: >>> - in newer standards it has some nontrivial mathematical functions: gamma, >>> bessel, etc. that numpy lacks right now >> >> That belongs to SciPy. > > I don't see exactly why. Why sh

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 21:12, skrev Sturla Molden: > > If you need to control the lifetime of an object, make an inner block > with curly brackets, and declare it on top of the block. Don't call new > and delete to control where you want it to be allocated and deallocated. > Nothing goes on the heap unless

Re: [Numpy-discussion] ndarray and lazy evaluation

2012-02-20 Thread James Bergstra
On Mon, Feb 20, 2012 at 2:57 PM, Lluís wrote: > James Bergstra writes: > [...] > > I should add that the biggest benefit of expressing things as compound > > expressions in this way is not in saving temporaries (though that is > nice) it's > > being able to express enough computation work at a ti

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 20:14, skrev Daniele Nicolodi: > Hello Sturla, unrelated to the numpy tewrite debate, can you please > suggest some resources you think can be used to learn how to program > C++ "the proper way"? Thank you. Cheers, This is totally OT on this list, however ... Scott Meyer's book

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread James Bergstra
Looks like Dag forked the discussion of lazy evaluation to a new thread ([Numpy-discussion] ndarray and lazy evaluation). There are actually several projects inspired by this sort of design: off the top of my head I can think of Theano, copperhead, numexpr, arguably sympy, and some non-public cod

Re: [Numpy-discussion] ndarray and lazy evaluation

2012-02-20 Thread Lluís
James Bergstra writes: [...] > I should add that the biggest benefit of expressing things as compound > expressions in this way is not in saving temporaries (though that is nice) > it's > being able to express enough computation work at a time that it offsets the > time > required to ship the arg

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Paul Anton Letnes
On 20. feb. 2012, at 16:29, Sturla Molden wrote: > Den 20.02.2012 08:35, skrev Paul Anton Letnes: >> In the language wars, I have one question. Why is Fortran not being >> considered? Fortran already implements many of the features that we want in >> NumPy: > > Yes ... but it does not make For

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Lluís
Lluís writes: > Francesc Alted writes: >> On Feb 20, 2012, at 6:18 PM, Dag Sverre Seljebotn wrote: >>> You need at least a slightly different Python API to get anywhere, so >>> numexpr/Theano is the right place to work on an implementation of this >>> idea. Of course it would be nice if numexpr

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Neal Becker
Charles R Harris wrote: > On Fri, Feb 17, 2012 at 12:09 PM, Benjamin Root wrote: > >> >> >> On Fri, Feb 17, 2012 at 1:00 PM, Christopher Jordan-Squire < >> cjord...@uw.edu> wrote: >> >>> On Fri, Feb 17, 2012 at 10:21 AM, Mark Wiebe wrote: >>> > On Fri, Feb 17, 2012 at 11:52 AM, Eric Firing >>>

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Lluís
Francesc Alted writes: > On Feb 20, 2012, at 6:18 PM, Dag Sverre Seljebotn wrote: >> You need at least a slightly different Python API to get anywhere, so >> numexpr/Theano is the right place to work on an implementation of this >> idea. Of course it would be nice if numexpr/Theano offered somet

Re: [Numpy-discussion] ndarray and lazy evaluation

2012-02-20 Thread James Bergstra
On Mon, Feb 20, 2012 at 1:01 PM, James Bergstra wrote: > On Mon, Feb 20, 2012 at 12:28 PM, Francesc Alted wrote: > >> On Feb 20, 2012, at 6:18 PM, Dag Sverre Seljebotn wrote: >> > You need at least a slightly different Python API to get anywhere, so >> > numexpr/Theano is the right place to work o

Re: [Numpy-discussion] ndarray and lazy evaluation

2012-02-20 Thread James Bergstra
On Mon, Feb 20, 2012 at 12:28 PM, Francesc Alted wrote: > On Feb 20, 2012, at 6:18 PM, Dag Sverre Seljebotn wrote: > > You need at least a slightly different Python API to get anywhere, so > > numexpr/Theano is the right place to work on an implementation of this > > idea. Of course it would be ni

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Matthieu Brucher
2012/2/20 Daniele Nicolodi > On 18/02/12 04:54, Sturla Molden wrote: > > This is not true. C++ can be much easier, particularly for those who > > already know Python. The problem: C++ textbooks teach C++ as a subset > > of C. Writing C in C++ just adds the complexity of C++ on top of C, > > for n

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Daniele Nicolodi
On 18/02/12 04:54, Sturla Molden wrote: > This is not true. C++ can be much easier, particularly for those who > already know Python. The problem: C++ textbooks teach C++ as a subset > of C. Writing C in C++ just adds the complexity of C++ on top of C, > for no good reason. I can write FORTRAN in a

Re: [Numpy-discussion] Forbidden charcter in the "names" argument of genfromtxt?

2012-02-20 Thread Adam Hughes
Thanks for clearing that up. On Mon, Feb 20, 2012 at 1:58 PM, Skipper Seabold wrote: > On Mon, Feb 20, 2012 at 1:35 PM, Brett Olsen > wrote: > > On Sat, Feb 18, 2012 at 8:12 PM, Adam Hughes > wrote: > >> Hey everyone, > >> > >> I have timeseries data in which the column label is simply a filena

Re: [Numpy-discussion] Forbidden charcter in the "names" argument of genfromtxt?

2012-02-20 Thread Skipper Seabold
On Mon, Feb 20, 2012 at 1:35 PM, Brett Olsen wrote: > On Sat, Feb 18, 2012 at 8:12 PM, Adam Hughes wrote: >> Hey everyone, >> >> I have timeseries data in which the column label is simply a filename from >> which the original data was taken.  Here's some sample data: >> >> name1.txt  name2.txt  n

Re: [Numpy-discussion] Forbidden charcter in the "names" argument of genfromtxt?

2012-02-20 Thread Adam Hughes
Thanks Brett. I appreciate you taking the time to help me out. In particular, I did not know the correct syntax for this: data.dtype.names = names Which is very helpful. If I would have known how to access data.dtype.names, I think it would have saved me a great deal of trouble. I guess it's a

Re: [Numpy-discussion] Forbidden charcter in the "names" argument of genfromtxt?

2012-02-20 Thread Brett Olsen
On Sat, Feb 18, 2012 at 8:12 PM, Adam Hughes wrote: > Hey everyone, > > I have timeseries data in which the column label is simply a filename from > which the original data was taken.  Here's some sample data: > > name1.txt  name2.txt  name3.txt > 32  34    953 > 32

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Francesc Alted
On Feb 20, 2012, at 7:08 PM, Dag Sverre Seljebotn wrote: > On 02/20/2012 09:34 AM, Christopher Jordan-Squire wrote: >> On Mon, Feb 20, 2012 at 9:18 AM, Dag Sverre Seljebotn >> wrote: >>> On 02/20/2012 08:55 AM, Sturla Molden wrote: Den 20.02.2012 17:42, skrev Sturla Molden: > There are

Re: [Numpy-discussion] ndarray and lazy evaluation

2012-02-20 Thread Dag Sverre Seljebotn
On 02/20/2012 10:04 AM, Francesc Alted wrote: > On Feb 20, 2012, at 6:46 PM, Dag Sverre Seljebotn wrote: > >> On 02/20/2012 09:24 AM, Olivier Delalleau wrote: >>> Hi Dag, >>> >>> Would you mind elaborating a bit on that example you mentioned at the >>> end of your email? I don't quite understand wh

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Dag Sverre Seljebotn
On 02/20/2012 09:34 AM, Christopher Jordan-Squire wrote: > On Mon, Feb 20, 2012 at 9:18 AM, Dag Sverre Seljebotn > wrote: >> On 02/20/2012 08:55 AM, Sturla Molden wrote: >>> Den 20.02.2012 17:42, skrev Sturla Molden: There are still other options than C or C++ that are worth considering. >>>

Re: [Numpy-discussion] ndarray and lazy evaluation (was: Proposed Rodmap Overview)

2012-02-20 Thread Olivier Delalleau
Never mind. The link Francesc posted answered my question :) -=- Olivier Le 20 février 2012 12:54, Olivier Delalleau a écrit : > Le 20 février 2012 12:46, Dag Sverre Seljebotn > a écrit : > > On 02/20/2012 09:24 AM, Olivier Delalleau wrote: >> > Hi Dag, >> > >> > Would you mind elaborating a b

Re: [Numpy-discussion] ndarray and lazy evaluation (was: Proposed Rodmap Overview)

2012-02-20 Thread Francesc Alted
On Feb 20, 2012, at 6:46 PM, Dag Sverre Seljebotn wrote: > On 02/20/2012 09:24 AM, Olivier Delalleau wrote: >> Hi Dag, >> >> Would you mind elaborating a bit on that example you mentioned at the >> end of your email? I don't quite understand what behavior you would like >> to achieve > > Sure, s

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 18:34, skrev Christopher Jordan-Squire: > I don't follow this. Could you expand a bit more? (Specifically, I > wasn't aware that numpy could be 10-20x slower than a cython loop, if > we're talking about the base numpy library--so core operations. I'm > also not totally sure why a

[Numpy-discussion] ndarray and lazy evaluation (was: Proposed Rodmap Overview)

2012-02-20 Thread Dag Sverre Seljebotn
On 02/20/2012 09:24 AM, Olivier Delalleau wrote: > Hi Dag, > > Would you mind elaborating a bit on that example you mentioned at the > end of your email? I don't quite understand what behavior you would like > to achieve Sure, see below. I think we should continue discussion on numpy-discuss. I w

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 18:18, skrev Dag Sverre Seljebotn: > > I think it is moot to focus on improving NumPy performance as long as in > practice all NumPy operations are memory bound due to the need to take a > trip through system memory for almost any operation. C/C++ is simply > "good enough". JIT is wh

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Christopher Jordan-Squire
On Mon, Feb 20, 2012 at 9:18 AM, Dag Sverre Seljebotn wrote: > On 02/20/2012 08:55 AM, Sturla Molden wrote: >> Den 20.02.2012 17:42, skrev Sturla Molden: >>> There are still other options than C or C++ that are worth considering. >>> One would be to write NumPy in Python. E.g. we could use LLVM as

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Matthieu Brucher
2012/2/19 Sturla Molden > Den 19.02.2012 10:28, skrev Mark Wiebe: > > > > Particular styles of using templates can cause this, yes. To properly > > do this kind of advanced C++ library work, it's important to think > > about the big-O notation behavior of your template instantiations, not > > jus

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Francesc Alted
On Feb 20, 2012, at 6:18 PM, Dag Sverre Seljebotn wrote: > You need at least a slightly different Python API to get anywhere, so > numexpr/Theano is the right place to work on an implementation of this > idea. Of course it would be nice if numexpr/Theano offered something as > convenient as > >

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Matthieu Brucher
2012/2/19 Nathaniel Smith > On Sun, Feb 19, 2012 at 9:16 AM, David Cournapeau > wrote: > > On Sun, Feb 19, 2012 at 8:08 AM, Mark Wiebe wrote: > >> Is there a specific > >> target platform/compiler combination you're thinking of where we can do > >> tests on this? I don't believe the compile tim

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 18:14, skrev Charles R Harris: > > Would that work for Ruby also? One of the advantages of C++ is that > the code doesn't need to be refactored to start with, just modified > step by step going into the future. I think PyPy is close to what you > are talking about. > If we plant

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Matthieu Brucher
2012/2/19 Matthew Brett > Hi, > > On Sat, Feb 18, 2012 at 8:38 PM, Travis Oliphant > wrote: > > > We will need to see examples of what Mark is talking about and clarify > some > > of the compiler issues. Certainly there is some risk that once code is > > written that it will be tempting to jus

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Matthieu Brucher
> Would it be fair to say then, that you are expecting the discussion > about C++ will mainly arise after the Mark has written the code? I > can see that it will be easier to specific at that point, but there > must be a serious risk that it will be too late to seriously consider > an alternative

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Matthieu Brucher
> C++11 has this option: > > for (auto& item : container) { > // iterate over the container object, > // get a reference to each item > // > // "container" can be an STL class or > // A C-style array with known size. > } > > Which does this: > > for item in container: > pass

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Dag Sverre Seljebotn
On 02/20/2012 08:55 AM, Sturla Molden wrote: > Den 20.02.2012 17:42, skrev Sturla Molden: >> There are still other options than C or C++ that are worth considering. >> One would be to write NumPy in Python. E.g. we could use LLVM as a >> JIT-compiler and produce the performance critical code we nee

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Charles R Harris
On Mon, Feb 20, 2012 at 9:55 AM, Sturla Molden wrote: > Den 20.02.2012 17:42, skrev Sturla Molden: > > There are still other options than C or C++ that are worth considering. > > One would be to write NumPy in Python. E.g. we could use LLVM as a > > JIT-compiler and produce the performance critic

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 17:42, skrev Sturla Molden: > There are still other options than C or C++ that are worth considering. > One would be to write NumPy in Python. E.g. we could use LLVM as a > JIT-compiler and produce the performance critical code we need on the fly. > > LLVM and its C/C++ frontend Cla

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 19.02.2012 00:09, skrev David Cournapeau: > There are better languages than C++ that has most of the technical > benefits stated in this discussion (rust and D being the most > "obvious" ones), but whose usage is unrealistic today for various > reasons: knowledge, availability on "esoteric"

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 08:35, skrev Paul Anton Letnes: > As far as I can understand, implementing element-wise operations, slicing, > and a host of other NumPy features is in some sense pointless - the Fortran > compiler authors have already done it for us. Only if you know the array dimensions in advan

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 08:35, skrev Paul Anton Letnes: > In the language wars, I have one question. Why is Fortran not being > considered? Fortran already implements many of the features that we want in > NumPy: Yes ... but it does not make Fortran a systems programming language. Making NumPy is differ

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 10:54, skrev Pauli Virtanen: > Fortran is OK for simple numerical algorithms, but starts to suck > heavily if you need to do any string handling, I/O, complicated logic, > or data structures For string handling, C is actually worse than Fortran. In Fortran a string can be sliced

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 12:43, skrev Charles R Harris: > > > There also used to be a problem with unsigned types not being > available. I don't know if that is still the case. > Fortran -- like Python and Java -- does not have built-in unsigned integer types. It is never really a problem though. One can

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Charles R Harris
On Mon, Feb 20, 2012 at 2:54 AM, Pauli Virtanen wrote: > 20.02.2012 08:35, Paul Anton Letnes kirjoitti: > > In the language wars, I have one question. > > Why is Fortran not being considered? > > Fortran is OK for simple numerical algorithms, but starts to suck > heavily if you need to do any str

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Stéfan van der Walt
On Mon, Feb 20, 2012 at 1:54 AM, Pauli Virtanen wrote: > 20.02.2012 08:35, Paul Anton Letnes kirjoitti: >> In the language wars, I have one question. >> Why is Fortran not being considered? > > Fortran is OK for simple numerical algorithms, but starts to suck > heavily if you need to do any string

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Pauli Virtanen
20.02.2012 08:35, Paul Anton Letnes kirjoitti: > In the language wars, I have one question. > Why is Fortran not being considered? Fortran is OK for simple numerical algorithms, but starts to suck heavily if you need to do any string handling, I/O, complicated logic, or data structures. Most of

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Samuel John
On 17.02.2012, at 21:46, Ralf Gommers wrote: > [...] > So far no one has managed to build the numpy/scipy combo with the LLVM-based > compilers, so if you were willing to have a go at fixing that it would be > hugely appreciated. See http://projects.scipy.org/scipy/ticket/1500 for > details. >