Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-18 Thread Antony Lee
In a sense this discussion is really about making np.array(iterable) more efficient, so I restarted the discussion at https://mail.scipy.org/pipermail/numpy-discussion/2016-February/075059.html Antony 2016-02-18 14:21 GMT-08:00 Chris Barker : > On Thu, Feb 18, 2016 at 10:15 AM, Antony Lee > wro

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-18 Thread Chris Barker
On Thu, Feb 18, 2016 at 10:15 AM, Antony Lee wrote: > Mostly so that there is no performance lost when someone passes range(...) > instead of np.arange(...). At least I had never realized that one is much > faster than the other and always just passed range() as a convenience. > Well, pretty m

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-18 Thread josef.pktd
On Thu, Feb 18, 2016 at 1:15 PM, Antony Lee wrote: > Mostly so that there is no performance lost when someone passes range(...) > instead of np.arange(...). At least I had never realized that one is much > faster than the other and always just passed range() as a convenience. > > Antony > > 2016

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-18 Thread Antony Lee
Mostly so that there is no performance lost when someone passes range(...) instead of np.arange(...). At least I had never realized that one is much faster than the other and always just passed range() as a convenience. Antony 2016-02-17 10:50 GMT-08:00 Chris Barker : > On Sun, Feb 14, 2016 at

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-17 Thread Chris Barker
On Sun, Feb 14, 2016 at 11:41 PM, Antony Lee wrote: > So how can np.array(range(...)) even work? > range() (in py3) is not a generator, nor is is a iterator. it is a range object, which is lazily evaluated, and satisfies both the iterator protocol and the sequence protocol (at least most of it:

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-15 Thread Robert Kern
On Mon, Feb 15, 2016 at 4:24 PM, Jeff Reback wrote: > > just an FYI. > > pandas implemented a RangeIndex in upcoming 0.18.0, mainly for memory savings, > see here, similar to how python range/xrange work. > > though there are substantial perf benefits, mainly with set operations, see here > though

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-15 Thread Jeff Reback
just an FYI. pandas implemented a RangeIndex in upcoming 0.18.0, mainly for memory savings, see here , similar to how python range/xrange work. though there are substantial perf benefits, mainly with set operations, see he

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-15 Thread Antony Lee
Indeed: In [1]: class C: def __getitem__(self, i): if i < 10: return i else: raise IndexError def __len__(self): return 10 ...: In [2]: np.array(C()) Out[2]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) (omitting __len__ results in the creation of an object array, co

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-15 Thread Nathaniel Smith
On Sun, Feb 14, 2016 at 11:41 PM, Antony Lee wrote: > I wonder whether numpy is using the "old" iteration protocol (repeatedly > calling x[i] for increasing i until StopIteration is reached?) A quick > timing shows that it is indeed slower. Yeah, I'm pretty sure that np.array doesn't know anythi

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-15 Thread Sebastian Berg
On So, 2016-02-14 at 23:41 -0800, Antony Lee wrote: > I wonder whether numpy is using the "old" iteration protocol > (repeatedly calling x[i] for increasing i until StopIteration is > reached?) A quick timing shows that it is indeed slower. > ... actually it's not even clear to me what qualifies a

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-14 Thread Antony Lee
I wonder whether numpy is using the "old" iteration protocol (repeatedly calling x[i] for increasing i until StopIteration is reached?) A quick timing shows that it is indeed slower. ... actually it's not even clear to me what qualifies as a sequence for `np.array`: class C: def __iter__(self

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-14 Thread Ralf Gommers
On Sun, Feb 14, 2016 at 10:36 PM, Charles R Harris < charlesr.har...@gmail.com> wrote: > > > On Sun, Feb 14, 2016 at 7:36 AM, Ralf Gommers > wrote: > >> >> >> On Sun, Feb 14, 2016 at 9:21 AM, Antony Lee >> wrote: >> >>> re: no reason why... >>> This has nothing to do with Python2/Python3 (I pers

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-14 Thread Charles R Harris
On Sun, Feb 14, 2016 at 7:36 AM, Ralf Gommers wrote: > > > On Sun, Feb 14, 2016 at 9:21 AM, Antony Lee > wrote: > >> re: no reason why... >> This has nothing to do with Python2/Python3 (I personally stopped using >> Python2 at least 3 years ago.) Let me put it this way instead: if >> Python3's

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-14 Thread Antony Lee
I was thinking (1) (special-case range()); however (2) may be more generally applicable and useful. Antony 2016-02-14 6:36 GMT-08:00 Ralf Gommers : > > > On Sun, Feb 14, 2016 at 9:21 AM, Antony Lee > wrote: > >> re: no reason why... >> This has nothing to do with Python2/Python3 (I personally s

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-14 Thread Ralf Gommers
On Sun, Feb 14, 2016 at 9:21 AM, Antony Lee wrote: > re: no reason why... > This has nothing to do with Python2/Python3 (I personally stopped using > Python2 at least 3 years ago.) Let me put it this way instead: if > Python3's "range" (or Python2's "xrange") was not a builtin type but a type >

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-14 Thread josef.pktd
On Sun, Feb 14, 2016 at 3:21 AM, Antony Lee wrote: > re: no reason why... > This has nothing to do with Python2/Python3 (I personally stopped using > Python2 at least 3 years ago.) Let me put it this way instead: if > Python3's "range" (or Python2's "xrange") was not a builtin type but a type >

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-14 Thread Antony Lee
re: no reason why... This has nothing to do with Python2/Python3 (I personally stopped using Python2 at least 3 years ago.) Let me put it this way instead: if Python3's "range" (or Python2's "xrange") was not a builtin type but a type provided by numpy, I don't think it would be controversial at a

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-13 Thread josef.pktd
On Sat, Feb 13, 2016 at 9:43 PM, wrote: > > > On Sat, Feb 13, 2016 at 8:57 PM, Antony Lee > wrote: > >> Compare (on Python3 -- for Python2, read "xrange" instead of "range"): >> >> In [2]: %timeit np.array(range(100), np.int64) >> 10 loops, best of 3: 156 ms per loop >> >> In [3]: %timeit np

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-13 Thread josef.pktd
On Sat, Feb 13, 2016 at 8:57 PM, Antony Lee wrote: > Compare (on Python3 -- for Python2, read "xrange" instead of "range"): > > In [2]: %timeit np.array(range(100), np.int64) > 10 loops, best of 3: 156 ms per loop > > In [3]: %timeit np.arange(100, dtype=np.int64) > 1000 loops, best of 3:

[Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

2016-02-13 Thread Antony Lee
Compare (on Python3 -- for Python2, read "xrange" instead of "range"): In [2]: %timeit np.array(range(100), np.int64) 10 loops, best of 3: 156 ms per loop In [3]: %timeit np.arange(100, dtype=np.int64) 1000 loops, best of 3: 853 µs per loop Note that while iterating over a range is not