Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

[Sebastian Berg]

On So, 2016-02-14 at 23:41 -0800, Antony Lee wrote:
> I wonder whether numpy is using the "old" iteration protocol
> (repeatedly calling x[i] for increasing i until StopIteration is
> reached?)  A quick timing shows that it is indeed slower.
> ... actually it's not even clear to me what qualifies as a sequence
> for `np.array`:
> 
> class C:   
> def __iter__(self):   
> return iter(range(10)) # [0... 9] under the new iteration
> protocol
> def __getitem__(self, i):
> raise IndexError # [] under the old iteration protocol
> 

Numpy currently uses PySequence_Fast, but it has to do a two pass
algorithm (find dtype+dims), and the range is converted twice to list
by this call. That explains the speed advantage of converting to list
manually.

- Sebastian


> np.array(C())
> ===> array(<__main__.C object at 0x7f3f2128>, dtype=object)
> 
> So how can np.array(range(...)) even work?
> 
> 2016-02-14 22:21 GMT-08:00 Ralf Gommers :
> > 
> > 
> > On Sun, Feb 14, 2016 at 10:36 PM, Charles R Harris <
> > charlesr.har...@gmail.com> wrote:
> > > 
> > > 
> > > On Sun, Feb 14, 2016 at 7:36 AM, Ralf Gommers <
> > > ralf.gomm...@gmail.com> wrote:
> > > > 
> > > > 
> > > > On Sun, Feb 14, 2016 at 9:21 AM, Antony Lee <
> > > > antony@berkeley.edu> wrote:
> > > > > re: no reason why...
> > > > > This has nothing to do with Python2/Python3 (I personally
> > > > > stopped using Python2 at least 3 years ago.)  Let me put it
> > > > > this way instead: if Python3's "range" (or Python2's
> > > > > "xrange") was not a builtin type but a type provided by
> > > > > numpy, I don't think it would be controversial at all to
> > > > > provide an `__array__` special method to efficiently convert
> > > > > it to a ndarray.  It would be the same if `np.array` used a
> > > > > `functools.singledispatch` dispatcher rather than an
> > > > > `__array__` special method (which is obviously not possible
> > > > > for chronological reasons).
> > > > > 
> > > > > re: iterable vs iterator: check for the presence of the
> > > > > __next__ special method (or isinstance(x, Iterable) vs.
> > > > > isinstance(x, Iterator) and not isinstance(x, Iterable))
> > > > > 
> > > > I think it's good to do something about this, but it's not
> > > > clear what the exact proposal is. I could image one or both of:
> > > > 
> > > >   - special-case the range() object in array (and
> > > > asarray/asanyarray?) such that array(range(N)) becomes as fast
> > > > as arange(N).
> > > >   - special-case all iterators, such that array(range(N))
> > > > becomes as fast as deque(range(N))
> > > > 
> > > I think the last wouldn't help much, as numpy would still need to
> > > determine dimensions and type.  I assume that is one of the
> > > reason sparse itself doesn't do that.
> > > 
> > Not orders of magnitude, but this shows that there's something to
> > optimize for iterators:
> > 
> > In [1]: %timeit np.array(range(10))
> > 100 loops, best of 3: 14.9 ms per loop
> > 
> > In [2]: %timeit np.array(list(range(10)))
> > 100 loops, best of 3: 9.68 ms per loop
> > 
> > Ralf
> >  
> > 
> > ___
> > NumPy-Discussion mailing list
> > NumPy-Discussion@scipy.org
> > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> > 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion

signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

[Nathaniel Smith]

On Sun, Feb 14, 2016 at 11:41 PM, Antony Lee  wrote:
> I wonder whether numpy is using the "old" iteration protocol (repeatedly
> calling x[i] for increasing i until StopIteration is reached?)  A quick
> timing shows that it is indeed slower.

Yeah, I'm pretty sure that np.array doesn't know anything about
"iterable", just about "sequence" (calling x[i] for 0 <= i <
i.__len__()).

(See Sequence vs Iterable:
https://docs.python.org/3/library/collections.abc.html)

Personally I'd like it if we could eventually make it so np.array
specifically looks for lists and only lists, because the way it has so
many different fallbacks right now creates all confusion between which
objects are elements. Compare:

In [5]: np.array([(1, 2), (3, 4)]).shape
Out[5]: (2, 2)

In [6]: np.array([(1, 2), (3, 4)], dtype="i4,i4").shape
Out[6]: (2,)

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numexpr-3.0 proposal

[Gregor Thalhammer]

> Am 14.02.2016 um 23:19 schrieb Robert McLeod :
> 
> Hello everyone,
> 
> I've done some work on making a new version of Numexpr that would fix some of 
> the limitations of the original virtual machine with regards to data types 
> and operation/function count. Basically I re-wrote the Python and C sides to 
> use 4-byte words, instead of null-terminated strings, for operations and 
> passing types.  This means the number of operations and types isn't 
> significantly limited anymore.
> 
> Francesc Alted suggested I should come here and get some advice from the 
> community. I wrote a short proposal on the Wiki here:
> 
> https://github.com/pydata/numexpr/wiki/Numexpr-3.0-Branch-Overview 
> 
> 
> One can see my branch here:
> 
> https://github.com/robbmcleod/numexpr/tree/numexpr-3.0 
> 
> 
> If anyone has any comments they'd be welcome. Questions from my side for the 
> group:
> 
> 1.) Numpy casting: I downloaded the Numpy source and after browsing it seems 
> the best approach is probably to just use 
> numpy.core.numerictypes.find_common_type?
> 
> 2.) Can anyone foresee any issues with casting build-in Python types (i.e. 
> float and integer) to their OS dependent numpy equivalents? Numpy already 
> seems to do this. 
> 
> 3.) Is anyone enabling the Intel VML library? There are a number of comments 
> in the code that suggest it's not accelerating the code. It also seems to 
> cause problems with bundling numexpr with cx_freeze.
> 
Dear Robert,

thanks for your effort on improving numexpr. Indeed, vectorized math libraries 
(VML) can give a large boost in performance (~5x), except for a couple of basic 
operations (add, mul, div), which current compilers are able to vectorize 
automatically. With recent gcc even more functions are vectorized, see 
https://sourceware.org/glibc/wiki/libmvec 
 But you need special flags 
depending on the platform (SSE, AVX present?), runtime detection of processor 
capabilities would be nice for distributing binaries. Some time ago, since I 
lost access to Intels MKL, I patched numexpr to use Accelerate/Veclib on os x, 
which is preinstalled on each mac, see https://github.com/geggo/numexpr.git 
 veclib_support branch.

As you increased the opcode size, I could imagine providing a bit to switch 
(during runtime) between internal functions and vectorized ones, that would be 
handy for tests and benchmarks.

Gregor

> 4.) I took a stab at converting from distutils to setuputils but this seems 
> challenging with numpy as a dependency. I wonder if anyone has tried 
> monkey-patching so that setup.py build_ext uses distutils and then pass the 
> interpreter.pyd/so as a data file, or some other such chicanery?   
> 
> (I was going to ask about attaching a debugger, but I just noticed: 
> https://wiki.python.org/moin/DebuggingWithGdb 
>    )
> 
> Ciao,
> 
> Robert
> 
> -- 
> Robert McLeod, Ph.D.
> Center for Cellular Imaging and Nano Analytics (C-CINA)
> Biozentrum der Universität Basel
> Mattenstrasse 26, 4058 Basel
> Work: +41.061.387.3225 
> robert.mcl...@unibas.ch 
> robert.mcl...@bsse.ethz.ch 
> robbmcl...@gmail.com 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

[Antony Lee]

Indeed:

In [1]: class C:
def __getitem__(self, i):
if i < 10: return i
else: raise IndexError
def __len__(self):
return 10
   ...:

In [2]: np.array(C())
Out[2]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])


(omitting __len__ results in the creation of an object array, consistently
with the fact that the sequence protocol requires __len__).
Meanwhile, I found a new way to segfault numpy :-)

In [3]: class C:
def __getitem__(self, i):
if i < 10: return i
else: raise IndexError
def __len__(self):
return 42
   ...:

In [4]: np.array(C())
Fatal Python error: Segmentation fault


2016-02-15 0:10 GMT-08:00 Nathaniel Smith :

> On Sun, Feb 14, 2016 at 11:41 PM, Antony Lee 
> wrote:
> > I wonder whether numpy is using the "old" iteration protocol (repeatedly
> > calling x[i] for increasing i until StopIteration is reached?)  A quick
> > timing shows that it is indeed slower.
>
> Yeah, I'm pretty sure that np.array doesn't know anything about
> "iterable", just about "sequence" (calling x[i] for 0 <= i <
> i.__len__()).
>
> (See Sequence vs Iterable:
> https://docs.python.org/3/library/collections.abc.html)
>
> Personally I'd like it if we could eventually make it so np.array
> specifically looks for lists and only lists, because the way it has so
> many different fallbacks right now creates all confusion between which
> objects are elements. Compare:
>
> In [5]: np.array([(1, 2), (3, 4)]).shape
> Out[5]: (2, 2)
>
> In [6]: np.array([(1, 2), (3, 4)], dtype="i4,i4").shape
> Out[6]: (2,)
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

[Jeff Reback]

just an FYI.

pandas implemented a RangeIndex in upcoming 0.18.0, mainly for memory
savings,
see here
,
similar to how python range/xrange work.

though there are substantial perf benefits, mainly with set operations, see
here

though didn't officially benchmark thes.

Jeff


On Mon, Feb 15, 2016 at 11:13 AM, Antony Lee 
wrote:

> Indeed:
>
> In [1]: class C:
> def __getitem__(self, i):
> if i < 10: return i
> else: raise IndexError
> def __len__(self):
> return 10
>...:
>
> In [2]: np.array(C())
> Out[2]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>
>
> (omitting __len__ results in the creation of an object array, consistently
> with the fact that the sequence protocol requires __len__).
> Meanwhile, I found a new way to segfault numpy :-)
>
> In [3]: class C:
> def __getitem__(self, i):
> if i < 10: return i
> else: raise IndexError
> def __len__(self):
> return 42
>...:
>
> In [4]: np.array(C())
> Fatal Python error: Segmentation fault
>
>
> 2016-02-15 0:10 GMT-08:00 Nathaniel Smith :
>
>> On Sun, Feb 14, 2016 at 11:41 PM, Antony Lee 
>> wrote:
>> > I wonder whether numpy is using the "old" iteration protocol (repeatedly
>> > calling x[i] for increasing i until StopIteration is reached?)  A quick
>> > timing shows that it is indeed slower.
>>
>> Yeah, I'm pretty sure that np.array doesn't know anything about
>> "iterable", just about "sequence" (calling x[i] for 0 <= i <
>> i.__len__()).
>>
>> (See Sequence vs Iterable:
>> https://docs.python.org/3/library/collections.abc.html)
>>
>> Personally I'd like it if we could eventually make it so np.array
>> specifically looks for lists and only lists, because the way it has so
>> many different fallbacks right now creates all confusion between which
>> objects are elements. Compare:
>>
>> In [5]: np.array([(1, 2), (3, 4)]).shape
>> Out[5]: (2, 2)
>>
>> In [6]: np.array([(1, 2), (3, 4)], dtype="i4,i4").shape
>> Out[6]: (2,)
>>
>> -n
>>
>> --
>> Nathaniel J. Smith -- https://vorpus.org
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Suggestion: special-case np.array(range(...)) to be faster

[Robert Kern]

On Mon, Feb 15, 2016 at 4:24 PM, Jeff Reback  wrote:
>
> just an FYI.
>
> pandas implemented a RangeIndex in upcoming 0.18.0, mainly for memory
savings,
> see here, similar to how python range/xrange work.
>
> though there are substantial perf benefits, mainly with set operations,
see here
> though didn't officially benchmark thes.

Since it is a numpy-aware object (unlike the builtins), you can (and have,
if I'm reading the code correctly) implement __array__() such that it does
the correctly performant thing and call np.arange(). RangeIndex won't be
adversely impacted by retaining the status quo.

--
Robert Kern
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Subclassing ma.masked_array, code broken after version 1.9

[Gutenkunst, Ryan N - (rgutenk)]

Thank Jonathan,

Good to confirm this isn't something inappropriate I'm doing. I give up 
transparency here in my application, so I'll just work around it. I leave it up 
to wiser numpy heads as to whether it's worth altering these numpy.ma functions 
to enable subclassing.

Best,
Ryan

On Feb 13, 2016, at 11:48 AM, Jonathan Helmus  wrote:

> 
> 
> On 2/12/16 6:06 PM, Gutenkunst, Ryan N - (rgutenk) wrote:
>> Hello all,
>> 
>> In 2009 I developed an application that uses a subclass of masked arrays as 
>> a central data object. My subclass Spectrum possesses additional attributes 
>> along with many custom methods. It was very convenient to be able to use 
>> standard numpy functions for doing arithmetic on these objects. However, my 
>> code broke with numpy 1.10. I've finally had a chance to track down the 
>> problem, and I am hoping someone can suggest a workaround.
>> 
>> See below for an example, which is as minimal as I could concoct. In this 
>> case, I have a Spectrum object that I'd like to take the logarithm of using 
>> numpy.ma.log, while preserving the value of the "folded" attribute. Up to 
>> numpy 1.9, this worked as expected, but in numpy 1.10 and 1.11 the attribute 
>> is not preserved.
>> 
>> The change in behavior appears to be driven by a commit made on Jun 16th, 
>> 2015 by Marten van Kerkwijk. In particular, the commit changed 
>> _MaskedUnaryOperation.__call__ so that the result array's update_from method 
>> is no longer called with the input array as the argument, but rather the 
>> result of the numpy UnaryOperation (old line 889, new line 885). Because 
>> that UnaryOperation doesn't carry my new attribute, it's not present for 
>> update_from to access. I notice that similar changes were made to 
>> MaskedBinaryOperation, although I haven't tested those. It's not clear to me 
>> from the commit message why this particular change was made, so I don't know 
>> whether this new behavior is intentional.
>> 
>> I know that subclassing arrays isn't widely encouraged, but it has been very 
>> convenient in my code. Is it still possible to subclass masked_array in such 
>> a way that functions like numpy.ma.log preserve additional attributes? If 
>> so, can someone point me in the right direction?
>> 
>> Thanks!
>> Ryan
>> 
>> *** Begin example
>> 
>> import numpy
>> print 'Working with numpy {0}'.format(numpy.__version__)
>> 
>> class Spectrum(numpy.ma.masked_array):
>> def __new__(cls, data, mask=numpy.ma.nomask, data_folded=None):
>> subarr = numpy.ma.masked_array(data, mask=mask, keep_mask=True,
>>shrink=True)
>> subarr = subarr.view(cls)
>> subarr.folded = data_folded
>> 
>> return subarr
>> 
>> def __array_finalize__(self, obj):
>> if obj is None:
>> return
>> numpy.ma.masked_array.__array_finalize__(self, obj)
>> self.folded = getattr(obj, 'folded', 'unspecified')
>> 
>> def _update_from(self, obj):
>> print('Input to update_from: {0}'.format(repr(obj)))
>> numpy.ma.masked_array._update_from(self, obj)
>> self.folded = getattr(obj, 'folded', 'unspecified')
>> 
>> def __repr__(self):
>> return 'Spectrum(%s, folded=%s)'\
>> % (str(self), str(self.folded))
>> 
>> fs1 = Spectrum([2,3,4.], data_folded=True)
>> fs2 = numpy.ma.log(fs1)
>> print('fs2.folded status: {0}'.format(fs2.folded))
>> print('Expectation is True, achieved with numpy 1.9')
>> 
>> *** End example
>> 
>> --
>> Ryan Gutenkunst
>> Assistant Professor
>> Molecular and Cellular Biology
>> University of Arizona
>> phone: (520) 626-0569, office LSS 325
>> http://gutengroup.mcb.arizona.edu
>> Latest paper: "Computationally efficient composite likelihood statistics for 
>> demographic inference"
>> Molecular Biology and Evolution; http://dx.doi.org/10.1093/molbev/msv255
> Ryan,
> 
> I'm not sure if you will be able to get this to work as in NumPy 1.9, but the 
> __array_wrap__ method is intended to be the mechanism for subclasses to set 
> their return type, adjust metadata, etc [1].  Unfortunately, the numpy.ma.log 
> function does not seem to make a call to  __array_wrap__ (at least in NumPy 
> 1.10.2) although numpy.log does:
> 
> from __future__ import print_function
> import numpy
> print('Working with numpy {0}'.format(numpy.__version__))
> 
> 
> class Spectrum(numpy.ma.masked_array):
>def __new__(cls, data, mask=numpy.ma.nomask, data_folded=None):
>subarr = numpy.ma.masked_array(data, mask=mask, keep_mask=True,
>   shrink=True)
>subarr = subarr.view(cls)
>subarr.folded = data_folded
> 
>return subarr
> 
>def __array_finalize__(self, obj):
>if obj is None:
>return
>numpy.ma.masked_array.__array_finalize__(self, obj)
>self.folded = getattr(obj, 'folded', 'unspecified')
> 
>def __array_wrap__(self, out_arr, context=None):
>print(

Re: [Numpy-discussion] ANN: pandas v0.18.0rc1 - RELEASE CANDIDATE

[Derek Homeier]

On 14 Feb 2016, at 1:53 am, Jeff Reback  wrote:
> 
> I'm pleased to announce the availability of the first release candidate of 
> Pandas 0.18.0.
> Please try this RC and report any issues here: Pandas Issues
> We will be releasing officially in 1-2 weeks or so.
> 
Thanks, looking forward to give this a try!
Do you have a download link to the source for non-Conda users and packagers?
Finding anything in the github source tarball repositories without having the 
exact
path seems hopeless.

Derek


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: pandas v0.18.0rc1 - RELEASE CANDIDATE

[Jeff Reback]

https://github.com/pydata/pandas/releases/tag/v0.18.0rc1

On Mon, Feb 15, 2016 at 12:51 PM, Derek Homeier <
de...@astro.physik.uni-goettingen.de> wrote:

> On 14 Feb 2016, at 1:53 am, Jeff Reback  wrote:
> >
> > I'm pleased to announce the availability of the first release candidate
> of Pandas 0.18.0.
> > Please try this RC and report any issues here: Pandas Issues
> > We will be releasing officially in 1-2 weeks or so.
> >
> Thanks, looking forward to give this a try!
> Do you have a download link to the source for non-Conda users and
> packagers?
> Finding anything in the github source tarball repositories without having
> the exact
> path seems hopeless.
>
> Derek
>
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Subclassing ma.masked_array, code broken after version 1.9

[Sebastian Berg]

On Mo, 2016-02-15 at 17:06 +, Gutenkunst, Ryan N - (rgutenk) wrote:
> Thank Jonathan,
> 
> Good to confirm this isn't something inappropriate I'm doing. I give
> up transparency here in my application, so I'll just work around it.
> I leave it up to wiser numpy heads as to whether it's worth altering
> these numpy.ma functions to enable subclassing.
> 

Frankly, when it comes to masked array stuff, at least I am puzzled
most of the time, so input is very welcome.
Most of the people currently contributing, barely use masked arrays as
far as I know, and sometimes it is hard to make good calls. It is a not
the easiest code base and any feedback or nudging is important. A new
release is about to come out, and if you feel it there is a serious
regression, we may want to push for fixing it (or even better, you may
have time to suggest a fix yourself).

- Sebastian


> Best,
> Ryan
> 
> On Feb 13, 2016, at 11:48 AM, Jonathan Helmus 
> wrote:
> 
> > 
> > 
> > On 2/12/16 6:06 PM, Gutenkunst, Ryan N - (rgutenk) wrote:
> > > Hello all,
> > > 
> > > In 2009 I developed an application that uses a subclass of masked
> > > arrays as a central data object. My subclass Spectrum possesses
> > > additional attributes along with many custom methods. It was very
> > > convenient to be able to use standard numpy functions for doing
> > > arithmetic on these objects. However, my code broke with numpy
> > > 1.10. I've finally had a chance to track down the problem, and I
> > > am hoping someone can suggest a workaround.
> > > 
> > > See below for an example, which is as minimal as I could concoct.
> > > In this case, I have a Spectrum object that I'd like to take the
> > > logarithm of using numpy.ma.log, while preserving the value of
> > > the "folded" attribute. Up to numpy 1.9, this worked as expected,
> > > but in numpy 1.10 and 1.11 the attribute is not preserved.
> > > 
> > > The change in behavior appears to be driven by a commit made on
> > > Jun 16th, 2015 by Marten van Kerkwijk. In particular, the commit
> > > changed _MaskedUnaryOperation.__call__ so that the result array's
> > > update_from method is no longer called with the input array as
> > > the argument, but rather the result of the numpy UnaryOperation
> > > (old line 889, new line 885). Because that UnaryOperation doesn't
> > > carry my new attribute, it's not present for update_from to
> > > access. I notice that similar changes were made to
> > > MaskedBinaryOperation, although I haven't tested those. It's not
> > > clear to me from the commit message why this particular change
> > > was made, so I don't know whether this new behavior is
> > > intentional.
> > > 
> > > I know that subclassing arrays isn't widely encouraged, but it
> > > has been very convenient in my code. Is it still possible to
> > > subclass masked_array in such a way that functions like
> > > numpy.ma.log preserve additional attributes? If so, can someone
> > > point me in the right direction?
> > > 
> > > Thanks!
> > > Ryan
> > > 
> > > *** Begin example
> > > 
> > > import numpy
> > > print 'Working with numpy {0}'.format(numpy.__version__)
> > > 
> > > class Spectrum(numpy.ma.masked_array):
> > > def __new__(cls, data, mask=numpy.ma.nomask,
> > > data_folded=None):
> > > subarr = numpy.ma.masked_array(data, mask=mask,
> > > keep_mask=True,
> > >shrink=True)
> > > subarr = subarr.view(cls)
> > > subarr.folded = data_folded
> > > 
> > > return subarr
> > > 
> > > def __array_finalize__(self, obj):
> > > if obj is None:
> > > return
> > > numpy.ma.masked_array.__array_finalize__(self, obj)
> > > self.folded = getattr(obj, 'folded', 'unspecified')
> > > 
> > > def _update_from(self, obj):
> > > print('Input to update_from: {0}'.format(repr(obj)))
> > > numpy.ma.masked_array._update_from(self, obj)
> > > self.folded = getattr(obj, 'folded', 'unspecified')
> > > 
> > > def __repr__(self):
> > > return 'Spectrum(%s, folded=%s)'\
> > > % (str(self), str(self.folded))
> > > 
> > > fs1 = Spectrum([2,3,4.], data_folded=True)
> > > fs2 = numpy.ma.log(fs1)
> > > print('fs2.folded status: {0}'.format(fs2.folded))
> > > print('Expectation is True, achieved with numpy 1.9')
> > > 
> > > *** End example
> > > 
> > > --
> > > Ryan Gutenkunst
> > > Assistant Professor
> > > Molecular and Cellular Biology
> > > University of Arizona
> > > phone: (520) 626-0569, office LSS 325
> > > http://gutengroup.mcb.arizona.edu
> > > Latest paper: "Computationally efficient composite likelihood
> > > statistics for demographic inference"
> > > Molecular Biology and Evolution; 
> > > http://dx.doi.org/10.1093/molbev/msv255
> > Ryan,
> > 
> > I'm not sure if you will be able to get this to work as in NumPy
> > 1.9, but the __array_wrap__ method is intended to be the mechanism
> > for subclasses to set their return type, adjust metadata, etc [1]. 

[Numpy-discussion] DyND 0.7.1 Release

[Irwin Zaid]

Hello everyone,

I'm pleased to announce the latest 0.7.1 release of DyND. The release notes
are at https://github.com/libdynd/libdynd/blob/master/docs/release_notes.txt
.

Over the last 6 months, DyND has really matured a lot and many features
that were "experimental" before are quite usable at the moment. At the same
time, we still have bugs and issues to sort out, so I wouldn't claim DyND
has reached stability just yet. Nevertheless, I'd encourage early adopters
to try it out. You can install it easily using conda, via "conda install -c
dynd/channel/dev dynd-python".

Presently, the core DyND team consists of myself, Mark Wiebe, and Ian
Henriksen, alongside several other contributors. Our focus is almost
entirely on gaps in stability and usability -- the novel features in DyND
that people find attractive (including missing values, ragged arrays,
variable-sized strings, dynamic custom types, and overloadable callables,
among others) are functioning pretty well now.

NumPy compatibility and interoperability is very important to us, and is
something we are constantly improving. We would eventually like to have an
optional NumPy-like API that is fully consistent with what a NumPy user
would expect, but we're not there just yet.

The DyND team would be happy to answer any questions people have about
DyND, like "what is working and what is not" or "what do we still need to
do to hit DyND 1.0".

All the best,

Irwin
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: pandas v0.18.0rc1 - RELEASE CANDIDATE

[Derek Homeier]

On 15 Feb 2016, at 6:55 pm, Jeff Reback  wrote:
> 
> https://github.com/pydata/pandas/releases/tag/v0.18.0rc1

Ah, think I forgot about the ‘releases’ pages.
Built on OS X 10.10 + 10.11 with python 2.7.11, 3.4.4 and 3.5.1.
17 errors in the test suite + 1 failure with python2.7 only; I can send
you details on the errors if desired, the majority seems to be generic
to a urllib problem with openssl on OS X anyway.

Thanks for the good work

Derek

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Suggestion] Labelled Array

[Lluís Vilanova]

Benjamin Root writes:

> Seems like you are talking about xarray: https://github.com/pydata/xarray

Oh, I wasn't aware of xarray, but there's also this:

  
https://people.gso.ac.upc.edu/vilanova/doc/sciexp2/user_guide/data.html#basic-indexing
  
https://people.gso.ac.upc.edu/vilanova/doc/sciexp2/user_guide/data.html#dimension-oblivious-indexing


Cheers,
  Lluis



> Cheers!
> Ben Root

> On Fri, Feb 12, 2016 at 9:40 AM, Sérgio  wrote:

> Hello,


> This is my first e-mail, I will try to make the idea simple.


> Similar to masked array it would be interesting to use a label array to
> guide operations.


> Ex.:
 x
> labelled_array(data = 

> [[0 1 2]
> [3 4 5]
> [6 7 8]],
> label =
> [[0 1 2]
> [0 1 2]
> [0 1 2]])


 sum(x)
> array([9, 12, 15])


> The operations would create a new axis for label indexing.


> You could think of it as a collection of masks, one for each label.


> I don't know a way to make something like this efficiently without a loop.
> Just wondering...


> Sérgio.

> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion




> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Suggestion] Labelled Array

[Paul Hobson]

Just for posterity -- any future readers to this thread who need to do
pandas-like on record arrays should look at matplotlib's mlab submodule.

I've been in situations (::cough:: Esri production ::cough::) where I've
had one hand tied behind my back and unable to install pandas. mlab was a
big help there.

https://goo.gl/M7Mi8B

-paul



On Mon, Feb 15, 2016 at 1:28 PM, Lluís Vilanova  wrote:

> Benjamin Root writes:
>
> > Seems like you are talking about xarray:
> https://github.com/pydata/xarray
>
> Oh, I wasn't aware of xarray, but there's also this:
>
>
> https://people.gso.ac.upc.edu/vilanova/doc/sciexp2/user_guide/data.html#basic-indexing
>
> https://people.gso.ac.upc.edu/vilanova/doc/sciexp2/user_guide/data.html#dimension-oblivious-indexing
>
>
> Cheers,
>   Lluis
>
>
>
> > Cheers!
> > Ben Root
>
> > On Fri, Feb 12, 2016 at 9:40 AM, Sérgio  wrote:
>
> > Hello,
>
>
> > This is my first e-mail, I will try to make the idea simple.
>
>
> > Similar to masked array it would be interesting to use a label array
> to
> > guide operations.
>
>
> > Ex.:
>  x
> > labelled_array(data =
>
> > [[0 1 2]
> > [3 4 5]
> > [6 7 8]],
> > label =
> > [[0 1 2]
> > [0 1 2]
> > [0 1 2]])
>
>
>  sum(x)
> > array([9, 12, 15])
>
>
> > The operations would create a new axis for label indexing.
>
>
> > You could think of it as a collection of masks, one for each label.
>
>
> > I don't know a way to make something like this efficiently without a
> loop.
> > Just wondering...
>
>
> > Sérgio.
>
> > ___
> > NumPy-Discussion mailing list
> > NumPy-Discussion@scipy.org
> > https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
>
> > ___
> > NumPy-Discussion mailing list
> > NumPy-Discussion@scipy.org
> > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] NumPy 1.11.0b3 released.

[josef.pktd]

On Mon, Feb 15, 2016 at 10:46 PM,  wrote:

>
>
> On Fri, Feb 12, 2016 at 4:19 PM, Nathan Goldbaum 
> wrote:
>
>> https://github.com/numpy/numpy/blob/master/doc/release/1.11.0-notes.rst
>>
>> On Fri, Feb 12, 2016 at 3:17 PM, Andreas Mueller 
>> wrote:
>>
>>> Hi.
>>> Where can I find the changelog?
>>> It would be good for us to know which changes are done one purpos
>>> without hunting through the issue tracker.
>>>
>>> Thanks,
>>> Andy
>>>
>>>
>>> On 02/09/2016 09:09 PM, Charles R Harris wrote:
>>>
>>> Hi All,
>>>
>>> I'm pleased to announce the release of NumPy 1.11.0b3. This beta
>>> contains additional bug fixes as well as limiting the number of
>>> FutureWarnings raised by assignment to masked array slices. One issue that
>>> remains to be decided is whether or not to postpone raising an error for
>>> floats used as indexes. Sources may be found on Sourceforge
>>>  and both
>>> sources and OS X wheels are availble on pypi. Please test, hopefully this
>>> will be that last beta needed.
>>>
>>> As a note on problems encountered, twine uploads continue to fail for
>>> me, but there are still variations to try. The wheeluploader downloaded
>>> wheels as it should, but could not upload them, giving the error message
>>> "HTTPError: 413 Client Error: Request Entity Too Large for url:
>>> https://www.python.org/pypi";. Firefox also
>>> complains that http://wheels.scipy.org is incorrectly configured with
>>> an invalid certificate.
>>>
>>> Enjoy,
>>>
>>> Chuck
>>>
>>>
>>> ___
>>> NumPy-Discussion mailing 
>>> listNumPy-Discussion@scipy.orghttps://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>>
>>> ___
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion@scipy.org
>>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
(try to send again)


>
> another indexing question:  (not covered by unit test but showed up in
> examples in statsmodels)
>
>
> This works in numpy at least 1.9.2 and 1.6.1   (python 2.7, and python 3.4)
>
> >>> list(range(5))[np.array([0])]
> 0
>
>
>
> on numpy 0.11.0b2   (I'm not yet at b3)   (python 3.4)
>
> I get the same exception as here but even if there is just one element
>
>
> >>> list(range(5))[np.array([0, 1])]
> Traceback (most recent call last):
>   File "", line 1, in 
> list(range(5))[np.array([0, 1])]
> TypeError: only integer arrays with one element can be converted to an
> index
>
>
> the actual code uses pop on a python list with a return from
> np.where(...)[0]   that returns a one element int64 array
>
> Josef
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] NumPy 1.11.0b3 released.

[Charles R Harris]

On Mon, Feb 15, 2016 at 8:50 PM,  wrote:

>
>
> On Mon, Feb 15, 2016 at 10:46 PM,  wrote:
>
>
>>
>> On Fri, Feb 12, 2016 at 4:19 PM, Nathan Goldbaum 
>> wrote:
>>
>>> https://github.com/numpy/numpy/blob/master/doc/release/1.11.0-notes.rst
>>>
>>> On Fri, Feb 12, 2016 at 3:17 PM, Andreas Mueller 
>>> wrote:
>>>
 Hi.
 Where can I find the changelog?
 It would be good for us to know which changes are done one purpos
 without hunting through the issue tracker.

 Thanks,
 Andy


 On 02/09/2016 09:09 PM, Charles R Harris wrote:

 Hi All,

 I'm pleased to announce the release of NumPy 1.11.0b3. This beta
 contains additional bug fixes as well as limiting the number of
 FutureWarnings raised by assignment to masked array slices. One issue that
 remains to be decided is whether or not to postpone raising an error for
 floats used as indexes. Sources may be found on Sourceforge
  and
 both sources and OS X wheels are availble on pypi. Please test, hopefully
 this will be that last beta needed.

 As a note on problems encountered, twine uploads continue to fail for
 me, but there are still variations to try. The wheeluploader downloaded
 wheels as it should, but could not upload them, giving the error message
 "HTTPError: 413 Client Error: Request Entity Too Large for url:
 https://www.python.org/pypi";. Firefox
 also complains that http://wheels.scipy.org is incorrectly configured
 with an invalid certificate.

 Enjoy,

 Chuck


 ___
 NumPy-Discussion mailing 
 listNumPy-Discussion@scipy.orghttps://mail.scipy.org/mailman/listinfo/numpy-discussion



 ___
 NumPy-Discussion mailing list
 NumPy-Discussion@scipy.org
 https://mail.scipy.org/mailman/listinfo/numpy-discussion


>>>
>>> ___
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion@scipy.org
>>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>
> (try to send again)
>
>
>>
>> another indexing question:  (not covered by unit test but showed up in
>> examples in statsmodels)
>>
>>
>> This works in numpy at least 1.9.2 and 1.6.1   (python 2.7, and python
>> 3.4)
>>
>> >>> list(range(5))[np.array([0])]
>> 0
>>
>>
>>
>> on numpy 0.11.0b2   (I'm not yet at b3)   (python 3.4)
>>
>> I get the same exception as here but even if there is just one element
>>
>>
>> >>> list(range(5))[np.array([0, 1])]
>> Traceback (most recent call last):
>>   File "", line 1, in 
>> list(range(5))[np.array([0, 1])]
>> TypeError: only integer arrays with one element can be converted to an
>> index
>>
>
Looks like a misleading error message. Apparently it requires scalar arrays
(ndim == 0)

In [3]: list(range(5))[np.array(0)]
Out[3]: 0

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Subclassing ma.masked_array, code broken after version 1.9

[Charles R Harris]

On Mon, Feb 15, 2016 at 10:06 AM, Gutenkunst, Ryan N - (rgutenk) <
rgut...@email.arizona.edu> wrote:

> Thank Jonathan,
>
> Good to confirm this isn't something inappropriate I'm doing. I give up
> transparency here in my application, so I'll just work around it. I leave
> it up to wiser numpy heads as to whether it's worth altering these
> numpy.ma functions to enable subclassing.
>

There is  a known bug MaskedArrays that might account for this.  It will
hopefully be fixed in the next beta.

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] NumPy 1.11.0b3 released.

[josef.pktd]

On Mon, Feb 15, 2016 at 11:05 PM, Charles R Harris <
charlesr.har...@gmail.com> wrote:

>
>
> On Mon, Feb 15, 2016 at 8:50 PM,  wrote:
>
>>
>>
>> On Mon, Feb 15, 2016 at 10:46 PM,  wrote:
>>
>>
>>>
>>> On Fri, Feb 12, 2016 at 4:19 PM, Nathan Goldbaum 
>>> wrote:
>>>
 https://github.com/numpy/numpy/blob/master/doc/release/1.11.0-notes.rst

 On Fri, Feb 12, 2016 at 3:17 PM, Andreas Mueller 
 wrote:

> Hi.
> Where can I find the changelog?
> It would be good for us to know which changes are done one purpos
> without hunting through the issue tracker.
>
> Thanks,
> Andy
>
>
> On 02/09/2016 09:09 PM, Charles R Harris wrote:
>
> Hi All,
>
> I'm pleased to announce the release of NumPy 1.11.0b3. This beta
> contains additional bug fixes as well as limiting the number of
> FutureWarnings raised by assignment to masked array slices. One issue that
> remains to be decided is whether or not to postpone raising an error for
> floats used as indexes. Sources may be found on Sourceforge
>  and
> both sources and OS X wheels are availble on pypi. Please test, hopefully
> this will be that last beta needed.
>
> As a note on problems encountered, twine uploads continue to fail for
> me, but there are still variations to try. The wheeluploader downloaded
> wheels as it should, but could not upload them, giving the error message
> "HTTPError: 413 Client Error: Request Entity Too Large for url:
> https://www.python.org/pypi";. Firefox
> also complains that http://wheels.scipy.org is incorrectly configured
> with an invalid certificate.
>
> Enjoy,
>
> Chuck
>
>
> ___
> NumPy-Discussion mailing 
> listNumPy-Discussion@scipy.orghttps://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>

 ___
 NumPy-Discussion mailing list
 NumPy-Discussion@scipy.org
 https://mail.scipy.org/mailman/listinfo/numpy-discussion


>>>
>> (try to send again)
>>
>>
>>>
>>> another indexing question:  (not covered by unit test but showed up in
>>> examples in statsmodels)
>>>
>>>
>>> This works in numpy at least 1.9.2 and 1.6.1   (python 2.7, and python
>>> 3.4)
>>>
>>> >>> list(range(5))[np.array([0])]
>>> 0
>>>
>>>
>>>
>>> on numpy 0.11.0b2   (I'm not yet at b3)   (python 3.4)
>>>
>>> I get the same exception as here but even if there is just one element
>>>
>>>
>>> >>> list(range(5))[np.array([0, 1])]
>>> Traceback (most recent call last):
>>>   File "", line 1, in 
>>> list(range(5))[np.array([0, 1])]
>>> TypeError: only integer arrays with one element can be converted to an
>>> index
>>>
>>
> Looks like a misleading error message. Apparently it requires scalar
> arrays (ndim == 0)
>
> In [3]: list(range(5))[np.array(0)]
> Out[3]: 0
>


We have a newer version of essentially same function a second time that
uses squeeze and that seems to work fine.

Just to understand

Why does this depend on the numpy version?  I would have understood that
this always failed, but this code worked for several years.
https://github.com/statsmodels/statsmodels/issues/2817

Josef




>
> Chuck
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] NumPy 1.11.0b3 released.

[Charles R Harris]

On Mon, Feb 15, 2016 at 9:15 PM,  wrote:

>
>
> On Mon, Feb 15, 2016 at 11:05 PM, Charles R Harris <
> charlesr.har...@gmail.com> wrote:
>
>>
>>
>> On Mon, Feb 15, 2016 at 8:50 PM,  wrote:
>>
>>>
>>>
>>> On Mon, Feb 15, 2016 at 10:46 PM,  wrote:
>>>
>>>

 On Fri, Feb 12, 2016 at 4:19 PM, Nathan Goldbaum >>> > wrote:

> https://github.com/numpy/numpy/blob/master/doc/release/1.11.0-notes.rst
>
> On Fri, Feb 12, 2016 at 3:17 PM, Andreas Mueller 
> wrote:
>
>> Hi.
>> Where can I find the changelog?
>> It would be good for us to know which changes are done one purpos
>> without hunting through the issue tracker.
>>
>> Thanks,
>> Andy
>>
>>
>> On 02/09/2016 09:09 PM, Charles R Harris wrote:
>>
>> Hi All,
>>
>> I'm pleased to announce the release of NumPy 1.11.0b3. This beta
>> contains additional bug fixes as well as limiting the number of
>> FutureWarnings raised by assignment to masked array slices. One issue 
>> that
>> remains to be decided is whether or not to postpone raising an error for
>> floats used as indexes. Sources may be found on Sourceforge
>>  and
>> both sources and OS X wheels are availble on pypi. Please test, hopefully
>> this will be that last beta needed.
>>
>> As a note on problems encountered, twine uploads continue to fail for
>> me, but there are still variations to try. The wheeluploader downloaded
>> wheels as it should, but could not upload them, giving the error message
>> "HTTPError: 413 Client Error: Request Entity Too Large for url:
>> https://www.python.org/pypi";. Firefox
>> also complains that http://wheels.scipy.org is incorrectly
>> configured with an invalid certificate.
>>
>> Enjoy,
>>
>> Chuck
>>
>>
>> ___
>> NumPy-Discussion mailing 
>> listNumPy-Discussion@scipy.orghttps://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>

>>> (try to send again)
>>>
>>>

 another indexing question:  (not covered by unit test but showed up in
 examples in statsmodels)


 This works in numpy at least 1.9.2 and 1.6.1   (python 2.7, and python
 3.4)

 >>> list(range(5))[np.array([0])]
 0



 on numpy 0.11.0b2   (I'm not yet at b3)   (python 3.4)

 I get the same exception as here but even if there is just one element


 >>> list(range(5))[np.array([0, 1])]
 Traceback (most recent call last):
   File "", line 1, in 
 list(range(5))[np.array([0, 1])]
 TypeError: only integer arrays with one element can be converted to an
 index

>>>
>> Looks like a misleading error message. Apparently it requires scalar
>> arrays (ndim == 0)
>>
>> In [3]: list(range(5))[np.array(0)]
>> Out[3]: 0
>>
>
>
> We have a newer version of essentially same function a second time that
> uses squeeze and that seems to work fine.
>
> Just to understand
>
> Why does this depend on the numpy version?  I would have understood that
> this always failed, but this code worked for several years.
> https://github.com/statsmodels/statsmodels/issues/2817
>

It's part of the indexing cleanup.

In [2]: list(range(5))[np.array([0])]
/home/charris/.local/bin/ipython:1: VisibleDeprecationWarning: converting
an array with ndim > 0 to an index will result in an error in the future
  #!/usr/bin/python
Out[2]: 0

The use of multidimensional arrays as indexes is likely a coding error. Or
so we hope...

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] NumPy 1.11.0b3 released.

[josef.pktd]

On Mon, Feb 15, 2016 at 11:31 PM, Charles R Harris <
charlesr.har...@gmail.com> wrote:

>
>
> On Mon, Feb 15, 2016 at 9:15 PM,  wrote:
>
>>
>>
>> On Mon, Feb 15, 2016 at 11:05 PM, Charles R Harris <
>> charlesr.har...@gmail.com> wrote:
>>
>>>
>>>
>>> On Mon, Feb 15, 2016 at 8:50 PM,  wrote:
>>>


 On Mon, Feb 15, 2016 at 10:46 PM,  wrote:


>
> On Fri, Feb 12, 2016 at 4:19 PM, Nathan Goldbaum <
> nathan12...@gmail.com> wrote:
>
>>
>> https://github.com/numpy/numpy/blob/master/doc/release/1.11.0-notes.rst
>>
>> On Fri, Feb 12, 2016 at 3:17 PM, Andreas Mueller 
>> wrote:
>>
>>> Hi.
>>> Where can I find the changelog?
>>> It would be good for us to know which changes are done one purpos
>>> without hunting through the issue tracker.
>>>
>>> Thanks,
>>> Andy
>>>
>>>
>>> On 02/09/2016 09:09 PM, Charles R Harris wrote:
>>>
>>> Hi All,
>>>
>>> I'm pleased to announce the release of NumPy 1.11.0b3. This beta
>>> contains additional bug fixes as well as limiting the number of
>>> FutureWarnings raised by assignment to masked array slices. One issue 
>>> that
>>> remains to be decided is whether or not to postpone raising an error for
>>> floats used as indexes. Sources may be found on Sourceforge
>>>  and
>>> both sources and OS X wheels are availble on pypi. Please test, 
>>> hopefully
>>> this will be that last beta needed.
>>>
>>> As a note on problems encountered, twine uploads continue to fail
>>> for me, but there are still variations to try. The wheeluploader 
>>> downloaded
>>> wheels as it should, but could not upload them, giving the error message
>>> "HTTPError: 413 Client Error: Request Entity Too Large for url:
>>> https://www.python.org/pypi";. Firefox
>>> also complains that http://wheels.scipy.org is incorrectly
>>> configured with an invalid certificate.
>>>
>>> Enjoy,
>>>
>>> Chuck
>>>
>>>
>>> ___
>>> NumPy-Discussion mailing 
>>> listNumPy-Discussion@scipy.orghttps://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>>
>>> ___
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion@scipy.org
>>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
 (try to send again)


>
> another indexing question:  (not covered by unit test but showed up in
> examples in statsmodels)
>
>
> This works in numpy at least 1.9.2 and 1.6.1   (python 2.7, and python
> 3.4)
>
> >>> list(range(5))[np.array([0])]
> 0
>
>
>
> on numpy 0.11.0b2   (I'm not yet at b3)   (python 3.4)
>
> I get the same exception as here but even if there is just one element
>
>
> >>> list(range(5))[np.array([0, 1])]
> Traceback (most recent call last):
>   File "", line 1, in 
> list(range(5))[np.array([0, 1])]
> TypeError: only integer arrays with one element can be converted to an
> index
>

>>> Looks like a misleading error message. Apparently it requires scalar
>>> arrays (ndim == 0)
>>>
>>> In [3]: list(range(5))[np.array(0)]
>>> Out[3]: 0
>>>
>>
>>
>> We have a newer version of essentially same function a second time that
>> uses squeeze and that seems to work fine.
>>
>> Just to understand
>>
>> Why does this depend on the numpy version?  I would have understood that
>> this always failed, but this code worked for several years.
>> https://github.com/statsmodels/statsmodels/issues/2817
>>
>
> It's part of the indexing cleanup.
>
> In [2]: list(range(5))[np.array([0])]
> /home/charris/.local/bin/ipython:1: VisibleDeprecationWarning: converting
> an array with ndim > 0 to an index will result in an error in the future
>   #!/usr/bin/python
> Out[2]: 0
>
> The use of multidimensional arrays as indexes is likely a coding error. Or
> so we hope...
>

Thanks for the explanation


Or, it forces everyone to watch out for the color of the ducks :)

It's just a number, whether it's python scalar, numpy scalar, 1D or 2D.
And once we squeeze, we cannot iterate over it anymore.


This looks like the last problem with have in statsmodels master.
Part of the reason that 0.10 hurt quite a bit is that we are using in
statsmodels some of the grey zones so we don't have to commit to a specific
usage. Even if a user or developer tries a "weird" case, it works for most
of the results, but breaks in some unknown places.

(In the current case a cryptic exception would be raised

Re: [Numpy-discussion] NumPy 1.11.0b3 released.

[josef.pktd]

On Tue, Feb 16, 2016 at 12:09 AM,  wrote:

>
>
> On Mon, Feb 15, 2016 at 11:31 PM, Charles R Harris <
> charlesr.har...@gmail.com> wrote:
>
>>
>>
>> On Mon, Feb 15, 2016 at 9:15 PM,  wrote:
>>
>>>
>>>
>>> On Mon, Feb 15, 2016 at 11:05 PM, Charles R Harris <
>>> charlesr.har...@gmail.com> wrote:
>>>


 On Mon, Feb 15, 2016 at 8:50 PM,  wrote:

>
>
> On Mon, Feb 15, 2016 at 10:46 PM,  wrote:
>
>
>>
>> On Fri, Feb 12, 2016 at 4:19 PM, Nathan Goldbaum <
>> nathan12...@gmail.com> wrote:
>>
>>>
>>> https://github.com/numpy/numpy/blob/master/doc/release/1.11.0-notes.rst
>>>
>>> On Fri, Feb 12, 2016 at 3:17 PM, Andreas Mueller 
>>> wrote:
>>>
 Hi.
 Where can I find the changelog?
 It would be good for us to know which changes are done one purpos
 without hunting through the issue tracker.

 Thanks,
 Andy


 On 02/09/2016 09:09 PM, Charles R Harris wrote:

 Hi All,

 I'm pleased to announce the release of NumPy 1.11.0b3. This beta
 contains additional bug fixes as well as limiting the number of
 FutureWarnings raised by assignment to masked array slices. One issue 
 that
 remains to be decided is whether or not to postpone raising an error 
 for
 floats used as indexes. Sources may be found on Sourceforge
  and
 both sources and OS X wheels are availble on pypi. Please test, 
 hopefully
 this will be that last beta needed.

 As a note on problems encountered, twine uploads continue to fail
 for me, but there are still variations to try. The wheeluploader 
 downloaded
 wheels as it should, but could not upload them, giving the error 
 message
 "HTTPError: 413 Client Error: Request Entity Too Large for url:
 https://www.python.org/pypi";. Firefox
 also complains that http://wheels.scipy.org is incorrectly
 configured with an invalid certificate.

 Enjoy,

 Chuck


 ___
 NumPy-Discussion mailing 
 listNumPy-Discussion@scipy.orghttps://mail.scipy.org/mailman/listinfo/numpy-discussion



 ___
 NumPy-Discussion mailing list
 NumPy-Discussion@scipy.org
 https://mail.scipy.org/mailman/listinfo/numpy-discussion


>>>
>>> ___
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion@scipy.org
>>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>
> (try to send again)
>
>
>>
>> another indexing question:  (not covered by unit test but showed up
>> in examples in statsmodels)
>>
>>
>> This works in numpy at least 1.9.2 and 1.6.1   (python 2.7, and
>> python 3.4)
>>
>> >>> list(range(5))[np.array([0])]
>> 0
>>
>>
>>
>> on numpy 0.11.0b2   (I'm not yet at b3)   (python 3.4)
>>
>> I get the same exception as here but even if there is just one element
>>
>>
>> >>> list(range(5))[np.array([0, 1])]
>> Traceback (most recent call last):
>>   File "", line 1, in 
>> list(range(5))[np.array([0, 1])]
>> TypeError: only integer arrays with one element can be converted to
>> an index
>>
>
 Looks like a misleading error message. Apparently it requires scalar
 arrays (ndim == 0)

 In [3]: list(range(5))[np.array(0)]
 Out[3]: 0

>>>
>>>
>>> We have a newer version of essentially same function a second time that
>>> uses squeeze and that seems to work fine.
>>>
>>> Just to understand
>>>
>>> Why does this depend on the numpy version?  I would have understood that
>>> this always failed, but this code worked for several years.
>>> https://github.com/statsmodels/statsmodels/issues/2817
>>>
>>
>> It's part of the indexing cleanup.
>>
>> In [2]: list(range(5))[np.array([0])]
>> /home/charris/.local/bin/ipython:1: VisibleDeprecationWarning: converting
>> an array with ndim > 0 to an index will result in an error in the future
>>   #!/usr/bin/python
>> Out[2]: 0
>>
>> The use of multidimensional arrays as indexes is likely a coding error.
>> Or so we hope...
>>
>
> Thanks for the explanation
>
>
> Or, it forces everyone to watch out for the color of the ducks :)
>
> It's just a number, whether it's python scalar, numpy scalar, 1D or 2D.
> And once we squeeze, we cannot iterate over it anymore.
>
>
> This looks like the last problem with have in statsmodels master.
> Part of the reason that 0.10 hurt quite a bit is that we are using in
> statsmodels some of the

[Numpy-discussion] cycler v.0.10.0 released

[Thomas Caswell]

Folks,

I am happy to announce the next release of Cycler.

This will become the minimal version for the upcoming mpl v2.0 release.

http://matplotlib.org/cycler/

Feature release for `cycler`.  This release includes a number of new
features:

 - `Cycler` objecst learned to generate an `itertools.cycle` by calling
them, a-la a generator.
 - `Cycler` objects learned to change the name of a  key via the  new
`.change_key(old_key, new_key)` method.
 - `Cycler` objects learned how to compare each other and determine if they
are equal or not (`==`).
 - `Cycler` objects learned how to join another `Cycler` to be concatenated
into a singel longer `Cycler` via `concat` method of function.
 `A.concat(B)` or `concat(A, B)`.
 - The `cycler` factory function learned to construct a complex `Cycler`
from iterables provided as keyword arguments.
 - `Cycler` objects learn do show their insides with the `by_key` method
which returns a dictionary of lists (instead of an iterable of
dictionaries).

Tom
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Proposal to add `weights` to `np.percentile` and `np.median`

[Joseph Fox-Rabinovitz]

I would like to add a `weights` keyword to `np.partition`,
`np.percentile` and `np.median`. My reason for doing so is to to allow
`np.histogram` to process automatic bin selection with weights.
Currently, weights are not supported for the automatic bin selection
and would be difficult to support in `auto` mode without having
`np.percentile` support a `weights` keyword. I suspect that there are
many other uses for such a feature.

I have taken a preliminary look at the C implementation of the
partition functions that are the basis for `partition`, `median` and
`percentile`. I think that it would be possible to add versions (or
just extend the functionality of existing ones) that check the ratio
of the weights below the partition point to the total sum of the
weights instead of just counting elements.

One of the main advantages of such an implementation is that it would
allow any real weights to be handled correctly, not just integers.
Complex weights would not be supported.

The purpose of this email is to see if anybody objects, has ideas or
cares at all about this proposal before I spend a significant amount
of time working on it. For example, did I miss any functions in my
list?

Regards,

-Joe
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion