[Numpy-discussion] Deprecating matrices.

[Charles R Harris]

Hi All,

Just throwing this click bait out for discussion. Now that the `@` operator
is available and things seem to be moving towards Python 3, especially in
the classroom, we should consider the real possibility of deprecating the
matrix type and later removing it. No doubt there are old scripts that
require them, but older versions of numpy are available for those who need
to run old scripts.

Thoughts?

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Deprecating matrices.

[Ralf Gommers]

On Tue, Jan 3, 2017 at 2:36 PM, Charles R Harris 
wrote:

> Hi All,
>
> Just throwing this click bait out for discussion. Now that the `@`
> operator is available and things seem to be moving towards Python 3,
> especially in the classroom, we should consider the real possibility of
> deprecating the matrix type and later removing it. No doubt there are old
> scripts that require them, but older versions of numpy are available for
> those who need to run old scripts.
>
> Thoughts?
>

Clearly deprecate in the docs now, and warn only later imho. We can't warn
before we have a good solution for scipy.sparse matrices, which have matrix
semantics and return matrix instances.

Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Deprecating matrices.

[josef . pktd]

On Mon, Jan 2, 2017 at 9:00 PM, Ralf Gommers  wrote:

>
>
> On Tue, Jan 3, 2017 at 2:36 PM, Charles R Harris <
> charlesr.har...@gmail.com> wrote:
>
>> Hi All,
>>
>> Just throwing this click bait out for discussion. Now that the `@`
>> operator is available and things seem to be moving towards Python 3,
>> especially in the classroom, we should consider the real possibility of
>> deprecating the matrix type and later removing it. No doubt there are old
>> scripts that require them, but older versions of numpy are available for
>> those who need to run old scripts.
>>
>> Thoughts?
>>
>
> Clearly deprecate in the docs now, and warn only later imho. We can't warn
> before we have a good solution for scipy.sparse matrices, which have matrix
> semantics and return matrix instances.
>
> Ralf
>

How about dropping python 2 support at the same time, then we can all be in
a @ world.

Josef



>
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Default type for functions that accumulate integers

[Charles R Harris]

Hi All,

Currently functions like trace use the C long type as the default
accumulator for integer types of lesser precision:

dtype : dtype, optional
> Determines the data-type of the returned array and of the accumulator
> where the elements are summed. If dtype has the value None and `a` is
> of integer type of precision less than the default integer
> precision, then the default integer precision is used. Otherwise,
> the precision is the same as that of `a`.
>

The problem with this is that the precision of long varies with the
platform so that the result varies,  see gh-8433
 for a complaint about this.
There are two possible alternatives that seem reasonable to me:


   1. Use 32 bit accumulators on 32 bit platforms and 64 bit accumulators
   on 64 bit platforms.
   2. Always use 64 bit accumulators.

Thoughts?

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Default type for functions that accumulate integers

[Nathaniel Smith]

On Mon, Jan 2, 2017 at 6:27 PM, Charles R Harris
 wrote:
> Hi All,
>
> Currently functions like trace use the C long type as the default
> accumulator for integer types of lesser precision:
>
>> dtype : dtype, optional
>> Determines the data-type of the returned array and of the accumulator
>> where the elements are summed. If dtype has the value None and `a` is
>> of integer type of precision less than the default integer
>> precision, then the default integer precision is used. Otherwise,
>> the precision is the same as that of `a`.
>
>
> The problem with this is that the precision of long varies with the platform
> so that the result varies,  see gh-8433 for a complaint about this. There
> are two possible alternatives that seem reasonable to me:
>
> Use 32 bit accumulators on 32 bit platforms and 64 bit accumulators on 64
> bit platforms.
> Always use 64 bit accumulators.

This is a special case of a more general question: right now we use
the default integer precision (i.e., what you get from np.array([1]),
or np.arange, or np.dtype(int)), and it turns out that the default
integer precision itself varies in confusing ways, and this is a
common source of bugs. Specifically: right now it's 32-bit on 32-bit
builds, and 64-bit on 64-bit builds, except on Windows where it's
always 32-bit. This matches the default precision of Python 2 'int'.

So some options include:
- make the default integer precision 64-bits everywhere
- make the default integer precision 32-bits on 32-bit systems, and
64-bits on 64-bit systems (including Windows)
- leave the default integer precision the same, but make accumulators
64-bits everywhere
- leave the default integer precision the same, but make accumulators
64-bits on 64-bit systems (including Windows)
- ...

Given the prevalence of 64-bit systems these days, and the fact that
the current setup makes it very easy to write code that seems to work
when tested on a 64-bit system but that silently returns incorrect
results on 32-bit systems, it sure would be nice if we could switch to
a 64-bit default everywhere. (You could still get 32-bit integers, of
course, you'd just have to ask for them explicitly.)

Things we'd need to know more about before making a decision:
- compatibility: if we flip this switch, how much code breaks? In
general correct numpy-using code has to be prepared to handle
np.dtype(int) being 64-bits, and in fact there might be more code that
accidentally assumes that np.dtype(int) is always 64-bits than there
is code that assumes it is always 32-bits. But that's theory; to know
how bad this is we would need to try actually running some projects
test suites and see whether they break or not.
- speed: there's probably some cost to using 64-bit integers on 32-bit
systems; how big is the penalty in practice?

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Deprecating matrices.

[Nathaniel Smith]

On Mon, Jan 2, 2017 at 6:26 PM,   wrote:
>
>
> On Mon, Jan 2, 2017 at 9:00 PM, Ralf Gommers  wrote:
>>
>>
>>
>> On Tue, Jan 3, 2017 at 2:36 PM, Charles R Harris
>>  wrote:
>>>
>>> Hi All,
>>>
>>> Just throwing this click bait out for discussion. Now that the `@`
>>> operator is available and things seem to be moving towards Python 3,
>>> especially in the classroom, we should consider the real possibility of
>>> deprecating the matrix type and later removing it. No doubt there are old
>>> scripts that require them, but older versions of numpy are available for
>>> those who need to run old scripts.
>>>
>>> Thoughts?
>>
>>
>> Clearly deprecate in the docs now, and warn only later imho. We can't warn
>> before we have a good solution for scipy.sparse matrices, which have matrix
>> semantics and return matrix instances.
>>
>> Ralf
>
>
> How about dropping python 2 support at the same time, then we can all be in
> a @ world.
>
> Josef

Let's not yoke together two (mostly) unrelated controversial
discussions? I doubt we'll be able to remove either Python 2 or matrix
support before 2020 at the earliest, so the discussion now is just
about how to communicate to users that they should not be using
'matrix'.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Deprecating matrices.

[Charles R Harris]

On Mon, Jan 2, 2017 at 7:26 PM,  wrote:

>
>
> On Mon, Jan 2, 2017 at 9:00 PM, Ralf Gommers 
> wrote:
>
>>
>>
>> On Tue, Jan 3, 2017 at 2:36 PM, Charles R Harris <
>> charlesr.har...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> Just throwing this click bait out for discussion. Now that the `@`
>>> operator is available and things seem to be moving towards Python 3,
>>> especially in the classroom, we should consider the real possibility of
>>> deprecating the matrix type and later removing it. No doubt there are old
>>> scripts that require them, but older versions of numpy are available for
>>> those who need to run old scripts.
>>>
>>> Thoughts?
>>>
>>
>> Clearly deprecate in the docs now, and warn only later imho. We can't
>> warn before we have a good solution for scipy.sparse matrices, which have
>> matrix semantics and return matrix instances.
>>
>> Ralf
>>
>
> How about dropping python 2 support at the same time, then we can all be
> in a @ world.
>
>
The "@" operator works with matrices already, what causes problems is the
combination  of matrices with 1-D arrays. That can be fixed, I think. The
big problem is probably the lack of "@" in Python 2.7. I wonder if there is
any chance of getting it backported to 2.7 before support is dropped in
2020? I expect it would be a fight, but I also suspect it would not be
difficult to do if the proposal was accepted. Then at some future date
sparse could simply start returning arrays.

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Deprecating matrices.

[Charles R Harris]

On Mon, Jan 2, 2017 at 8:12 PM, Charles R Harris 
wrote:

>
>
> On Mon, Jan 2, 2017 at 7:26 PM,  wrote:
>
>>
>>
>> On Mon, Jan 2, 2017 at 9:00 PM, Ralf Gommers 
>> wrote:
>>
>>>
>>>
>>> On Tue, Jan 3, 2017 at 2:36 PM, Charles R Harris <
>>> charlesr.har...@gmail.com> wrote:
>>>
 Hi All,

 Just throwing this click bait out for discussion. Now that the `@`
 operator is available and things seem to be moving towards Python 3,
 especially in the classroom, we should consider the real possibility of
 deprecating the matrix type and later removing it. No doubt there are old
 scripts that require them, but older versions of numpy are available for
 those who need to run old scripts.

 Thoughts?

>>>
>>> Clearly deprecate in the docs now, and warn only later imho. We can't
>>> warn before we have a good solution for scipy.sparse matrices, which have
>>> matrix semantics and return matrix instances.
>>>
>>> Ralf
>>>
>>
>> How about dropping python 2 support at the same time, then we can all be
>> in a @ world.
>>
>>
> The "@" operator works with matrices already, what causes problems is the
> combination  of matrices with 1-D arrays. That can be fixed, I think. The
> big problem is probably the lack of "@" in Python 2.7. I wonder if there is
> any chance of getting it backported to 2.7 before support is dropped in
> 2020? I expect it would be a fight, but I also suspect it would not be
> difficult to do if the proposal was accepted. Then at some future date
> sparse could simply start returning arrays.
>

Hmm, matrix-scalar multiplication will be a problem.

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Deprecating matrices.

[Nathaniel Smith]

On Mon, Jan 2, 2017 at 7:12 PM, Charles R Harris
 wrote:
>
>
> On Mon, Jan 2, 2017 at 7:26 PM,  wrote:
[...]
>> How about dropping python 2 support at the same time, then we can all be
>> in a @ world.
>>
>
> The "@" operator works with matrices already, what causes problems is the
> combination  of matrices with 1-D arrays. That can be fixed, I think. The
> big problem is probably the lack of "@" in Python 2.7. I wonder if there is
> any chance of getting it backported to 2.7 before support is dropped in
> 2020? I expect it would be a fight, but I also suspect it would not be
> difficult to do if the proposal was accepted. Then at some future date
> sparse could simply start returning arrays.

Unfortunately the chance of Python 2.7 adding support for "@" is best
expressed as a denormal.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Deprecating matrices.

[Charles R Harris]

On Mon, Jan 2, 2017 at 8:29 PM, Nathaniel Smith  wrote:

> On Mon, Jan 2, 2017 at 7:12 PM, Charles R Harris
>  wrote:
> >
> >
> > On Mon, Jan 2, 2017 at 7:26 PM,  wrote:
> [...]
> >> How about dropping python 2 support at the same time, then we can all be
> >> in a @ world.
> >>
> >
> > The "@" operator works with matrices already, what causes problems is the
> > combination  of matrices with 1-D arrays. That can be fixed, I think. The
> > big problem is probably the lack of "@" in Python 2.7. I wonder if there
> is
> > any chance of getting it backported to 2.7 before support is dropped in
> > 2020? I expect it would be a fight, but I also suspect it would not be
> > difficult to do if the proposal was accepted. Then at some future date
> > sparse could simply start returning arrays.
>
> Unfortunately the chance of Python 2.7 adding support for "@" is best
> expressed as a denormal.
>

That's what I figured ;) Hmm, matrices would work fine with the current
combination of '*' (works for scalar muiltiplication) and '@' (works for
matrices). So for Python3 code currently written for matrices can be
reformed to be array compatible. But '@' for Python 2.7 would sure help...

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Deprecating matrices.

[Nathaniel Smith]

On Mon, Jan 2, 2017 at 7:54 PM, Charles R Harris
 wrote:
>
>
> On Mon, Jan 2, 2017 at 8:29 PM, Nathaniel Smith  wrote:
>>
>> On Mon, Jan 2, 2017 at 7:12 PM, Charles R Harris
>>  wrote:
>> >
>> >
>> > On Mon, Jan 2, 2017 at 7:26 PM,  wrote:
>> [...]
>> >> How about dropping python 2 support at the same time, then we can all
>> >> be
>> >> in a @ world.
>> >>
>> >
>> > The "@" operator works with matrices already, what causes problems is
>> > the
>> > combination  of matrices with 1-D arrays. That can be fixed, I think.
>> > The
>> > big problem is probably the lack of "@" in Python 2.7. I wonder if there
>> > is
>> > any chance of getting it backported to 2.7 before support is dropped in
>> > 2020? I expect it would be a fight, but I also suspect it would not be
>> > difficult to do if the proposal was accepted. Then at some future date
>> > sparse could simply start returning arrays.
>>
>> Unfortunately the chance of Python 2.7 adding support for "@" is best
>> expressed as a denormal.
>
>
> That's what I figured ;) Hmm, matrices would work fine with the current
> combination of '*' (works for scalar muiltiplication) and '@' (works for
> matrices). So for Python3 code currently written for matrices can be
> reformed to be array compatible. But '@' for Python 2.7 would sure help...

I mean, it can just use arrays + dot(). It's not as elegant as '@',
but given that almost everyone has already switched it's clearly not
*that* bad...

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion