[Numpy-discussion] [RFC] - numpy/SVML appears to be poorly optimized

[Noah Goldstein]

The numpy SVML library: https://github.com/numpy/SVML

appears to be poorly optimized. Since its just the raw assembly dump
this also makes it quite difficult to improve (with either a better compiler
or by hand).

Some of the glaring issues are:
1. register allocation / spilling
2. rodata layouts / const-propagation of the values.
3. Very odd use of internal functions that really ought to be inlined.

Are these functions meant to be heavily optimized?

If so, are people open to patches that optimize them (either with new C
implementations are in the current assembly implementations).
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] ndenumerate for masked arrays

[Jouke Witteveen]

Hi all,

Using `np.ndenumerate` on masked arrays gives unexpected results. For instance,

next(np.ndenumerate(np.ma.masked_all((

does *not* equal `((), np.ma.masked)`, but instead contains whatever
value happens to be in the uninitialized data part of the masked
array.

Even better than yielding `((), np.ma.masked)` would be for
`ndenumerate` to skip masked entries altogether. After all, we are
also getting the index of the entries we are iterating over.

I went ahead and implemented this specialization of `ndenumerate` for
masked arrays: https://github.com/numpy/numpy/pull/20020
If there is anything I can do to advance this pull request, please let me know.

Regards,
- Jouke
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: [RFC] - numpy/SVML appears to be poorly optimized

[Devulapalli, Raghuveer]

They are meant to be optimized. Any contribution to improve them further is 
more than welcome. 

Raghuveer

-Original Message-
From: Noah Goldstein  
Sent: Thursday, November 4, 2021 10:46 AM
To: numpy-discussion@python.org
Subject: [Numpy-discussion] [RFC] - numpy/SVML appears to be poorly optimized

The numpy SVML library: https://github.com/numpy/SVML

appears to be poorly optimized. Since its just the raw assembly dump this also 
makes it quite difficult to improve (with either a better compiler or by hand).

Some of the glaring issues are:
1. register allocation / spilling
2. rodata layouts / const-propagation of the values.
3. Very odd use of internal functions that really ought to be inlined.

Are these functions meant to be heavily optimized?

If so, are people open to patches that optimize them (either with new C 
implementations are in the current assembly implementations).
___
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe 
send an email to numpy-discussion-le...@python.org 
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: raghuveer.devulapa...@intel.com
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: Revert the return of a single NaN for `np.unique` with floating point numbers?

[Ralf Gommers]

On Mon, Aug 2, 2021 at 7:49 PM Ralf Gommers  wrote:

>
>
> On Mon, Aug 2, 2021 at 7:04 PM Sebastian Berg 
> wrote:
>
>> Hi all,
>>
>> In NumPy 1.21, the output of `np.unique` changed in the presence of
>> multiple NaNs.  Previously, all NaNs were returned when we now only
>> return one (all NaNs were considered unique):
>>
>> a = np.array([1, 1, np.nan, np.nan, np.nan])
>>
>> Before 1.21:
>>
>> >>> np.unique(a)
>> array([ 1., nan, nan, nan])
>>
>> After 1.21:
>>
>> array([ 1., nan])
>>
>>
>> This change was requested in an old issue:
>>
>>  https://github.com/numpy/numpy/issues/2111
>>
>> And happened here:
>>
>>  https://github.com/numpy/numpy/pull/18070
>>
>> While, it has a release note.  I am not sure the change got the
>> attention it deserved.  This would be especially worrying if it is a
>> regression for anyone?
>>
>
> I think it's now the expected answer, not a regression. `unique` is not an
> elementwise function that needs to adhere to IEEE-754 where nan != nan. I
> can't remember reviewing this change, but it makes perfect sense to me.
>

It turns out there's a lot more to this story. The short summary of it is
that:
- use case wise it's still true that considering NaN's equal makes more
sense.
- scikit-learn has a custom implementation which removes duplicate NaN's:
https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/utils/_encode.py#L7
- all other array/tensor libraries match NumPy's old behavior (return
multiple NaN's)
- there's a performance and implementation cost for CuPy et al. to match
NumPy's changed behavior (multiple NaN's is the natural result, no extra
checks needed)
- There is also a significant performance cost for NumPy, ~25% for
small/medium-size arrays with no/few NaN's (say the benchmarks in
https://github.com/numpy/numpy/pull/18070 - which is a common case, and
that's not negligible like the PR description claims.
- the "single NaN" behavior is easy to get as a utility function on top of
the "multiple NaN' (like scikit-learn does), the opposite is much harder
- for the array API standard, the final decision was to go with the
"multiple NaN' behavior, so we'll need that in `numpy.array_api`.
- more discussion in https://github.com/data-apis/array-api/issues/249

It would be good to make up our minds before 1.22.0. Two choices:
1. We can leave it as it is now and work around the issue in
`numpy.array_api`. It would also require CuPy and others to make changes,
which probably cost performance.
2. We can revert it in 1.22.0, and possibly in 1.21.5 if such a release
will be made.

There is something to say for either option. I'd personally lean slightly
towards reverting because of both consistency with other implementations
and performance. But it seems like there's no optimal solution here, it's a
fairly hairy issue.

Thoughts?

Ralf
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: Revert the return of a single NaN for `np.unique` with floating point numbers?

[Juan Nunez-Iglesias]

I agree with the argument to revert. Consistency among libraries should be near 
the top of the list of priorities, as should performance. Whether the new 
behaviour "makes more sense", meanwhile, is debatable.

On Fri, 5 Nov 2021, at 4:08 PM, Ralf Gommers wrote:
> 
> 
> On Mon, Aug 2, 2021 at 7:49 PM Ralf Gommers  wrote:
>> 
>> 
>> On Mon, Aug 2, 2021 at 7:04 PM Sebastian Berg  
>> wrote:
>>> Hi all,
>>> 
>>> In NumPy 1.21, the output of `np.unique` changed in the presence of
>>> multiple NaNs.  Previously, all NaNs were returned when we now only
>>> return one (all NaNs were considered unique):
>>> 
>>> a = np.array([1, 1, np.nan, np.nan, np.nan])
>>> 
>>> Before 1.21:
>>> 
>>> >>> np.unique(a)
>>> array([ 1., nan, nan, nan])
>>> 
>>> After 1.21:
>>> 
>>> array([ 1., nan])
>>> 
>>> 
>>> This change was requested in an old issue:
>>> 
>>>  https://github.com/numpy/numpy/issues/2111
>>> 
>>> And happened here:
>>> 
>>>  https://github.com/numpy/numpy/pull/18070
>>> 
>>> While, it has a release note.  I am not sure the change got the
>>> attention it deserved.  This would be especially worrying if it is a
>>> regression for anyone?
>> 
>> I think it's now the expected answer, not a regression. `unique` is not an 
>> elementwise function that needs to adhere to IEEE-754 where nan != nan. I 
>> can't remember reviewing this change, but it makes perfect sense to me.
> 
> It turns out there's a lot more to this story. The short summary of it is 
> that:
> - use case wise it's still true that considering NaN's equal makes more sense.
> - scikit-learn has a custom implementation which removes duplicate NaN's: 
> https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/utils/_encode.py#L7
> - all other array/tensor libraries match NumPy's old behavior (return 
> multiple NaN's)
> - there's a performance and implementation cost for CuPy et al. to match 
> NumPy's changed behavior (multiple NaN's is the natural result, no extra 
> checks needed)
> - There is also a significant performance cost for NumPy, ~25% for 
> small/medium-size arrays with no/few NaN's (say the benchmarks in 
> https://github.com/numpy/numpy/pull/18070 - which is a common case, and 
> that's not negligible like the PR description claims.
> - the "single NaN" behavior is easy to get as a utility function on top of 
> the "multiple NaN' (like scikit-learn does), the opposite is much harder
> - for the array API standard, the final decision was to go with the "multiple 
> NaN' behavior, so we'll need that in `numpy.array_api`.
> - more discussion in https://github.com/data-apis/array-api/issues/249
> 
> It would be good to make up our minds before 1.22.0. Two choices:
> 1. We can leave it as it is now and work around the issue in 
> `numpy.array_api`. It would also require CuPy and others to make changes, 
> which probably cost performance.
> 2. We can revert it in 1.22.0, and possibly in 1.21.5 if such a release will 
> be made.
> 
> There is something to say for either option. I'd personally lean slightly 
> towards reverting because of both consistency with other implementations and 
> performance. But it seems like there's no optimal solution here, it's a 
> fairly hairy issue.
> 
> Thoughts?
> 
> Ralf
> 
> ___
> NumPy-Discussion mailing list -- numpy-discussion@python.org
> To unsubscribe send an email to numpy-discussion-le...@python.org
> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
> Member address: j...@fastmail.com
> 
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] ANN: SciPy 1.7.2

[Tyler Reddy]

Hi all,

On behalf of the SciPy development team I'm pleased to announce the release
of SciPy 1.7.2, which is a bug fix release that includes wheels for Python
3.10 on many platforms. Many thanks to upstream developers in the ecosystem
for their assistance in making this possible.

Sources and binary wheels can be found at:
https://pypi.org/project/scipy/
and at: https://github.com/scipy/scipy/releases/tag/v1.7.2



One of a few ways to install this release with pip:

pip install scipy==1.7.2


SciPy 1.7.2 Release Notes


SciPy 1.7.2 is a bug-fix release with no new features
compared to 1.7.1. Notably, the release includes wheels
for Python 3.10, and wheels are now built with a newer
version of OpenBLAS, 0.3.17. Python 3.10 wheels are provided
for MacOS x86_64 (thin, not universal2 or arm64 at this time),
and Windows/Linux 64-bit. Many wheels are now built with newer
versions of manylinux, which may require newer versions of pip.

Authors
==

* Peter Bell
* da-woods +
* Isuru Fernando
* Ralf Gommers
* Matt Haberland
* Nicholas McKibben
* Ilhan Polat
* Judah Rand +
* Tyler Reddy
* Pamphile Roy
* Charles Harris
* Matti Picus
* Hugo van Kemenade
* Jacob Vanderplas

A total of 14 people contributed to this release.
People with a "+" by their names contributed a patch for the first time.
This list of names is automatically generated, and may not be fully
complete.

Issues closed for 1.7.2
--

* `#6019 `__: minimize_scalar
doesn't seem to honor "disp" option
* `#14321 `__: BUG: Indexing
of CSR matrices with many rows is much slower than...
* `#14465 `__: BUG: n-d
interpolation parameter provided to geometric_slerp
* `#14599 `__: SciPy 1.7
builds as zipped egg, ruining imports
* `#14606 `__: BUG: crash /
core dump when calling scipy.stats.beta.ppf with...
* `#14732 `__: CI, TST:
pre-release failures for scipy/interpolate/tests/test_rbfinterp.py
* `#14802 `__: CI: Azure Main
coverage job failure
* `#14829 `__: macOS CI
failing with \`ld: library not found for -lSystem\`
* `#14887 `__: BUG:
scipy.stats.multivariate_normal.logpdf mutates some inputs

Pull requests for 1.7.2
--

* `#14207 `__: DOC: stats:
remove 'Methods' section from \`binomtest\` docstring...
* `#14316 `__: MAINT: Update
\`openblas_support.py\` to support Apple Silicon
* `#14323 `__: BUG: Speed up
sparse compressed indexing with very many rows
* `#14333 `__: MAINT: Use
/usr/bin/linux32 so that sysconfig.get_platform()...
* `#14478 `__: BUG:
geometric_slerp t ndim guard
* `#14605 `__: MAINT: Skip some
interpolative decomposition tests
* `#14616 `__: REL: update build
dependency versions in pyproject.toml for 1.7.2
* `#14618 `__: FIX: raise
RuntimeWarning when Boost evaluation_error is encountered
* `#14672 `__: BLD: add
\`zip_safe=False\` to the \`setup()\` call
* `#14791 `__: MAINT: SciPy
1.7.2 prep/backports
* `#14803 `__: MAINT: disable
include/source coverage warning.
* `#14813 `__: Added missing
np.import_array()
* `#14831 `__: CI: Add stdlib to
LD_LIBRARY_PATH
* `#14893 `__: BUG: Fix
alignment errors due to relaxed stride checking
* `#14897 `__: BUG: avoid
mutating inputs in multivariate distributions
* `#14921 `__: MAINT: "backport"
3.10 support
* `#14937 `__: MAINT: backports
for 1.7.2, plus update Pythran min version to...
* `#14938 `__: TST: silence test
failures on macOS for \`beta.ppf\` overflow

Checksums
=

MD5
~~~

45913b1797379cc2e258b0d544276cc8
 scipy-1.7.2-cp310-cp310-macosx_10_9_x86_64.whl
443a83b81b59b627aa63bdfaa66d8f62
 scipy-1.7.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
cf5762eba17598002d2d310a5a6b6f9f
 scipy-1.7.2-cp310-cp310-manylinux_2_17_