[Numpy-discussion] Floating point warnings/errors for comparisons, etc.?

[Sebastian Berg]

Hi all,

I am trying to clean up our floating point warning handling:

https://github.com/numpy/numpy/pull/19316

And an upcoming PR to remove most floating point error clearing.  There
are some things I am unsure about, though.

Part of why it got so confusing, is that GCC seemed to have
fixed/changed their behaviour for comparisons with NaN.  In GCC 7 it
did not give the warning, but GCC 8 does (correctly).


Comparison with NaN
---

IEEE says that the default comparisons should give warnings for
comparison with NaN (except == and !=).  And notes that an alternative
should be provided (C99 does this with `isless`, etc.).

We currently break this by suppressing invalid value warnings for all
comparisons (e.g. also `NaN > 0.`).

We can easily do either version (aside possibly compiler issues).
Making it give warnings had one test case fail for `quantile`, which
uses the pattern:

if not (np.all(q >= 0) and np.all(q <= 1)):
raise ValueError("bad q")

This would additionally (and first) give an "invalid value" warning and
require `np.errstate(invalid="ignore") to suppress it.

I dislike diverging from IEEE, but Python also does not warn for [1]:

float("nan") >= 0

and presumably the user either explicitly created the NaN or has seen a
warning earlier during computation when the NaN was first created.

(IEEE does not distinguish creating a new NaN with `0./0.` from a
comparison with `NaN > 0.` [2].  So we can't easily make this settable
via `np.errstate` or so.)

So, should we ignore the warning here?


Compiler Issues
---

Some compilers may get flags wrong.  How much effort do we want to
spend on details few users will notice?

My current problem is `1 % 0` and `divmod(1, 0)`.  The MacOS/clang CI
does not set the correct "invalid value" warning flag.  (The remainder
is NaN, so a new NaN is created and that should be indicated but the
C99 `fmod` does not set it.)


Signalling NaNs
---

I propose dropping any special concern for signalling NaNs.  Which
means they raise almost always.  Although, rarely we might suppress the
warning if we do it manually for normal NaNs [0].

We have two tests which check for behaviour on signalling NaNs. I could
not find having any logic to them besides someone being surprised at
signalling NaN behaviour at the time – not based on use-cases.
Even functions like `isnan` give a warning for signalling NaNs!

The "fix" for anyone having sNaN's is to convert them to qNaNs as early
as possible.  Which e.g. `np.positive(arr, out=arr)` should probably
do.  If this becomes an issue, maybe we could have an explicit ufunc.


Cheers,

Sebastian



[0] Mainly it seems SSE2 does not provide some non-error comparisons. 
So trying to avoid manually clearing errors might make some SSE code
considerable slower (e.g. `isfinite`, `np.min`).

[1] Probably Python just does not check the CPU warning flags

[2] https://www.gnu.org/software/libc/manual/html_node/FP-Exceptions.html

___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] NumPy's BLAS library on macOS?

[Jerry Morrison]

Would someone please answer installation questions about NumPy's BLAS on
macOS? I'm not finding the answers in the release notes
, the PR
 source, the docs
, or Stack Overflow
.


Q1. The NumPy 1.21.0 release note
 says "This change
enables the Accelerate Framework as an option on macOS." How to set
that option on/off?

Q2. How to determine if NumPy uses Accelerate vs. its internal copy of
OpenBLAS?
After installing a wheel, `numpy.show_config()` shows the openblas_info
library_dirs et al as '/usr/local/lib'. Neither '/usr/local/lib/' nor
'site-packages/numpy/' contains a *blas*.so library (for Python 3.8.* on
macOS 10.14.6) but the doc  says "The OpenBLAS
libraries are included in the wheel."

Q3. How to pip install NumPy 1.21.0 in a way that ensures it uses its
embedded OpenBLAS on macOS as on Linux? I'm aiming for as portable results
as possible. Or should we link NumPy to an external OpenBLAS via `pip
install numpy --no-binary numpy==1.21.0` with `~/.numpy-site.cfg`? (Ditto
for SciPy.)

Q4. Can the new NPY_* environment variables select specific BLAS & LAPACK
libraries through pip install, and perhaps install faster than building
NumPy, SciPy, etc. from source? How to do that?

Q5. Is NumPy's embedded OpenBLAS compiled by gcc or clang? Is that
controllable via `pip install`?

Thank you!
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] NumPy's BLAS library on macOS?

[Ralf Gommers]

On Wed, Jul 7, 2021 at 9:56 PM Jerry Morrison <
jerry.morrison+nu...@gmail.com> wrote:

> Would someone please answer installation questions about NumPy's BLAS on
> macOS? I'm not finding the answers in the release notes
> , the PR
>  source, the docs
> , or Stack Overflow
> .
>
>
> Q1. The NumPy 1.21.0 release note
>  says "This change
> enables the Accelerate Framework as an option on macOS." How to set
> that option on/off?
>

It's autodetected at build time. If you have no other BLAS installed, it
will be used. Or explicitly select it with NPY_BLAS_ORDER/NPY_LAPACK_ORDER


> Q2. How to determine if NumPy uses Accelerate vs. its internal copy of
> OpenBLAS?
> After installing a wheel, `numpy.show_config()` shows the openblas_info
> library_dirs et al as '/usr/local/lib'. Neither '/usr/local/lib/' nor
> 'site-packages/numpy/' contains a *blas*.so library (for Python 3.8.* on
> macOS 10.14.6) but the doc  says "The
> OpenBLAS libraries are included in the wheel."
>

It's a build-time option, you cannot select it at runtime.


> Q3. How to pip install NumPy 1.21.0 in a way that ensures it uses its
> embedded OpenBLAS on macOS as on Linux? I'm aiming for as portable results
> as possible. Or should we link NumPy to an external OpenBLAS via `pip
> install numpy --no-binary numpy==1.21.0` with `~/.numpy-site.cfg`? (Ditto
> for SciPy.)
>

If you install a wheel, you will always get the bundled OpenBLAS on every
platform for which we have binary wheels.


>
> Q4. Can the new NPY_* environment variables select specific BLAS & LAPACK
> libraries through pip install, and perhaps install faster than building
> NumPy, SciPy, etc. from source? How to do that?
>

This question seems a little bit confused. Those env vars just select the
BLAS/LAPACK library. It will not affect build time - we're never building
BLAS or LAPACK itself from source.


> Q5. Is NumPy's embedded OpenBLAS compiled by gcc or clang? Is that
> controllable via `pip install`?
>

gcc/gfortran. and no, you cannot control it through pip

Cheers,
Ralf


> Thank you!
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] NumPy's BLAS library on macOS?

[Jerry Morrison]

Got it!

*Summary:*
* Installing a numpy wheel (e.g. `pip install numpy==1.21.0`) uses its
embedded OpenBLAS on every platform that has a wheel.
  That OpenBLAS is always compiled with gcc/gfortran.
  In this case, `np.show_config()` reports `library_dirs =
['/usr/local/lib']` even though there's no libblas in that directory.

* Installing numpy from source (e.g. `pip install numpy==1.21.0 --no-binary
numpy)` looks for BLAS & LAPACK libraries at build time as influenced by
the environment vars NPY_BLAS_ORDER/NPY_LAPACK_ORDER or by the file
~/.numpy-site.cfg.
  On macOS, 'accelerate' is in the default search order after 'openblas'.
  On macOS < 11.3, importing numpy that's linked to Accelerate will detect
an Accelerate bug and raise a RuntimeError.

On Wed, Jul 7, 2021 at 1:32 PM Ralf Gommers  wrote:

>
>
> On Wed, Jul 7, 2021 at 9:56 PM Jerry Morrison <
> jerry.morrison+nu...@gmail.com> wrote:
>
>> Would someone please answer installation questions about NumPy's BLAS on
>> macOS? I'm not finding the answers in the release notes
>> , the PR
>>  source, the docs
>> , or Stack Overflow
>> .
>>
>>
>> Q1. The NumPy 1.21.0 release note
>>  says "This change
>> enables the Accelerate Framework as an option on macOS." How to set
>> that option on/off?
>>
>
> It's autodetected at build time. If you have no other BLAS installed, it
> will be used. Or explicitly select it with NPY_BLAS_ORDER/NPY_LAPACK_ORDER
>
>
>> Q2. How to determine if NumPy uses Accelerate vs. its internal copy of
>> OpenBLAS?
>> After installing a wheel, `numpy.show_config()` shows the openblas_info
>> library_dirs et al as '/usr/local/lib'. Neither '/usr/local/lib/' nor
>> 'site-packages/numpy/' contains a *blas*.so library (for Python 3.8.* on
>> macOS 10.14.6) but the doc  says "The
>> OpenBLAS libraries are included in the wheel."
>>
>
> It's a build-time option, you cannot select it at runtime.
>
>
>> Q3. How to pip install NumPy 1.21.0 in a way that ensures it uses its
>> embedded OpenBLAS on macOS as on Linux? I'm aiming for as portable results
>> as possible. Or should we link NumPy to an external OpenBLAS via `pip
>> install numpy --no-binary numpy==1.21.0` with `~/.numpy-site.cfg`? (Ditto
>> for SciPy.)
>>
>
> If you install a wheel, you will always get the bundled OpenBLAS on every
> platform for which we have binary wheels.
>
>
>>
>> Q4. Can the new NPY_* environment variables select specific BLAS & LAPACK
>> libraries through pip install, and perhaps install faster than building
>> NumPy, SciPy, etc. from source? How to do that?
>>
>
> This question seems a little bit confused. Those env vars just select the
> BLAS/LAPACK library. It will not affect build time - we're never building
> BLAS or LAPACK itself from source.
>
>
>> Q5. Is NumPy's embedded OpenBLAS compiled by gcc or clang? Is that
>> controllable via `pip install`?
>>
>
> gcc/gfortran. and no, you cannot control it through pip
>
> Cheers,
> Ralf
>
>
>> Thank you!
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] NumPy's BLAS library on macOS?

[Matti Picus]

On 8/7/21 2:23 am, Jerry Morrison wrote:

Got it!

*Summary:*
* Installing a numpy wheel (e.g. `pip install numpy==1.21.0`) uses its 
embedded OpenBLAS on every platform that has a wheel.

  That OpenBLAS is always compiled with gcc/gfortran.
  In this case, `np.show_config()` reports `library_dirs = 
['/usr/local/lib']` even though there's no libblas in that directory.


* Installing numpy from source (e.g. `pip install numpy==1.21.0 
--no-binary numpy)` looks for BLAS & LAPACK libraries at build time as 
influenced by the environment vars NPY_BLAS_ORDER/NPY_LAPACK_ORDER or 
by the file ~/.numpy-site.cfg.

  On macOS, 'accelerate' is in the default search order after 'openblas'.
  On macOS < 11.3, importing numpy that's linked to Accelerate will 
detect an Accelerate bug and raise a RuntimeError.



That seems correct, although admittedly show_config could do a better 
job. The problem is that not every BLAS implementation provides a 
convenient method to self-report.


It might be nice to document all this somewhere more permanent, the 
docstring for show_config might be a good place to start.



Matti

___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion