Re: [Numpy-discussion] Floor divison on int returns float

[Antoine Pitrou]

On Wed, 13 Apr 2016 15:49:15 -0600
Charles R Harris  wrote:
> 
> I looked this up once, `C` returns unsigned in the scalar case when both
> operands have the same width. See Usual Arithmetic Conversions
> .
> I think that is not a bad choice, but there is the back compatibility
> problem, plus it is a bit exceptional.

It may be a worse choice for Python. In the original use case (indexing
with an integer), losing the sign is a bug since negative indices have
a well-defined meaning in Python. This is a far more likely issue than
magnitude loss on a 64-bit integer.

In Numba, we decided that combining signed and unsigned would return
signed (see
http://numba.pydata.org/numba-doc/dev/proposals/integer-typing.html#proposal-predictable-width-conserving-typing).

Regards

Antoine.


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] linux wheels coming soon

[Jens Nielsen]

I have tried testing the wheels in a project that runs tests on Travis's
Trusty infrastructure which. The wheels work great for python 3.5 and saves
us several minuts of runtime.

However, I am having trouble using the wheels on python 2.7 on the same
Trusty machines. It seems to be because the wheels are tagged as
cp27-cp27mu  (numpy-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl) where as
pip.pep425tags.get_abi_tag()
returns cp27m on this particular python version. (Stock python 2.7
installed on Travis 14.04 VMs) Any chance of a cp27m compatible wheel build?

best
Jens

On Thu, 14 Apr 2016 at 01:46 Nathaniel Smith  wrote:

> https://github.com/numpy/numpy/issues/7545
>
> On Wed, Apr 13, 2016 at 3:38 PM, Nathaniel Smith  wrote:
> > I can reproduce in self-compiled 1.9, so it's not a new bug.
> >
> > I think something's going wrong with NPY_SIGINT_ON / NPY_SIGINT_OFF,
> > where our special sigint handler is getting left in place even after
> > our code finishes running.
> >
> > Skimming the code, my best guess is that this is due to a race
> > condition in how we save/restore the original signal handler, when
> > multiple threads are running numpy fftpack code at the same time (and
> > thus using NPY_SIGINT_{ON,OFF} from multiple threads).
> >
> > -n
> >
> > On Wed, Apr 13, 2016 at 1:47 PM, Matthew Brett 
> wrote:
> >> On Wed, Apr 13, 2016 at 1:29 PM, Oscar Benjamin
> >>  wrote:
> >>> On 13 April 2016 at 20:15, Matthew Brett 
> wrote:
>  Done.  If y'all are on linux, and you have pip >= 8.11,  you should
>  now see this kind of thing:
> >>>
> >>> That's fantastic. Thanks Matt!
> >>>
> >>> I just test installed this and ran numpy.test(). All tests passed but
> >>> then I got a segfault at the end by (semi-accidentally) hitting Ctrl-C
> >>> at the prompt:
> >>>
> >>> $ python
> >>> Python 2.7.9 (default, Apr  2 2015, 15:33:21)
> >>> [GCC 4.9.2] on linux2
> >>> Type "help", "copyright", "credits" or "license" for more information.
> >> import numpy
> >> numpy.test()
> >>> Running unit tests for numpy
> >>> 
> >>> Ran 5781 tests in 72.238s
> >>>
> >>> OK (KNOWNFAIL=6, SKIP=15)
> >>> 
> >> Segmentation fault (core dumped)
> >>>
> >>> It was stopped at the prompt and then I did Ctrl-C and then the
> >>> seg-fault message.
> >>>
> >>> $ uname -a
> >>> Linux vnwulf 3.19.0-15-generic #15-Ubuntu SMP Thu Apr 16 23:32:37 UTC
> >>> 2015 x86_64 x86_64 x86_64 GNU/Linux
> >>> $ lsb_release -a
> >>> No LSB modules are available.
> >>> Distributor ID:Ubuntu
> >>> Description:Ubuntu 15.04
> >>> Release:15.04
> >>> Codename:vivid
> >>>
> >>
> >> Thanks so much for testing - that's very useful.
> >>
> >> I get the same thing on my Debian Sid machine.
> >>
> >> Actually I also get the same thing with a local compile against Debian
> >> ATLAS, here's the stack trace after:
> >>
> > import numpy; numpy.test()
> > # Ctrl-C
> >>
> >> https://gist.github.com/f6d8fb42f24689b39536a2416d717056
> >>
> >> Do you get this as well?
> >>
> >> Cheers,
> >>
> >> Matthew
> >> ___
> >> NumPy-Discussion mailing list
> >> NumPy-Discussion@scipy.org
> >> https://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
> >
> > --
> > Nathaniel J. Smith -- https://vorpus.org
>
>
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] linux wheels coming soon

[Matthew Brett]

Hi,

On Thu, Apr 14, 2016 at 8:02 AM, Jens Nielsen  wrote:
> I have tried testing the wheels in a project that runs tests on Travis's
> Trusty infrastructure which. The wheels work great for python 3.5 and saves
> us several minuts of runtime.
>
> However, I am having trouble using the wheels on python 2.7 on the same
> Trusty machines. It seems to be because the wheels are tagged as cp27-cp27mu
> (numpy-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl) where as
> pip.pep425tags.get_abi_tag() returns cp27m on this particular python
> version. (Stock python 2.7 installed on Travis 14.04 VMs) Any chance of a
> cp27m compatible wheel build?

Ouch - do you know where travis-ci's Python 2.7 comes from?I see
that the standard apt-get install -y python is a wide (mu) build...

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] linux wheels coming soon

[Benjamin Root]

Are we going to have to have documentation somewhere making it clear that
the numpy wheel shouldn't be used in a conda environment? Not that I would
expect this issue to come up all that often, but I could imagine a scenario
where a non-scientist is simply using a base conda distribution because
that is what IT put on their system. Then they do "pip install ipython"
that indirectly brings in numpy (through the matplotlib dependency), and
end up with an incompatible numpy because they would have been linked
against different pythons?

Or is this not an issue?

Ben Root


On Thu, Apr 14, 2016 at 2:04 PM, Matthew Brett 
wrote:

> Hi,
>
> On Thu, Apr 14, 2016 at 8:02 AM, Jens Nielsen 
> wrote:
> > I have tried testing the wheels in a project that runs tests on Travis's
> > Trusty infrastructure which. The wheels work great for python 3.5 and
> saves
> > us several minuts of runtime.
> >
> > However, I am having trouble using the wheels on python 2.7 on the same
> > Trusty machines. It seems to be because the wheels are tagged as
> cp27-cp27mu
> > (numpy-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl) where as
> > pip.pep425tags.get_abi_tag() returns cp27m on this particular python
> > version. (Stock python 2.7 installed on Travis 14.04 VMs) Any chance of a
> > cp27m compatible wheel build?
>
> Ouch - do you know where travis-ci's Python 2.7 comes from?I see
> that the standard apt-get install -y python is a wide (mu) build...
>
> Cheers,
>
> Matthew
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] linux wheels coming soon

[Matthew Brett]

Hi,

On Thu, Apr 14, 2016 at 11:11 AM, Benjamin Root  wrote:
> Are we going to have to have documentation somewhere making it clear that
> the numpy wheel shouldn't be used in a conda environment? Not that I would
> expect this issue to come up all that often, but I could imagine a scenario
> where a non-scientist is simply using a base conda distribution because that
> is what IT put on their system. Then they do "pip install ipython" that
> indirectly brings in numpy (through the matplotlib dependency), and end up
> with an incompatible numpy because they would have been linked against
> different pythons?
>
> Or is this not an issue?

I'm afraid I don't know conda at all, but I'm guessing that pip will
not install numpy when it is installed via conda.

So the potential difference is that, pre-wheel, if numpy was not
installed in your conda environment, then pip would build numpy from
source, whereas now you'll get a binary install.

I _think_ that Python's binary API specification
(pip.pep425tags.get_abi_tag()) should prevent pip from installing an
incompatible wheel.  Are there any conda experts out there who can
give more detail, or more convincing assurance?

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] linux wheels coming soon

[G Young]

Actually, conda pip will install the wheels that you put up.  The good news
is: they all (by which I mean *numpy* and *scipy* both on 2.7 and 3.5) pass!

On Thu, Apr 14, 2016 at 7:26 PM, Matthew Brett 
wrote:

> Hi,
>
> On Thu, Apr 14, 2016 at 11:11 AM, Benjamin Root 
> wrote:
> > Are we going to have to have documentation somewhere making it clear that
> > the numpy wheel shouldn't be used in a conda environment? Not that I
> would
> > expect this issue to come up all that often, but I could imagine a
> scenario
> > where a non-scientist is simply using a base conda distribution because
> that
> > is what IT put on their system. Then they do "pip install ipython" that
> > indirectly brings in numpy (through the matplotlib dependency), and end
> up
> > with an incompatible numpy because they would have been linked against
> > different pythons?
> >
> > Or is this not an issue?
>
> I'm afraid I don't know conda at all, but I'm guessing that pip will
> not install numpy when it is installed via conda.
>
> So the potential difference is that, pre-wheel, if numpy was not
> installed in your conda environment, then pip would build numpy from
> source, whereas now you'll get a binary install.
>
> I _think_ that Python's binary API specification
> (pip.pep425tags.get_abi_tag()) should prevent pip from installing an
> incompatible wheel.  Are there any conda experts out there who can
> give more detail, or more convincing assurance?
>
> Cheers,
>
> Matthew
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] linux wheels coming soon

[Nathaniel Smith]

On Apr 14, 2016 11:11 AM, "Benjamin Root"  wrote:
>
> Are we going to have to have documentation somewhere making it clear that
the numpy wheel shouldn't be used in a conda environment? Not that I would
expect this issue to come up all that often, but I could imagine a scenario
where a non-scientist is simply using a base conda distribution because
that is what IT put on their system. Then they do "pip install ipython"
that indirectly brings in numpy (through the matplotlib dependency), and
end up with an incompatible numpy because they would have been linked
against different pythons?
>
> Or is this not an issue?

There are always issues when you have two different package managers
maintaining separate and out-of-sync metadata about what they think is
installed, but that's true for any mixed use of conda and pip.

But:
- pip won't install a numpy that is incompatible with your python, unless
Anaconda is actively breaking cpython's standard abi (they aren't) or
there's a bug in pip (possible, but no reports yet).
- conda packages for python packages like numpy do generally include the
.egg-info / .dist-info directories that pip uses to store its installation
metadata, so pip can "see" packages installed by conda (but not
vice-versa). So "pip install matplotlib" won't drag in a pypi numpy if
there's already a conda numpy installed.

AFAIK the one case that's nasty is if you first install a conda X, and then
install a pypi X, and then try to use conda to (explicitly, or implicitly
via dependencies) upgrade X. And maybe this is particularly nasty for
X=numpy just because numpy is so low in the stack, but it's not really
numpy specific. (NB I'm not an expert on the internals of conda though :-).)

Actually, from the numpy developer point of view, one of the major
advantages of having wheels is that we can ask people to test prereleases
with 'pip install -U --pre numpy'. If you're a conda user you should only
do this in a temporary environment (like any use of pip really), but I
definitely hope that some conda users will do exactly that to test things
:-).

Also note that there's nothing Linux specific about this scenario. We've
been shipping osx wheels for ages, and AFAIK it hasn't caused any disaster.

-n
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] linux wheels coming soon

[Jonathan Helmus]

On 4/14/16 1:26 PM, Matthew Brett wrote:

Hi,

On Thu, Apr 14, 2016 at 11:11 AM, Benjamin Root  wrote:

Are we going to have to have documentation somewhere making it clear that
the numpy wheel shouldn't be used in a conda environment? Not that I would
expect this issue to come up all that often, but I could imagine a scenario
where a non-scientist is simply using a base conda distribution because that
is what IT put on their system. Then they do "pip install ipython" that
indirectly brings in numpy (through the matplotlib dependency), and end up
with an incompatible numpy because they would have been linked against
different pythons?

Or is this not an issue?

I'm afraid I don't know conda at all, but I'm guessing that pip will
not install numpy when it is installed via conda.
Correct, pip will not (or at least should not, and did not in my tests) 
install numpy over top of an existing conda installed numpy. 
Unfortunately from my testing, conda will install a conda version of 
numpy over top of a pip installed version.  This may be the expected 
behavior as conda maintains its own list of installed packages.

So the potential difference is that, pre-wheel, if numpy was not
installed in your conda environment, then pip would build numpy from
source, whereas now you'll get a binary install.

I _think_ that Python's binary API specification
(pip.pep425tags.get_abi_tag()) should prevent pip from installing an
incompatible wheel.  Are there any conda experts out there who can
give more detail, or more convincing assurance?
I tested "pip install numpy" in conda environments (conda's equivalent 
to virtualenvs) which did not have numpy installed previously for Python 
2.7, 3.4 and 3.5 in a Ubuntu 14.04 Docker container.  In all cases numpy 
was installed from the whl file and appeared to be functional.  Running 
the numpy test suite found three failing tests for Python 2.7 and 3.5 
and 21 errors in Python 3.4. The 2.7 and 3.5 failures do not look 
concerning but the 3.4 errors are a bit strange.
Logs are in 
https://gist.github.com/jjhelmus/a433a66d56fb0e39b8ebde248ad3fe36



Cheers,

- Jonathan Helmus


Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] linux wheels coming soon

[Matthew Brett]

On Thu, Apr 14, 2016 at 12:25 PM, Jonathan Helmus  wrote:
>
>
> On 4/14/16 1:26 PM, Matthew Brett wrote:
>>
>> Hi,
>>
>> On Thu, Apr 14, 2016 at 11:11 AM, Benjamin Root 
>> wrote:
>>>
>>> Are we going to have to have documentation somewhere making it clear that
>>> the numpy wheel shouldn't be used in a conda environment? Not that I
>>> would
>>> expect this issue to come up all that often, but I could imagine a
>>> scenario
>>> where a non-scientist is simply using a base conda distribution because
>>> that
>>> is what IT put on their system. Then they do "pip install ipython" that
>>> indirectly brings in numpy (through the matplotlib dependency), and end
>>> up
>>> with an incompatible numpy because they would have been linked against
>>> different pythons?
>>>
>>> Or is this not an issue?
>>
>> I'm afraid I don't know conda at all, but I'm guessing that pip will
>> not install numpy when it is installed via conda.
>
> Correct, pip will not (or at least should not, and did not in my tests)
> install numpy over top of an existing conda installed numpy. Unfortunately
> from my testing, conda will install a conda version of numpy over top of a
> pip installed version.  This may be the expected behavior as conda maintains
> its own list of installed packages.
>>
>> So the potential difference is that, pre-wheel, if numpy was not
>> installed in your conda environment, then pip would build numpy from
>> source, whereas now you'll get a binary install.
>>
>> I _think_ that Python's binary API specification
>> (pip.pep425tags.get_abi_tag()) should prevent pip from installing an
>> incompatible wheel.  Are there any conda experts out there who can
>> give more detail, or more convincing assurance?
>
> I tested "pip install numpy" in conda environments (conda's equivalent to
> virtualenvs) which did not have numpy installed previously for Python 2.7,
> 3.4 and 3.5 in a Ubuntu 14.04 Docker container.  In all cases numpy was
> installed from the whl file and appeared to be functional.  Running the
> numpy test suite found three failing tests for Python 2.7 and 3.5 and 21
> errors in Python 3.4. The 2.7 and 3.5 failures do not look concerning but
> the 3.4 errors are a bit strange.
> Logs are in
> https://gist.github.com/jjhelmus/a433a66d56fb0e39b8ebde248ad3fe36

Thanks for testing.  For:

docker run -ti --rm ubuntu:14.04 /bin/bash

apt-get update && apt-get install -y curl
curl -LO https://bootstrap.pypa.io/get-pip.py
python3 get-pip.py
pip install numpy nose
python3 -c "import numpy; numpy.test()"

I get:

FAILED (KNOWNFAIL=7, SKIP=17, errors=21)

This is stock Python 3.4 - so not a conda issue.  It is definitely a
problem with the wheel because a compiled numpy wheel on the same
docker image:

apt-get update && apt-get install -y curl python3-dev
curl -LO https://bootstrap.pypa.io/get-pip.py
python3 get-pip.py
pip install --no-binary=:all: numpy nose
python3 -c "import numpy; numpy.test()"

gives no test errors.

It looks like we have some more work to do...

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] linux wheels coming soon

[Paul Hobson]

On Thu, Apr 14, 2016 at 12:07 PM, Nathaniel Smith  wrote:

> On Apr 14, 2016 11:11 AM, "Benjamin Root"  wrote:
> >
> > Are we going to have to have documentation somewhere making it clear
> that the numpy wheel shouldn't be used in a conda environment? Not that I
> would expect this issue to come up all that often, but I could imagine a
> scenario where a non-scientist is simply using a base conda distribution
> because that is what IT put on their system. Then they do "pip install
> ipython" that indirectly brings in numpy (through the matplotlib
> dependency), and end up with an incompatible numpy because they would have
> been linked against different pythons?
> >
> > Or is this not an issue?
>
> There are always issues when you have two different package managers
> maintaining separate and out-of-sync metadata about what they think is
> installed, but that's true for any mixed use of conda and pip.
>
> But:
> - pip won't install a numpy that is incompatible with your python, unless
> Anaconda is actively breaking cpython's standard abi (they aren't) or
> there's a bug in pip (possible, but no reports yet).
> - conda packages for python packages like numpy do generally include the
> .egg-info / .dist-info directories that pip uses to store its installation
> metadata, so pip can "see" packages installed by conda (but not
> vice-versa). So "pip install matplotlib" won't drag in a pypi numpy if
> there's already a conda numpy installed.
>
Minor clarification:. I believe conda can see pip-installed packages.

If I execute "conda list" in an environment, I can see packaged installed
by both pip, conda, and locally (i.e., "pip install . -e").

-paul
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] linux wheels coming soon

[Benjamin Root]

I am honestly surprised that these worked (I haven't gotten around to
testing for myself). I could have sworn there was a difference in how
Continuum compiled python such that any binaries built against a stock
python would not work in a conda environment. I ran into issues a couple
years ago where a modwsgi package provided through yum wouldn't work with
miniconda because of link-time differences.

I cannot for the life of me remember the error message, though.

Ben Root


On Thu, Apr 14, 2016 at 3:59 PM, Paul Hobson  wrote:

>
>
> On Thu, Apr 14, 2016 at 12:07 PM, Nathaniel Smith  wrote:
>
>> On Apr 14, 2016 11:11 AM, "Benjamin Root"  wrote:
>> >
>> > Are we going to have to have documentation somewhere making it clear
>> that the numpy wheel shouldn't be used in a conda environment? Not that I
>> would expect this issue to come up all that often, but I could imagine a
>> scenario where a non-scientist is simply using a base conda distribution
>> because that is what IT put on their system. Then they do "pip install
>> ipython" that indirectly brings in numpy (through the matplotlib
>> dependency), and end up with an incompatible numpy because they would have
>> been linked against different pythons?
>> >
>> > Or is this not an issue?
>>
>> There are always issues when you have two different package managers
>> maintaining separate and out-of-sync metadata about what they think is
>> installed, but that's true for any mixed use of conda and pip.
>>
>> But:
>> - pip won't install a numpy that is incompatible with your python, unless
>> Anaconda is actively breaking cpython's standard abi (they aren't) or
>> there's a bug in pip (possible, but no reports yet).
>> - conda packages for python packages like numpy do generally include the
>> .egg-info / .dist-info directories that pip uses to store its installation
>> metadata, so pip can "see" packages installed by conda (but not
>> vice-versa). So "pip install matplotlib" won't drag in a pypi numpy if
>> there's already a conda numpy installed.
>>
> Minor clarification:. I believe conda can see pip-installed packages.
>
> If I execute "conda list" in an environment, I can see packaged installed
> by both pip, conda, and locally (i.e., "pip install . -e").
>
> -paul
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] linux wheels coming soon

[Matthew Brett]

On Thu, Apr 14, 2016 at 12:57 PM, Matthew Brett  wrote:
> On Thu, Apr 14, 2016 at 12:25 PM, Jonathan Helmus  wrote:
>>
>>
>> On 4/14/16 1:26 PM, Matthew Brett wrote:
>>>
>>> Hi,
>>>
>>> On Thu, Apr 14, 2016 at 11:11 AM, Benjamin Root 
>>> wrote:

 Are we going to have to have documentation somewhere making it clear that
 the numpy wheel shouldn't be used in a conda environment? Not that I
 would
 expect this issue to come up all that often, but I could imagine a
 scenario
 where a non-scientist is simply using a base conda distribution because
 that
 is what IT put on their system. Then they do "pip install ipython" that
 indirectly brings in numpy (through the matplotlib dependency), and end
 up
 with an incompatible numpy because they would have been linked against
 different pythons?

 Or is this not an issue?
>>>
>>> I'm afraid I don't know conda at all, but I'm guessing that pip will
>>> not install numpy when it is installed via conda.
>>
>> Correct, pip will not (or at least should not, and did not in my tests)
>> install numpy over top of an existing conda installed numpy. Unfortunately
>> from my testing, conda will install a conda version of numpy over top of a
>> pip installed version.  This may be the expected behavior as conda maintains
>> its own list of installed packages.
>>>
>>> So the potential difference is that, pre-wheel, if numpy was not
>>> installed in your conda environment, then pip would build numpy from
>>> source, whereas now you'll get a binary install.
>>>
>>> I _think_ that Python's binary API specification
>>> (pip.pep425tags.get_abi_tag()) should prevent pip from installing an
>>> incompatible wheel.  Are there any conda experts out there who can
>>> give more detail, or more convincing assurance?
>>
>> I tested "pip install numpy" in conda environments (conda's equivalent to
>> virtualenvs) which did not have numpy installed previously for Python 2.7,
>> 3.4 and 3.5 in a Ubuntu 14.04 Docker container.  In all cases numpy was
>> installed from the whl file and appeared to be functional.  Running the
>> numpy test suite found three failing tests for Python 2.7 and 3.5 and 21
>> errors in Python 3.4. The 2.7 and 3.5 failures do not look concerning but
>> the 3.4 errors are a bit strange.
>> Logs are in
>> https://gist.github.com/jjhelmus/a433a66d56fb0e39b8ebde248ad3fe36
>
> Thanks for testing.  For:
>
> docker run -ti --rm ubuntu:14.04 /bin/bash
>
> apt-get update && apt-get install -y curl
> curl -LO https://bootstrap.pypa.io/get-pip.py
> python3 get-pip.py
> pip install numpy nose
> python3 -c "import numpy; numpy.test()"
>
> I get:
>
> FAILED (KNOWNFAIL=7, SKIP=17, errors=21)
>
> This is stock Python 3.4 - so not a conda issue.  It is definitely a
> problem with the wheel because a compiled numpy wheel on the same
> docker image:
>
> apt-get update && apt-get install -y curl python3-dev
> curl -LO https://bootstrap.pypa.io/get-pip.py
> python3 get-pip.py
> pip install --no-binary=:all: numpy nose
> python3 -c "import numpy; numpy.test()"
>
> gives no test errors.
>
> It looks like we have some more work to do...

Actually, I can solve these errors by first doing:

apt-get install gcc

I think these must be bugs in the numpy tests where numpy is assuming
a functional compiler.

Does the conda numpy give test errors when there is no compiler?

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] linux wheels coming soon

[Matthew Brett]

On Thu, Apr 14, 2016 at 8:02 AM, Jens Nielsen  wrote:
> I have tried testing the wheels in a project that runs tests on Travis's
> Trusty infrastructure which. The wheels work great for python 3.5 and saves
> us several minuts of runtime.
>
> However, I am having trouble using the wheels on python 2.7 on the same
> Trusty machines. It seems to be because the wheels are tagged as cp27-cp27mu
> (numpy-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl) where as
> pip.pep425tags.get_abi_tag() returns cp27m on this particular python
> version. (Stock python 2.7 installed on Travis 14.04 VMs) Any chance of a
> cp27m compatible wheel build?

Nathaniel / other pip experts - I can't remember the history of these tags.

Is there any danger that an older pip will install a cp27m wheel on a
cp27mu system?

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] linux wheels coming soon

[Jonathan Helmus]

On 4/14/16 3:11 PM, Matthew Brett wrote:

On Thu, Apr 14, 2016 at 12:57 PM, Matthew Brett  wrote:

On Thu, Apr 14, 2016 at 12:25 PM, Jonathan Helmus  wrote:


On 4/14/16 1:26 PM, Matthew Brett wrote:

Hi,

On Thu, Apr 14, 2016 at 11:11 AM, Benjamin Root 
wrote:

Are we going to have to have documentation somewhere making it clear that
the numpy wheel shouldn't be used in a conda environment? Not that I
would
expect this issue to come up all that often, but I could imagine a
scenario
where a non-scientist is simply using a base conda distribution because
that
is what IT put on their system. Then they do "pip install ipython" that
indirectly brings in numpy (through the matplotlib dependency), and end
up
with an incompatible numpy because they would have been linked against
different pythons?

Or is this not an issue?

I'm afraid I don't know conda at all, but I'm guessing that pip will
not install numpy when it is installed via conda.

Correct, pip will not (or at least should not, and did not in my tests)
install numpy over top of an existing conda installed numpy. Unfortunately
from my testing, conda will install a conda version of numpy over top of a
pip installed version.  This may be the expected behavior as conda maintains
its own list of installed packages.

So the potential difference is that, pre-wheel, if numpy was not
installed in your conda environment, then pip would build numpy from
source, whereas now you'll get a binary install.

I _think_ that Python's binary API specification
(pip.pep425tags.get_abi_tag()) should prevent pip from installing an
incompatible wheel.  Are there any conda experts out there who can
give more detail, or more convincing assurance?

I tested "pip install numpy" in conda environments (conda's equivalent to
virtualenvs) which did not have numpy installed previously for Python 2.7,
3.4 and 3.5 in a Ubuntu 14.04 Docker container.  In all cases numpy was
installed from the whl file and appeared to be functional.  Running the
numpy test suite found three failing tests for Python 2.7 and 3.5 and 21
errors in Python 3.4. The 2.7 and 3.5 failures do not look concerning but
the 3.4 errors are a bit strange.
Logs are in
https://gist.github.com/jjhelmus/a433a66d56fb0e39b8ebde248ad3fe36

Thanks for testing.  For:

docker run -ti --rm ubuntu:14.04 /bin/bash

apt-get update && apt-get install -y curl
curl -LO https://bootstrap.pypa.io/get-pip.py
python3 get-pip.py
pip install numpy nose
python3 -c "import numpy; numpy.test()"

I get:

FAILED (KNOWNFAIL=7, SKIP=17, errors=21)

This is stock Python 3.4 - so not a conda issue.  It is definitely a
problem with the wheel because a compiled numpy wheel on the same
docker image:

apt-get update && apt-get install -y curl python3-dev
curl -LO https://bootstrap.pypa.io/get-pip.py
python3 get-pip.py
pip install --no-binary=:all: numpy nose
python3 -c "import numpy; numpy.test()"

gives no test errors.

It looks like we have some more work to do...

Actually, I can solve these errors by first doing:

apt-get install gcc

I think these must be bugs in the numpy tests where numpy is assuming
a functional compiler.

Does the conda numpy give test errors when there is no compiler?

Cheers,

Matthew


Yes, both the wheel and conda numpy packages give errors when there is 
not a compiler.  These errors clear when gcc is installed.  Looks like 
the wheels are fine, just forgot about a compiler.


Cheers,

- Jonathan Helmus
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] linux wheels coming soon

[Matthew Brett]

On Thu, Apr 14, 2016 at 1:47 PM, Jonathan Helmus  wrote:
>
>
> On 4/14/16 3:11 PM, Matthew Brett wrote:
>>
>> On Thu, Apr 14, 2016 at 12:57 PM, Matthew Brett 
>> wrote:
>>>
>>> On Thu, Apr 14, 2016 at 12:25 PM, Jonathan Helmus 
>>> wrote:


 On 4/14/16 1:26 PM, Matthew Brett wrote:
>
> Hi,
>
> On Thu, Apr 14, 2016 at 11:11 AM, Benjamin Root 
> wrote:
>>
>> Are we going to have to have documentation somewhere making it clear
>> that
>> the numpy wheel shouldn't be used in a conda environment? Not that I
>> would
>> expect this issue to come up all that often, but I could imagine a
>> scenario
>> where a non-scientist is simply using a base conda distribution
>> because
>> that
>> is what IT put on their system. Then they do "pip install ipython"
>> that
>> indirectly brings in numpy (through the matplotlib dependency), and
>> end
>> up
>> with an incompatible numpy because they would have been linked against
>> different pythons?
>>
>> Or is this not an issue?
>
> I'm afraid I don't know conda at all, but I'm guessing that pip will
> not install numpy when it is installed via conda.

 Correct, pip will not (or at least should not, and did not in my tests)
 install numpy over top of an existing conda installed numpy.
 Unfortunately
 from my testing, conda will install a conda version of numpy over top of
 a
 pip installed version.  This may be the expected behavior as conda
 maintains
 its own list of installed packages.
>
> So the potential difference is that, pre-wheel, if numpy was not
> installed in your conda environment, then pip would build numpy from
> source, whereas now you'll get a binary install.
>
> I _think_ that Python's binary API specification
> (pip.pep425tags.get_abi_tag()) should prevent pip from installing an
> incompatible wheel.  Are there any conda experts out there who can
> give more detail, or more convincing assurance?

 I tested "pip install numpy" in conda environments (conda's equivalent
 to
 virtualenvs) which did not have numpy installed previously for Python
 2.7,
 3.4 and 3.5 in a Ubuntu 14.04 Docker container.  In all cases numpy was
 installed from the whl file and appeared to be functional.  Running the
 numpy test suite found three failing tests for Python 2.7 and 3.5 and 21
 errors in Python 3.4. The 2.7 and 3.5 failures do not look concerning
 but
 the 3.4 errors are a bit strange.
 Logs are in
 https://gist.github.com/jjhelmus/a433a66d56fb0e39b8ebde248ad3fe36
>>>
>>> Thanks for testing.  For:
>>>
>>> docker run -ti --rm ubuntu:14.04 /bin/bash
>>>
>>> apt-get update && apt-get install -y curl
>>> curl -LO https://bootstrap.pypa.io/get-pip.py
>>> python3 get-pip.py
>>> pip install numpy nose
>>> python3 -c "import numpy; numpy.test()"
>>>
>>> I get:
>>>
>>> FAILED (KNOWNFAIL=7, SKIP=17, errors=21)
>>>
>>> This is stock Python 3.4 - so not a conda issue.  It is definitely a
>>> problem with the wheel because a compiled numpy wheel on the same
>>> docker image:
>>>
>>> apt-get update && apt-get install -y curl python3-dev
>>> curl -LO https://bootstrap.pypa.io/get-pip.py
>>> python3 get-pip.py
>>> pip install --no-binary=:all: numpy nose
>>> python3 -c "import numpy; numpy.test()"
>>>
>>> gives no test errors.
>>>
>>> It looks like we have some more work to do...
>>
>> Actually, I can solve these errors by first doing:
>>
>> apt-get install gcc
>>
>> I think these must be bugs in the numpy tests where numpy is assuming
>> a functional compiler.
>>
>> Does the conda numpy give test errors when there is no compiler?
>>
>> Cheers,
>>
>> Matthew
>
>
> Yes, both the wheel and conda numpy packages give errors when there is not a
> compiler.  These errors clear when gcc is installed.  Looks like the wheels
> are fine, just forgot about a compiler.

Thanks for checking.  I think the problem is fixed here:

https://github.com/numpy/numpy/pull/7549

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] linux wheels coming soon

[Nathaniel Smith]

On Thu, Apr 14, 2016 at 1:22 PM, Matthew Brett  wrote:
> On Thu, Apr 14, 2016 at 8:02 AM, Jens Nielsen  wrote:
>> I have tried testing the wheels in a project that runs tests on Travis's
>> Trusty infrastructure which. The wheels work great for python 3.5 and saves
>> us several minuts of runtime.
>>
>> However, I am having trouble using the wheels on python 2.7 on the same
>> Trusty machines. It seems to be because the wheels are tagged as cp27-cp27mu
>> (numpy-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl) where as
>> pip.pep425tags.get_abi_tag() returns cp27m on this particular python
>> version. (Stock python 2.7 installed on Travis 14.04 VMs) Any chance of a
>> cp27m compatible wheel build?
>
> Nathaniel / other pip experts - I can't remember the history of these tags.
>
> Is there any danger that an older pip will install a cp27m wheel on a
> cp27mu system?

No, support for cp27m/cp27mu tags went in before support for manylinux
tags. And in any case, a pip that doesn't know about cp27m/cp27mu will
just not install such wheels.

The dangerous case is if you were to use an old version of bdist_wheel
that generated a wheel with the "none" abi tag instead of a
cp27m/cp27mu abi tag -- this will mess up all versions of pip, new and
old. But the manylinux docker image definitely has a new enough
version of the wheel package that this is not a problem. ...I guess
the other dangerous case is if you generate a wheel that simply has
the wrong name -- this happened to the gevent packager due to some
distutils brokenness involving using the same source directory to
build both wheels. So don't do that :-). (IIRC there's an open bug
against auditwheel to check for all these problems -- belt *and*
suspenders -- but that hasn't been implemented yet.)

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Please Unsubscribe

[Colin J. Williams]

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion