On Fri, Mar 4, 2016 at 4:42 AM, Matthew Brett <matthew.br...@gmail.com> wrote:
> Hi, > > Summary: > > I propose that we upload Windows wheels to pypi. The wheels are > likely to be stable and relatively easy to maintain, but will have > slower performance than other versions of numpy linked against faster > BLAS / LAPACK libraries. > > Background: > > There's a long discussion going on at issue github #5479 [1], where > the old problem of Windows wheels for numpy came up. > > For those of you not following this issue, the current situation for > community-built numpy Windows binaries is dire: > > * We have not so far provided windows wheels on pypi, so `pip install > numpy` on Windows will bring you a world of pain; > * Until recently we did provide .exe "superpack" installers on > sourceforge, but these became increasingly difficult to build and we > gave up building them as of the latest (1.10.4) release. > > Despite this, popularity of Windows wheels on pypi is high. A few > weeks ago, Donald Stufft ran a query for the binary wheels most often > downloaded from pypi, for any platform [2] . The top five most > downloaded were (n_downloads, name): > > 6646, > numpy-1.10.4-cp27-none-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl > 5445, cryptography-1.2.1-cp27-none-win_amd64.whl > 5243, matplotlib-1.4.0-cp34-none-win32.whl > 5241, scikit_learn-0.15.1-cp34-none-win32.whl > 4573, pandas-0.17.1-cp27-none-win_amd64.whl > > So a) the OSX numpy wheel is very popular and b) despite the fact that > we don't provide a numpy wheel for Windows, matplotlib, sckit_learn > and pandas, that depend on numpy, are the 3rd, 4th and 5th most > downloaded wheels as of a few weeks ago. > > So, there seems to be a large appetite for numpy wheels. > > Current proposal: > > I have now built numpy wheels, using the ATLAS blas / lapack library - > the build is automatic and reproducible [3]. > > I chose ATLAS to build against, rather than, say OpenBLAS, because > we've had some significant worries in the past about the reliability > of OpenBLAS, and I thought it better to err on the side of > correctness. > > However, these builds are relatively slow for matrix multiply and > other linear algebra routines compared numpy built against OpenBLAS or > MKL (which we cannot use because of its license) [4]. In my very > crude array test of a dot product and matrix inversion, the ATLAS > wheels were 2-3 times slower than MKL. Other benchmarks on Julia > found about the same result for ATLAS vs OpenBLAS on 32-bit bit, but a > much bigger difference on 64-bit (for an earlier version of ATLAS than > we are currently using) [5]. > > So, our numpy wheels likely to be stable and give correct results, but > will be somewhat slow for linear algebra. > I would not worry too much about this: at worst, this gives us back the situation where we were w/ so-called superpack, which have been successful in the past to spread numpy use on windows. My main worry is whether this locks us into ATLAS for a long time because of package depending on numpy blas/lapack (scipy, scikit learn). I am not sure how much this is the case. David > > I propose that we upload these ATLAS wheels to pypi. The upside is > that this gives our Windows users a much better experience with pip, > and allows other developers to build Windows wheels that depend on > numpy. The downside is that these will not be optimized for > performance on modern processors. In order to signal that, I propose > adding the following text to the numpy pypi front page: > > ``` > All numpy wheels distributed from pypi are BSD licensed. > > Windows wheels are linked against the ATLAS BLAS / LAPACK library, > restricted to SSE2 instructions, so may not give optimal linear > algebra performance for your machine. See > http://docs.scipy.org/doc/numpy/user/install.html for alternatives. > ``` > > In a way this is very similar to our previous situation, in that the > superpack installers also used ATLAS - in fact an older version of > ATLAS. > > Once we are up and running with numpy wheels, we can consider whether > we should switch to other BLAS libraries, such as OpenBLAS or BLIS > (see [6]). > > I'm posting here hoping for your feedback... > > Cheers, > > Matthew > > > [1] https://github.com/numpy/numpy/issues/5479 > [2] https://gist.github.com/dstufft/1dda9a9f87ee7121e0ee > [3] https://ci.appveyor.com/project/matthew-brett/np-wheel-builder > [4] http://mingwpy.github.io/blas_lapack.html#intel-math-kernel-library > [5] https://github.com/numpy/numpy/issues/5479#issuecomment-185033668 > [6] https://github.com/numpy/numpy/issues/7372 > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > https://mail.scipy.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
