Re: [Numpy-discussion] Numpy 1.11.0b2 released
> An extra ~2 hours of tests / 6-way parallelism is not that big a deal > in the grand scheme of things (and I guess it's probably less than > that if we can take advantage of existing binary builds) If we set up a numpy-testing conda channel, it could be used to cache binary builds for all he versions of everything we want to test against. Conda-build-all could make it manageable to maintain that channel. -CHB ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy 1.11.0b2 released
On Feb 5, 2016 8:28 AM, "Chris Barker - NOAA Federal" wrote: > > > An extra ~2 hours of tests / 6-way parallelism is not that big a deal > > in the grand scheme of things (and I guess it's probably less than > > that if we can take advantage of existing binary builds) > > If we set up a numpy-testing conda channel, it could be used to cache > binary builds for all he versions of everything we want to test > against. > > Conda-build-all could make it manageable to maintain that channel. What would be the advantage of maintaining that channel ourselves instead of using someone else's binary builds that already exist (e.g. Anaconda's, or official project wheels)? -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy 1.11.0b2 released
05.02.2016, 19:55, Nathaniel Smith kirjoitti: > On Feb 5, 2016 8:28 AM, "Chris Barker - NOAA Federal" > wrote: >> >>> An extra ~2 hours of tests / 6-way parallelism is not that big a deal >>> in the grand scheme of things (and I guess it's probably less than >>> that if we can take advantage of existing binary builds) >> >> If we set up a numpy-testing conda channel, it could be used to cache >> binary builds for all he versions of everything we want to test >> against. >> >> Conda-build-all could make it manageable to maintain that channel. > > What would be the advantage of maintaining that channel ourselves instead > of using someone else's binary builds that already exist (e.g. Anaconda's, > or official project wheels)? ABI compatibility. However, as I understand it, not many backward ABI incompatible changes in Numpy are not expected in future. If they were, I note that if you work in the same environment, you can push repeated compilation times to zero compared to the time it takes to run tests in a way that requires less configuration, by enabling ccache/f90cache. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy 1.11.0b2 released
On Fri, Feb 5, 2016 at 9:55 AM, Nathaniel Smith wrote: > > If we set up a numpy-testing conda channel, it could be used to cache > > binary builds for all he versions of everything we want to test > > against. > > > > Conda-build-all could make it manageable to maintain that channel. > > What would be the advantage of maintaining that channel ourselves instead > of using someone else's binary builds that already exist (e.g. Anaconda's, > or official project wheels)? > other's binary wheels are only available for the versions that are supported. Usually the latest releases, but Anaconda doesn't always have the latest builds of everything. Maybe we want to test against matplotlib master (or a release candidate, or??), for instance. And when we are testing a numpy-abi-breaking release, we'll need to have everything tested against that release. Usually, when you set up a conda environment, it preferentially pulls from the default channel anyway (or any other channel you set up) , so we'd only maintain stuff that wasn't readily available elsewhere. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy 1.11.0b2 released
On Fri, Feb 5, 2016 at 3:24 PM, wrote: > > > On Fri, Feb 5, 2016 at 1:14 PM, Pauli Virtanen wrote: > >> 05.02.2016, 19:55, Nathaniel Smith kirjoitti: >> > On Feb 5, 2016 8:28 AM, "Chris Barker - NOAA Federal" < >> chris.bar...@noaa.gov> >> > wrote: >> >> >> >>> An extra ~2 hours of tests / 6-way parallelism is not that big a deal >> >>> in the grand scheme of things (and I guess it's probably less than >> >>> that if we can take advantage of existing binary builds) >> >> >> >> If we set up a numpy-testing conda channel, it could be used to cache >> >> binary builds for all he versions of everything we want to test >> >> against. >> >> >> >> Conda-build-all could make it manageable to maintain that channel. >> > >> > What would be the advantage of maintaining that channel ourselves >> instead >> > of using someone else's binary builds that already exist (e.g. >> Anaconda's, >> > or official project wheels)? >> >> ABI compatibility. However, as I understand it, not many backward ABI >> incompatible changes in Numpy are not expected in future. >> >> If they were, I note that if you work in the same environment, you can >> push repeated compilation times to zero compared to the time it takes to >> run tests in a way that requires less configuration, by enabling >> ccache/f90cache. >> > > > control of fortran compiler and libraries > > I was just looking at some new test errors on TravisCI in unchanged code > of statsmodels, and it looks like conda switched from openblas to mkl > yesterday. > > (statsmodels doesn't care when compiling which BLAS/LAPACK is used as long > as they work because we don't have Fortran code.) > > Josef > > (sending again, delivery refused) > > >> >> >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> https://mail.scipy.org/mailman/listinfo/numpy-discussion >> > > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy 1.11.0b2 released
On Fri, Feb 5, 2016 at 1:16 PM, Chris Barker wrote: > On Fri, Feb 5, 2016 at 9:55 AM, Nathaniel Smith wrote: >> >> > If we set up a numpy-testing conda channel, it could be used to cache >> > binary builds for all he versions of everything we want to test >> > against. >> > >> > Conda-build-all could make it manageable to maintain that channel. >> >> What would be the advantage of maintaining that channel ourselves instead >> of using someone else's binary builds that already exist (e.g. Anaconda's, >> or official project wheels)? > > other's binary wheels are only available for the versions that are > supported. Usually the latest releases, but Anaconda doesn't always have the > latest builds of everything. True, though official project wheels will hopefully solve that soon. > Maybe we want to test against matplotlib master (or a release candidate, > or??), for instance. Generally I think for numpy's purposes we want to test against the latest released version, because it doesn't do end-users much good if a numpy release breaks their environment, and the only fix is hiding in some git repo somewhere :-). But yeah. > And when we are testing a numpy-abi-breaking release, we'll need to have > everything tested against that release. There aren't any current plans to have such a release, but true. -n -- Nathaniel J. Smith -- https://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
