[Numpy-discussion] Re: No credits left in travis for 1.22.0 release

[James Tocknell]

I'll need to check with the other maintainers (see
https://github.com/h5py/h5py/issues/2026), but I believe h5py
shouldn't need the x86 builds any more (I'm less sure about the other
systems)?

James

On Fri, 31 Dec 2021 at 17:03, Matti Picus  wrote:
>
>
> On 31/12/21 3:50 am, Charles R Harris wrote:
> >
> >
> > On Thu, Dec 30, 2021 at 6:26 PM Charles R Harris
> > mailto:charlesr.har...@gmail.com>> wrote:
> >
> > Hi All,
> >
> > We need to get more credits in order to run MacPython/numpy-wheels
> > builds on travis. Anyone recall how to go about this?
> >
> > Chuck
> >
> >
> > MacPython has used 800550 of 80 credits.  NumPy has used 625860 of
> > 80 credits, so should be good for a couple of months. SciPy has
> > one credit left, but doesn't use travis. I think the MacPython credits
> > covers everyone who uses multibuild and travis, so NumPy is likely not
> > the only one affected.
> >
> > Chuck
> >
> I requested more OpenSource credits for MacPython. The MacPython credits
> cover many of the wheel-building projects, the main repos (numpy/numpy)
> are under different accounts. I last reqeusted credits for MacPython in
> October.
>
>
> A bit more information:
>
> Travis-ci.com bills differently for each platform.
>
> For x86, they bill
>
> 10 credits per minute on linux
>
> 20 credits per minute on windows
>
> 50 credits per minute on macos
>
> aarch64 (arm linux), ppc, and s390x are **not billed** against credits.
> I think this is because other companies provide the actual hardware runners.
>
>
> Some of the projects under MacPython like matplotlib have migrated off
> travis and use cibuildwheel in their main repo [0], using qemu for arm64.
>
>
> Others  like numpy-wheels and openblas-libs moved their x86 builds to
> other providers, their travis builds actually consume 0 credits.
>
>
> h5py-wheels, scipy-wheels, pandas-wheels and others are still consuming
> credits for their x86 builds. If all the projects moved to other
> providers it would save opening requests for more credits, admittedly it
> is easier to request credits than to move the build :). I could also be
> more proactive in tracking the credit situation and request a renewal
> before they run out, I will set a reminder to do so 2 1/2 months from now.
>
>
> Matti
>
>
> [0]
> https://github.com/matplotlib/matplotlib/blob/main/.github/workflows/cibuildwheel.yml
>
> ___
> NumPy-Discussion mailing list -- numpy-discussion@python.org
> To unsubscribe send an email to numpy-discussion-le...@python.org
> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
> Member address: aragilar+nu...@gmail.com
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: status of long double support and what to do about it

[James Tocknell]

Around 2015-2016 I created a fork of numpy with IEEE 128bit float
support, but I never upstreamed it because I couldn't see a way to
make it not depend on GCC/libquadmath (the licensing issue Chuck
brought up). I wonder if it's possible now with the dtype changes to
have different dtypes on different platforms (it didn't appear to be
the case when I looked then), which would avoid the need to distribute
third-party libraries to cover certain platforms.

Regards
James

On Fri, 18 Nov 2022 at 12:28, Scott Ransom  wrote:
>
>
>
> On 11/17/22 7:13 PM, Charles R Harris wrote:
> >
> >
> > On Thu, Nov 17, 2022 at 3:15 PM Ralf Gommers  > > wrote:
> >
> > Hi all,
> >
> > We have to do something about long double support. This is something I 
> > wanted to propose a long
> > time ago already, and moving build systems has resurfaced the pain yet 
> > again.
> >
> > This is not a full proposal yet, but the start of a discussion and 
> > gradual plan of attack.
> >
> > The problem
> > ---
> > The main problem is that long double support is *extremely* painful to 
> > maintain, probably far
> > more than justified. I could write a very long story about that, but 
> > instead I'll just
> > illustrate with some of the key points:
> >
> > (1) `long double` is the main reason why we're having such a hard time 
> > with building wheels on
> > Windows, for SciPy in particular. This is because MSVC makes long 
> > double 64-bit, and Mingw-w64
> > defaults to 80-bit. So we have to deal with Mingw-w64 toolchains, 
> > proposed compiler patches,
> > etc. This alone has been a massive time sink. A couple of threads:
> > https://github.com/numpy/numpy/issues/20348 
> > 
> > 
> > https://discuss.scientific-python.org/t/releasing-or-not-32-bit-windows-wheels/282
> > 
> > 
> > The first issue linked above is one of the key ones, with a lot of 
> > detail about the fundamental
> > problems with `long double`. The Scientific Python thread focused more 
> > on Fortran, however that
> > Fortran + Windows problem is at least partly the fault of `long 
> > double`. And Fortran may be
> > rejuvenated with LFortran and fortran-lang.org 
> > ; `long double` is a
> > dead end.
> >
> > (2) `long double` is not a well-defined format. We support 9 specific 
> > binary representations,
> > and have a ton of code floating around to check for those, and manually 
> > fiddle with individual
> > bits in long double numbers. Part of that is the immediate pain point 
> > for me right now: in the
> > configure stage of the build we consume object files produced by the 
> > compiler and parse them,
> > matching some known bit patterns. This check is so weird that it's the 
> > only one that I cannot
> > implement in Meson (short of porting the hundreds of lines of Python 
> > code for it to C), see
> > https://github.com/mesonbuild/meson/issues/11068
> >  for details. To get 
> > an idea of the complexity
> > here:
> > 
> > https://github.com/numpy/numpy/blob/9e144f7c1598221510d49d8c6b79c66dc000edf6/numpy/core/setup_common.py#L264-L434
> >  
> > 
> > 
> > https://github.com/numpy/numpy/blob/9e144f7c1598221510d49d8c6b79c66dc000edf6/numpy/core/src/npymath/npy_math_private.h#L179-L484
> >  
> > 
> > 
> > https://github.com/numpy/numpy/blob/main/numpy/core/src/npymath/npy_math_complex.c.src#L598-L619
> > 
> > 
> > 
> > https://github.com/numpy/numpy/blob/main/numpy/core/src/multiarray/dragon4.c#L2480-L3052
> > 
> > 
> > Typically `long double` has multiple branches, and requires more code 
> > than float/double.
> >
> > (3) We spend a lot of time dealing with issues and PR to keep `long 
> > double` working, as well as
> > dealing with hard-to-diagnose build issues. Which sometimes even stops 
> > people from
> > building/contributing, especially on Windows. Some recent examples:
> > https://github.com/numpy/numpy/pull/20360 
> > 
> > https://github.com/numpy/numpy/pull/18536 
> > 
> > https://github.com/numpy/numpy/pull/21813 
> > 
> > https://github.com/numpy/numpy/pull/22405 
> >