[Numpy-discussion] Numpy binary wheels and CI for win/arm64 platform

[Niyas Sait]

Hello,

Numpy v1.21.2  added
support for windows/arm64 platforms but we still don't have any systems in
place to produce binary wheels or test win/arm64 packages. I think it will
be good to start looking into this. CPython has an official buildbot worker
running for win/arm64 and official python support for the platform will be
available from the 3.11 release.

It is not yet clear to me how the build and CI system for numpy is deployed
and how to enable support for a new platform like win/arm64.

One of the main issues in supporting win/arm64 build would be due to the
lack of win/arm64 VMs available on the cloud. But I see we have been
producing binary wheels for Apple M1 platforms on pypi and conda repository
for some time which also lacks the cloud VM support. I think we could take
some learnings from Apple M1 support and look at how a similar strategy can
be used for win/arm64.

I would like to hear if anyone has any thoughts on this topic. Also, any
pointers to understand numpy wheel generation and CI flow for similar
platforms would be helpful as well.

Thanks,
Niyas
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: Numpy binary wheels and CI for win/arm64 platform

[Ralf Gommers]

On Tue, Nov 2, 2021 at 1:54 PM Niyas Sait  wrote:

> Hello,
>
> Numpy v1.21.2  added
> support for windows/arm64 platforms but we still don't have any systems in
> place to produce binary wheels or test win/arm64 packages. I think it will
> be good to start looking into this. CPython has an official buildbot worker
> running for win/arm64 and official python support for the platform will be
> available from the 3.11 release.
>
> It is not yet clear to me how the build and CI system for numpy is
> deployed and how to enable support for a new platform like win/arm64.
>

For CI, it should all be publicly available (free for open source) CI. We
use GitHub Actions and Azure DevOps for the heavy lifting, plus CircleCI
(doc builds), and TravisCI (arm64, ppc64le, s390x).


> One of the main issues in supporting win/arm64 build would be due to the
> lack of win/arm64 VMs available on the cloud. But I see we have been
> producing binary wheels for Apple M1 platforms on pypi and conda repository
> for some time which also lacks the cloud VM support. I think we could take
> some learnings from Apple M1 support and look at how a similar strategy can
> be used for win/arm64.
>

Hi Niyas, thanks for asking. The learnings are mainly that without regular
CI resources being available, this is a huge pain (it's been a year since
hardware became available, and support is still spotty). Apple M1 is a bit
of a special case, because many users and contributors, and even a few
maintainers, now have that hardware locally. On the other hand, Windows on
arm64 is not really a thing yet that a significant amount of people have.
So probably our stance is that we'll accept patches for it, but cannot
think about custom CI build agents (too much maintenance overhead).


> I would like to hear if anyone has any thoughts on this topic. Also, any
> pointers to understand numpy wheel generation and CI flow for similar
> platforms would be helpful as well.
>

Our current wheel build machinery is at
https://github.com/MacPython/numpy-wheels/, but please ignore that. We just
merged a PR which uses cibuildwheel into the main repo. That should be the
target. If cibuildwheel has/gains the ability to produce win-arm64 wheels,
that'd be a good first step towards support.

More generally: it'd be good to look at support in conda-forge, that had
across-the-board macOS M1 support fairly soon, and is in general a much
more capable setup for supporting new hardware like this.

Cheers,
Ralf
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: Numpy binary wheels and CI for win/arm64 platform

[Peter Cock]

On Tue, Nov 2, 2021 at 1:07 PM Ralf Gommers  wrote:
>
> Our current wheel build machinery is at 
> https://github.com/MacPython/numpy-wheels/,
> but please ignore that. We just merged a PR which uses cibuildwheel into the 
> main repo.
> That should be the target. If cibuildwheel has/gains the ability to produce 
> win-arm64 wheels,
> that'd be a good first step towards support.
>

That would be https://github.com/numpy/numpy/pull/20102 I presume?

I expect quite a few projects using numpy will follow your lead here. Thanks!

Peter
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: Numpy binary wheels and CI for win/arm64 platform

[Ralf Gommers]

On Tue, Nov 2, 2021 at 3:15 PM Peter Cock  wrote:

> 
>
> On Tue, Nov 2, 2021 at 1:07 PM Ralf Gommers 
> wrote:
> >
> > Our current wheel build machinery is at
> https://github.com/MacPython/numpy-wheels/,
> > but please ignore that. We just merged a PR which uses cibuildwheel into
> the main repo.
> > That should be the target. If cibuildwheel has/gains the ability to
> produce win-arm64 wheels,
> > that'd be a good first step towards support.
> >
>
> That would be https://github.com/numpy/numpy/pull/20102 I presume?
>

Yes indeed.

I expect quite a few projects using numpy will follow your lead here.
> Thanks!
>

It's not quite our lead - quite a few other projects have been using
cibuildwheel already for the last year or longer. It's just continued to
improve, and it is clear now that cibuildwheel is the best supported way to
build wheels.

Cheers,
Ralf
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] NumPy Development Meeting Wednesday - Triage Focus

[Sebastian Berg]

Hi all,

Our bi-weekly triage-focused NumPy development meeting is Wednesday,
November 3rd at 16:30 UTC (9:30am Pacific Time).
Everyone is invited to join in and edit the work-in-progress meeting
topics and notes:
https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg

I encourage everyone to notify us of issues or PRs that you feel should
be prioritized, discussed, or reviewed.

Best regards

Sebastian




signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com