[Numpy-discussion] Re: Fwd: Research Opportunity: Can we monitor your projects?
"Lab rat retires on crypto mine in 'research app', drinks at 6." -- Phobrain.com On 2025-04-29 23:24, Ralf Gommers via NumPy-Discussion wrote: > On Wed, Apr 30, 2025 at 6:08 AM Bill Ross wrote: > >> Why run someone else's code? Can't they monitor through git? >> >> [reads more] .. why? > > Agreed, this request is badly thought out. The amount of research requests > we're receiving privately is going up in my experience. In this case it seems > to be coupled with a particularly bad idea on how to actually do the > research, so I filed https://github.com/unl-pal/typechange-bot/issues/1 to > ask them to stop. > > We don't want any bot commenting on NumPy PRs for any purpose, and certainly > not for someone else's projects that doesn't benefit NumPy itself. > > Cheers, > Ralf > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: bross_phobr...@sonic.net___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Fwd: Research Opportunity: Can we monitor your projects?
Why run someone else's code? Can't they monitor through git? [reads more] .. why? --- -- Phobrain.com On 2025-04-29 15:38, Charles R Harris via NumPy-Discussion wrote: > Just thought I'd pass this along for discussion. > > Chuck > > -- Forwarded message - > From: > Date: Tue, Apr 29, 2025 at 4:09 PM > Subject: Research Opportunity: Can we monitor your projects? > To: Charles Harris > > Hello Charles Harris, > > I'm Samuel Flint, and I'm working with Dr. Robert Dyer from the University of > Nebraska-Lincoln on a study about type annotations in dynamic languages. We > would like your permission to monitor the following project(s), which are > highly relevant to our research, and for which you are maintainer: > > - numpy/numpy (https://github.com/numpy/numpy) > > If you agree, we will provide a bot that tracks changes related to type > annotations. The bot will notify committers after relevant changes, asking > them to participate in the study (no more than once every 24 hours, and they > can opt-out or request data removal at any time). > > Would you be willing to have us monitor your project(s)? If so, please > install the bot using the instructions here: > https://github.com/apps/typechangebot > > Additional help is available from GitHub > (https://docs.github.com/en/apps/using-github-apps/installing-a-github-app-from-a-third-party). > > Thanks for considering! > > Samuel W. Flint and Robert Dyer > > This study as been approved by the University of Nebraska-Lincoln > Institutional Review Board > (https://research.unl.edu/researchcompliance/human-subjects-research/) with > the title "Understanding Developers' Addition and Removal of Type > Annotations", IRB# 23988. > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: bross_phobr...@sonic.net___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Better compatibility of the Python scientific/data stack with fast Python interpreters
On Tue, Apr 29, 2025 at 11:24 AM PIERRE AUGIER < pierre.aug...@univ-grenoble-alpes.fr> wrote: > Dear Numpy community members and Numpy developers, > > This email is to get the points of view of the Numpy community members and > developers about a subject that I find very important. I'm going to > introduce it just in few lines so I write few statements without backing > them with proper arguments and without giving links and word definitions. I > assume that most people registered to the numpy-discussion list are at > least familiar to the subject. > Thanks for thinking about this Pierre. > I think getting proper compatibility of the Python scientific/data stack > with fast Python interpreters is very important for the long-term future of > Python for science and data. > I'm not sure it is, it wouldn't rank high on my wish list. PyPy is nearing its end-of-life, and GraalPy is really only for Java devs it seems - I've never seen it in the real world. With CPython performance being worked hard on, the performance gap is shrinking every year. More importantly, none of these efforts (including the "faster CPython" project), seem critical to numerical/scientific users. We're still talking about pure Python code that gets maybe up to 5x faster, while the gains from doing things in compiled languages are a lot higher. So the benefits are more important for small packages if it moves the threshold at which it becomes necessary for them to write zero extension modules. For core libraries like NumPy, pure Python performance isn't super critical. When thinking about overall performance improvements, I'd say that the two most promising large developments are: (1) making it easier to use accelerator libraries (PyTorch, CuPy et al.), and (2) free-threaded CPython for enabling Python-level threading. > Nowadays, fast Python implementations like PyPy and GraalPy are > > - complicated to use (need local builds of wheels) > - slow as soon as the code involves Python extensions (because extensions > work through an emulation layer, like cpyext in PyPy) > > Therefore, these fast Python implementations are not as popular as they > should be. However, they are really great tools which should be used in > particular for scientific/data applications. > > Unfortunately, if we just follow the pace of the CPython C API evolution, > we will basically never have an ecosystem natively compatible with fast > Python implementations. > > The more I think and learn about this subject, the more I think that Numpy > has to stop using directly the CPython C API and to be ported to HPy > (though there are other alternatives - in particular Nanobind and Cython - > that could be discussed). Numpy 1 has been ported to HPy but unfortunately, > there have been deep changes with Numpy 2 (and Meson) so it seems that one > should restart the porting. > > Moreover, unfortunately, HPy does not currently receive as much care as it > should. > > It seems to me that the project of fixing the roots of the Python > ecosystem has to be relaunched. I think that the dynamics has to come from > the Python scientific/data community and in particular Numpy. It is > unfortunately outside of the C API working group's scope (see > https://discuss.python.org/t/c-api-working-group-and-plan-to-get-a-python-c-api-compatible-with-alternative-python-implementations/89477 > ). > > It seems to me that it is necessary (and possible) to get some founding > for such an impactful project so that we can get people working on it. > > I'd like to write a long and serious text on this subject, and I first try > to get the points of view of the different people and projects involved. > > I guess I should write explicit questions: > > - What do you think about the project of fixing the Python scientific/data > stack so that it becomes natively compatible (hence fast and convenient, > with easy installations) with alternative and fast Python interpreters? > - Do you have points of view on how this should be done, technically > (HPy?, something else?) and on other aspects (community, NEP?, founding, > ...). > - Anything else interesting on this subject? > Having HPy or something like it will be very nice and lead ot long-term benefits. The problem with HPy seems to be more a social one at this point: if CPython core devs don't want to adopt it but do their own "make the C API more opaque" strategy, then more effort on HPy isn't going to help. If you're going to dig into this more, I suggest trying to get a very good sense of what the CPython core dev team, and in particular its C API Workgroup, is thinking/planning. That will inform whether the right strategy is to help their efforts along, or work on HPy. Cheers, Ralf > > Best regards, > Pierre > > -- > Pierre Augier - CR CNRS http://www.legi.grenoble-inp.fr > LEGI (UMR 5519) Laboratoire des Ecoulements Geophysiques et Industriels > BP53, 38041 Grenoble Cedex, Francetel:+33.4.56.52.86.16 >
[Numpy-discussion] Re: Fwd: Research Opportunity: Can we monitor your projects?
Where can I find the source code of that app? I wasn't able to find it on the github organization or on the website. ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] next NumPy triage meeting - Wednesday, April 30th, 2025 at 5 pm UTC
The next NumPy triage meeting will be held TOMORROW, April 30th at 17:00 (5 pm) UTC. This is a meeting where we synchronously triage prioritized PRs and issues. Everyone is welcome to attend and contribute to a conversation. Join us via Zoom: https://numfocus-org.zoom.us/j/82096749952?pwd=MW9oUmtKQ1c3a2gydGk1RTdYUUVXZz09 Please notify us of issues or PRs that you’d like to have reviewed by adding a GitHub link to them in the meeting agenda: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg -- Cheers, Inessa Inessa Pawson GitHub: inessapawson ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Fwd: Research Opportunity: Can we monitor your projects?
Just thought I'd pass this along for discussion. Chuck -- Forwarded message - From: Date: Tue, Apr 29, 2025 at 4:09 PM Subject: Research Opportunity: Can we monitor your projects? To: Charles Harris Hello Charles Harris, I'm Samuel Flint, and I'm working with Dr. Robert Dyer from the University of Nebraska-Lincoln on a study about type annotations in dynamic languages. We would like your permission to monitor the following project(s), which are highly relevant to our research, and for which you are maintainer: - numpy/numpy (https://github.com/numpy/numpy) If you agree, we will provide a bot that tracks changes related to type annotations. The bot will notify committers after relevant changes, asking them to participate in the study (no more than once every 24 hours, and they can opt-out or request data removal at any time). Would you be willing to have us monitor your project(s)? If so, please install the bot using the instructions here: https://github.com/apps/typechangebot Additional help is available from GitHub ( https://docs.github.com/en/apps/using-github-apps/installing-a-github-app-from-a-third-party ). Thanks for considering! Samuel W. Flint and Robert Dyer This study as been approved by the University of Nebraska-Lincoln Institutional Review Board ( https://research.unl.edu/researchcompliance/human-subjects-research/) with the title "Understanding Developers' Addition and Removal of Type Annotations", IRB# 23988. ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Fwd: Research Opportunity: Can we monitor your projects?
On Wed, Apr 30, 2025, 08:41 Charles R Harris via NumPy-Discussion < numpy-discussion@python.org> wrote: > Just thought I'd pass this along for discussion. > > Chuck > > -- Forwarded message - > From: > Date: Tue, Apr 29, 2025 at 4:09 PM > Subject: Research Opportunity: Can we monitor your projects? > To: Charles Harris > > > Hello Charles Harris, > > I'm Samuel Flint, and I'm working with Dr. Robert Dyer from the University > of Nebraska-Lincoln on a study about type annotations in dynamic > languages. We would like your permission to monitor the following > project(s), which are highly relevant to our research, and for which you > are maintainer: > > - numpy/numpy (https://github.com/numpy/numpy) > > If you agree, we will provide a bot that tracks changes related to type > annotations. The bot will notify committers after relevant changes, asking > them to participate in the study (no more than once every 24 hours, and > they can opt-out or request data removal at any time). > > Would you be willing to have us monitor your project(s)? If so, please > install the bot using the instructions here: > https://github.com/apps/typechangebot > I'd prefer if people weren't contacted at all after making commits. It should be an opt in thing. ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Fwd: Research Opportunity: Can we monitor your projects?
On Wed, Apr 30, 2025 at 6:08 AM Bill Ross wrote: > Why run someone else's code? Can't they monitor through git? > > [reads more] .. why? > Agreed, this request is badly thought out. The amount of research requests we're receiving privately is going up in my experience. In this case it seems to be coupled with a particularly bad idea on how to actually do the research, so I filed https://github.com/unl-pal/typechange-bot/issues/1 to ask them to stop. We don't want any bot commenting on NumPy PRs for any purpose, and certainly not for someone else's projects that doesn't benefit NumPy itself. Cheers, Ralf ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Better compatibility of the Python scientific/data stack with fast Python interpreters
- Mail original - > De: "Stefan Krah" > À: "numpy-discussion" > Envoyé: Mardi 29 Avril 2025 12:09:26 > Objet: [Numpy-discussion] Re: Better compatibility of the Python > scientific/data stack with fast Python interpreters > On Tue, Apr 29, 2025 at 11:20:28AM +0200, PIERRE AUGIER wrote: >> - slow as soon as the code involves Python extensions (because extensions >> work >> through an emulation layer, like cpyext in PyPy) > [...] >> Moreover, unfortunately, HPy does not currently receive as much care as it >> should. > > Does HPy solve the speed issue for C extensions? I thought it was more of a > compatibility layer so that things similar to CFFI could be used > automatically. > > CFFI/PyPy on the other hand was slower for the _decimal extension than > CPython. Yes, HPy does solve the speed issue for C extensions with PyPy and GraalPy. HPy can be seen as the native C API for PyPy and GraalPy, so that the emulation layer cpyext (https://pypy.org/posts/2018/09/inside-cpyext-why-emulating-cpython-c-8083064623681286567.html) can completely be avoided. With HPy you get typically the same speed for extensions than with CPython. Of course, the pure Python parts are much faster than with CPython, typically more than one order of magnitude faster for CPU bounded pure Python. Regarding C-extension speed, with HPy it could be possible to avoid boxing/unboxing (see for example https://github.com/hpyproject/hpy/issues/474), which could dramatically improve interaction between Python JIT and C code. Another huge advantage of HPy compared to the historical C API is that you can build "universal" wheels that can be used on all Python implementations and all Python versions. > CPython's C-extension speed is actually quite reasonable. The only garbage > collected language that achieves the _same_ extension speed that I know of > is OCaml (which is quite an achievement). Yes, but CPython is extremely slow on pure Python. And with objects fixed in memory and reference counting (or emulation of that), it's highly probable that it will stay very inefficient compared to alternative implementations and interpreters for comparable languages, for example JavaScript/TypeScript and Julia. > Does the Graal VM solve this issue? Yes, with Graal, the JIT can even act across the boundaries of languages. > > Stefan Krah > > > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: pierre.aug...@univ-grenoble-alpes.fr ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Better compatibility of the Python scientific/data stack with fast Python interpreters
Dear Numpy community members and Numpy developers, This email is to get the points of view of the Numpy community members and developers about a subject that I find very important. I'm going to introduce it just in few lines so I write few statements without backing them with proper arguments and without giving links and word definitions. I assume that most people registered to the numpy-discussion list are at least familiar to the subject. I think getting proper compatibility of the Python scientific/data stack with fast Python interpreters is very important for the long-term future of Python for science and data. Nowadays, fast Python implementations like PyPy and GraalPy are - complicated to use (need local builds of wheels) - slow as soon as the code involves Python extensions (because extensions work through an emulation layer, like cpyext in PyPy) Therefore, these fast Python implementations are not as popular as they should be. However, they are really great tools which should be used in particular for scientific/data applications. Unfortunately, if we just follow the pace of the CPython C API evolution, we will basically never have an ecosystem natively compatible with fast Python implementations. The more I think and learn about this subject, the more I think that Numpy has to stop using directly the CPython C API and to be ported to HPy (though there are other alternatives - in particular Nanobind and Cython - that could be discussed). Numpy 1 has been ported to HPy but unfortunately, there have been deep changes with Numpy 2 (and Meson) so it seems that one should restart the porting. Moreover, unfortunately, HPy does not currently receive as much care as it should. It seems to me that the project of fixing the roots of the Python ecosystem has to be relaunched. I think that the dynamics has to come from the Python scientific/data community and in particular Numpy. It is unfortunately outside of the C API working group's scope (see https://discuss.python.org/t/c-api-working-group-and-plan-to-get-a-python-c-api-compatible-with-alternative-python-implementations/89477). It seems to me that it is necessary (and possible) to get some founding for such an impactful project so that we can get people working on it. I'd like to write a long and serious text on this subject, and I first try to get the points of view of the different people and projects involved. I guess I should write explicit questions: - What do you think about the project of fixing the Python scientific/data stack so that it becomes natively compatible (hence fast and convenient, with easy installations) with alternative and fast Python interpreters? - Do you have points of view on how this should be done, technically (HPy?, something else?) and on other aspects (community, NEP?, founding, ...). - Anything else interesting on this subject? Best regards, Pierre -- Pierre Augier - CR CNRS http://www.legi.grenoble-inp.fr LEGI (UMR 5519) Laboratoire des Ecoulements Geophysiques et Industriels BP53, 38041 Grenoble Cedex, Francetel:+33.4.56.52.86.16 ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Better compatibility of the Python scientific/data stack with fast Python interpreters
On Tue, Apr 29, 2025 at 11:20:28AM +0200, PIERRE AUGIER wrote: > - slow as soon as the code involves Python extensions (because extensions > work through an emulation layer, like cpyext in PyPy) [...] > Moreover, unfortunately, HPy does not currently receive as much care as it > should. Does HPy solve the speed issue for C extensions? I thought it was more of a compatibility layer so that things similar to CFFI could be used automatically. CFFI/PyPy on the other hand was slower for the _decimal extension than CPython. CPython's C-extension speed is actually quite reasonable. The only garbage collected language that achieves the _same_ extension speed that I know of is OCaml (which is quite an achievement). SBCL Lisp, which has a great compiler for pure Lisp, also does not have really fast extensions. Does the Graal VM solve this issue? Stefan Krah ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
