Re: [Numpy-discussion] Scipy 2017 NumPy sprint

[Peter Cock]

Note that TravisCI does not yet have official Python support on Mac OS X,

https://github.com/travis-ci/travis-ci/issues/2312

I believe it is possible to do anyway by faking it under another setting
(e.g. pretend to be a generic language build, and use the system Python
or install your own specific version of Python as needed), so that may be
worth trying during a sprint.

Peter

On Wed, Jul 5, 2017 at 10:43 AM, Ralf Gommers  wrote:
>
> Better platform test coverage would be a useful topic if someone is willing
> to work on that. NumPy needs OS X testing enabled on TravisCI, SciPy needs
> OS X and a 32-bit test (steal from NumPy). And if someone really feels
> ambitious: replace ATLAS by OpenBLAS in one of the test matrix entries.
>
> Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Scipy 2017 NumPy sprint

[Peter Cock]

On Wed, Jul 5, 2017 at 11:25 AM, Ralf Gommers  wrote:
>
>
> On Wed, Jul 5, 2017 at 10:14 PM, Peter Cock 
> wrote:
>>
>> Note that TravisCI does not yet have official Python support on Mac OS X,
>>
>> https://github.com/travis-ci/travis-ci/issues/2312
>>
>> I believe it is possible to do anyway by faking it under another setting
>> (e.g. pretend to be a generic language build, and use the system Python
>> or install your own specific version of Python as needed), so that may be
>> worth trying during a sprint.
>
>
> That approach has worked reliably for
> https://github.com/MacPython/numpy-wheels for a while now, so should be
> straightforward.
>
> Ralf

Thanks for that link - I'm going off topic but the MacPython wiki page goes
into more background about how they build wheels for PyPI which I'm
very interested to read up on:

https://github.com/MacPython/wiki/wiki/Spinning-wheels

Peter
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy wheels, openBLAS and threading

[Peter Cock]

This came up for Biopython recently (someone using our
library on a cluster ran into thread limits triggered by the
importing of NumPy), and suggested something like this:

import os
try:
os.environ["OMP_NUM_THREADS"] = "1"
import numpy
finally:
   del os.environ["OMP_NUM_THREADS"]

Or MKL_NUM_THREADS, or apparently also it might
be OPENBLAS_NUM_THREADS as well:

https://github.com/biopython/biopython/pull/1401

Peter

On Thu, Sep 28, 2017 at 2:26 PM, Jean-Christophe Houde
 wrote:
> Hi all,
>
> not sure if this is the best place to ask for this. If not, please advise on
> the correct place.
>
> Since the numpy wheels internally use openBLAS, operations can be implicitly
> multithreaded directly by openBLAS.
>
> This, of course, can clash with multithreading or parallel processing. The
> recommended practice in this case is to set
>
> export OPENBLAS_NUM_THREADS=1
>
> in the environment. However, I would like to be able to adjust this directly
> in my python code.
>
> Is there a way to control this directly through Python, whether through
> numpy or not?
>
> Thanks for your time!
>
> --
> Jean-Christophe Houde
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support

[Peter Cock]

On Tue, Nov 7, 2017 at 11:40 PM, Nathaniel Smith  wrote:
>
> 
>
> Right now, the decision in front of us is what to tell people who ask about
> numpy's py2 support plans, so that they can make their own plans. Given what
> we know right now, I don't think we should promise to keep support past
> 2018. If we get there and the situation's changed, and there's both desire
> and means to extend support we can revisit that. But's better to
> under-promise and possibly over-deliver, instead of promising to support py2
> until after it becomes a millstone around our necks and then realizing we
> haven't warned anyone and are stuck supporting it another year beyond
> that...
>
> -n

NumPy (and to a lesser extent SciPy) is in a tough position being at the
bottom of many scientific Python programming stacks. Whenever you
drop Python 2 support is going to upset someone.

It is too ambitious to pledge to drop support for Python 2.7 no later than
2020, coinciding with the Python development team’s timeline for dropping
support for Python 2.7?

If that looks doable, NumPy could sign up to http://www.python3statement.org/

Regards,

Peter
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support

[Peter Cock]

On Wed, Nov 8, 2017 at 10:17 PM, Bryan Van de ven  wrote:
>
>> On Nov 8, 2017, at 10:50, Peter Cock  wrote:
>>
>> NumPy (and to a lesser extent SciPy) is in a tough position being at the
>> bottom of many scientific Python programming stacks. Whenever you
>> drop Python 2 support is going to upset someone.
>
> Existing versions of NumPy will still exist and continue to work with Python 
> 2.7. If users want to say with Python 2.7, that's fine, they will just have 
> to rely on those older/LTS versions. I personally would be happy for projects 
> at the bottom of stacks to take an activist stance and make decisions to 
> actively encourage movement to Python 3.
>
>> It is too ambitious to pledge to drop support for Python 2.7 no later than
>> 2020, coinciding with the Python development team’s timeline for dropping
>> support for Python 2.7?
>
> Developing NumPy is hard, as it is. Everything that can be done to simplify 
> things for the current maintainers and help attract new contributors should 
> be done. It is not reasonable to ask a few (largely volunteer) people to 
> shoulder the burden and difficulties of supporting Python 2.7 for several 
> additional *years* of their life.
>
> I agree entirely with Nick Coghlan's comments from another discussion, and 
> think they apply equally well in this instance:
>
> """
> While it's entirely admirable that many upstream developers are generous 
> enough to help their end users work around this inertia, in the long run 
> doing so is detrimental for everyone concerned, as long term sustaining 
> engineering for old releases is genuinely demotivating for upstream 
> developers (it's a good job, but a lousy way to spend your free time) and for 
> end users, working around institutional inertia this way reduces the pressure 
> to actually get the situation addressed properly.
> """
>
> Thanks,
>
> Bryan

I agree too - I was trying to phrase that email neutrally as I am not
a direct NumPy contributor, but to be more explicit, as someone
invested in this ecosystem:

I'd fully support NumPy pledging to drop Python 2.7 support no later
than 2020. I see signing up to http://www.python3statement.org/ as
being about helping publicise this choice.

(This is not to say dropping Python 2.7 support in NumPy couldn't
happen much sooner than 2020 - the C99 compiler issues sounds like a
strong pressure to do so.)

Peter
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support

[Peter Cock]

Since Konrad Hinsen no longer follows the NumPy discussion list
for lack of time, he has not posted here - but he has commented
about this on Twitter and written up a good blog post:

http://blog.khinsen.net/posts/2017/11/16/a-plea-for-stability-in-the-scipy-ecosystem/

In a field where scientific code is expected to last and be developed
on a timescale of decades, the change of pace with Python 2 and 3
is harder to handle.

Regards,

Peter

On Wed, Nov 15, 2017 at 2:19 AM, Nathaniel Smith  wrote:
> Apparently this is actually uncontroversial, the discussion's died
> down (see also the comments on Chuck's PR [1]), and anyone who wanted
> to object has had more than a week to do so, so... I guess we can say
> this is what's happening and start publicizing it to our users!
>
> A direct link to the rendered NEP in the repo is:
> https://github.com/numpy/numpy/blob/master/doc/neps/dropping-python2.7-proposal.rst
>
> (I guess that at some point it will also show up on docs.scipy.org.)
>
> -n
>
> [1] https://github.com/numpy/numpy/pull/10006
>
> On Thu, Nov 9, 2017 at 5:52 PM, Nathaniel Smith  wrote:
>> Fortunately we can wait until we're a bit closer before we have to
>> make any final decision on the version numbering :-)
>>
>> Right now though it would be good to start communicating to
>> users/downstreams about whatever our plans our though, so they can
>> make plans. Here's a first attempt at some text we can put in the
>> documentation and point people to -- any thoughts, on either the plan
>> or the wording?
>>
>>  DRAFT TEXT - NOT FINAL - DO NOT POST THIS TO HACKERNEWS OK? OK 
>>
>> The Python core team plans to stop supporting Python 2 in 2020. The
>> NumPy project has supported both Python 2 and Python 3 in parallel
>> since 2010, and has found that supporting Python 2 is an increasing
>> burden on our limited resources; thus, we plan to eventually drop
>> Python 2 support as well. Now that we're entering the final years of
>> community-supported Python 2, the NumPy project wants to clarify our
>> plans, with the goal of to helping our downstream ecosystem make plans
>> and accomplish the transition with as little disruption as possible.
>>
>> Our current plan is as follows:
>>
>> Until **December 31, 2018**, all NumPy releases will fully support
>> both Python 2 and Python 3.
>>
>> Starting on **January 1, 2019**, any new feature releases will support
>> only Python 3.
>>
>> The last Python-2-supporting release will be designated as a long-term
>> support (LTS) release, meaning that we will continue to merge
>> bug-fixes and make bug-fix releases for a longer period than usual.
>> Specifically, it will be supported by the community until **December
>> 31, 2019**.
>>
>> On **January 1, 2020** we will raise a toast to Python 2, and
>> community support for the last Python-2-supporting release will come
>> to an end. However, it will continue to be available on PyPI
>> indefinitely, and if any commercial vendors wish to extend the LTS
>> support past this point then we are open to letting them use the LTS
>> branch in the official NumPy repository to coordinate that.
>>
>> If you are a NumPy user who requires ongoing Python 2 support in 2020
>> or later, then please contact your vendor. If you are a vendor who
>> wishes to continue to support NumPy on Python 2 in 2020+, please get
>> in touch; ideally we'd like you to get involved in maintaining the LTS
>> before it actually hits end-of-life, so we can make a clean handoff.
>>
>> To minimize disruption, running 'pip install numpy' on Python 2 will
>> continue to give the last working release in perpetuity; but after
>> January 1, 2019 it may not contain the latest features, and after
>> January 1, 2020 it may not contain the latest bug fixes.
>>
>> For more information on the scientific Python ecosystem's transition
>> to Python-3-only, see: http://www.python3statement.org/
>>
>> For more information on porting your code to run on Python 3, see:
>> https://docs.python.org/3/howto/pyporting.html
>>
>> 
>>
>> Thoughts?
>>
>> -n
>>
>> On Thu, Nov 9, 2017 at 12:53 PM, Marten van Kerkwijk
>>  wrote:
>>> In astropy we had a similar discussion about version numbers, and
>>> decided to make 2.0 the LTS that still supports python 2.7 and 3.0 the
>>> first that does not.  If we're discussing jumping a major number, we
>>> could do the same for numpy.  (Admittedly, it made a bit more sense
>>> with the numbering scheme astropy had adopted anyway.) -- Marten
>>> ___
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion@python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
>>
>>
>> --
>> Nathaniel J. Smith -- https://vorpus.org
>
>
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Hoop jumping, and other sports

[Peter Cock]

On Wed, Feb 7, 2018 at 11:04 PM, Andrew Nelson  wrote:
> Other factors hindering new contributors:
>
> 1) Being unfamiliar with git. e.g. knowing that you have to fork numpy
> first, then clone from your fork, then create a feature branch. That's just
> the """straightforward bit""". It's hell the first time you have to rebase
> something/correct commit messages/squash, etc.

Is there anything specific to NumPy's use of git which is unsual
which needs particular warning for newcomers?

> 2) I have a mindblock on the correct way to use ReStructured Text in
> docstrings. When do I use backticks/double backticks, etc. Getting
> documentation changes to perfection is hard.

Me too. I still regularly copy-paste my docstrings from the Python
code into an RST editor to confirm the formatting, or do any
non-trivial editing.

I also found I'd often only discover invalid RST when trying to
build a project's documentation (using Sphinx or epydoc as
appropriate for the project at hand), so I wrong a flake8 plugin
to do basic reStructuredText validation:

https://pypi.python.org/pypi/flake8-rst-docstrings
https://github.com/peterjc/flake8-rst-docstrings

One could imagine checking NumPy's specific docstring
style with a more complicated flake8 plugin?

Peter
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Hoop jumping, and other sports

[Peter Cock]

On Thu, Feb 8, 2018 at 11:22 AM, Ralf Gommers  wrote:
> On Wed, Feb 7, 2018 at 11:29 PM, Stefan van der Walt 
> wrote:
>>
>> For scikit-image, we've started a policy of pushing minor edits
>> (spelling corrections, sentence restructuring, etc.) directly to the PR
>> branch, instead of flagging those during a review (we also have a PEP8
>> bot that mentions PEP8 issues, which makes the human reviews less
>> nitpicky).
>
> +1. Especially useful for docstring formatting, that doesn't get picked up
> by a PEP8 bot
>
> Ralf

Were is the current NumPy PEP8 bot setup? I don't see flake8 or
similar under the TravisCI or AppVeyor setup - which is where I am
using a flake8 plugin to at least partially validate RST docstrings [*]
(and catch things automatically in pull requests).

Peter

[*] My plugin:

https://github.com/peterjc/flake8-rst-docstrings
https://pypi.python.org/pypi/flake8-rst-docstrings
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Top level release index page

[Peter Cock]

On Tue, Jul 2, 2019 at 11:19 PM Matti Picus  wrote:

> In PR 13886 I reworked the way the link to the release notes is
> generated. The current page is
>
>
> http://www.numpy.org/devdocs/release.html
>
>
> and  the new page is
>
>
>
> https://8001-908607-gh.circle-artifacts.com/0/home/circleci/repo/doc/build/html/release.html
>
>
> I don't like the "wall of text" in the current page. On the other hand,
> it is nice to use CTRL-F to search the page itself for answers to
> questions like "what release deprecated indexing by float", which is why
> I left the level-of-detail at 3 (1 - release, 2 - sections, 3 - item
> header).
>
>
That seems a good compromise - certainly I have often used CTRL-F
on release notes or change notes while tracing regressions - and having
all the releases on one page makes this *so* much easier.

Due to the way I typically use release notes, I would perhaps have
just left the wall of text as is.


>
> Should the level-of-detail be reduced to only single links to the
> release document?
>
>
I'm unclear what you are suggesting here.


>
> An alternative would be to render the contents as a collapsible list,
> which would require some javascript.
>
>
Collapsible lists (which worked with CTRL-F searching) would be even
better for balancing readability with utility, provided it was not too much
trouble to implement and maintain of course.

Peter
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Stricter numpydoc validation

[Peter Cock]

I’d find this sort of (stricter) numpydoc validation tool very useful,
especially if the different codes can be selectively enforced while
bringing a large code base into compliance (as pandas seems to have used
this).

A stand alone tool would be fine, a flake8 plug-in perhaps even better -
see also my
https://github.com/peterjc/flake8-rst-docstrings for doing basic RST
validation of docstrings (but not their contents as we are discussing
here). The nice thing with writing a flake8 plugin is that handles
include/excluding of codes and file names for you.

Regards,

Peter

On Tue, 16 Jul 2019 at 16:13, Ralf Gommers  wrote:

>
>
> On Tue, Jul 16, 2019 at 4:23 AM Gael Varoquaux <
> gael.varoqu...@normalesup.org> wrote:
>
>> > The one thing I worry about is maintenance burden, where numpydoc is
>> already
>> > spread a little bit thin -- would any of the Pandas developers be
>> willing to
>> > maintain it?
>>
>> Any reason that this is not done in sphinx, with the napoleon extension?
>> https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html
>>
>> I would hope that this can increase the number of people likely to help
>> maintaining.
>>
>
> Just history. Numpydoc came first, most projects rely on it. Napoleon came
> way later and then did its own numpy docstring support rather than
> contribute to numpydoc or propose a merge.
>
> If someone wants to figure out how compatible these two implementations
> are and whether they can be merged, that would be really welcome.
>
> Cheers,
> Ralf
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Accuracy improvement in np.mean

[Peter Cock]

To save anyone else looking, it was pull request 15185:

https://github.com/numpy/numpy/pull/15185

Peter

On Mon, Jan 13, 2020 at 5:25 PM mpro  wrote:
>
> Hi,
>
> 18 days ago I submitted pull request that improves accuracy of mean function
> by calculating mean sequentially along certain axis.
> Happy to get some feedback.
>
> Magdalena
>
>
>
> --
> Sent from: http://numpy-discussion.10968.n7.nabble.com/
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Replacement for Rackspace

[Peter Cock]

Ah - this is unwelcome news.

See https://mail.python.org/pipermail/scipy-dev/2020-February/023990.html
and https://github.com/matthew-brett/multibuild/issues/304

There are quite a few project's using the multibuild system now...

Peter

On Fri, Aug 7, 2020 at 2:01 PM Kevin Sheppard 
wrote:

> The Rackspace hosted wheel endpoints at
>
>
>
>
> https://7933911d6844c6c53a7d-47bd50c35cd79bd838daf386af554a83.ssl.cf2.rackcdn.com/
>
>
>
> and
>
>
>
>
> https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a43.ssl.cf2.rackcdn.com/
>
>
>
> seem to not be working.  I know NumPy, SciPy, pandas and scikit-learn are
> all using a common end point on anacona.org.  Statsmodels is preparing
> for  release, and the wheel builder at
> https://github.com/MacPython/statsmodels-wheels is failing at upload.  Is
> there any shared resource for uploading nightlies and release wheels?  Or
> should we just use a separate account on anaconda.org?
>
>
>
> Thanks,
>
> Kevin
>
>
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Replacement for Rackspace

[Peter Cock]

Hi Matti,

Is this an open invitation to the wider Numpy ecosystem? I am
interested on behalf of Biopython which was using the donated
Rackspace for multibuild wheel staging prior to PyPy release
(although having weekly test releases sounds interesting too).

I would be happy to continue this discussion off list if you prefer,

Thank you,

Peter

On Mon, Aug 10, 2020 at 9:20 PM Matti Picus  wrote:

>
> On 8/10/20 10:54 PM, Peter Wang wrote:
> > FWIW, we're happy to provide wheel hosting for statsmodels on
> > anaconda.org .
> >
> > -Peter
> >
> > On Fri, Aug 7, 2020 at 8:01 AM Kevin Sheppard
> > mailto:kevin.k.shepp...@gmail.com>> wrote:
> >
> > The Rackspace hosted wheel endpoints at
> >
> >
> https://7933911d6844c6c53a7d-47bd50c35cd79bd838daf386af554a83.ssl.cf2.rackcdn.com/
> >
> > and
> >
> >
> https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a43.ssl.cf2.rackcdn.com/
> >
> > seem to not be working.  I know NumPy, SciPy, pandas and
> > scikit-learn are all using a common end point on anacona.org
> > . Statsmodels is preparing for  release, and
> > the wheel builder at
> > https://github.com/MacPython/statsmodels-wheels is failing at
> > upload.  Is there any shared resource for uploading nightlies and
> > release wheels?  Or should we just use a separate account on
> > anaconda.org ?
> >
> > Thanks,
> >
> > Kevin
> >
> > ___
> > NumPy-Discussion mailing list
> > NumPy-Discussion@python.org 
> > https://mail.python.org/mailman/listinfo/numpy-discussion
> >
> >
>
> Thanks Peter, anaconda is generously hosting projects at
> https://anaconda.org/scipy-wheels-nightly/ (for weekly development
> releases that can be used to test downstream projects) and
> https://anaconda.org/multibuild-wheels-staging (for staging wheels to be
> tested for release on PyPI).
>
>
> The trick is that CI needs a token so it can upload to those
> organizations. Kevin, we can either add you to the groups you can create
> a token, or one of the current members could create tokens and transport
> them safely to Kevin. Please disucss it with me (or one of the other
> members https://anaconda.org/multibuild-wheels-staging/groups).
>
> Matti
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Staging Biopython wheels on Anaconda.org, was: Replacement for Rackspace

[Peter Cock]

Hi Matti,

If your offer still stands, I'd like to transition Biopython from the
donated Rackspace storage to Anaconda as discussed a few weeks back.

I presume this means updating the WHEELHOUSE credentials in our multi-wheel
repository, both for TravisCI and AppVeyor?

https://github.com/biopython/biopython-wheels

Do you need any other information from me, or the Biopython team?

Thank you,

Peter,
(On behalf of Biopython)

On Tue, Aug 11, 2020 at 6:34 AM Matti Picus  wrote:

>
> On 8/11/20 12:39 AM, Peter Cock wrote:
>
> Hi Matti,
>
> Is this an open invitation to the wider Numpy ecosystem? I am
> interested on behalf of Biopython which was using the donated
> Rackspace for multibuild wheel staging prior to PyPy release
> (although having weekly test releases sounds interesting too).
>
> I would be happy to continue this discussion off list if you prefer,
>
> Thank you,
>
> Peter
>
> On Mon, Aug 10, 2020 at 9:20 PM Matti Picus  wrote:
>
>> anaconda is generously hosting projects at
>> https://anaconda.org/scipy-wheels-nightly/ (for weekly development
>> releases that can be used to test downstream projects) and
>> https://anaconda.org/multibuild-wheels-staging (for staging wheels to be
>> tested for release on PyPI).
>>
>>
>> The trick is that CI needs a token so it can upload to those
>> organizations. Kevin, we can either add you to the groups you can create
>> a token, or one of the current members could create tokens and transport
>> them safely to Kevin. Please disucss it with me (or one of the other
>> members https://anaconda.org/multibuild-wheels-staging/groups).
>>
>> Matti
>
>
> Yes, it is a general invitation. My mail is in the message.
>
> I guess we should set up some kind of janitor task to remove older
> packages from the hosting space as usage goes up.
>
> Matti
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.20.1 availability

[Peter Cock]

I would recommend using the community run conda-forge as one of your
default conda channels. They have a very slick largely automated system
to update recipes when upstream makes a release. The default Anaconda
channel from Anaconda, Inc. (formerly Continuum Analytics, Inc.) is
comparatively slow.

You may recognise some of the maintainers of the conda-forge numpy
recipe? https://github.com/conda-forge/numpy-feedstock/

I'm impressed to see 17 million conda-forge numpy downloads, vs
'just' 2.5 million downloads of the default channel's package:

https://anaconda.org/conda-forge/numpy
https://anaconda.org/anaconda/numpy

Regards,

Peter

On Sun, Mar 14, 2021 at 8:06 AM Matti Picus  wrote:
>
> On 3/14/21 6:12 AM, dan_patterson wrote:
>
> Any idea why the most recent version isn't available on the main anaconda
> channel.  conda-forge and building are not options for a number of reasons.
> I posted a package request there but double digit days have gone by it just
> got a thumbs up and package-request tag
> https://github.com/ContinuumIO/anaconda-issues/issues/12309
> I realize it could be the "times" or maybe no one is aware of its absence.
>
>
> NumPy does not control the packages on the main anaconda channel, so a 
> request here is likely to go unanswered. The package has been updated in the 
> conda-forge channel.
>
>
> Matti
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Re: Numpy binary wheels and CI for win/arm64 platform

[Peter Cock]

On Tue, Nov 2, 2021 at 1:07 PM Ralf Gommers  wrote:
>
> Our current wheel build machinery is at 
> https://github.com/MacPython/numpy-wheels/,
> but please ignore that. We just merged a PR which uses cibuildwheel into the 
> main repo.
> That should be the target. If cibuildwheel has/gains the ability to produce 
> win-arm64 wheels,
> that'd be a good first step towards support.
>

That would be https://github.com/numpy/numpy/pull/20102 I presume?

I expect quite a few projects using numpy will follow your lead here. Thanks!

Peter
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: No credits left in travis for 1.22.0 release

[Peter Cock]

Hello Matti, Charles,

I'm a minor numpy contributor but mostly follow the mailing list with an eye
on any impacts downstream for Biopython. Like numpy and scipy etc, we
had been using TravisCI to build our wheels:

https://github.com/biopython/biopython-wheels

Lots of other projects using multibuild are in a similar situation. It
looks like
we will probably migrate to GitHub actions, but using TravisCI for one
more release is tempting if getting credits is straight forward.

Do you have any advice on how to request TravisCI credits (I'd be happy to
discuss off list), or would you advise us to focus on the migration now (which
pushes back any release plans).

Thank you,

Peter

On Fri, Dec 31, 2021 at 8:36 PM Charles R Harris
 wrote:
>
>
>
> On Fri, Dec 31, 2021 at 12:30 PM matti picus  wrote:
>>
>> We are back in business with 100k credits.
>> Matti
>>
>
> Thanks Matti.
>
> 
>
> Chuck
>
> ___
> NumPy-Discussion mailing list -- numpy-discussion@python.org
> To unsubscribe send an email to numpy-discussion-le...@python.org
> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
> Member address: p.j.a.c...@googlemail.com
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: No credits left in travis for 1.22.0 release

[Peter Cock]

That's great, thanks! Hopefully I'll get time to sort that out soon.

Peter

On Thu, Jan 6, 2022 at 3:23 PM Matti Picus  wrote:
>
> On 6/1/22 2:15 pm, Peter Cock wrote:
>
> > Hello Matti, Charles,
> >
> > ...
> >
> > Do you have any advice on how to request TravisCI credits (I'd be happy to
> > discuss off list), or would you advise us to focus on the migration now 
> > (which
> > pushes back any release plans).
> >
> > Thank you,
> >
> > Peter
> >
> Here is what travis-ci wrote me when I asked about free credits for
> multibuild:
>
> ---
>
> We offer an Open Source Subscription for free to non-commercial
> open-source projects. To qualify for an Open Source subscription, the
> project must meet the following requirements:
>
>   * You are a project lead or regular committer (latest commit in the
> last month)
>   * Project must be at least 3 months old and is in active development
> (with regular commits and activity)
>   * Project meets the OSD <https://opensource.org/docs/osd> specification
>   * Project must not be an expressly commercial project
>   * Project can not provide commercial services or distribute paid
> versions of the software
>
> 
>
>
>
> So I wrote to supp...@travis-ci.com:
>
> -
>
> I am one of the project leads and the number 4 contributor to the
> project in 2021
>
> https://github.com/multi-build/multibuild/graphs/contributors?from=2021-01-04&to=2021-10-13&type=c
>
>
> The graph above shows that the project is under active development.
>
> The license for the project is BSD 2
>
> https://github.com/multi-build/multibuild/blob/devel/LICENSE
>
> The project is a support framework for open source projects and provides
> no commercial services or paid versions.
> 
> Matti
>
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: NumPy 1.22.1 has been released.

[Peter Cock]

Dear Charles,

Thank you for your work on the numpy releases, including v1.22.1.

I noticed that for Windows Python 3.10, there is only a 64-bit wheel:

numpy-1.22.1-cp310-cp310-win_amd64.whl

That is true both on PyPI and in the list of checksums here:

https://github.com/numpy/numpy/releases/tag/v1.22.1

However, for older versions of Python there are also 32-bit Windows wheels,
e.g.

numpy-1.22.1-cp39-cp39-win32.whl
numpy-1.22.1-cp39-cp39-win_amd64.whl

numpy-1.22.1-cp38-cp38-win32.whl
numpy-1.22.1-cp38-cp38-win_amd64.whl

Is this expected? My interest is in building Windows Wheels for packages
compiling against NumPy.

Thank you,

Peter

On Fri, Jan 14, 2022 at 8:29 PM Charles R Harris 
wrote:

> Hi All,
>
> On behalf of the NumPy team, I'm pleased to announce the release of NumPy
> 1.22.1. NumPy 1.22.1 fixes several bugs discovered after the 1.22.0
> release. Notable fixes are:
>
>- Fix for f2PY docstring problems (SciPy)
>- Fix for reduction type problems (AstroPy)
>- Fixes for various typing bugs.
>
> The Python versions supported in this release are 3.8-3.10.  Wheels can be
> downloaded from PyPI ; source
> archives, release notes, and wheel hashes are available on Github
> . Linux users will
> need pip >= 0.19.3 in order to install the manylinux2014 wheels. A recent
> version of pip is needed to install the universal2 macos wheels.
>
> *Contributors*
>
> A total of 14 people contributed to this release.  People with a "+" by
> their
> names contributed a patch for the first time.
>
>- Arryan Singh
>- Bas van Beek
>- Charles Harris
>- Denis Laxalde
>- Isuru Fernando
>- Kevin Sheppard
>- Matthew Barber
>- Matti Picus
>- Melissa Weber Mendonça
>- Mukulika Pahari
>- Omid Rajaei +
>- Pearu Peterson
>- Ralf Gommers
>- Sebastian Berg
>
>
> *Pull requests merged*
> A total of 20 pull requests were merged for this release.
>
>- #20702: MAINT, DOC: Post 1.22.0 release fixes.
>- #20703: DOC, BUG: Use pngs instead of svgs.
>- #20704: DOC: Fixed the link on user-guide landing page
>- #20714: BUG: Restore vc141 support
>- #20724: BUG: Fix array dimensions solver for multidimensional
>arguments...
>- #20725: TYP: change type annotation for ``__array_namespace__`` to
>ModuleType
>- #20726: TYP, MAINT: Allow ``ndindex`` to accept integer tuples
>- #20757: BUG: Relax dtype identity check in reductions
>- #20763: TYP: Allow time manipulation functions to accept ``date``
>and ``timedelta``...
>- #20768: TYP: Relax the type of ``ndarray.__array_finalize__``
>- #20795: MAINT: Raise RuntimeError if setuptools version is too
>recent.
>- #20796: BUG, DOC: Fixes SciPy docs build warnings
>- #20797: DOC: fix OpenBLAS version in release note
>- #20798: PERF: Optimize array check for bounded 0,1 values
>- #20805: BUG: Fix that reduce-likes honor out always (and live in
>the...
>- #20806: BUG: ``array_api.argsort(descending=True)`` respects
>relative...
>- #20807: BUG: Allow integer inputs for pow-related functions in
>``array_api``
>- #20814: DOC: Refer to NumPy, not pandas, in main page
>- #20815: DOC: Update Copyright to 2022 [License]
>- #20819: BUG: Return correctly shaped inverse indices in array_api
>set...
>
> Cheers,
>
> Charles Harris
> ___
> NumPy-Discussion mailing list -- numpy-discussion@python.org
> To unsubscribe send an email to numpy-discussion-le...@python.org
> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
> Member address: p.j.a.c...@googlemail.com
>
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: NumPy 1.22.1 has been released.

[Peter Cock]

That's helpful and informative, thank you Matti.

It will also be interesting to see how many calls you get for the
missing wheel (if any). I note that the Python.org download
page says quite clearly that the 64-bit Windows installer is
recommended.

Kind regards,

Peter

On Tue, Jan 18, 2022 at 7:24 PM Matti Picus  wrote:

> On 18/1/22 5:37 pm, Peter Cock wrote:
>
> > Dear Charles,
> >
> > Thank you for your work on the numpy releases, including v1.22.1.
> >
> > I noticed that for Windows Python 3.10, there is only a 64-bit wheel:
> >
> >
> > ...
> >
> > Is this expected? My interest is in building Windows Wheels for
> > packages compiling against NumPy.
> >
> > Thank you,
> >
> > Peter
> >
> >
>
> Hi Peter. This is intentional, as our wheel building system is based on
> Azure CI, and they did not provide 32-bit python for 3.10 for windows.
> We decided to "solve" this and other wheel building issues by moving to
> cibuildwheel in github actions in the main repo (building on travis-ci
> for the aarch64 wheels) [1] for the 1.23 release. Digging around in the
> azure image repo issues, I see they may have restored 32-bit python [2]
> in mid Nov 2021, when we were already well into the 1.22 release cycle.
> I guess we could try to see if it works now. A PR to modify the azure
> pipelines yml would move us in this direction. I don't know that even if
> the PR is successful if we would put the wheels up on PyPI, that
> decision is up to the release manager.
>
> Matti
>
>
> [1] https://github.com/numpy/numpy/issues/20277
>
> [2] https://github.com/actions/virtual-environments/issues/4400
>
> [3]
> https://github.com/MacPython/numpy-wheels/blob/main/azure-pipelines.yml
>
>
> ___
> NumPy-Discussion mailing list -- numpy-discussion@python.org
> To unsubscribe send an email to numpy-discussion-le...@python.org
> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
> Member address: p.j.a.c...@googlemail.com
>
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: Automatic formatters for only changed lines

[Peter Cock]

On Fri, May 6, 2022 at 4:59 PM Charles R Harris 
wrote:

>
> Heh. I think automatic formatting is the future and was happy to see the
> PR. The only question is if black is the way we want to go. IIRC, the main
> sticking points were
>
>- Line length (<= 79).
>- Quotation mark changes (noisy).
>- Formatting of  '*', '**', and '/'
>
> Even if the result isn't perfect by our current standards, I suspect we
> will get used to it and just not worry anymore.
>

On that note, in black v22.1.0 (the first non-beta release), one of the
changes was to the ** operator to better suit mathematical conventions:

 "Remove spaces around power operators if both operands are simple"
https://github.com/psf/black/pull/2726

Either way I agree with you, most people seem to get used to black and stop
worrying about it.

Peter
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: Representation of NumPy scalars

[Peter Cock]

Hello Sebastian,

I rarely use NumPy scalars directly, but the repr change could
have impact in assorted downstream projects' documentation.

For clarity, this idea would not alter how NumPy arrays print,
would it - since they already include the type information?

>>> np.array([34.3, 10.1, -0.5], np.float32)
array([34.3, 10.1, -0.5], dtype=float32)
>>> np.array([5, 10, 0], np.uint8)
array([ 5, 10,  0], dtype=uint8)

Thanks,

Peter

On Thu, Sep 8, 2022 at 10:42 AM Sebastian Berg 
wrote:

>
> TL;DR:  NumPy scalars representation is e.g. `34.3` instead of
> `float32(34.3)`.  So the representation is missing the type
> information.  What are your thoughts on changing that?
>
>
> Hi all,
>
> I am thinking about the next steps for NEP 50 (The NEP wants to fix the
> NumPy promotion rules, especially with respect to scalars):
>
> https://numpy.org/neps/nep-0050-scalar-promotion.html
>
> In relation to that, there was one point that Stéfan brought up
> previously.
>
> The NumPy scalars (representation) currently print as numbers:
>
> >>> np.float32(34.3)
> 34.3
> >>> np.uint8(5)
> 5
>
> That can already be confusing now.  However, it gets more problematic
> if NEP 50 is introduced since the behavior between a Python `34.3` and
> `np.float32(34.3)` would differ more than it does now (please refer to
> the NEP).
>
> The change would be that we should print as:
>
> float64(34.3)  (or similar?)
>
> This Email is mainly to ask for any feedback or concern on such a
> change.  I suspect we may have to write a very brief NEP about it.
>
> If there is little concern, maybe we could move forward such a change
> promptly.  Otherwise it could be moved forward together with NEP 50 and
> take effect in a "major" release [1].
>
> Cheers,
>
> Sebastian
>
>
>
> [1] Note that for me, even a major release would hopefully not affect
> the majority of users or be very disruptive.
>
> ___
> NumPy-Discussion mailing list -- numpy-discussion@python.org
> To unsubscribe send an email to numpy-discussion-le...@python.org
> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
> Member address: p.j.a.c...@googlemail.com
>
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: Change in numpy.percentile

[Peter Cock via NumPy-Discussion]

On Tue, Oct 10, 2023 at 6:32 PM Matthew Brett 
wrote:

> Hi,
>
>
> On Tue, 10 Oct 2023 at 00:55, Andrew Nelson  wrote:
> >
> >
> > On Mon, 9 Oct 2023 at 23:50, Matthew Brett 
> wrote:
> >>
> >> Hi,
> >>
> >> On Mon, Oct 9, 2023 at 11:49 AM Andrew Nelson 
> wrote:
> >> Could you say more about why you consider:
> >> np.mean(x, dropna=True)
> >> to be less clear in intent than:
> >> np.nanmean(x)
> >> ?  Is it just that someone could accidentally forget that the default
> >
> >
> > The discussion isn't a deal breaker for me, I just wanted to put out a
> different POV.
> > The name of the function encodes what it does. By putting them both in
> the function name it's clear what the function does.
> >
> > ...
> >
> > Imagine that one has a large codebase and you have to find all the
> locations where nans could affect a mean. There may be lots of prod, sum,
> etc, also distributed within the codebase. You wouldn't want to search for
> `dropna` because you get every function that handles a nan. If you search
> for nanmean you only get the locations you want.
>
> So, is this the more or less the difference between:
>
> grep 'np\.nanmean' *.py
>
> and
>
> grep 'np\.mean(.*,\s*dropna\s*=\s*True' *.py
>
> ?
>
> Cheers,
>
> Matthew
>
>
Keep in mind that the dropna argument might very well be on a different
line (especially with black formatting), so searches could be much harder
than looking for the nanmean function.

(I do not deal with enough NaN data to have a strong view either way here)

Peter
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Black style as applied to np.array(...) and the ruff formatter

[Peter Cock via NumPy-Discussion]

Hello all,

I imagine there are many people here using the black coding style as
implemented by the tool black, albeit with reservations about how it
lays out arrays by default (often therefore wrapped in a format off/on
block to exclude the array from automatic layout to allow for manual
column based layouts).

You may already be aware of the tool ruff as a fast alternative to flake8,
but it now has a formatter which implements the black format (with some
minor divergences):

https://astral.sh/blog/the-ruff-formatter

The authors are open to exploring special casing how it autoformats
arrays, and I think input now from the numpy community would be a
good idea:

https://github.com/astral-sh/ruff/discussions/8452

Kind regards,

Peter
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: New Ruff rule for migrating to NumPy 2.0

[Peter Cock via NumPy-Discussion]

This looks handy - I used the following to try it:

$ pip install -U ruff
$ ruff --preview --select NPY201 --fix 

Happily nothing to address on the code baseI tried.

Thanks,

Peter

On Thu, Jan 11, 2024 at 11:32 AM Mateusz Sokol  wrote:
>
> Hi all!
>
> Some time ago we added a new rule to Ruff linter, "NPY201", which updates the 
> codebase to a NumPy 2.0 compatible version.
>
> You can read about it in the migration guide: 
> https://numpy.org/devdocs/numpy_2_0_migration_guide.html#ruff-plugin
> And on the Ruff docs website: 
> https://docs.astral.sh/ruff/rules/numpy2-deprecation/
> (it's still in a "preview" mode but available since 0.1.4 release).
>
> Best regards,
> Mateusz
> ___
> NumPy-Discussion mailing list -- numpy-discussion@python.org
> To unsubscribe send an email to numpy-discussion-le...@python.org
> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
> Member address: p.j.a.c...@googlemail.com
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: What should remain on PyPi

[Peter Cock via NumPy-Discussion]

If I recall correctly, people were building against the Numpy 2.0.0 release
candidates in particular. In hindsight keeping those on PyPI might have
been better. A formal NEP/SPEC seems a good idea.

Peter

On Tue, Sep 3, 2024 at 6:20 PM matti picus  wrote:

> I would prefer we never delete packages once we upload them to PyPI,
> unless there are security issues with them. As Sean demonstrated,
> someone somewhere is going to be using them, and deleting packages
> will inevitably break something.
> Matti
>
>
> On Tue, Sep 3, 2024 at 7:44 PM Sean Gillies 
> wrote:
> >
> > Hi Chuck,
> >
> > I've got a version of a package on PyPI that requires Numpy 2.0.0rc1 at
> build time. Not the best decision in hindsight, but I assumed that Numpy
> was the kind of project that wouldn't remove published distributions unless
> there were security issues. It had not up today, right? Would it be
> possible to restore 2.0.0rc1?
> >
> > On Tue, Sep 3, 2024 at 9:20 AM Charles R Harris <
> charlesr.har...@gmail.com> wrote:
> >>
> >> Hi All,
> >>
> >> I just got through deleting a bunch of pre-releases on PyPi and it
> occurred to me that we should have a policy as to what releases should be
> kept. I think that reproducibility requires that we keep all the major and
> micro versions, but if so, we should make that an official guarantee.
> Perhaps a short NEP? This might even qualify for an SPEC. Thoughts?
> >>
> >> Chuck
> >
> >
> > --
> > Sean Gillies
> > ___
> > NumPy-Discussion mailing list -- numpy-discussion@python.org
> > To unsubscribe send an email to numpy-discussion-le...@python.org
> > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
> > Member address: matti.pi...@gmail.com
> ___
> NumPy-Discussion mailing list -- numpy-discussion@python.org
> To unsubscribe send an email to numpy-discussion-le...@python.org
> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
> Member address: p.j.a.c...@googlemail.com
>
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: PR-23061

[Peter Cock via NumPy-Discussion]

On Sat, Mar 25, 2023 at 12:35 PM Matteo Raso via NumPy-Discussion
 wrote:
>
> P.S. I originally tried to send this message as an email, but it was instantly
> rejected because I'm not a list member. That's a pretty serious error for a
> public mailing list.

That's very normal on a mailing list. Even if you disagree with the design,
the immediate failure message was very clear so you know how to fix it
(sign up first, then resend your message).

And the URL to the issue in question, certainly not a trivial issue where
a quick review and resolution might be expected:

https://github.com/numpy/numpy/pull/23061

Peter
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com