Re: [Numpy-discussion] Proposal to accept NEP 49: Data allocation strategies

[Eric Wieser]

> The Python version of this does have a `void *ctx`, but I am not sure if
the use for this is actually valuable for the NumPy use-cases.

Do you mean "the CPython version"? If so, can you link a reference?

>  While I like the `PyObject *` idea, I am also not sure that it helps
much.  If we want allocation specific state, the user should overallocate
and save it before the actual allocation.

I was talking about allocator- not allocation- specific state. I agree that
the correct place to store the latter is by overallocating, but it doesn't
make much sense to me to duplicate state about the allocator itself in each
allocation.

> But if we don't mind the churn it creates, the only serious idea I would
have right now is using a `FromSpec` API. We could allow get/set functions
on it though

We don't even need to go as far as a flexible `FromSpec` API. Simply having
a function to allocate (and free) the opaque struct and a handful of
getters ought to be enough to let us change the allocator to be stateful in
future.
On the other hand, this is probably about as much work as just making it a
PyObject in the first place.

Eric
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] GSoD'21- Submitting first draft of the Statement of Interest.

[Melissa Mendonça]

Thank you, Ayush! We will take your project into consideration and let you
all know of our decision shortly.

Cheers!

- Melissa

On Mon, May 10, 2021 at 8:26 AM Ayush Verma  wrote:

> I have updated the statement with links to a personal project's
> documentation following the Diataxis framework guidelines. I am also
> attaching the links below in the mail itself.
> Please find the updated document and attached links below.
> Github repo link of the project: www.github.com/verma16Ayush/profpoll
> Hosted documentation: profpoll.readthedocs.io/en/latest
> Thanks,
> Ayush
>
> On Fri, May 7, 2021 at 6:37 PM Ayush Verma 
> wrote:
>
>> Hello everyone,
>> I have attached my Statement of Interest for GSoD'21 below. I will also
>> be adding a couple more links to the statement within a day or two. Till
>> then, I would love to get reviews from the community on the proposed
>> changes.
>> Cheers.
>> Ayush.
>>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] GSoD 2021 - Statement of Interest

[Melissa Mendonça]

Hello, Mahesh!

We will take your project into consideration and let you know of our
decision shortly.

Cheers!

Melissa

On Sat, May 8, 2021 at 4:19 PM Mahesh S  wrote:

> Hi there everyone,
>
> I am attaching my in depth proposal for Google Season of Docs 2021 with
> NumPy along with this mail. Kindly share your thoughts on it
>
> Thanks & Regards
> Mahesh S Nair
>
> On Wed, May 5, 2021 at 2:06 AM Melissa Mendonça 
> wrote:
>
>> Hello, Mahesh
>>
>> Yes, the timeline looks fine.
>>
>> Cheers,
>>
>> Melissa
>>
>> On Sun, May 2, 2021 at 2:08 PM Mahesh S  wrote:
>>
>>> Hello there Melissa,
>>>
>>> Thank You for your valuable suggestion.Actually , I am planning for a
>>> high-impact work. I have already started reading the User-Guide and working
>>> out small projects using NumPy to understand it further, to prepare an
>>> in-depth proposal, which will include changes which I mentioned in my brief
>>> proposal, reorganizing content, addressing issues in the GitHub tracker and
>>> all. One doubt from my side is regarding the timeline. Currently I am
>>> planning to prepare the proposal with the GSoD Timeline that is from June
>>> 16 2021 and ends on November 16th 2021 with four evaluation phases ,
>>> Is there any specific timeline from the community or Am I free to follow
>>> the same?
>>>
>>> And  as of now, I am planning to set an order in which the tasks needed
>>> to be done , which I will share in the first draft of my proposal which I
>>> will submit by this week, later changes can be done as per suggestions from
>>> the mentors and community.
>>>
>>> Thank You
>>> Mahesh
>>>
>>> On Sun, May 2, 2021 at 7:05 PM Melissa Mendonça 
>>> wrote:
>>>
 Hello, Mahesh

 While you work on this, it may be interesting to keep in mind that we
 are looking for high-impact work that can be done in the timeframe of the
 GSoD program - examples would be reorganizing content in a section of the
 documentation, creating new complete document pages on some subject or
 concept. It may be worth familiarizing yourself with the documentation (I
 won't suggest reading all of it as it's huge!) to get an idea for the
 different sections. One idea would be to focus on the user guide, which
 contains several sub-pages, and check for improvements that can be done
 there.

 Cheers,

 Melissa

 On Fri, Apr 30, 2021 at 9:28 AM Mahesh S 
 wrote:

> Hi there,
>
> The current documentation of NumPy is really good but a bit more
> improvement can be made to it, which is the prime objective of my project.
> The improvements which I mentioned in my brief proposal are strategies 
> that
> can be applied to every documentation to make it better.
>
> Apart from general improvements most the documentation related issues
> in the NumPy's GitHub issue tracker
> 
>  will
> be addressed. . Some needs more technical information and help from the
> community. Some are due to the lack of visual aids. Most of them will be
> addressed and improvements will be made such that similar issues will not
> be generated in future.
> Rearranging of sections in the User Guide
>  can be done
> after further discussions
>
> Some examples regarding the need of restructuring and duplication are
> given in the attached document. Apologies in advance if my observations 
> are
> inaccurate or nitpicks. The given doc is just a very brief one. An 
> in-depth
> proposal with all the planned changes along with solutions to close as 
> many
> issues in the tracker will be prepared. I am getting familiar with the
> community and NumPy.
>
>
> On Thu, Apr 29, 2021 at 9:54 PM Melissa Mendonça 
> wrote:
>
>> Hello, Mahesh
>>
>> Thank you for your proposal. One thing I would say is that some of
>> the things you are suggesting are already there - for example, the 
>> How-tos
>> and Explanations section - but I agree they can be improved!
>>
>> Another question I could pose is about duplication of content: can
>> you give examples of duplicated content that you found in the docs?
>>
>> Cheers,
>>
>> Melissa
>>
>> On Wed, Apr 28, 2021 at 12:33 AM Mahesh S 
>> wrote:
>>
>>> Hello,
>>>
>>> I am Mahesh from India. I am interested in doing Google Season of
>>> Docs with NumPy on the project HIGH-LEVEL RESTRUCTURING AND END-USER 
>>> FOCUS
>>> - NumPy
>>>
>>> I have past experience in documentation writing with WordPress and
>>> have completed *Google Summer of Code 2018 *with KDE. I have been
>>> an open source enthusiast and contributor since 2017.My past experience
>>> with coding ,code documentation , understa

[Numpy-discussion] NumPy 1.20.3 release

[Charles R Harris]

Hi All,

On behalf of the NumPy team I am pleased to announce the release of NumPy
1.20.3. NumPy 1,20.3 is a bugfix release containing several fixes merged to
the main branch after the NumPy 1.20.2 release. The Python versions
supported for this release are 3.7-3.9. Wheels can be downloaded from PyPI
; source archives, release notes,
and wheel hashes are available on Github
. Linux users will
need pip >= 0.19.3 in  order to install manylinux2010 and manylinux2014
wheels.

*Contributors*

A total of 7 people contributed to this release.  People with a "+" by their
names contributed a patch for the first time.

   - Anne Archibald
   - Bas van Beek
   - Charles Harris
   - Dong Keun Oh +
   - Kamil Choudhury +
   - Sayed Adel
   - Sebastian Berg


*Pull requests merged*
A total of 15 pull requests were merged for this release.

   - #18763: BUG: Correct ``datetime64`` missing type overload for
   ``datetime.date``...
   - #18764: MAINT: Remove ``__all__`` in favor of explicit re-exports
   - #18768: BLD: Strip extra newline when dumping gfortran version on MacOS
   - #18769: BUG: fix segfault in object/longdouble operations
   - #18794: MAINT: Use towncrier build explicitly
   - #18887: MAINT: Relax certain integer-type constraints
   - #18915: MAINT: Remove unsafe unions and ABCs from return-annotations
   - #18921: MAINT: Allow more recursion depth for scalar tests.
   - #18922: BUG: Initialize the full nditer buffer in case of error
   - #18923: BLD: remove unnecessary flag ``-faltivec`` on macOS
   - #18924: MAINT, CI: treats _SIMD module build warnings as errors
   through...
   - #18925: BUG: for MINGW, threads.h existence test requires GLIBC > 2.12
   - #18941: BUG: Make changelog recognize gh- as a PR number prefix.
   - #18948: REL, DOC: Prepare for the NumPy 1.20.3 release.
   - #18953: BUG: Fix failing mypy test in 1.20.x.


Cheers,

Charles Harris
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposal to accept NEP 49: Data allocation strategies

[Sebastian Berg]

On Mon, 2021-05-10 at 10:01 +0100, Eric Wieser wrote:
> > The Python version of this does have a `void *ctx`, but I am not
> > sure if
> the use for this is actually valuable for the NumPy use-cases.
> 
> Do you mean "the CPython version"? If so, can you link a reference?

Yes, sorry, had been a while since I had looked it up:

https://docs.python.org/3/c-api/memory.html#c.PyMemAllocatorEx

That all looks like it can be customized in theory. But I am not sure
that it is practical, except for hooking and calling the previous one.
(But we also have tracemalloc anyway?)  I have to say it feels a bit
like exposing things publicly, that are really mainly used internally,
but not sure...  Presumably Python uses the `ctx` for something though.


> 
> >  While I like the `PyObject *` idea, I am also not sure that it
> > helps
> much.  If we want allocation specific state, the user should
> overallocate
> and save it before the actual allocation.
> 
> I was talking about allocator- not allocation- specific state. I
> agree that
> the correct place to store the latter is by overallocating, but it
> doesn't
> make much sense to me to duplicate state about the allocator itself
> in each
> allocation.
> 

Right, I don't really know a use-case right now.  But I am fine with
saying: lets pass in some state anyway, to future-proof.  Although if
we ensure that the API can be extended, even that is probably not
really necessary, unless we have a faint idea how it would be used?

(I guess the C++ similarity may be a reason, but I am not familiar with
that.)
  

> > But if we don't mind the churn it creates, the only serious idea I
> > would
> have right now is using a `FromSpec` API. We could allow get/set
> functions
> on it though
> 
> We don't even need to go as far as a flexible `FromSpec` API. Simply
> having
> a function to allocate (and free) the opaque struct and a handful of
> getters ought to be enough to let us change the allocator to be
> stateful in
> future.
> On the other hand, this is probably about as much work as just making
> it a
> PyObject in the first place.

Yeah, if we don't expect things to grow often/much, we can just use
what we have now and either add `NULL` argument at the end and/or just
make a new function when we need it.

The important part would be returning a new struct.  I think even
opaque is not necessary!
If we return the new struct, we can extend it freely and return NULL to
indicate an error (thus being able to deprecate if we have to).

Right now we don't even have getters in the proposal IIRC, so that part
probably just doesn't matter either.  (If we want to allow to fall back
to the previous allocator this would have to be expanded.)


I agree that `PyObject *` is probably just as well if you want the
struct to be free'able since then you suddenly need reference counting
or similar!
But right now the proposal says this is static, and I honestly don't
see much reason for it to be freeable?  The current use-cases `cupy` or
`pnumpy` don't not seem to need it.

If we return a new struct (I do not care if opaque or not), all of that
can still be expanded.  Should we just do that?
Or can we think of any downside to that or use-case where this is
clearly too limiting right now?

Cheers,

Sebastian


> 
> Eric
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion


___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposal to accept NEP 49: Data allocation strategies

[Matti Picus]

On 10/5/21 8:43 pm, Sebastian Berg wrote:


But right now the proposal says this is static, and I honestly don't
see much reason for it to be freeable?



I think this is the crux of the issue. The current design is for a 
singly-allocated struct to be passed around since it is just an 
aggregate of functions. If someone wants a different strategy (i.e. 
different alignment) they create a new policy: there are no additional 
parameters or data associated with the struct. I don't really see an ask 
from possible users for anything more, and so would prefer to remain 
with the simplest possible design. If the need arises in the future for 
additional data, which is doubtful, I am confident we can expand this as 
needed, and do not want to burden the current design with unneeded 
optional features.



It would be nice to hear from some actual users if they need the 
flexibility.



In any case I would like to resolve this quickly and get it into the 
next release, so if Eric is adamant that the advanced design is needed I 
will accept his proposal, since that seems easier than any of the 
alternatives so far.



Matti

___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion