Re: [Cython] Python array support (#113)
On Sat, May 5, 2012 at 10:42 PM, Stefan Behnel wrote: > Stefan Behnel, 05.05.2012 21:50: >>> https://github.com/cython/cython/pull/113 >> >> This looks ok to me now. There have been objections back when we discussed >> the initial patch for array.array support, so what do you think about >> merging this in? > > One think I'm not sure about is how to deal with the header file. It would > be nice to not rely on an external dependency that users need to ship with > their code. Moving this into our utility code to write it into the C file > would remove that need, but we don't currently have a way to trigger > utility code insertion from .pxd files explicitly. Should we special case > "cimport cpython.array" for this? That's my biggest concern as well (though I only quickly skimmed through the code). I would be OK with special casing this Python builtin. > Oh, and maybe we should also provide a fused type for the supported array > item types to make it easier for users to write generic array code? > (Although the mass of types may be overkill for most users...) I think it's easy enough for the end user to make a fused type of the specializations they care about. - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Python array support (#113)
On 5 May 2012 20:50, Stefan Behnel wrote: >> https://github.com/cython/cython/pull/113 > > This looks ok to me now. There have been objections back when we discussed > the initial patch for array.array support, so what do you think about > merging this in? > > Stefan > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel Great, I think it can be quite useful for some people. I think only some documentation is missing. Also, very minor complaint, the way it allocates shape, strides and the format string in __getbuffer__ is weird and complicated for no good reason. I think it's better to declare two Py_ssize_t scalars or one-sized arrays as class attributes and one char[2] array, and use those (then you can also get rid of __releasebuffer__). ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Python array support (#113)
mark florisson, 06.05.2012 10:56: > On 5 May 2012 20:50, Stefan Behnel wrote: >>> https://github.com/cython/cython/pull/113 >> >> This looks ok to me now. There have been objections back when we discussed >> the initial patch for array.array support, so what do you think about >> merging this in? > > Great, I think it can be quite useful for some people. I think only > some documentation is missing. > > Also, very minor complaint, the way it allocates shape, strides and > the format string in __getbuffer__ is weird and complicated for no > good reason. Maybe the reason is just that it wasn't written by you. ;) I take it that it's best to merge this pull request then, and to further fix it up afterwards. > I think it's better to declare two Py_ssize_t scalars or > one-sized arrays as class attributes and one char[2] array, and use > those (then you can also get rid of __releasebuffer__). Yes, that would be good. Note that itemsize is specifically designed so that it can be pointed to by strides for 1D arrays, and I guess shape can similarly just point to ob_size. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Python array support (#113)
On 6 May 2012 09:56, mark florisson wrote: > On 5 May 2012 20:50, Stefan Behnel wrote: >>> https://github.com/cython/cython/pull/113 >> >> This looks ok to me now. There have been objections back when we discussed >> the initial patch for array.array support, so what do you think about >> merging this in? >> >> Stefan >> ___ >> cython-devel mailing list >> cython-devel@python.org >> http://mail.python.org/mailman/listinfo/cython-devel > > Great, I think it can be quite useful for some people. I think only > some documentation is missing. > > Also, very minor complaint, the way it allocates shape, strides and > the format string in __getbuffer__ is weird and complicated for no > good reason. I think it's better to declare two Py_ssize_t scalars or > one-sized arrays as class attributes and one char[2] array, and use > those (then you can also get rid of __releasebuffer__). Or hm, that might be a problem with their variable nature? Then the shape of the buffer would suddenly change... Maybe malloc is better in __getbuffer__ then. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Python array support (#113)
On 6 May 2012 10:05, Stefan Behnel wrote: > mark florisson, 06.05.2012 10:56: >> On 5 May 2012 20:50, Stefan Behnel wrote: https://github.com/cython/cython/pull/113 >>> >>> This looks ok to me now. There have been objections back when we discussed >>> the initial patch for array.array support, so what do you think about >>> merging this in? >> >> Great, I think it can be quite useful for some people. I think only >> some documentation is missing. >> >> Also, very minor complaint, the way it allocates shape, strides and >> the format string in __getbuffer__ is weird and complicated for no >> good reason. > > Maybe the reason is just that it wasn't written by you. ;) > > I take it that it's best to merge this pull request then, and to further > fix it up afterwards. > Definitely, +1. >> I think it's better to declare two Py_ssize_t scalars or >> one-sized arrays as class attributes and one char[2] array, and use >> those (then you can also get rid of __releasebuffer__). > > Yes, that would be good. Note that itemsize is specifically designed so > that it can be pointed to by strides for 1D arrays, and I guess shape can > similarly just point to ob_size. > > Stefan > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Python array support (#113)
mark florisson, 06.05.2012 11:05:
> On 6 May 2012 09:56, mark florisson wrote:
>> On 5 May 2012 20:50, Stefan Behnel wrote:
https://github.com/cython/cython/pull/113
>>>
>>> This looks ok to me now. There have been objections back when we discussed
>>> the initial patch for array.array support, so what do you think about
>>> merging this in?
>>
>> Great, I think it can be quite useful for some people. I think only
>> some documentation is missing.
>>
>> Also, very minor complaint, the way it allocates shape, strides and
>> the format string in __getbuffer__ is weird and complicated for no
>> good reason. I think it's better to declare two Py_ssize_t scalars or
>> one-sized arrays as class attributes and one char[2] array, and use
>> those (then you can also get rid of __releasebuffer__).
>
> Or hm, that might be a problem with their variable nature? Then the
> shape of the buffer would suddenly change...
I'm fine with saying that any user who changes the size of an array while a
buffer view on it is being held (and used) is just plain out of warranty.
After all, a realloc() is allowed to move the memory buffer around and may
really do it in some cases, so the length is the least of our problems,
even if the array doesn't shrink but only grows.
I just noticed that the array module supports the buffer interface natively
in Python 3. That makes this whole patch somewhat less interesting, because
it's essentially just a work-around for a missing feature in Py2.
Py3 does the setup like this:
view->buf = (void *)self->ob_item;
view->obj = (PyObject*)self;
Py_INCREF(self);
if (view->buf == NULL)
view->buf = (void *)emptybuf;
view->len = (Py_SIZE(self)) * self->ob_descr->itemsize;
view->readonly = 0;
view->ndim = 1;
view->itemsize = self->ob_descr->itemsize;
view->suboffsets = NULL;
view->shape = NULL;
if ((flags & PyBUF_ND)==PyBUF_ND) {
view->shape = &((Py_SIZE(self)));
}
view->strides = NULL;
if ((flags & PyBUF_STRIDES)==PyBUF_STRIDES)
view->strides = &(view->itemsize);
view->format = NULL;
view->internal = NULL;
if ((flags & PyBUF_FORMAT) == PyBUF_FORMAT)
view->format = self->ob_descr->formats;
It also counts the number of exports and prevents resizing while a buffer
is being exported. The current .pxd implementation cannot achieve that,
which is really unfortunate. ISTM that it should fall through to the native
buffer interface if the underlying array supports it.
Stefan
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
[Cython] 0.17
Hey, I think we already have quite a bit of functionality (nearly) ready, after merging some pending pull requests maybe it will be a good time for a 0.17 release? I think it would be good to also document to what extent pypy support works, what works and what doesn't. Stefan, since you added a large majority of the features, would you want to be the release manager? In summary, the following pull requests should likely go in - array.array support (unless further discussion prevents that) - fused types runtime buffer dispatch - newaxis - more? The memoryview documentation should also be reworked a bit. Matthew, are you still willing to have a go at that? Otherwise I can clean up the mess first, some things are no longer true and simply outdated, and then have a second opinion. Mark ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] 0.17
On 05/06/2012 04:28 PM, mark florisson wrote: Hey, I think we already have quite a bit of functionality (nearly) ready, after merging some pending pull requests maybe it will be a good time for a 0.17 release? I think it would be good to also document to what extent pypy support works, what works and what doesn't. Stefan, since you added a large majority of the features, would you want to be the release manager? In summary, the following pull requests should likely go in - array.array support (unless further discussion prevents that) - fused types runtime buffer dispatch - newaxis - more? Sounds more like a 0.16.1? (Did we have any rules for that -- except the obvious one that breaking backwards compatibility in noticeable ways has to increment the major?) Dag The memoryview documentation should also be reworked a bit. Matthew, are you still willing to have a go at that? Otherwise I can clean up the mess first, some things are no longer true and simply outdated, and then have a second opinion. Mark ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] 0.17
Dag Sverre Seljebotn, 06.05.2012 19:51: > On 05/06/2012 04:28 PM, mark florisson wrote: >> I think we already have quite a bit of functionality (nearly) ready, >> after merging some pending pull requests maybe it will be a good time >> for a 0.17 release? I think it would be good to also document to what >> extent pypy support works, what works and what doesn't. Stefan, since >> you added a large majority of the features, would you want to be the >> release manager? >> >> In summary, the following pull requests should likely go in >> - array.array support (unless further discussion prevents that) >> - fused types runtime buffer dispatch >> - newaxis >> - more? > > > Sounds more like a 0.16.1? (Did we have any rules for that -- except the > obvious one that breaking backwards compatibility in noticeable ways has to > increment the major?) Those are only the pending pull requests, the current feature set in the master branch is way larger than that. I'll start writing up the release notes soon. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] 0.17
2012/5/6 Dag Sverre Seljebotn : > On 05/06/2012 04:28 PM, mark florisson wrote: >> >> Hey, >> >> I think we already have quite a bit of functionality (nearly) ready, >> after merging some pending pull requests maybe it will be a good time >> for a 0.17 release? I think it would be good to also document to what >> extent pypy support works, what works and what doesn't. Stefan, since >> you added a large majority of the features, would you want to be the >> release manager? >> >> In summary, the following pull requests should likely go in >> - array.array support (unless further discussion prevents that) >> - fused types runtime buffer dispatch >> - newaxis >> - more? > > > > Sounds more like a 0.16.1? (Did we have any rules for that -- except the > obvious one that breaking backwards compatibility in noticeable ways has to > increment the major?) > > Dag > > >> >> The memoryview documentation should also be reworked a bit. Matthew, >> are you still willing to have a go at that? Otherwise I can clean up >> the mess first, some things are no longer true and simply outdated, >> and then have a second opinion. >> +1, I think that 0.16 has some bugs and we should make a bugfix release. -- vitja. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] 0.17
mark florisson, 06.05.2012 16:28: > I think we already have quite a bit of functionality (nearly) ready, > after merging some pending pull requests maybe it will be a good time > for a 0.17 release? I think it would be good to also document to what > extent pypy support works, what works and what doesn't. Sure, although it's basically just "what works, works". However, there are certainly things that users must know in order to make their own code work. > Stefan, since > you added a large majority of the features, would you want to be the > release manager? I agree that it would make sense, but I'll be head under water for the next month or so. Maybe it would be better to put out a 0.16.1 with only selected fixes in the meantime? Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] 0.17
On 6 May 2012 19:29, Vitja Makarov wrote: > 2012/5/6 Dag Sverre Seljebotn : >> On 05/06/2012 04:28 PM, mark florisson wrote: >>> >>> Hey, >>> >>> I think we already have quite a bit of functionality (nearly) ready, >>> after merging some pending pull requests maybe it will be a good time >>> for a 0.17 release? I think it would be good to also document to what >>> extent pypy support works, what works and what doesn't. Stefan, since >>> you added a large majority of the features, would you want to be the >>> release manager? >>> >>> In summary, the following pull requests should likely go in >>> - array.array support (unless further discussion prevents that) >>> - fused types runtime buffer dispatch >>> - newaxis >>> - more? >> >> >> >> Sounds more like a 0.16.1? (Did we have any rules for that -- except the >> obvious one that breaking backwards compatibility in noticeable ways has to >> increment the major?) >> >> Dag >> >> >>> >>> The memoryview documentation should also be reworked a bit. Matthew, >>> are you still willing to have a go at that? Otherwise I can clean up >>> the mess first, some things are no longer true and simply outdated, >>> and then have a second opinion. >>> > > +1, I think that 0.16 has some bugs and we should make a bugfix release. > > -- > vitja. > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel Stefan bumped the version number to 0.17pre a while back, we had a discussion about this before that. I think the features are large enough to warrant a major release. If we do want a bugfix release, we'll probably have to cherrypick the fixes over, that would be fine as well. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] 0.17
On 6 May 2012 19:38, Stefan Behnel wrote: > mark florisson, 06.05.2012 16:28: >> I think we already have quite a bit of functionality (nearly) ready, >> after merging some pending pull requests maybe it will be a good time >> for a 0.17 release? I think it would be good to also document to what >> extent pypy support works, what works and what doesn't. > > Sure, although it's basically just "what works, works". However, there are > certainly things that users must know in order to make their own code work. > > >> Stefan, since >> you added a large majority of the features, would you want to be the >> release manager? > > I agree that it would make sense, but I'll be head under water for the next > month or so. > > Maybe it would be better to put out a 0.16.1 with only selected fixes in > the meantime? Ok, if no one else wants to take it up (please do speak up if you do), I could give it another go. > Stefan > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] 0.17
Hi, On Sun, May 6, 2012 at 7:28 AM, mark florisson wrote: > Hey, > > I think we already have quite a bit of functionality (nearly) ready, > after merging some pending pull requests maybe it will be a good time > for a 0.17 release? I think it would be good to also document to what > extent pypy support works, what works and what doesn't. Stefan, since > you added a large majority of the features, would you want to be the > release manager? > > In summary, the following pull requests should likely go in > - array.array support (unless further discussion prevents that) > - fused types runtime buffer dispatch > - newaxis > - more? > > The memoryview documentation should also be reworked a bit. Matthew, > are you still willing to have a go at that? Otherwise I can clean up > the mess first, some things are no longer true and simply outdated, > and then have a second opinion. Yes, sorry, I have been taken up by releasing my own project. What's the deadline do you think? I have another big release to do for the end of next week, but I might be able to carve out some time, See you, Matthew ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] 0.17
On 6 May 2012 20:41, Matthew Brett wrote: > Hi, > > On Sun, May 6, 2012 at 7:28 AM, mark florisson > wrote: >> Hey, >> >> I think we already have quite a bit of functionality (nearly) ready, >> after merging some pending pull requests maybe it will be a good time >> for a 0.17 release? I think it would be good to also document to what >> extent pypy support works, what works and what doesn't. Stefan, since >> you added a large majority of the features, would you want to be the >> release manager? >> >> In summary, the following pull requests should likely go in >> - array.array support (unless further discussion prevents that) >> - fused types runtime buffer dispatch >> - newaxis >> - more? >> >> The memoryview documentation should also be reworked a bit. Matthew, >> are you still willing to have a go at that? Otherwise I can clean up >> the mess first, some things are no longer true and simply outdated, >> and then have a second opinion. > > Yes, sorry, I have been taken up by releasing my own project. What's > the deadline do you think? I have another big release to do for the > end of next week, but I might be able to carve out some time, > > See you, > > Matthew Great, I'd say we're probably not going to release anything within the next two weeks, so take your time, there is no hurry really :). ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
