Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-04 Thread Stefan Krah
Thomas Wouters wrote: > Do you test against pydebug builds of Python, or otherwise a build that > actually enables asserts? Yes, I do (and much more than that): http://hg.python.org/features/cdecimal/file/40917e4b51aa/Modules/_decimal/python/runall-memorydebugger.sh http://hg.python.org/features

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-03 Thread Thomas Wouters
On Sat, Mar 3, 2012 at 03:08, Stefan Krah wrote: > Stefan Behnel wrote: > > > 1. assert() is the wrong tool for this job > > > > Absolutely. > > I disagree. This assert() is meant for extension authors and not end > users. I > can't see how a reasonable release procedure would fail to trigger th

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-03 Thread Stefan Krah
Nick Coghlan wrote: > 2. the current check is too strict (it should just check for obj != > NULL, not obj == &exporter) Yes. For anyone who is interested, see issue #14181. > 3. the current check is in the wrong place (it should be in > PyObject_GetBuffer) Agreed, since it's not memoryview sp

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-03 Thread Stefan Krah
Stefan Behnel wrote: > > 1. assert() is the wrong tool for this job > > Absolutely. I disagree. This assert() is meant for extension authors and not end users. I can't see how a reasonable release procedure would fail to trigger the assert(). My procedure as a C extension author is to test agai

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-03 Thread Stefan Behnel
Nick Coghlan, 03.03.2012 00:49: > On Sat, Mar 3, 2012 at 3:14 AM, Stefan Behnel wrote: >> Stefan Krah, 02.03.2012 17:42: >>> The reason why this scheme was not chosen for a chain of memoryviews >>> was that 'exporter' (in theory) could implement a slideshow of buffers, >>> which means that in the f

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-03 Thread Stefan Krah
Stefan Behnel wrote: > Yes, that's a suitable example. It would take the ndarray out of the loop - > after all, it has nothing to do with what the memoryview wants, and won't > need to do any cleanup for the memoryview's buffer view either. Keeping it > explicitly alive in the memoryview is just a

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-02 Thread Nick Coghlan
On Sat, Mar 3, 2012 at 3:14 AM, Stefan Behnel wrote: > Stefan Krah, 02.03.2012 17:42: >> The reason why this scheme was not chosen for a chain of memoryviews >> was that 'exporter' (in theory) could implement a slideshow of buffers, >> which means that in the face of redirecting requests m might n

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-02 Thread Thomas Wouters
On Fri, Mar 2, 2012 at 05:22, Nick Coghlan wrote: > On Fri, Mar 2, 2012 at 10:55 PM, Stefan Krah wrote: > > Nick Coghlan wrote: > >> However, given the lack of control, an assert() isn't the appropriate > >> tool here - PyObject_GetBuffer itself should be *checking* the > >> constraint and then

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-02 Thread Stefan Behnel
Stefan Krah, 02.03.2012 17:42: > Stefan Krah wrote: >>> Why would the object that bf_getbuffer() is being called on have to >>> be identical with the one that exports the buffer? >> >> It doesn't have to be. This is now possible: >> >> >>> from _testbuffer import * >> >>> exporter = b'123' >> >>> n

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-02 Thread Stefan Krah
Stefan Krah wrote: > > Why would the object that bf_getbuffer() is being called on have to > > be identical with the one that exports the buffer? > > It doesn't have to be. This is now possible: > > >>> from _testbuffer import * > >>> exporter = b'123' > >>> nd = ndarray(exporter) > >>> m = memo

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-02 Thread Stefan Krah
Stefan Behnel wrote: > I keep failing to see the interest in making this an error in the first > place. First, it is meant to guard against random pointers in the view.obj field, precisely because view.obj was undocumented and exporters might not fill in the field. Then, as I said, the exporter

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-02 Thread Nick Coghlan
On Sat, Mar 3, 2012 at 12:39 AM, Stefan Behnel wrote: >> I keep failing to see the interest in making this an error in the first > place. Why would the object that bf_getbuffer() is being called on have to > be identical with the one that exports the buffer? OK, I misunderstood your suggestion. S

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-02 Thread Stefan Behnel
Nick Coghlan, 02.03.2012 14:22: > On Fri, Mar 2, 2012 at 10:55 PM, Stefan Krah wrote: >> Nick Coghlan wrote: >>> However, given the lack of control, an assert() isn't the appropriate >>> tool here - PyObject_GetBuffer itself should be *checking* the >>> constraint and then reporting an error if the

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-02 Thread Stefan Krah
Stefan Krah wrote: > > Careful. There are tons of code out there that use the buffer interface, > > and the "obj" field has been the way to handle the buffer release ever > > since the interface actually worked (somewhere around the release of Py3.0, > > IIRC). > > > > Personally, I never read th

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-02 Thread Stefan Krah
Stefan Behnel wrote: > > http://docs.python.org/dev/c-api/buffer.html#Py_buffer > > http://docs.python.org/dev/c-api/typeobj.html#buffer-object-structures > > > > Since the Py_buffer.obj filed was undocumented in 3.2, I think we're within > > out rights to restrict the field to the exporter. > >

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-02 Thread Nick Coghlan
On Fri, Mar 2, 2012 at 10:55 PM, Stefan Krah wrote: > Nick Coghlan wrote: >> However, given the lack of control, an assert() isn't the appropriate >> tool here - PyObject_GetBuffer itself should be *checking* the >> constraint and then reporting an error if the check fails. Otherwise a >> misbeha

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-02 Thread Stefan Krah
Nick Coghlan wrote: > However, given the lack of control, an assert() isn't the appropriate > tool here - PyObject_GetBuffer itself should be *checking* the > constraint and then reporting an error if the check fails. Otherwise a > misbehaving extension module could trivially crash the Python > in

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-02 Thread Stefan Behnel
Stefan Krah, 02.03.2012 12:53: > Stefan Behnel wrote: >> if (PyObject_GetBuffer(base, &mbuf->master, PyBUF_FULL_RO) < 0) { >> /* mbuf->master.obj must be NULL. */ >> Py_DECREF(mbuf); >> return NULL; >> } >> >> /* Assume that master.obj is a new reference to base.

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-02 Thread Nick Coghlan
On Fri, Mar 2, 2012 at 8:19 PM, Stefan Behnel wrote: > I'm not saying that this is likely to happen, but I could imagine code that > wants to use a different object for the cleanup than itself, possibly for > keeping a certain kind of state when it delivers more than one buffer, or > for rememberi

Re: [Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-02 Thread Stefan Krah
Stefan Behnel wrote: > if (PyObject_GetBuffer(base, &mbuf->master, PyBUF_FULL_RO) < 0) { > /* mbuf->master.obj must be NULL. */ > Py_DECREF(mbuf); > return NULL; > } > > /* Assume that master.obj is a new reference to base. */ > assert(mbuf->master.obj == b

[Python-Dev] Assertion in _PyManagedBuffer_FromObject()

2012-03-02 Thread Stefan Behnel
Hi, I just stumbled over this assertion in _PyManagedBuffer_FromObject() in the latest Py3.3 branch: """ static PyObject * _PyManagedBuffer_FromObject(PyObject *base) { _PyManagedBufferObject *mbuf; mbuf = mbuf_alloc(); if (mbuf == NULL) return NULL; if (PyObject_GetBuff