This patch is a very rough attempt at adding support for read-only
buffer objects to typed memoryviews. It is incomplete, has bugs, and
is very dirty. I'm positing it anyway to solicit help in figuring out
the right way to add read-only buffer support. The 'XXX' comments
mark issues I'm not sure about.
The problem:
The following example function raises an exception when passed a bytes
object or any other read-only buffer object:
cpdef object printbuf(unsigned char[:] buf):
chars = [chr(x) for x in buf]
print(repr(''.join(chars)))
This is what happens:
$ python -c 'import test; test.printbuf("test\0ing")'
Traceback (most recent call last):
File "", line 1, in
File "test.pyx", line 1, in test.printbuf (test.c:1417)
File "stringsource", line 614, in View.MemoryView.memoryview_cwrapper
(test.c:6795)
File "stringsource", line 321, in View.MemoryView.memoryview.__cinit__
(test.c:3341)
BufferError: Object is not writable.
The exception is raised because the code generated for typed
memoryviews always passes the PyBUF_WRITABLE flag to
PyObject_GetBuffer(), and "test\0ing" is a read-only bytes object.
The bytes class raises an exception when it sees the PyBUF_WRITABLE
flag.
In this example case, there's no need to pass PyBUF_WRITABLE, and if
it wasn't passed then the code would work as expected.
The approach this patch takes is:
* Never pass PyBUF_WRITABLE to PyObject_GetBuffer() just in case the
underlying object is read-only.
* If it turns out that we do need write access to the buffer (e.g.,
to handle 'buf[0] = x'), then check to see if the Py_buffer
structure's readonly member is false. If so, it's safe to write.
Otherwise, raise a BufferError exception ("read-only object").
This approach is not ideal because it doesn't support copy-on-write
objects or similarly complicated buffer objects, but a better approach
would be a more invasive change. See the comments I added in
check_buffer_writable().
Feedback would be appreciated.
Thanks,
Richard
---
Cython/Compiler/ExprNodes.py | 100 ++
Cython/Compiler/MemoryView.py | 12 ++---
Cython/Utility/MemoryView.pyx | 14 +-
tests/memoryview/memslice.pyx | 12 ++---
4 files changed, 124 insertions(+), 14 deletions(-)
diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py
index ac28d03..13017a9 100644
--- a/Cython/Compiler/ExprNodes.py
+++ b/Cython/Compiler/ExprNodes.py
@@ -3642,8 +3642,106 @@ class IndexNode(ExprNode):
self.extra_index_params(code),
code.error_goto(self.pos)))
+def check_buffer_writable(self, code):
+"""Raise BufferError if the Py_buffer struct's readonly flag is set"""
+
+# XXX This is called from:
+# * generate_buffer_setitem_code()
+# * generate_memoryviewslice_setslice_code()
+# * generate_memoryviewslice_assign_scalar_code()
+# Are these sufficient? Is there another function called to
+# generate code to directly modify a buffer via Cython syntax?
+
+# XXX is it safe to call self.buffer_entry() multiple times?
+buffer_entry = self.buffer_entry()
+
+# XXX is this the right way to check to see if the object
+# being modified is a memoryview slice object and not some
+# other object type?
+if not buffer_entry.type.is_buffer:
+
+# XXX Currently the PyBUF_WRITABLE flag is never passed
+# when calling PyObject_GetBuffer() on the underlying
+# buffer object, so we never asked the buffer object to
+# provide us with a writable chunk of memory. Even if the
+# Py_buffer struct's readonly flag is unset, might it be
+# inappropriate to write to the buffer since we never
+# passed PyBUF_WRITABLE? The Python 3 documentation seems
+# to imply that this is OK, but it's not entirely clear.
+
+# XXX This code simply checks to see whether the Py_buffer
+# view's readonly flag is unset. This design does not
+# support copy-on-write or other similarly complicated
+# buffer objects.
+#
+# For example, imagine a class that does copy-on-write.
+# Suppose there is an instance of that type, and a copy of
+# it is made. The copy hasn't been modified yet, so it
+# shares a buffer with the original object. If
+# PyBUF_WRITABLE is not passed to PyObject_GetBuffer(),
+# the returned Py_buffer simply points to the shared
+# memory and the readonly flag is set to avoid corrupting
+# the original object. If PyBUF_WRITABLE is passed to
+# PyObject_GetBuffer(), the object makes its own copy of
+# the memory and the returned Py_buffer points to the
+# private copy with the readonly flag uns
