Re: [Python-Dev] ctypes, memory mapped files and context manager

[Armin Rigo]

Hi Hans-Peter,

On 6 January 2017 at 00:28, Hans-Peter Jansen  wrote:
> Leaves the question, how stable this "interface" is?

Another way to jump through hoops:

c_raw = ctypes.PYFUNCTYPE(ctypes.c_void_p, ctypes.c_void_p)(lambda p: p)

addr = c_raw(ctypes.pointer(T.from_buffer(m)))
b = ctypes.cast(addr, ctypes.POINTER(T)).contents

These lines give an object 'b' that is equivalent to
'T.from_buffer(m)', but doesn't hold any reference or any "opened
buffer" state to the original 'm'.  Your context manager can yield
that.  It should prevent all BufferErrors, at the price of segfaulting
if used incorrectly.  This means in your case that ``with
map_struct(..) as a:`` should not continue to use ``a`` after the
``with`` statement, which is pretty natural anyway.

(The same issue occurs with cffi instead of ctypes, but in this case a
simple cast is enough to detach the memoryview, instead of the hack
above.)


A bientôt,

Armin.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ctypes, memory mapped files and context manager

[eryk sun]

On Sun, Jan 8, 2017 at 8:25 AM, Armin Rigo  wrote:
>
> c_raw = ctypes.PYFUNCTYPE(ctypes.c_void_p, ctypes.c_void_p)(lambda p: p)

Use ctypes.addressof.

> addr = c_raw(ctypes.pointer(T.from_buffer(m)))
> b = ctypes.cast(addr, ctypes.POINTER(T)).contents

ctypes.cast uses an FFI call. In this case you can more simply use from_address:

b = T.from_address(ctypes.addressof(T.from_buffer(m)))

There's no supporting connection between b and m. If m was allocated
from a heap/pool/freelist, as opposed to a separate mmap
(VirtualAlloc) call, then you won't necessarily get a segfault (access
violation) if b is used after m has been deallocated or internally
realloc'd. It can lead to corrupt data and difficult to diagnose
errors. You're lucky if it segfaults.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com