STINNER Victor added the comment:
I attached a patch that I wrote for Wyplay: py_setallocators.patch. The patch
adds two functions:
PyAPI_FUNC(int) Py_GetAllocators(
char api,
void* (**malloc_p) (size_t),
void* (**realloc_p) (void*, size_t),
void (**free_p) (void*)
);
PyAPI_FUNC(int) Py_SetAllocators(
char api,
void* (*malloc) (size_t),
void* (*realloc) (void*, size_t),
void (*free) (void*)
);
Where api is one of these values:
- PY_ALLOC_SYSTEM_API: the system API (malloc, realloc, free)
- PY_ALLOC_MEM_API: the PyMem_Malloc() API
- PY_ALLOC_OBJECT_API: the PyObject_Malloc() API
These functions are used by the pytracemalloc project to hook PyMem_Malloc()
and PyObject_Malloc() API. pytracemalloc traces all Python memory allocations
to compute statistics per Python file.
https://pypi.python.org/pypi/pytracemalloc
Wyplay is also using Py_SetAllocators() internally to replace completly system
allocators *before* Python is started. We have another private patch on Python
adding a function. This function sets its own memory allocators, it is called
before the start of Python thanks to an "__attribute__((constructor))"
attribute.
--
If you use Py_SetAllocators() to replace completly a memory allocator (any
memory allocation API), you have to do it before the first Python memory
allocation (before Py_Main()) *or* your memory allocator must be able to
recognize if a pointer was not allocated by him and pass the operation (realloc
or free) to the previous memory allocator.
For example, PyObject_Free() is able to recognize that a pointer is part of its
memory pool, or fallback to the system allocator (extract of the original code):
if (Py_ADDRESS_IN_RANGE(p, pool)) {
...
return;
}
free(p);
--
If you use Py_SetAllocators() to hook memory allocators (do something before
and/or after calling the previous function, *without* touching the pointer nor
the size), you can do it anytime.
--
I didn't run a benchmark yet to measure the overhead of the patch on Python
performances.
New functions are not documented nor tested yet. If we want to test these new
functions, we can write a simple hook tracing calls to the memory allocators
and call the memory allocator.
----------
keywords: +patch
nosy: +haypo
Added file: http://bugs.python.org/file29324/py_setallocators.patch
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue3329>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
