On 13 Jun 2013 09:09, "Victor Stinner" <victor.stin...@gmail.com> wrote: > > Hi, > > I would like to improve memory allocators of Python. My two use cases > are replacing memory allocators with custom allocators in embedded > system and hooking allocators to track usage of memory. > > I wrote a patch for this, I'm going to commit it if nobody complains: > http://bugs.python.org/issue3329 > > Using this patch, detecting memory corruptions (buffer underflow and > overflow) can be done without recompilation. We may add an environment > variable to enable Python debug functions at runtime, example: > PYDEBUGMALLOC=1. There is just a restriction: the environment variable > would not be ignored with -E command line option, because command line > options are parsed after the first memory allocation. What do you > think?
The rest of it sounds fine, but please don't add the runtime switching support to our existing main function. Interpreter startup is a mess already. If you were interested in helping directly with PEP 432, though, that would be good - I haven't been able to spend much time on it lately. Cheers, Nick. > > ***** > > The patch adds the following functions: > > void PyMem_GetAllocators( > void **ctx_p, > void* (**malloc_p) (void *ctx, size_t size), > void* (**realloc_p) (void *ctx, void *ptr, size_t size), > void (**free_p) (void *ctx, void *ptr)); > > void PyMem_SetAllocators( > void *ctx, > void* (*malloc) (void *ctx, size_t size), > void* (*realloc) (void *ctx, void *ptr, size_t size), > void (*free) (void *ctx, void *ptr)); > > It adds 4 similar functions (get/set) for PyObject_Malloc() and > allocators of pymalloc arenas. > > ***** > > For the "track usage of memory" use case, see the following project > which hooks memory allocators using PyMem_SetAllocators() and > PyObject_SetAllocators() to get allocated bytes per filename and line > number. > https://pypi.python.org/pypi/pytracemalloc > > ***** > > Another issue proposes to use VirtualAlloc() and VirtualFree() for > pymalloc arenas, see: > http://bugs.python.org/issue13483 > > I don't know if it would be interesting, but it would now possible to > choose the memory allocator (malloc, mmap, HeapAlloc, VirtualAlloc, > ...) at runtime, with an environment variable for example. > > Victor > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
