[issue35056] Test leaks of memory not managed by Python allocator

Erlend Egeberg Aasland added the comment: > This issue is about C extension modules which don't use the Python memory > allocator. Yes, I know. Your proposal in msg328533 is to continue the implementation of bpo-18227: > _sqlite: call sqlite3_config(SQLITE_CONFIG_MALLOC, pMem) to use > PyM

[issue35056] Test leaks of memory not managed by Python allocator

STINNER Victor added the comment: patch-with-simple-msize.diff: I suggest you opening a new issue to propose using the Python memory allocators in the sqlite module. This issue is about C extension modules which don't use the Python memory allocator. -- _

[issue35056] Test leaks of memory not managed by Python allocator

Change by Erlend Egeberg Aasland : Added file: https://bugs.python.org/file49963/patch-with-simple-msize.diff ___ Python tracker ___ ___ Pyt

[issue35056] Test leaks of memory not managed by Python allocator

Erlend Egeberg Aasland added the comment: The msize() talk is referring to msg389039 and msg328533. If we are to use Python memory allocators for the sqlite3 extension module, we need some sort of msize() function; overriding SQLite's memory functions requires msize() support. -- __

[issue35056] Test leaks of memory not managed by Python allocator

Serhiy Storchaka added the comment: No, I did not mean using msize() or something like. Since memory is managed outside of Python, we have no a list of allocated blocks. I meant that we can get the total memory used by the Python process (using OS-specific methods) and compare it between ite

[issue35056] Test leaks of memory not managed by Python allocator

STINNER Victor added the comment: Multiple C library don't provide msize() function. If you seriously consider to add it, you should conduct a study on all platforms. -- ___ Python tracker _

[issue35056] Test leaks of memory not managed by Python allocator

Erlend Egeberg Aasland added the comment: Victor, Serhiy: Would this issue be "large" or important enough to re-raise the debate about implementing an msize() function in the PyMem_ API, or is it not worth it? I guess no; it has been discussed numerous times before. Else, I can start a threa

[issue35056] Test leaks of memory not managed by Python allocator

Erlend Egeberg Aasland added the comment: > _sqlite: call sqlite3_config(SQLITE_CONFIG_MALLOC, pMem) to use > PyMem_RawMalloc() SQLite requires the xSize member of sqlite3_mem_methods to be implemented for this to work, so we'd have to implement msize(). The msize() idea seems to have been

[issue35056] Test leaks of memory not managed by Python allocator

Change by Erlend Egeberg Aasland : -- nosy: +erlendaasland ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: ht

[issue35056] Test leaks of memory not managed by Python allocator

STINNER Victor added the comment: It seems like the easiest thing to do thta would directly benefit (to tracemalloc users) is to continue the implementation of bpo-18227: * _sqlite: call sqlite3_config(SQLITE_CONFIG_MALLOC, pMem) to use PyMem_RawMalloc() * _ssl: try again CRYPTO_set_mem_func

[issue35056] Test leaks of memory not managed by Python allocator

STINNER Victor added the comment: About automation, regrtest -R checks memory leaks using sys.getallocatedblocks(). But this function only tracks PyMem_Malloc() and PyObject_Malloc() (which is now technically the same memory allocator, since Python 3.6: https://docs.python.org/dev/c-api/mem

[issue35056] Test leaks of memory not managed by Python allocator

STINNER Victor added the comment: Before seeing how to automate it, first someone should try to detect the bpo-34794 memory leak manually ;-) By the way, when I implemented the PEP 445, I tried to configure OpenSSL to use Python memory allocators (to benefit of tracemalloc), but the OpenSSL

[issue35056] Test leaks of memory not managed by Python allocator

STINNER Victor added the comment: > Can it be used on buildbots and in CI tests? These kind of tool produces a lot of false alarms :-( They are also hard to use. -- ___ Python tracker __

[issue35056] Test leaks of memory not managed by Python allocator

Serhiy Storchaka added the comment: Can it be used on buildbots and in CI tests? -- ___ Python tracker ___ ___ Python-bugs-list mai

[issue35056] Test leaks of memory not managed by Python allocator

Change by Xiang Zhang : -- nosy: +xiang.zhang ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.py

[issue35056] Test leaks of memory not managed by Python allocator

STINNER Victor added the comment: You can try to use Valgrind for that? -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue35056] Test leaks of memory not managed by Python allocator

New submission from Serhiy Storchaka : Would be nice to add a possibility to test memory leaks if memory is allocated not by Python allocators, but inside external libraries. This would allow to catch leaks on the bridge between Python and external libraries. See for example issue34794.