[Martin v. Löwis] > Several places in the documentation claim that Py_Finalize will > release all memory: > > http://docs.python.org/api/embedding.html > > says that literally,
It's wrong ;-). > and > > http://docs.python.org/api/initialization.html#l2h-778 > > suggests it is a bug when memory is not freed in Py_Finalize. > > This has left people to believe that this is a bug: > > <https://sourceforge.net/tracker/index.php?func=detail&aid=1445210&group_id=5470&atid=105470> Well, there may well be a bug (or multiple bugs) underlying that one too. It's one thing for Py_Finalize() not to release all memory (it doesn't and probably never will), but it's not necessarily the same thing if running Py_Initialize() ... Py_Finalize() repeatedly keeps leaking more and more memory. > However, I don't see any chance to make this promise even remotely. > Objects allocated in extension modules, and held in global variables > (e.g. socketmodule.c:socket_error, socket_herror, socket_gaierror, > socket_timeout) will never be released, right? Not unless the module has a finalization function called by Py_Finalize() that frees such things (like PyString_Fini and PyInt_Fini). Other globals allocated via a static PyObject *someglobal = NULL; ... if (someglobal == NULL) someglobal = allocate_an_object_somehow(); pattern shouldn't contribute to continuing leaks across Py_Initialize() ... Py_Finalize() loops. > And because of the small objects allocator, their pool will remain > allocated, right? And, then, the arena. Before Python 2.5, arenas are never freed, period. In Python 2.5, an arena will be freed if and only if it contains no allocated object by the time Py_Finalize ends. There may also be trash cycles that aren't collected during Py_Finalize because I had to comment out Py_Finalize's second call to PyGC_Collect(); new-style class objects are among the trash thingies leaked (although if Py_Initialize() is called again, it's possible that they'll get cleaned up by cyclic gc after all). See Misc/SpecialBuilds.txt, section Py_TRACE_REFS, entry PYTHONDUMPREFS, for a way to get a dump of all heap objects Py_Finalize leaves alive. I doubt that's been run in years; Guido and I used it in 2.3b1 to cure some unreasonably high finalization leakage at the time. > So ISTM that invoking Py_Finalize after importing socket will yield > atleast 256KiB garbage. Of course, that's not real garbage, because > the next Py_Initialize'd interpreter will continue to allocate from > the arenas. I'm not clear on whether, e.g., init_socket() may get called more than once if socket-slinging code appears in a Py_Initialize() ... Py_Finalize(). If it doesn't, then, e.g., the unconditional socket_gaierror = PyErr_NewException("socket.gaierror", ...); won't contribute to ongoing leaks. But if it does, then we'll systematically leak the exception object on each loop trip. > But still, the actual objects that the modules hold on to will not be > reclaimed > until the process terminates. Without a _Fini() function called by Py_Finalize, that's correct. > Please correct me if I'm wrong. If you ever are, I will ;-) _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
