Thanks Victor, I have a better understanding of tracemalloc internals. On Tue, Nov 19, 2019 at 10:07 PM Victor Stinner <vstin...@python.org> wrote:
> Hi, > > Le mar. 19 nov. 2019 à 18:33, Sümer Cip <sum...@gmail.com> a écrit : > > First of all, I would like to thank for such a useful tool for debugging > memory issues. I am pretty proud as a Python lover that we have such a > tool: tracemalloc:) > > As the module author: you're welcome ;-) > > > > AFAIU, tracemalloc holds a single traceback per-object: > > "tracemalloc" name means that a "trace" is stored in mapping > associated to an allocated memory block. A trace is just: (size, > traceback): memory block in bytes, and the Python traceback where the > memory block has been allocated. > > tracemalloc is not aware of Python objects at all. For example, it is > unable to associate an object __dict__ to the object. It works at > malloc() and free() level, not at the _PyObject_GC_Malloc() level. > > One advantage is that memory which is not tracked by the GC is tracked > by tracemalloc. Another is that the implementation is "simple" (from > my point of view :-)). > > One drawback is that tracemalloc traces are harder to analyze. > > > > """ > > Output: > > ... > > 5.py:38: size=9264 B, count=1, average=9264 B > > [' File "5.py", line 38', ' a.alloc_more()', ' File "5.py", line > 32', ' self._d += [1] * 512'] > > """ > > When the internal storage of a Python list grows, the old memory block > is released and a new memory block is allocated. From the tracemalloc > point of view, the old trace is removed, a new trace is created. There > is no relationship between the two traces, even if realloc() has been > used. In tracemalloc, realloc() traces work as two isolated > operations: free() + malloc(). > > > > Now, allocation of alloc and alloc_more functions seems to be merged > into a single traceback. Is it possible we add (maybe a constant number of > tracebacks) per object? This might be pretty useful to find the responsible > leak in situations where same object is modified in many places? > > You might try to take snapshots frequently and write an heuristic to > link traces which look similar... good luck with that :-) > > -- > > There are other tools to track Python memory usage which may help you > to provide a view closer to what you are looking for. > > Victor > -- > Night gathers, and now my watch begins. It shall not end until my death. > -- Sümer Cip
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7ECDFRQBT2ZHVCEOTNP6XPZ7NERIXZJF/ Code of Conduct: http://python.org/psf/codeofconduct/
