[issue31558] gc.freeze() - an API to mark objects as uncollectable
Zekun Li added the comment: > This is only useful if the parent process has a lot of memory that's never > used by the child processes right? Otherwise, you would lose via refcounting > COWs. What we saw in prod is that memory fragmentation caused by gc is the main reason of shared memory shrink. The memory fragmentation is figured out by doing a full collection before fork and keep it disabled, it'll make a bunch of copy-on-write in child process. This can't solve the copy-on-write caused by ref count, but we're thinking about freezing the ref count on those permanent objects too. So this is useful if you did some warm-up work in parent process. Also it could speedup gc if you have large amount of permanent objects. -- nosy: +brainfvck ___ Python tracker <https://bugs.python.org/issue31558> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31558] gc.freeze() - an API to mark objects as uncollectable
Zekun Li added the comment: So what we did is: We keep gc **disabled** on parent process and freeze after warmup, enable gc on child process. The reason not to do a full collection is mentioned in previous comments/original ticket - (I called it) memory fragmentation. The observation is - We keep gc disabled on both parent and child process and did a full collection before fork, it makes the shared memory shrink a lot compared to no collection. - There's no way for disabled gc to touch the head to make copy-on-write. Of course, enable gc will make the shared memory shrink more. But the former case is accounting more than latter one. So my understand is that gc frees some objects and makes some memory pages becomes available to allocate in child process. Allocation on the shared memory pages will cause the copy-on-write even without gc. Though this behavior may have better name? -- ___ Python tracker <https://bugs.python.org/issue31558> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
