[issue35715] ProcessPool workers hold onto return value of last task in memory
New submission from David Chevell :
ProcessPoolExecutor workers will hold onto the return value of their last task
in memory until the next task is received. Since the return value has already
been propagated to the parent process's `Future` or else effectively discarded,
this is holding onto objects unnecessarily.
Simple case to reproduce:
import concurrent.futures
import time
executor = concurrent.futures.ProcessPoolExecutor(max_workers=1)
def big_val():
return [{1:1} for i in range(1, 100)]
executor.submit(big_val)
# Observe the memory usage of the process worker during the sleep interval
time.sleep(10)
This should be easily fixed by having the worker explicitly `del r` after
calling `_sendback_result` as it already does this for `call_item`
--
components: Library (Lib)
messages: 333444
nosy: dchevell
priority: normal
severity: normal
status: open
title: ProcessPool workers hold onto return value of last task in memory
versions: Python 3.8
___
Python tracker
<https://bugs.python.org/issue35715>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35715] ProcessPool workers hold onto return value of last task in memory
Change by David Chevell : -- keywords: +patch pull_requests: +11075 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue35715> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35715] ProcessPool workers hold onto return value of last task in memory
Change by David Chevell : -- keywords: +patch, patch, patch, patch pull_requests: +11075, 11076, 11077, 11078 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue35715> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35715] ProcessPool workers hold onto return value of last task in memory
Change by David Chevell : -- keywords: +patch, patch, patch pull_requests: +11075, 11076, 11077 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue35715> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35715] ProcessPool workers hold onto return value of last task in memory
Change by David Chevell : -- keywords: +patch, patch pull_requests: +11075, 11076 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue35715> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
