[issue28663] Higher virtual memory usage on recent Linux versions

INADA Naoki added the comment: I close this issue, because there are no enough evidence it's Python's issue. When hit ulimit is just a OS's detail. Please ping or file a new issue when memory usage is really grown. -- resolution: -> not a bug stage: -> resolved status: open -> closed

[issue28663] Higher virtual memory usage on recent Linux versions

ProgVal added the comment: > On Linux 3.16? 4.7? What is difference between 3.16 and 4.7? On 4.8 (which has the same issue as 4.7). The difference is that on 4.7, the rlimit has to be set incredibly high for the child process not to hit the limit. > Again, I doubt Python's memory consumption

[issue28663] Higher virtual memory usage on recent Linux versions

INADA Naoki added the comment: On Linux 3.16? 4.7? What is difference between 3.16 and 4.7? Again, I doubt Python's memory consumption increased by Linux version. Isn't rlimit more strict for now? Even if memory usage is really grow, I don't think it's a Python's issue. Maybe, environment issu

[issue28663] Higher virtual memory usage on recent Linux versions

ProgVal added the comment: first command: (0, 0) 0.14user 0.01system 0:00.94elapsed 16%CPU (0avgtext+0avgdata 18484maxresident)k 560inputs+0outputs (0major+2702minor)pagefaults 0swaps second command: (6663744, 6732519) 0.18user 0.01system 0:00.96elapsed 20%CPU (0avgtext+0avgdata 22572maxresident

[issue28663] Higher virtual memory usage on recent Linux versions

INADA Naoki added the comment: @ProgVal, could you try following? I doubt that this difference just shows rlimit behavior, not memory usage. --- $ cat rlimit_difference_linux_versions.py import sys import queue import multiprocessing from urllib.request import Request, urlopen import resource i

[issue28663] Higher virtual memory usage on recent Linux versions

ProgVal added the comment: Sorry. That's the result of |||get_traced_memory|on Python 3.5 and Linux 4.7. -- ___ Python tracker ___ ___

[issue28663] Higher virtual memory usage on recent Linux versions

STINNER Victor added the comment: ProgVal: "(4350362, 4376669)" is it the same result on Python 3.4 and Python 3.5? I don't understand your comment :-/ Can you please elaborate? -- ___ Python tracker _

[issue28663] Higher virtual memory usage on recent Linux versions

ProgVal added the comment: (4350362, 4376669) -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue28663] Higher virtual memory usage on recent Linux versions

STINNER Victor added the comment: rlimit_tracemalloc.txt: Oh, my idea was only to see the total, https://docs.python.org/dev/library/tracemalloc.html#tracemalloc.get_traced_memory Current and peak memory usage. -- ___ Python tracker

[issue28663] Higher virtual memory usage on recent Linux versions

STINNER Victor added the comment: It's very hard to estimate the water high-mark for memory because the memory includes different things: read-only memory, mmap() on files, read-only memory pages shared between multiple processes for libraries, etc. I suggest to use the tracemalloc module to h

[issue28663] Higher virtual memory usage on recent Linux versions

Antoine Pitrou added the comment: Your script works fine here (Ubuntu 16.04). Besides, if you're seeing differences between two Linux versions with the same Python version, it is quite unlikely to be a Python issue. -- nosy: +pitrou ___ Python track

[issue28663] Higher virtual memory usage on recent Linux versions

Changes by James Lu : -- nosy: +James Lu ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.or

[issue28663] Higher virtual memory usage on recent Linux versions

Changes by ProgVal : -- versions: +Python 3.6, Python 3.7 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https

[issue28663] Higher virtual memory usage on recent Linux versions

New submission from ProgVal: Hi, I use `resource.setrlimit(resource.RLIMIT_DATA, ...)` to limit the memory usage of a child process. I noticed that on recent Linux versions, my processes are much more likely to raise MemoryError or behave inconsistently (which I believe is caused by a MemoryE