Setting a size limit works around this, because the go() method calls the callback with None when the object does not fit into memory (and no specific member was requested in the go call).
-- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to python-apt in Ubuntu. https://bugs.launchpad.net/bugs/2105503 Title: Uncontrolled memory usage Status in python-apt package in Ubuntu: Triaged Bug description: Here is a simplified snippet of the `filelist` function in the `debfile.py` file. I got there while debugging OOM kill triggering on cnf-extractor, that makes use of that function. ``` $ cat apt_file_0ad-data.py import apt_inst files = [] debfile = apt_inst.DebFile("0ad-data_0.27.0-1_all.deb") debfile.data.go(lambda item, data: files.append(item.name)) print(files) ``` To run this, you need the latest `0ad-data` package downloaded next to the script: `apt download 0ad-data`. When running this "normally", I can see very high memory usage, sometimes leading to OOM kill (how I digged into that in the first place): python3 apt_file_0ad-data.py & ; htop -p $! Interestingly, when wrapping that with some kind of constraints, the issue doesn't appear anymore at all, and the memory usage remains low: prlimit -v536870912 python3 apt_file_0ad-data.py & ; htop -p $! With that previous command, even the time to finish is divided by two (~consistently 8s vs 15s without wrapping on my machine), but I didn't take precise measurements there, so might be noise, but still... And then one final interesting result: systemd-run --scope -p MemoryMax=500M --user python3 apt_file_0ad-data.py Sometimes this works just as with `prlimit`, with slightly higher memory usage, and sometimes it correctly triggers oom kill, after more than one minute. Again, the timing benchmark is inaccurate, but still... ProblemType: Bug DistroRelease: Ubuntu 25.04 Package: python3-apt 2.9.9build1 ProcVersionSignature: Ubuntu 6.12.0-12.12-generic 6.12.4 Uname: Linux 6.12.0-12-generic x86_64 NonfreeKernelModules: zfs ApportVersion: 2.32.0-0ubuntu3 Architecture: amd64 CasperMD5CheckResult: unknown CurrentDesktop: sway Date: Mon Mar 31 15:58:25 2025 InstallationDate: Installed on 2023-08-02 (607 days ago) InstallationMedia: Ubuntu 23.04 "Lunar Lobster" - Release amd64 (20230418) ProcEnviron: LANG=fr_FR.UTF-8 PATH=(custom, no user) SHELL=/bin/zsh TERM=alacritty XDG_RUNTIME_DIR=<set> SourcePackage: python-apt UpgradeStatus: Upgraded to plucky on 2024-06-12 (292 days ago) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-apt/+bug/2105503/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
