The deb contains a 3.5GB file. The python-apt code reads the file into
memory into a C++ array and then creates a PyBytes object from it,
copying the data, causing 7GB of memory usage.
** Changed in: python-apt (Ubuntu)
Status: New => Triaged
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to python-apt in Ubuntu.
https://bugs.launchpad.net/bugs/2105503
Title:
Uncontrolled memory usage
Status in python-apt package in Ubuntu:
Triaged
Bug description:
Here is a simplified snippet of the `filelist` function in the
`debfile.py` file. I got there while debugging OOM kill triggering on
cnf-extractor, that makes use of that function.
```
$ cat apt_file_0ad-data.py
import apt_inst
files = []
debfile = apt_inst.DebFile("0ad-data_0.27.0-1_all.deb")
debfile.data.go(lambda item, data: files.append(item.name))
print(files)
```
To run this, you need the latest `0ad-data` package downloaded next to
the script: `apt download 0ad-data`.
When running this "normally", I can see very high memory usage, sometimes
leading to OOM kill (how I digged into that in the first place):
python3 apt_file_0ad-data.py & ; htop -p $!
Interestingly, when wrapping that with some kind of constraints, the issue
doesn't appear anymore at all, and the memory usage remains low:
prlimit -v536870912 python3 apt_file_0ad-data.py & ; htop -p $!
With that previous command, even the time to finish is divided by two
(~consistently 8s vs 15s without wrapping on my machine), but I didn't take
precise measurements there, so might be noise, but still...
And then one final interesting result:
systemd-run --scope -p MemoryMax=500M --user python3 apt_file_0ad-data.py
Sometimes this works just as with `prlimit`, with slightly higher memory
usage, and sometimes it correctly triggers oom kill, after more than one
minute. Again, the timing benchmark is inaccurate, but still...
ProblemType: Bug
DistroRelease: Ubuntu 25.04
Package: python3-apt 2.9.9build1
ProcVersionSignature: Ubuntu 6.12.0-12.12-generic 6.12.4
Uname: Linux 6.12.0-12-generic x86_64
NonfreeKernelModules: zfs
ApportVersion: 2.32.0-0ubuntu3
Architecture: amd64
CasperMD5CheckResult: unknown
CurrentDesktop: sway
Date: Mon Mar 31 15:58:25 2025
InstallationDate: Installed on 2023-08-02 (607 days ago)
InstallationMedia: Ubuntu 23.04 "Lunar Lobster" - Release amd64 (20230418)
ProcEnviron:
LANG=fr_FR.UTF-8
PATH=(custom, no user)
SHELL=/bin/zsh
TERM=alacritty
XDG_RUNTIME_DIR=<set>
SourcePackage: python-apt
UpgradeStatus: Upgraded to plucky on 2024-06-12 (292 days ago)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python-apt/+bug/2105503/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
