I have observed similar issue as Nevo Hed described in https://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg1892339.html but with fakeroot-1.32.2 on Gentoo system instead. Fixed it in similar ways like Nevo by adding this to daemon.json for dockerd:
"default-ulimits": {
"nofile": {
"Hard": 524288,
"Name": "nofile",
"Soft": 524288
}
}
