clone 695839 -1 retitle -1 lintian: regression in memory usage or memory leak found -1 2.5.11 notfound -1 2.5.10.2 severity -1 serious retitle 695839 [reporting/harness]: Schedule check packages in chucks tags 695839 pending thanks
On 2012-12-13 10:55, Niels Thykier wrote: > [...] > > Currently Lintian on lintian.d.o is doing a full run and top says: > > [...] VIRT RES SHR S %CPU %MEM TIME+ COMMAND > [...] 2567m 818m 1528 R 81 40.6 432:36.97 lintian > > The run started on the 11th and I remember seeing it at 900ish VIRT/RES. > Yesterday, it was at 2.5 G VIRT (don't remember the RES). Today, we are > still at 2.5 G VIRT and RES has been 800-1000 MB for the past 30 minutes. > > We could be looking at a leak, because the VIRT is still growing a > bit. Though, if so, only a subset of the packages triggers the leak. > I am more inclined to believe it is long-living objects that we no > longer have use, but still have a reference to. > > L::Tags keeps a table of statistics for each file processed that is > not cleaned (afaict), so a slow grow is expected I guess. However, > I have a feeling this is not the (only) cause of grow. > > During the write up of this email the stats has changed to: > > [...] VIRT RES SHR S %CPU %MEM TIME+ COMMAND > [...] 2699m 977m 1484 S 0 48.6 442:20.99 lintian > > > If it is not a leak and just Perl not releasing memory back > (regardless of reason), I think we should look at splitting > full runs into smaller "short" lived processes. > > ~Niels > > The bad news first; Lintian 2.5.11 was unable to finish the full run. Around gnome-scan, Lintian started spewing errors - first a fork failed and then a couple of: """[...] cannot open X: Cannot allocate memory at ... """ After that, it just said "collect info X of ... failed" a couple of times for each of the remaining packages. The VIRT memory usage had grown over 3GB, so I now strongly suspect that 2.5.11 introduces a memory leak. On the plus side, I have managed to finish up a set of patches to make harness schedule groups in smaller chucks. I pulled them onto lintian.d.o and restarted to run as a work-around for now. ~Niels -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
