Hello Antoine, > On 24 Feb 2025, at 16:51, Antoine Beaupré <anar...@debian.org> wrote: > > On 2025-02-24 15:56:57, Michael Tremer wrote: > > [...] > >> So, has this solved it all for good for you guys? What release of xapian are >> you on? > > In the end, no, not really. Things have *improved*: we used to have 4-5 > OOM/day, with peaks at 15, 120 when reindexing, and this is down to 1-5 > a day, depending on the day. Kind of hard to track discrete events like > this... > > We had a single OOM in the last 48h. That's "nice".
Okay, this might still be a slight step in the right direction. > We also have stupidly large Xapian indexes now, it's ridiculous. Clearly > something wrong either with the haystack or the hyperkitty > implementation. So far I've filed it in the latter: > > https://gitlab.com/mailman/hyperkitty/-/issues/533 > > So, TL;DR: improved, but not fixed. I suspect we had a multi-dimensional > issue, of which search/whoosh *was* a part of, because we would see a > huge increase in OOMs when rebuilding the indexes. But we're still > having an issue, so perhaps there's something else. This is my experience with Xapian and I have found confirmation that this is supposed to be normal. My mailbox indexes were massive and there was no point having them any more. So I can confirm that this looked very similar in Dovecot, too. > We tried to hookup a memory profiler (austin) but it failed because it > didn't work with Python 3.11... so maybe that's something we'll try to > revisit after our trixie upgrades (hopefully soon!). I tried that but I was struggling with a missing sssd and some other things. Not sure I am ready to try again. It would also not help us to find out where exactly this went wrong if it were fixed :( > > a. > > -- > On reconnait la grandeur et la valeur d'une nation à la façon dont > celle-ci traite ses animaux. > - Mahatma Gandhi
