Hi Michael, On Wed, Jan 16, 2019, at 10:12 PM, Michael Menge wrote: > Hi, > > because conversations db seems to be required for search indexes, I > enabled this option > on our production servers today and tried to rebuild the conversations > db for all users with > > ctl_conversationsdb -v -b UID > > For most users this did take less than a second. But for some users > this process would > not finish. I did kill one process after about 30 Minutes (most others > after 3 Minutes). > The UserID.conversations has grown over 2 GB (the mailbox itself has > only ~700 MB of mails, > and the conversations files from finnished rebuilds are less then 20 MB)
So, a 2GB-and-growing conversations db for a 700MB mailbox is a ratio of ~3:1, which seems obscene... For comparison, the Conversations.append_reply_200 cassandane test (two messages with 100 replies each) produces an 87Kb conversations db for a 1016Kb mailbox (ratio ~1:12, n.b inverted!), which makes your ~3:1 and growing ratio seem even more obscene... Obviously you can't share the bad conversations db because conversations db's are full of personally-identifying information. But I wonder if looking through it with `cyr_dbtool ... show` turns up any unusual patterns, especially in comparison to one of the good users? I wonder if it's somehow repeating itself? Bron, does this sound like anything you've seen before, that maybe got fixed on master but not backported to 3.0? > Cyrus Logs show many "ctl_conversationsdb[933]: mailbox: longlock > user.UserID for 1.5 seconds" > every few seconds. This message is logged when the lock is released, if that lock had been held for >1s. It's reporting that it held the lock for longer than it would like to have, but at least you know the lock has been released. I think this is telling you that user-visible performance on the mailbox would have been impacted while this was occurring (because imapd/lmtpd wouldn't be able to access their mailbox while ctl_conversationsdb was holding those locks), but also tells you that ctl_conversationsdb was doing the right thing, and not just holding a single lock for the entire job. Though, that raises the question: was the user accessing the mailbox while the rebuild was in progress? I wonder if their client is doing something funny and tripping things up? Cheers, ellie > I did try reconstruct -r -G user/UserID, followed by > ctl_conversationsdb -v -z UID > and tried to rebuild the db again. Reconstruct reported no errors, and > the new rebuild > hat the same problem. > > Has anybody seen a similar problem? > > Any ideas how to fix this? > > > We are running cyrus-imapd 3.0.8 > > > Kind regards > > Michael Menge > > -------------------------------------------------------------------------------- > M.Menge Tel.: (49) 7071/29-70316 > Universität Tübingen Fax.: (49) 7071/29-5912 > Zentrum für Datenverarbeitung mail: > michael.me...@zdv.uni-tuebingen.de > Wächterstraße 76 > 72074 Tübingen ---- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
