Running cyrus-imapd-2.2.3 on Linux Fedora Core 1, with Postfix-2.0.16 It's a small system, i'm the only user, mail gets delivered via SMTP to Postfix locally, which delivers it to Cyrus. I'm also using Sieve. Evolution is the principal client, with some webmailers used occasionally. The load is not much, i'm a very busy email user (bout 1k messages / day), but still i'm only one person, and i'm the sole user of that system, so the system is quite idle.
This is the second time the Cyrus IMAP server crashes spectacularly. First time, months ago, i was able to somehow regenerate mailboxes.db - don't ask me how, i only remember i found a script on the Internet that creates a text version of it which can be imported. Even so, since that crash it wouldn't let me create new folders. But that was ok, temporarily. Now it crashed again. I've found a huge email queue in the local Postfix server this morning - all messages waiting to be delivered to me locally. According to the logs, it happened Saturday early in the morning (that's when the messages started to queue up), now it's Monday at noon and everything is broken. I tried to run reconstruct, ctl_cyrusdb, ctl_cyrusdb_all, i moved away deliver.db temporarily... nothing helped. I cannot read my email, i have no idea how to fix it. I'm in a pretty bad situation. I'm writing this message from my other (personal) account. The cyrus documentation is not very helpful either. Please understand that i'm making efforts to stay rational. I'll provide you guys with all information about my system that you will require, i'm willing to run scripts and whatnot. I hope we could work together to fix it. But having the IMAP server crash twice in a few months with no apparent reason, and not having documentation available for what to do in such situations - it's not encouraging. In any case, i'm still willing to put up with it and do my best to fix it, provided that i get some kind of guidance. Logs are below. Please let me know what else you need from me. Thank you in advance. ########################################################## May 10 12:07:24 stantz master[19933]: process started May 10 12:07:24 stantz master[19934]: about to exec /usr/lib/cyrus-imapd/ctl_cyrusdb May 10 12:07:26 stantz ctl_cyrusdb[19934]: DBERROR db4: Improper file close at 5/9465323 May 10 12:07:26 stantz ctl_cyrusdb[19934]: DBERROR db4: Recovery function for LSN 5 9465323 failed on backward pass May 10 12:07:26 stantz ctl_cyrusdb[19934]: DBERROR db4: PANIC: Invalid argument May 10 12:07:26 stantz ctl_cyrusdb[19934]: DBERROR: critical database situation May 10 12:07:26 stantz master[19933]: process 19934 exited, status 75 May 10 12:07:26 stantz master[19937]: about to exec /usr/lib/cyrus-imapd/idled May 10 12:07:26 stantz idled[19937]: DBERROR db4: fatal region error detected; run recovery May 10 12:07:26 stantz idled[19937]: DBERROR: dbenv->open '/var/lib/imap/db' failed: DB_RUNRECOVERY: Fatal error, run database recovery May 10 12:07:26 stantz idled[19937]: DBERROR: init() on berkeley May 10 12:07:26 stantz master[19933]: ready for work May 10 12:07:26 stantz master[19942]: about to exec /usr/lib/cyrus-imapd/ctl_cyrusdb May 10 12:07:26 stantz ctl_cyrusdb[19942]: DBERROR db4: fatal region error detected; run recovery May 10 12:07:26 stantz ctl_cyrusdb[19942]: DBERROR: dbenv->open '/var/lib/imap/db' failed: DB_RUNRECOVERY: Fatal error, run database recovery May 10 12:07:26 stantz ctl_cyrusdb[19942]: DBERROR: init() on berkeley May 10 12:07:26 stantz master[19943]: about to exec /usr/lib/cyrus-imapd/imapd May 10 12:07:26 stantz master[19944]: about to exec /usr/lib/cyrus-imapd/imapd May 10 12:07:26 stantz master[19945]: about to exec /usr/lib/cyrus-imapd/lmtpd May 10 12:07:26 stantz ctl_cyrusdb[19942]: checkpointing cyrus databases May 10 12:07:26 stantz imap[19943]: DBERROR db4: fatal region error detected; run recovery May 10 12:07:26 stantz imaps[19944]: DBERROR db4: fatal region error detected; run recovery May 10 12:07:26 stantz lmtpunix[19945]: DBERROR db4: fatal region error detected; run recovery May 10 12:07:26 stantz ctl_cyrusdb[19942]: archiving database file: /var/lib/imap/annotations.db May 10 12:07:26 stantz imap[19943]: DBERROR: dbenv->open '/var/lib/imap/db' failed: DB_RUNRECOVERY: Fatal error, run database recovery May 10 12:07:26 stantz imaps[19944]: DBERROR: dbenv->open '/var/lib/imap/db' failed: DB_RUNRECOVERY: Fatal error, run database recovery May 10 12:07:27 stantz lmtpunix[19945]: DBERROR: dbenv->open '/var/lib/imap/db' failed: DB_RUNRECOVERY: Fatal error, run database recovery May 10 12:07:27 stantz ctl_cyrusdb[19942]: DBERROR db4: txn_checkpoint interface requires an environment configured for the transaction subsystem May 10 12:07:27 stantz imap[19943]: DBERROR: init() on berkeley May 10 12:07:27 stantz imaps[19944]: DBERROR: init() on berkeley May 10 12:07:27 stantz lmtpunix[19945]: DBERROR: init() on berkeley May 10 12:07:27 stantz ctl_cyrusdb[19942]: DBERROR: couldn't checkpoint: Invalid argument May 10 12:07:27 stantz imap[19943]: executed May 10 12:07:27 stantz imaps[19944]: executed May 10 12:07:27 stantz lmtpunix[19945]: executed May 10 12:07:27 stantz ctl_cyrusdb[19942]: DBERROR: sync /var/lib/imap/db: cyrusdb error May 10 12:07:27 stantz lmtpunix[19945]: DBERROR db4: environment not yet opened May 10 12:07:27 stantz ctl_cyrusdb[19942]: DBERROR db4: DB_ENV->log_archive interface requires an environment configured for the logging subsystem May 10 12:07:27 stantz lmtpunix[19945]: DBERROR: opening /var/lib/imap/deliver.db: Invalid argument May 10 12:07:27 stantz ctl_cyrusdb[19942]: DBERROR: error listing log files: Invalid argument May 10 12:07:27 stantz lmtpunix[19945]: DBERROR: opening /var/lib/imap/deliver.db: cyrusdb error May 10 12:07:27 stantz ctl_cyrusdb[19942]: DBERROR: archive /var/lib/imap/db: cyrusdb error May 10 12:07:27 stantz lmtpunix[19945]: FATAL: lmtpd: unable to init duplicate delivery database May 10 12:07:28 stantz ctl_cyrusdb[19942]: archiving database file: /var/lib/imap/mailboxes.db May 10 12:07:28 stantz master[19933]: process 19945 exited, status 75 May 10 12:07:28 stantz ctl_cyrusdb[19942]: DBERROR db4: txn_checkpoint interface requires an environment configured for the transaction subsystem May 10 12:07:28 stantz master[19933]: service lmtpunix pid 19945 in READY state: terminated abnormally May 10 12:07:28 stantz ctl_cyrusdb[19942]: DBERROR: couldn't checkpoint: Invalid argument May 10 12:07:28 stantz master[19946]: about to exec /usr/lib/cyrus-imapd/imapd May 10 12:07:28 stantz master[19947]: about to exec /usr/lib/cyrus-imapd/lmtpd May 10 12:07:28 stantz ctl_cyrusdb[19942]: DBERROR: sync /var/lib/imap/db: cyrusdb error May 10 12:07:28 stantz imap[19946]: DBERROR db4: fatal region error detected; run recovery May 10 12:07:28 stantz lmtpunix[19947]: DBERROR db4: fatal region error detected; run recovery May 10 12:07:28 stantz ctl_cyrusdb[19942]: DBERROR db4: DB_ENV->log_archive interface requires an environment configured for the logging subsystem May 10 12:07:28 stantz imap[19946]: DBERROR: dbenv->open '/var/lib/imap/db' failed: DB_RUNRECOVERY: Fatal error, run database recovery May 10 12:07:28 stantz lmtpunix[19947]: DBERROR: dbenv->open '/var/lib/imap/db' failed: DB_RUNRECOVERY: Fatal error, run database recovery May 10 12:07:28 stantz master[19933]: exiting on SIGTERM/SIGINT ######################################################## -- Florin Andrei http://florin.myip.org/ --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
