Dear All
Mea Culpa !!
I think I was assuming that replication would work the same in 2.4.5 as
it was in 2.3.16 - but it has had a major rewrite.
I think the problem is I had this in the replica's cyrus.conf
purgeall cmd="ipurge -f -d 150" at=2200
So it impacted on the client / master but the replica the logs report
Dec 5 01:00:02 proxy cyr_expire[25067]: Expunged 0 out of 640465
messages from 1479 mailboxes
Dec 6 01:01:27 proxy cyr_expire[29917]: Expunged 0 out of 588837
messages from 1431 mailboxes
The reason I suspect this has an impact is the -d 150 and I now suspect
what happens on the replica is reflected on the master.
I had this parameter as I did not want the amount of emails to get
massive - if an email was more than 150 days old it was lost.
Now I know I was not using the replica as a true "image" of the master
in that I have in the replica's imapd.conf the following
expunge_mode: delayed
delete_mode: delayed
The reason for this is that I have had instances of users misconfiguring
POP or deleting emails and then emptying Trash and asking where is the
email from John Citizen.
I have had to restore whole mail folders for a few users.
So I was using the replica as a emergency backup store of emails.
What do you recommend ?
I have disabled the purgeall command.
Thanks
Stephen Carr
Stephen Carr wrote:
Dear Bron
Yes ant = any - see comments below as well.
Any chance the reconstruct runs could have had an impact ?
I have restored all the emails on one account and have run reconstruct
- r -f -G on that account that had about 70,000 emails rediscovered.
Will watch what happens overnight.
PS Most users POP theie emails off the server only a few use pure IMAP.
Thanks again.
Stephen Carr
Bron Gondwana wrote:
On Mon, Dec 06, 2010 at 08:22:57AM +1030, Stephen Carr wrote:
After fixing sync problems with the help of Bron this morning I
discovered that ant email older that 150 days has been expunged.
What alerted me was the size of the Full backup done one day after
the upgrade.
"ant email" - do you mean "any email"?
Here is part of the log
Dec 5 04:00:02 brooks cyr_expire[23141]: Expunged 0 out of 208877
messages from 1453 mailboxes
No messages expired.
I did an ls -lR of the users directory and this morning got 77,179
where as before it was around 230,000
I restored the maildata from the backup prior to the upgrade to
another system as a check and ran the command below result was 235,362
The command is
ls -lR /maildata/cyrus/user/ | grep " [1-9]*\." | wc -l
Dec 5 21:00:00 brooks ipurge[4410]: Expunged 2 messages from
user.user1.Trash
Dec 5 21:00:00 brooks ipurge[4410]: Expunged 1 messages from
user.user2.Trash
Dec 5 21:00:01 brooks ipurge[4410]: Expunged 25 messages from
user.user3.Trash
...
Dec 6 03:00:00 brooks ipurge[18145]: Expunged 1 messages from
user.user4.quarantine
Dec 6 03:00:00 brooks ipurge[18145]: Expunged 5 messages from
user.user5.quarantine
....
Dec 6 04:00:00 brooks ipurge[20370]: Expunged 95 messages from
user.support.Bacula
Lots of messages cleaned up by ipurge.
Some of these emails are temporary especially from Bacula where I get
150+ emails a day as I backup clients and servers.
Dec 6 04:00:02 brooks cyr_expire[20368]: Expunged 0 out of 113053
messages from 1453 mailboxes
Still no messages expired.
Notice the 208877 is now 113053 and I suspect it now more like 77000
since it is 2 days after the upgrade.
Is there a default Expunge ?
It's looking to me more like ipurge used to be broken, and now it's
working!
expunge_mode: immediate
delete_mode: immediate
I strongly recommend removing the "expunge_mode: immediate" and either
setting it explicitly to "expunge_mode: default" or even just not
specifying it. "expunge_mode: immediate" doesn't work nicely with
replication - you wind up having to run inefficient sync all the time
because it cleaned out the expunge information too quickly.
Change ran kill -HUP of master process ID.
sync_password: 30cyrus51
Opps will change
In general I wouldn't recommend posting this bit to a public
mailing list...
# this is only necessary if using duplicate delivery suppression,
# Sieve or NNTP
delprune cmd="cyr_expire -E 3" at=0400
You don't have any "-X" in there, which means that delayed expunge
will be running immediately for all mailboxes. Again, -X 1 along
with the expunge_mode changes above will make your replication
experience nicer.
Updated cyrus config
trashprune1 cmd="ipurge -f -d 21 user.%.Trash" at=2100
trashprune2 cmd="ipurge -f -d 21 user.%.trash" at=2130
# Purge quarantine mailboxes older than 14 days at 3 am
purgespam cmd="ipurge -f -d 14 user.%.quarantine" at=0300
purgespam2 cmd="ipurge -f -d 2 user.%.spam" at=0330
purgebacula cmd="ipurge -f -d 28 user.support.Bacula" at=0400
Yeah, it's those ipurge commands that are doing it. I suspect
there used to be bugs in ipurge with immediate expunge, and
you just didn't notice that it wasn't actually being cleaned up
because you didn't have a -X specified in you cyr_expire...
Bron.
--
Stephen Carr
School of Civil, Environmental & Mining Engineering
The University of Adelaide
Adelaide, South Australia, 5005
Australia
Tel +618-8303-4313
Fax +618-8303-4359
Email sgc...@civeng.adelaide.edu.au
CRICOS Provider Number 00123M
-----------------------------------------------------------
This email message is intended only for the addressee(s) and
contains information that may be confidential and/or copyright.
If you are not the intended recipient please notify the sender
by reply email and immediately delete this email. Use, disclosure
or reproduction of this email by anyone other than the intended
recipient(s) is strictly prohibited. No representation is made
that this email or any attachments are free of viruses. Virus
scanning is recommended and is the responsibility of the recipient.
------------------------------------------------------------------------
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
--
Stephen Carr
School of Civil, Environmental & Mining Engineering
The University of Adelaide
Adelaide, South Australia, 5005
Australia
Tel +618-8303-4313
Fax +618-8303-4359
Email sgc...@civeng.adelaide.edu.au
CRICOS Provider Number 00123M
-----------------------------------------------------------
This email message is intended only for the addressee(s) and
contains information that may be confidential and/or copyright.
If you are not the intended recipient please notify the sender
by reply email and immediately delete this email. Use, disclosure
or reproduction of this email by anyone other than the intended
recipient(s) is strictly prohibited. No representation is made
that this email or any attachments are free of viruses. Virus
scanning is recommended and is the responsibility of the recipient.
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
