Re: lmtpd: deliver.db checkpointing

[David Mayo]

On 15/11/2010 14:46, Patrick Boutilier wrote:
> On 11/15/2010 10:36 AM, David Mayo wrote:
>>
>> Having looked at the source code, we can see that ctl_cyrusdb does not
>> touch deliver.db and it is not possible to force checkpoints on or off
>> within lmtpd. Ideally we want to schedule the checkpointing to a "less
>> sociable" time.
>>
>> On our setup, the checkpoint runs every couple of days and usually takes
>> 4-6 minutes[1]. We run cyr_expire every night to remove entries older
>> than 3 days from the duplicate deliveries database but this is the only
>> maintenance we can schedule on this database.
>
> No solution, but I wonder why your checkpoint takes so long? With
> similar DB size it take only seconds here.
>
> Nov 8 21:18:04 student2 lmtpunix[6541]: skiplist: checkpointed
> /var/imap/deliver.db (650493 records, 67180160 bytes) in 5 seconds
>
> Nov 12 23:14:06 student2 lmtpunix[26425]: skiplist: checkpointed
> /var/imap/deliver.db (768262 records, 80010936 bytes) in 11 seconds

Looking back further through our logs, I found a checkpoint that was run 
overnight which completes somewhat faster, but nowhere near your times! 
Our IMAP back-end is running on a 4x Quad-Core AMD Opteron 2.3 GHz 
processor with 16 GB RAM:

Oct 14 00:17:47 imap.bath.ac.uk lmtp[29924]: [ID 301543 mail.info] 
skiplist: checkpointed /opt/etc/imapd/deliver.db (715978 records, 
68761076 bytes) in 76 seconds

This presumably doesn't cause anyone much trouble overnight, but it's a 
bit more noticeable during the day.

Regards,


Dave.

David Mayo
Networks/Systems Administrator
University of Bath Computing Services, UK

>> Do other people suffer from this? Is there a patch that anyone has
>> written and/or will be submitting upstream any time soon?
>>
>> We are running Cyrus 2.3.13 across a front end, a back end and a
>> replication host for around 25,000 users.
>>
>> Regards,
>>
>>
>> Dave.
>>
>> David Mayo
>> Networks/Systems Administrator
>> University of Bath Computing Services, UK
>>
>> [1]
>> Nov 4 15:32:33 imap.bath.ac.uk lmtp[29595]: [ID 301543 mail.info]
>> skiplist: checkpointed /opt/etc/imapd/deliver.db (787618 records,
>> 74775536 bytes) in 386 seconds
>> Nov 9 14:26:06 imap.bath.ac.uk lmtp[1676]: [ID 301543 mail.info]
>> skiplist: checkpointed /opt/etc/imapd/deliver.db (386167 records,
>> 36638408 bytes) in 232 seconds
>> Nov 11 12:45:35 imap.bath.ac.uk lmtp[2497]: [ID 301543 mail.info]
>> skiplist: checkpointed /opt/etc/imapd/deliver.db (657254 records,
>> 62540568 bytes) in 396 seconds

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: replication

[Michael Menge]

Quoting Shuvam Misra :


Quoting Bron Gondwana :
>
> It's getting better, but it's still not 100% reliable to have
> master/master replication between two servers with interactions
> going to both sides.
>
> It SHOULD be safe now to have a single master/master setup with
> individual users on one side or the other - but note that nobody
> is known to be running that setup successfully yet.
>
> As for what the point is?  I don't know about you, but I run a
> 24hr/day shop, and I like to be able to take a server down for
> maintainence in about 2 minutes, with users seeing a brief
> disconnection and then being able to keep using the service
> with minimal disruption.
>
> Bron.

As Bron already mentioned the problems of master/master mode
you can easy live without.

We run multiple servers, these are paired, each server is running one
cyrus instance in as master and one as slave, so that the pairs
replicate each other. In case of a crash one server would run two
master instances.

You only need a way of splitting the users between the  servers.
That could be DNS, a proxy or murder setup.


Are you using local storage on each server for spool and metadata?


We have all cyrus storage on iSCSI-Systems


How good/bad is the idea of using shared storage (an external SAN
chassis) and letting multiple servers keep their spool areas there? Can
one set up, say, half a dozen servers in a pool, each using a separate
LUN for spool+data on a common back-end SAN chassis? Out of the six
servers, one would be a hot spare, standing by. If any of the five active
servers failed, the standby would be told to mount the failed server's
LUN, borrow the failed server's IP address, and start offering services?



That would work, but you would still have a single point of failure
if the SAN system chraches or if the filesystem of one backend gets
corrupted.

We have 6 Servers and 2 independent iSCSI-Systems. Each iSCSI-System
holds 3 partitions for active servers and 3 partiotins for replications.


In this proposed model, each user's account is on one "physical" server
(i.e. bound to a specific IP address). No load balancing or connection
spreading is needed when clients connect. If the site chooses to use
Murder, then the proposed model can apply to the back-end while the
multiplexer can take care of the front-end.

The only thing I'm not sure about is the file system corruption when a
node goes down and the time taken for an fsck before the standby node can
assume the failed node's role. I wonder whether something like the ext4
will help reduce fsck timings to acceptable levels.


The time checking is one thing, but if you lose data in one partition
you have a problem. Restoring files from filebased backup is a pain
if you have many small files like cyrus has.



Is this a good idea for a scalable fault-tolerant Cyrus setup? I've been
toying with this approach for some time, for a proposed large-system design.



We are testing cyrus murder to ease the work of switching to a replication
and back.




M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen

smime.p7s
Description: S/MIME Signatur

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Does anyone allow unlimited or extremely large quotas?

[Dave McMurtrie]

Good morning,

This may be slightly off-topic, so apologies in advance.  Is there 
anyone out there who allows unlimited quota for their users or provides 
extremely large quotas when asked for?

If so, can you describe any problems you've had with this?

Thanks,

Dave

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Gavin McCullagh]

Hi,

On Tue, 16 Nov 2010, Dave McMurtrie wrote:

> This may be slightly off-topic, so apologies in advance.  Is there 
> anyone out there who allows unlimited quota for their users or provides 
> extremely large quotas when asked for?

What do you regard as extremely large?  10GB, 100GB, 1TB, ...?

Gavin


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Dave McMurtrie]

On 11/16/2010 06:45 AM, Gavin McCullagh wrote:
> Hi,
>
> On Tue, 16 Nov 2010, Dave McMurtrie wrote:
>
>> This may be slightly off-topic, so apologies in advance.  Is there
>> anyone out there who allows unlimited quota for their users or provides
>> extremely large quotas when asked for?
>
> What do you regard as extremely large?  10GB, 100GB, 1TB, ...?

Well, unlimited was the largest I had in mind.  Short of that, sure, 
10GB, 100GB 1TB would all be large.

Thanks,

Dave

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Simon Matter]

> On 11/16/2010 06:45 AM, Gavin McCullagh wrote:
>> Hi,
>>
>> On Tue, 16 Nov 2010, Dave McMurtrie wrote:
>>
>>> This may be slightly off-topic, so apologies in advance.  Is there
>>> anyone out there who allows unlimited quota for their users or provides
>>> extremely large quotas when asked for?
>>
>> What do you regard as extremely large?  10GB, 100GB, 1TB, ...?
>
> Well, unlimited was the largest I had in mind.  Short of that, sure,
> 10GB, 100GB 1TB would all be large.

More than 10GB and unlimited is used in environments I've been busy with.
Of course it really depends on the nature of service you're providing.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Fwd: Re: Does anyone allow unlimited or extremely large quotas?

[Dave McMurtrie]

I didn't realize that I only responded to Rob here.  Perhaps my 
additional information will shed some light on the kind of information 
I'm looking for.

 Original Message 
Subject: Re: Does anyone allow unlimited or extremely large quotas?
Date: Tue, 16 Nov 2010 07:06:53 -0500
From: Dave McMurtrie 
To: Rob Mueller 

On 11/16/2010 06:45 AM, Rob Mueller wrote:
>
>> This may be slightly off-topic, so apologies in advance. Is there
>> anyone out there who allows unlimited quota for their users or provides
>> extremely large quotas when asked for?
>
> What do you consider extremely large? And what sort of problems are you
> referring to?

I don't actually know what sort of problems I'm referring to, hence the
question.  The big problem I can imagine would be opendir() and
readdir() with a huge number of files in a directory, but the cyrus code
doesn't appear to do that in a lot of places that would matter to a user
(deleting an entire folder, delete sieve scripts, etc) in the course of
normal operations.

My manager has asked me how well Cyrus will cope with large (100GB+) or
unlimited quotas.  My answer to him was that it should be okay, but I
have very little practical experience with such so I wanted to ask on
the list.

> The usual issue is just the huge number of emails and thus files that
> accumulate. Creating a fresh replica, body searching, reconstructing,
> etc all take quite a bit of time because of the large amount of random
> IOs. Apart from that, everything does actually work ok...

The only issue we ever had was with a bboard that our network group
sends automated system messages to.  Something in their environment went
haywire and we ended up with ~1.5 million messages in that bboard.  They
were unable to find a client that was willing to deal with the folder to
be able to clean it up.  I was able to connect using imtest and SELECT
and FETCH messages without any problems, though.  I also recall that
replication was broken by this folder, but I don't remember exactly why.

So basically, I have this tiny amount of practical experience that tells
me if there are 1.5 million files in a single folder, clients may be
unhappy and replication may break but the server was still generally
working.

Any anecdotal evidence I can collect in addition to this would be
helpful for me to be able to go back to my manager with.

Thanks!

Dave

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Rudy Gevaert]

On 11/16/2010 12:30 PM, Dave McMurtrie wrote:
> Good morning,
>
> This may be slightly off-topic, so apologies in advance.  Is there
> anyone out there who allows unlimited quota for their users or provides
> extremely large quotas when asked for?
>
> If so, can you describe any problems you've had with this?


We have users with > 5 GB.

We haven't seen any problems with them.  The only general problem we 
face is taking backups of a full store.  It just takes very long to 
complete full backups.  (Of course that isn't tied to large or no quota.)

Rudy

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Fwd: Re: Does anyone allow unlimited or extremely large quotas?

[Simon Fraser]

> I don't actually know what sort of problems I'm referring to, hence the
> question.  The big problem I can imagine would be opendir() and
> readdir() with a huge number of files in a directory, but the cyrus code
> doesn't appear to do that in a lot of places that would matter to a user
> (deleting an entire folder, delete sieve scripts, etc) in the course of
> normal operations.

The number of files in a directory certainly seems to be the performance
factor for us.  We don't enforce quotas, but our largest mailboxes are
only about 15Gb. Deleting large folders (~10 messages) does take
some time. The only event that has troubled other users of the system
was one user who had added 7.2 million messages to their trash folder,
and then emptied their trash.  It took the better part of a day to
finish, and impacted both read and write performance for other users
(nexsan providing storage over fibre, xfs on top) but the service kept
going.  For what it's worth the Trash folder was only a few Gb. 

Simon.



-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE. 

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Adam Tauno Williams]

On Tue, 2010-11-16 at 14:12 +0100, Rudy Gevaert wrote: 
> On 11/16/2010 12:30 PM, Dave McMurtrie wrote:
> > Good morning,
> > This may be slightly off-topic, so apologies in advance.  Is there
> > anyone out there who allows unlimited quota for their users or provides
> > extremely large quotas when asked for?
> > If so, can you describe any problems you've had with this?
> We have users with > 5 GB.

Our largest quota's a 4GB; without any issues.

I think the issue you will encounter first is clients will start to fall
down when folders exceed a 'reasonable' number of messages.  Common IMAP
clients I've seen start to exhibit severe performance issues beyond a
few hundred thousand messages.

> We haven't seen any problems with them.  The only general problem we 
> face is taking backups of a full store.  It just takes very long to 
> complete full backups.  (Of course that isn't tied to large or no quota.)




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Mike Eggleston]

On Tue, 16 Nov 2010, Adam Tauno Williams might have said:

> On Tue, 2010-11-16 at 14:12 +0100, Rudy Gevaert wrote: 
> > On 11/16/2010 12:30 PM, Dave McMurtrie wrote:
> > > Good morning,
> > > This may be slightly off-topic, so apologies in advance.  Is there
> > > anyone out there who allows unlimited quota for their users or provides
> > > extremely large quotas when asked for?
> > > If so, can you describe any problems you've had with this?
> > We have users with > 5 GB.
> 
> Our largest quota's a 4GB; without any issues.
> 
> I think the issue you will encounter first is clients will start to fall
> down when folders exceed a 'reasonable' number of messages.  Common IMAP
> clients I've seen start to exhibit severe performance issues beyond a
> few hundred thousand messages.
> 
> > We haven't seen any problems with them.  The only general problem we 
> > face is taking backups of a full store.  It just takes very long to 
> > complete full backups.  (Of course that isn't tied to large or no quota.)
> 

I have two users that have over 4GB of email. I manage a small shop,
around 25 users depending on the time of year. The support and sales
people insist on keeping all mail messages ad infitium.

I have no quotas.

Mike

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Simon Amor]

On 16 Nov 2010, at 13:38, Adam Tauno Williams wrote:
>
> Our largest quota's a 4GB; without any issues.
>
> I think the issue you will encounter first is clients will start to  
> fall
> down when folders exceed a 'reasonable' number of messages.  Common  
> IMAP
> clients I've seen start to exhibit severe performance issues beyond a
> few hundred thousand messages.

Is that with the server and client on the same LAN or with the client  
on a low speed WAN connection? We find that 50,000 messages in a  
folder is more than enough to make Thunderbird/Outlook unresponsive  
for minutes at a time when connecting to a remote server.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Fwd: Re: Does anyone allow unlimited or extremely large quotas?

[Simon Matter]

>
>
>> I don't actually know what sort of problems I'm referring to, hence the
>> question.  The big problem I can imagine would be opendir() and
>> readdir() with a huge number of files in a directory, but the cyrus code
>> doesn't appear to do that in a lot of places that would matter to a user
>> (deleting an entire folder, delete sieve scripts, etc) in the course of
>> normal operations.
>
> The number of files in a directory certainly seems to be the performance
> factor for us.  We don't enforce quotas, but our largest mailboxes are
> only about 15Gb. Deleting large folders (~10 messages) does take
> some time. The only event that has troubled other users of the system
> was one user who had added 7.2 million messages to their trash folder,
> and then emptied their trash.  It took the better part of a day to
> finish, and impacted both read and write performance for other users
> (nexsan providing storage over fibre, xfs on top) but the service kept

Speaking of XFS it was quite slow in deleting large number of small files
some years ago. If that's still true it may be what you saw.
Since delayed delete has been enabled those issues have not be seen over
here.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Shuvam Misra]

> > I think the issue you will encounter first is clients will start to fall
> > down when folders exceed a 'reasonable' number of messages.  Common IMAP
> > clients I've seen start to exhibit severe performance issues beyond a
> > few hundred thousand messages.
> 
> As far as I'm aware (the helpdesk guys know better than me so I'm parroting
> their reply), Outlook 2003's PST file has a limit of 2GB so if it's locally
> caching folders, you may run into that.
> 
> If you use Outlook 2007 or later, the limit is more like 20GB.  BUT, if you
> upgrade from 2003 and use the same PST, that PST may continue with the same
> 2GB limit.  Apparently you might need to create a new PST file and move the
> mail into it?.  Some big users have been moved to Thunderbird to avoid this
> and to improve performance.
> 
> Gavin
> 
> ? To be honest, I haven't personally dealt with this issue, but this
>   paraphrases the knowledge of those here who have.  I'd think of it as
>   having "[citation required]" beside it.

I'm in an almost-identical position w.r.t. lack of direct knowledge, but
our Support guys say exactly the same things about Outlook 2003 and
Outlook 2007 and size limits of PST files.

That said, we have users of our product who have 40GB mailboxes.
Cyrus works perfectly happily with all this. The problem is the number
of messages in the current folder, as many have mentioned before me. We
keep telling users to clean up their Inboxes and keep a max of 1,000
msgs there. We know things will be fine with 10,000 messages too, but
100,000 msgs in a folder is pushing things.

We find that Webmail chokes server performance much earlier than normal
IMAP clients do. I know this has nothing to do with Cyrus, but I just
thought I'd mention it. Most programming environments in which such
Webmail thingies are written (mostly PHP on the server and nowadays
lots of Javascript on the browser) cannot keep an IMAP connection to the
Cyrus server open between pages, therefore each time a user clicks on a
folder or does any other operation, there's this fresh IMAP connection
and a huge surge of IMAP operations while the folder contents are listed
afresh, etc. This puts a lot of load on the server. I guess Webmail is
OT on a Cyrus mailing list, but can't help asking: any suggestions for
improving Webmail performance? (Admission: we haven't yet tried imapproxy
-- it appears to be a good piece of C which will help things.)

regards,
Shuvam

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Joseph Brennan]

> We know things will be fine with 10,000 messages too, but
> 100,000 msgs in a folder is pushing things.

My inbox is 17,338 at the moment, and it's still fast.  But I'm using
an old copy of Mulberry, which was designed for IMAP.

The problem area is clients designed for POP and adapted to IMAP.  I'd
say this relevant to Cyrus, because if we have a great server side but
weak user-facing software, we lose the game.

I wish we'd somehow financed a native Cyrus webmail interface, that is
not using IMAP but built into Cyrus.  I don't think users know how good
Cyrus is because they look at it through a weak intermediary.



> (Admission: we haven't yet tried imapproxy
> -- it appears to be a good piece of C which will help things.)

Do it.  It makes a huge difference.  You go from crawling to just slow.


Joseph Brennan
Columbia University Information Technology


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Kenneth Marshall]

On Tue, Nov 16, 2010 at 08:41:46PM +0530, Shuvam Misra wrote:
> > > I think the issue you will encounter first is clients will start to fall
> > > down when folders exceed a 'reasonable' number of messages.  Common IMAP
> > > clients I've seen start to exhibit severe performance issues beyond a
> > > few hundred thousand messages.
> > 
> > As far as I'm aware (the helpdesk guys know better than me so I'm parroting
> > their reply), Outlook 2003's PST file has a limit of 2GB so if it's locally
> > caching folders, you may run into that.
> > 
> > If you use Outlook 2007 or later, the limit is more like 20GB.  BUT, if you
> > upgrade from 2003 and use the same PST, that PST may continue with the same
> > 2GB limit.  Apparently you might need to create a new PST file and move the
> > mail into it?.  Some big users have been moved to Thunderbird to avoid this
> > and to improve performance.
> > 
> > Gavin
> > 
> > ? To be honest, I haven't personally dealt with this issue, but this
> >   paraphrases the knowledge of those here who have.  I'd think of it as
> >   having "[citation required]" beside it.
> 
> I'm in an almost-identical position w.r.t. lack of direct knowledge, but
> our Support guys say exactly the same things about Outlook 2003 and
> Outlook 2007 and size limits of PST files.
> 
> That said, we have users of our product who have 40GB mailboxes.
> Cyrus works perfectly happily with all this. The problem is the number
> of messages in the current folder, as many have mentioned before me. We
> keep telling users to clean up their Inboxes and keep a max of 1,000
> msgs there. We know things will be fine with 10,000 messages too, but
> 100,000 msgs in a folder is pushing things.
> 
> We find that Webmail chokes server performance much earlier than normal
> IMAP clients do. I know this has nothing to do with Cyrus, but I just
> thought I'd mention it. Most programming environments in which such
> Webmail thingies are written (mostly PHP on the server and nowadays
> lots of Javascript on the browser) cannot keep an IMAP connection to the
> Cyrus server open between pages, therefore each time a user clicks on a
> folder or does any other operation, there's this fresh IMAP connection
> and a huge surge of IMAP operations while the folder contents are listed
> afresh, etc. This puts a lot of load on the server. I guess Webmail is
> OT on a Cyrus mailing list, but can't help asking: any suggestions for
> improving Webmail performance? (Admission: we haven't yet tried imapproxy
> -- it appears to be a good piece of C which will help things.)
> 
> regards,
> Shuvam


We use imapproxy here to avoid exactly this situation with webmail.

Cheers,
Ken

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Wesley Craig]

Didn't Dave write up.imapproxy?  It makes a huge difference for, e.g., IMP & 
roundcube.  Also, configuring client to not retrieve the LIST of mailboxes 
during every transaction is a big win.

:wes

On 16 Nov 2010, at 10:11, Shuvam Misra wrote:

> Most programming environments in which such
> Webmail thingies are written (mostly PHP on the server and nowadays
> lots of Javascript on the browser) cannot keep an IMAP connection to the
> Cyrus server open between pages, therefore each time a user clicks on a
> folder or does any other operation, there's this fresh IMAP connection
> and a huge surge of IMAP operations while the folder contents are listed
> afresh, etc. This puts a lot of load on the server. I guess Webmail is
> OT on a Cyrus mailing list, but can't help asking: any suggestions for
> improving Webmail performance? 


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Dave McMurtrie]

On 11/16/2010 10:36 AM, Wesley Craig wrote:
> Didn't Dave write up.imapproxy?  It makes a huge difference for, e.g., IMP&  
> roundcube.  Also, configuring client to not retrieve the LIST of mailboxes 
> during every transaction is a big win.

Coincidentally, yes I did originally write that :)

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Adam Tauno Williams]

> > We find that Webmail chokes server performance much earlier than normal
> > IMAP clients do. I know this has nothing to do with Cyrus, but I just
> > thought I'd mention it. Most programming environments in which such
> > Webmail thingies are written (mostly PHP on the server and nowadays
> > lots of Javascript on the browser) cannot keep an IMAP connection to the
> > Cyrus server open between pages, therefore each time a user clicks on a
> > folder or does any other operation, there's this fresh IMAP connection
> > and a huge surge of IMAP operations while the folder contents are listed
> > afresh, etc. This puts a lot of load on the server. I guess Webmail is
> > OT on a Cyrus mailing list, but can't help asking: any suggestions for
> > improving Webmail performance? (Admission: we haven't yet tried imapproxy
> > -- it appears to be a good piece of C which will help things.)
> We use imapproxy here to avoid exactly this situation with webmail.

+1

We use up.imapproxy; since installing that our Horde/IMP servers *fly*.
The performance difference is really impressive.

-- 
Adam Tauno Williams  LPIC-1, Novell CLA

OpenGroupware, Cyrus IMAPd, Postfix, OpenLDAP, Samba


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

sieve not filing messages?

[Mike Eggleston]

Morning,

I have a user saying that email messages are not being filed as desired. I have 
copied the user's script into Sieve Test 
 and the user's sieve rules. 
Sieve Test says the message should go to where is expected, but the message 
isn't in the cyrus message store.

The time stamps have changed in the folder where the message should be, so I 
can't tell if the message was there and deleted or never filed there.

The log entries of that day when the message was received from another internal 
user has an odd entry:

Nov  9 10:31:51 elo master[10189]: about to exec /usr/lib/cyrus-imapd/lmtpd
Nov  9 10:31:51 elo lmtpunix[10189]: executed
Nov  9 10:33:07 elo sendmail[10210]: oA9GX7ER010210: from=<$us...@$domain.com>, 
size=975, class=0, nrcpts=1, msgid=<005a01cb802b$ca410940$5ec31b...@com>, 
proto=ESMTP, daemon=MTA, relay=ANAKIN.$domain.com [10.1.2.194]
Nov  9 10:33:07 elo lmtpunix[10189]: accepted connection
Nov  9 10:33:07 elo lmtpunix[10189]: lmtp connection preauth'd as postman
Nov  9 10:33:07 elo lmtpunix[10189]: duplicate_check: 
<005a01cb802b$ca410940$5ec31b...@com>user.$user2.backup  0
Nov  9 10:33:07 elo lmtpunix[10189]: seen_db: user $user2 opened 
/var/lib/imap/user/u/$user2.seen
Nov  9 10:33:08 elo lmtpunix[10189]: duplicate_mark: 
<005a01cb802b$ca410940$5ec31b...@com>user.$user2.backup  1289320387 852209
Nov  9 10:33:08 elo lmtpunix[10189]: duplicate_check:  ^A| }^\Ö7Ã^W^_JÍ&cXC 
$user2  1289839869
Nov  9 10:33:08 elo lmtpunix[10189]: duplicate_mark: 
<005a01cb802b$ca410940$5ec31b...@com>.$use...@.sieve.1289320388 0
Nov  9 10:33:08 elo sendmail[10212]: oA9GX7ER010210: to=<$us...@$domain.com>, 
delay=00:00:01, xdelay=00:00:01, mailer=cyrusv2, pri=120975, relay=localhost, 
dsn=2.0.0, stat=Sent
Nov  9 10:34:08 elo master[5702]: process 10189 exited, status 0

The entry about the duplicate_check has some really wierd characters. Is this 
normal?

Where else can I check to trace why the message wasn't filed into the folder, 
was filed into the folder, was deleted from the folder, etc?

Mike

$ rpm -qa | grep cyrus
cyrus-imapd-devel-2.3.1-2.8.fc5
cyrus-sasl-lib-2.1.21-10
cyrus-sasl-2.1.21-10
cyrus-imapd-utils-2.3.1-2.8.fc5
cyrus-sasl-ldap-2.1.21-10
cyrus-imapd-2.3.1-2.8.fc5
cyrus-sasl-devel-2.1.21-10

$ cat /etc/redhat-release
Fedora Core release 5 (Bordeaux)

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Leena Heino]

On 16.11.2010 17:11, Shuvam Misra wrote:
> I guess Webmail is OT on a Cyrus mailing list, but can't help asking: any 
> suggestions for
> improving Webmail performance? (Admission: we haven't yet tried imapproxy
> -- it appears to be a good piece of C which will help things.)

You should install imapproxy. Also make sure you do a lot of caching on
the webmail side for example if the webmail is programmed with php use
apc and cache mailbox listing and message headers if possible.

Cache also things on the imap server, if your cyrus version supports it
you should try to use status cache and test whether it makes any
difference in your environment.

-- 
  Leena Heino  University of Tampere / Computer Centre
  ( liinu at uta.fi )  ( http://www.uta.fi/laitokset/tkk )

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Wesley Craig]

On 16 Nov 2010, at 10:32, Joseph Brennan wrote:
> I wish we'd somehow financed a native Cyrus webmail interface, that is
> not using IMAP but built into Cyrus.  I don't think users know how good
> Cyrus is because they look at it through a weak intermediary.

I don't think a Cyrus-specific web interface is the answer to that question.  
IMP performance is not great, but it's the http paradigm that slows it.  Check 
out roundcube, utilizing AJAX it's way more responsive to the user.

:wes

(UMich runs IMP & roundcube both pointing to unlimited quota Cyrus servers.  
Roundcube is the win.)

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

sync_client IOERROR

[Michael D. Sofka]

I am seeing occasional messages of the form:

Nov 16 13:01:36 imap-be4 sync_client[21977]: IOERROR: index record 0 for 
user. past end of file

There are two messages from around the time of the error message that 
were not synchronized.  But, these also appear to have been expunged by 
pop3.

Mike

-- 
Michael D. Sofka   sof...@rpi.edu
C&MT Sr. Systems Programmer,   Email, HPC, TeX, Epistemology
Rensselaer Polytechnic Institute, Troy, NY.  http://www.rpi.edu/~sofkam/

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Shuvam Misra]

> Didn't Dave write up.imapproxy?  It makes a huge difference for, e.g.,
> IMP & roundcube.  Also, configuring client to not retrieve the LIST of
> mailboxes during every transaction is a big win.

Thanks a lot -- will definitely incorporate it into our setup. How does
one configure the client not to retrieve the LIST of mailboxes? Can these
be set for, say, roundcube or SqMail?

Shuvam

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Fwd: Re: Does anyone allow unlimited or extremely large quotas?

[David Lang]

On Tue, 16 Nov 2010, Dave McMurtrie wrote:

> I didn't realize that I only responded to Rob here.  Perhaps my
> additional information will shed some light on the kind of information
> I'm looking for.
>
>  Original Message 
> Subject: Re: Does anyone allow unlimited or extremely large quotas?
> Date: Tue, 16 Nov 2010 07:06:53 -0500
> From: Dave McMurtrie 
> To: Rob Mueller 
>
> On 11/16/2010 06:45 AM, Rob Mueller wrote:
>>
>>> This may be slightly off-topic, so apologies in advance. Is there
>>> anyone out there who allows unlimited quota for their users or provides
>>> extremely large quotas when asked for?
>>
>> What do you consider extremely large? And what sort of problems are you
>> referring to?
>
> I don't actually know what sort of problems I'm referring to, hence the
> question.  The big problem I can imagine would be opendir() and
> readdir() with a huge number of files in a directory, but the cyrus code
> doesn't appear to do that in a lot of places that would matter to a user
> (deleting an entire folder, delete sieve scripts, etc) in the course of
> normal operations.

this depends on what filesystem you are useing, I have mailboxes with hundreds 
of thousands of messages in them on XFS and have no problems, but on ext3 I 
start seeing slowdowns with a bit over ten thousand messages.

>> The usual issue is just the huge number of emails and thus files that
>> accumulate. Creating a fresh replica, body searching, reconstructing,
>> etc all take quite a bit of time because of the large amount of random
>> IOs. Apart from that, everything does actually work ok...
>
> The only issue we ever had was with a bboard that our network group
> sends automated system messages to.  Something in their environment went
> haywire and we ended up with ~1.5 million messages in that bboard.  They
> were unable to find a client that was willing to deal with the folder to
> be able to clean it up.  I was able to connect using imtest and SELECT
> and FETCH messages without any problems, though.  I also recall that
> replication was broken by this folder, but I don't remember exactly why.

alpine and mulberry have no problem with huge numbers of messages.

David Lang

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[David Lang]

On Tue, 16 Nov 2010, Ciprian wrote:

> Adam Tauno Williams wrote:
>> I think the issue you will encounter first is clients will start to fall
>> down when folders exceed a 'reasonable' number of messages.  Common IMAP
>> clients I've seen start to exhibit severe performance issues beyond a
>> few hundred thousand messages.
>>
> Older versions of Outlook (e.g. Office 2003) will choke well before that
> (I think the limit was around 35.000 on XP SP2 ) also couple that with
> the local Outlook file store for that IMAP account which used to be
> limited to 2G. We generally advice our users to avoid going past 20.000
> messages in one folder.

the nice thing is that this is per-folder, and generally shows up as a slowdown 
as you get large, so the user can just create a subfolder and move messages in 
to it to work around the client issues.

This should mostly be self-regulating as a result.

David Lang

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: sieve not filing messages?

[Dan White]

On 16/11/10 10:33 -0600, Mike Eggleston wrote:
>Morning,
>
>I have a user saying that email messages are not being filed as desired. I 
>have copied the user's script into Sieve Test 
> and the user's sieve rules. 
>Sieve Test says the message should go to where is expected, but the message 
>isn't in the cyrus message store.
>
>The time stamps have changed in the folder where the message should be, so I 
>can't tell if the message was there and deleted or never filed there.
>
>The log entries of that day when the message was received from another 
>internal user has an odd entry:
>
>Nov  9 10:31:51 elo master[10189]: about to exec /usr/lib/cyrus-imapd/lmtpd
>Nov  9 10:31:51 elo lmtpunix[10189]: executed
>Nov  9 10:33:07 elo sendmail[10210]: oA9GX7ER010210: 
>from=<$us...@$domain.com>, size=975, class=0, nrcpts=1, 
>msgid=<005a01cb802b$ca410940$5ec31b...@com>, proto=ESMTP, daemon=MTA, 
>relay=ANAKIN.$domain.com [10.1.2.194]
>Nov  9 10:33:07 elo lmtpunix[10189]: accepted connection
>Nov  9 10:33:07 elo lmtpunix[10189]: lmtp connection preauth'd as postman
>Nov  9 10:33:07 elo lmtpunix[10189]: duplicate_check: 
><005a01cb802b$ca410940$5ec31b...@com>user.$user2.backup  0
>Nov  9 10:33:07 elo lmtpunix[10189]: seen_db: user $user2 opened 
>/var/lib/imap/user/u/$user2.seen
>Nov  9 10:33:08 elo lmtpunix[10189]: duplicate_mark: 
><005a01cb802b$ca410940$5ec31b...@com>user.$user2.backup  1289320387 852209
>Nov  9 10:33:08 elo lmtpunix[10189]: duplicate_check:  ^A| }^\Ö7Ã^W^_JÍ&cXC
> $user2  1289839869
>Nov  9 10:33:08 elo lmtpunix[10189]: duplicate_mark: 
><005a01cb802b$ca410940$5ec31b...@com>.$use...@.sieve.1289320388 0
>Nov  9 10:33:08 elo sendmail[10212]: oA9GX7ER010210: to=<$us...@$domain.com>, 
>delay=00:00:01, xdelay=00:00:01, mailer=cyrusv2, pri=120975, relay=localhost, 
>dsn=2.0.0, stat=Sent
>Nov  9 10:34:08 elo master[5702]: process 10189 exited, status 0
>
>The entry about the duplicate_check has some really wierd characters. Is this 
>normal?

Seems like there should be a:

   dupelim: eliminated duplicate message to...

or a

   Delivered: ...

line in your syslog. Maybe you have an invalidly formatted message that
sieve is throwing away, or may have triggered a bug.

The logs you have might indicate 8-bit headers in the email, which
shouldn't be valid, although Cyrus should work with them in some
circumstances.  See imapd.conf(5) for options munge8bit and reject8bit.

-- 
Dan White

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

ext3 / XFS [Was: Re: Does anyone allow unlimited or extremely large quotas?]

[Adam Tauno Williams]

On Tue, 2010-11-16 at 11:25 -0800, David Lang wrote: 
> > I don't actually know what sort of problems I'm referring to, hence the
> > question.  The big problem I can imagine would be opendir() and
> > readdir() with a huge number of files in a directory, but the cyrus code
> > doesn't appear to do that in a lot of places that would matter to a user
> > (deleting an entire folder, delete sieve scripts, etc) in the course of
> > normal operations.
> This is depends on what filesystem you are useing, I have mailboxes with 
> hundreds 
> of thousands of messages in them on XFS and have no problems, but on ext3 I 
> start seeing slowdowns with a bit over ten thousand messages.

Was dir_index enabled on that ext3 filesystem?  Prior to dir-index ext3
was very slow for large folders. dir_index is not enabled by default in
ext3.


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: ext3 / XFS [Was: Re: Does anyone allow unlimited or extremely large quotas?]

[David Lang]

On Tue, 16 Nov 2010, Adam Tauno Williams wrote:

> On Tue, 2010-11-16 at 11:25 -0800, David Lang wrote:
>>> I don't actually know what sort of problems I'm referring to, hence the
>>> question.  The big problem I can imagine would be opendir() and
>>> readdir() with a huge number of files in a directory, but the cyrus code
>>> doesn't appear to do that in a lot of places that would matter to a user
>>> (deleting an entire folder, delete sieve scripts, etc) in the course of
>>> normal operations.
>> This is depends on what filesystem you are useing, I have mailboxes with 
>> hundreds
>> of thousands of messages in them on XFS and have no problems, but on ext3 I
>> start seeing slowdowns with a bit over ten thousand messages.
>
> Was dir_index enabled on that ext3 filesystem?  Prior to dir-index ext3
> was very slow for large folders. dir_index is not enabled by default in
> ext3.

yes, even with dir-index I see slowdown on large, busy folders. not as bad as 
without them, but still there.

without dir-index a folder that at one time had 10K files in it becomes 
unusuably slow forever, with dir-index the slowdown isn't as bad, but it's 
still 
there.

remember that dir-index only helps for the case where you are looking for a 
single file, if you are walking the entire directory it has little, if any 
effect.

XFS is slow deleting large numbers of files as noted by others, but delayed 
expunge sidesteps that issue.

David Lang

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Bron Gondwana]

On Tue, Nov 16, 2010 at 08:38:49AM -0500, Adam Tauno Williams wrote:
> On Tue, 2010-11-16 at 14:12 +0100, Rudy Gevaert wrote: 
> > On 11/16/2010 12:30 PM, Dave McMurtrie wrote:
> > > Good morning,
> > > This may be slightly off-topic, so apologies in advance.  Is there
> > > anyone out there who allows unlimited quota for their users or provides
> > > extremely large quotas when asked for?
> > > If so, can you describe any problems you've had with this?
> > We have users with > 5 GB.
> 
> Our largest quota's a 4GB; without any issues.

Our biggest currently is about 30GB I think.

> I think the issue you will encounter first is clients will start to fall
> down when folders exceed a 'reasonable' number of messages.  Common IMAP
> clients I've seen start to exhibit severe performance issues beyond a
> few hundred thousand messages.

On a 32 bit architecture: we had one folder with over a million messages
which was causing processes to run out of virtual memory trying to map
the cache file in.  This wouldn't be a problem with a 64 bit userland.
 
> > We haven't seen any problems with them.  The only general problem we 
> > face is taking backups of a full store.  It just takes very long to 
> > complete full backups.  (Of course that isn't tied to large or no quota.)

Nup - full backup would be a problem.  We don't do them any more:
everything is context aware backups.

Bron.

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: ext3 / XFS [Was: Re: Does anyone allow unlimited or extremely large quotas?]

[Robert Mueller]

> > This is depends on what filesystem you are useing, I have mailboxes with 
> > hundreds 
> > of thousands of messages in them on XFS and have no problems, but on ext3 I 
> > start seeing slowdowns with a bit over ten thousand messages.
> 
> Was dir_index enabled on that ext3 filesystem?  Prior to dir-index ext3
> was very slow for large folders. dir_index is not enabled by default in
> ext3.

FYI our experience at Fastmail 2 years back was that reiserfs still much
better than ext3 (even with dir_index) at handling large numbers of
files in folders. We tried switching one server to ext3, but after a
week or two it was being crushed by load and we switched back to
reiserfs.

However we've recently found that ext4 is at least as good as reiserfs
at handling large directories, so we've started switching everything to
ext4 and so far the migration is going well.

So don't use ext3, but ext4 is ok.

Oh, and we recently setup a spare machine with btrfs and tried
replicating a few partitions to it. That wasn't good. Started off
promising, but by the time it was 1/3 full, the machine was utterly
crawling. Clearly not ready for production yet.

Rob

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Shuvam Misra]

> On 16.11.2010 17:11, Shuvam Misra wrote:
> > I guess Webmail is OT on a Cyrus mailing list, but can't help asking: any 
> > suggestions for
> > improving Webmail performance? (Admission: we haven't yet tried imapproxy
> > -- it appears to be a good piece of C which will help things.)
> 
> You should install imapproxy. Also make sure you do a lot of caching on
> the webmail side for example if the webmail is programmed with php use
> apc and cache mailbox listing and message headers if possible.
> 
> Cache also things on the imap server, if your cyrus version supports it
> you should try to use status cache and test whether it makes any
> difference in your environment.

Thanks a lot, Leena. :) You've actually tickled some suspicions we have
also had, that to deliver good Webmail service to our users, we may have
to figure out PHP acceleration.

I sometimes wish the world wasn't so desperately pushing the one-size-
fits-all paradigm of browser-based apps for each and every thing in
sight... all this problem with Webmail is primarily due to using a
connectionless run-time system for a connection-oriented service (IMAP).

Will look into PHP cacheing and other stuff.

Shuvam

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Shuvam Misra]

> On 16 Nov 2010, at 10:32, Joseph Brennan wrote:
> > I wish we'd somehow financed a native Cyrus webmail interface, that is
> > not using IMAP but built into Cyrus.  I don't think users know how good
> > Cyrus is because they look at it through a weak intermediary.
> 
> I don't think a Cyrus-specific web interface is the answer to that
> question.  IMP performance is not great, but it's the http paradigm that
> slows it.  Check out roundcube, utilizing AJAX it's way more responsive
> to the user.

Is this a general observation? We used to offer SquirrelMail with our
product, and we recently moved to RoundCube because we thought the
heavier use of Ajax would make the user experience more responsive. One
by one, all our more demanding clients have begun to ask for SqMail back.
They say (i) SqMail used to work on the browser of mobile phones but RC
doesn't, and (ii) RC is too slow, SqMail was just fine. This second point
has caught us by surprise and we are in the process of setting up SqMail
again for these customers. Personally we find the SqMail interface quite
dated.

We were toying with the idea of actually buying and supplying the Tuxedo
mail client to some of these customers to see if they'd find it better.

Haven't tried IMP.

Shuvam

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: ext3 / XFS

[Shuvam Misra]

> > This is depends on what filesystem you are useing, I have mailboxes with 
> > hundreds 
> > of thousands of messages in them on XFS and have no problems, but on ext3 I 
> > start seeing slowdowns with a bit over ten thousand messages.
> 
> Was dir_index enabled on that ext3 filesystem?  Prior to dir-index ext3
> was very slow for large folders. dir_index is not enabled by default in
> ext3.

Maybe it's better to move to ext4... we're going to do that with the next
release of our product too. This article:

http://www.linux-mag.com/id/7876

based on Ric Wheeler's now-well-known presentation at Linuxcon threw up
interesting comparative figures between ext3 and ext4 performance. Caught
us by surprise -- we didn't expect this kind of a performance jump.

Shuvam

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Does anyone allow unlimited or extremely large quotas?

[Pascal Gienger]

Am 16.11.10 19:08, schrieb Wesley Craig:
> On 16 Nov 2010, at 10:32, Joseph Brennan wrote:
>> I wish we'd somehow financed a native Cyrus webmail interface, that is
>> not using IMAP but built into Cyrus.  I don't think users know how good
>> Cyrus is because they look at it through a weak intermediary.
>
> I don't think a Cyrus-specific web interface is the answer to that question.  
> IMP performance is not great, but it's the http paradigm that slows it.  
> Check out roundcube, utilizing AJAX it's way more responsive to the user.


We started using SOGo here. It just loads a reasonable number of mail 
items for the index view and continues to load when you scroll down (or 
up) to get more.

As for the FS, we still use Sun aaah Oracle ZFS. Mailboxes with 500,000 
messages (postfix mailing list :-)  ) are just as SELECTable as empty 
mailboxes - no difference in speed or access time when retrieving 
messages from there.

The smell of Oracle still gets bitter and bitter compared to Sun, but 
especially this cookie (zfs) still tastes too well.

SOGo is slower (as it has another paradigma as Horde/IMP or Squirrel) 
but our users seems not to have a problem with it.


-- 
Pascal Gienger Jabber/XMPP/Mail: pascal.gien...@uni-konstanz.de
University of Konstanz, IT Services Department ("Rechenzentrum")
Electronic Communications and Web Services
Building V, Room V404, Phone +49 7531 88 5048, Fax +49 7531 88 3739

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

move default domain mailboxes

[Stephen Ingram]

After setting up murder, is it possible to move mailboxes that use the
default domain from one backend server to another? The new back-end
server has a different default domain. After which the default domain
on the original backend server will be changed.

before:

server 1 default domain: domainA.tld

mailbox for user st...@domaina.tld is user.steve

server 2 default domain: domainB.tld


next:

server 1 default domain: domainA.tld

move mailbox for user st...@domaina.tld to server 2

server 2 default domain: domainB.tld

mailbox becomes user.st...@domaina.tld


finally:

server 1 default domain: domainC.tld

server 2 default domain: domainB.tld

mailbox user st...@domaina.tld is still user.st...@domaina.tld

Steve

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: ext3 / XFS [Was: Re: Does anyone allow unlimited or extremely large quotas?]

[LALOT Dominique]

We use ext4 for more than one year now. Efficient and stable. A good choise
12 spool of 250GB over 10 FC disks using metalun.

Dom

2010/11/16 Robert Mueller 

>
> > > This is depends on what filesystem you are useing, I have mailboxes
> with hundreds
> > > of thousands of messages in them on XFS and have no problems, but on
> ext3 I
> > > start seeing slowdowns with a bit over ten thousand messages.
> >
> > Was dir_index enabled on that ext3 filesystem?  Prior to dir-index ext3
> > was very slow for large folders. dir_index is not enabled by default in
> > ext3.
>
> FYI our experience at Fastmail 2 years back was that reiserfs still much
> better than ext3 (even with dir_index) at handling large numbers of
> files in folders. We tried switching one server to ext3, but after a
> week or two it was being crushed by load and we switched back to
> reiserfs.
>
> However we've recently found that ext4 is at least as good as reiserfs
> at handling large directories, so we've started switching everything to
> ext4 and so far the migration is going well.
>
> So don't use ext3, but ext4 is ok.
>
> Oh, and we recently setup a spare machine with btrfs and tried
> replicating a few partitions to it. That wasn't good. Started off
> promising, but by the time it was 1/3 full, the machine was utterly
> crawling. Clearly not ready for production yet.
>
> Rob
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>



-- 
Dominique LALOT
Ingénieur Systèmes et Réseaux
http://annuaire.univmed.fr/showuser.php?uid=lalot

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/