Re: cyrus perdtion , connections not dying

[Robert Banz]

Why are you using perdition, when you could be using Cyrus'  "murder"  
clustering which will work to the same end, with added bonuses of  
being able to share folders between users on both servers, easier  
administration, etc.

-rob


On Apr 4, 2008, at 9:51 AM, Andrew Morgan wrote:

> On Fri, 4 Apr 2008, ram wrote:
>
>> I have a 40k user cyrus server , and I am trying to  scale up by  
>> trying
>> to add a perdition server infront of this server so that we can add  
>> more
>> servers ( Both servers dell 2950 with 8GB RAM )
>>
>>
>> But the problem I have the moment I install a perdition server is  
>> that ,
>> the number of pop connections do not seem to reduce at all once  
>> started.
>> I had 10 connections in morning and with 15 minutes the number of
>> connections went to 3000. And on the real server the connections  
>> are all
>> full
>>
>> Usually we have around 700k simultaneous pop connections  on the real
>> servers now with perdition we have 3000+ connections
>>
>>
>>
>> How do I get perdition close connections quickly
>
> Doesn't Perdition hold the backend connection open so that it can be
> reused the next time the same client connects?
>
>   Andy
> 
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Cyrus infrastructure performance less than expected

[Robert Banz]

>
> I don't have any system log that complains about something and the  
> only
> Cyrus message I got is a DB4 warning about lockers. I think the DB4 in
> question is the TLS sessions cache DB and in my case the number of
> lockers can be as high as 8 000...

Dump Berkeley DB with a quickness, and switch to skiplist.

That said, there's something to be said about not putting all of your  
eggs onto one server -- virtualized or not.

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: choosing a file system

[Robert Banz]

On Dec 30, 2008, at 8:49 AM, LALOT Dominique wrote:

> Hello,
>
> We are using cyrus-imap for a long time. Our architecture is a SAN  
> from EMC and thanks to our "DELL support" we are obliged to install  
> redhat. The only option we have is to use ext3fs on rather old  
> kernels. We have 4000 accounts for staff and 2 for students
> The system is rather fast and reliable. BUT..
>
> Once, there was a bad shutdown corrupting ext3fs and we spent 6  
> hours on an fsck.
> Next we discovered that our backup system was going slower and  
> slower. We just pointed out that it was due to fragmentation, and  
> guess what, there's no online defrag tool for ext3.
>
> I'm looking for other solutions:
> ext4fs (does somebody use such filesystem?), xfs
> zfs (but we should switch to solaris or freebsd and throw away our  
> costly SAN)
> use a NetApp Appliance (are you using such a device?, NFS seems to  
> be tricky with cyrus..)

Run Solaris, but keep a machine on the SAN with that old version of  
RedHat that you can use to replicate any problems you have? ;)

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: choosing a file system

[Robert Banz]

On Dec 30, 2008, at 9:06 AM, Pascal Gienger wrote:

> LALOT Dominique  wrote:
>
>> zfs (but we should switch to solaris or freebsd and throw away our  
>> costly
>> SAN)
>
> Why that? SAN volumes are running very fine with Solaris 10 hosts  
> (SPARC
> and x86). You have extended multipathing (symmetric and asymmetric)  
> onboard.
> Solaris accepts nearly all Q-Logic FC cards  (according to my  
> experience).


At my last job, we had explored a Dell/EMC SAN at one point. Those  
folks don't seem to understand the idea that Fibre Channel is a well  
established standard -- they only expect you to connect their  
supported stack of hardware and software, otherwise they don't wanna  
talk.

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: choosing a file system

[Robert Banz]

On Jan 8, 2009, at 4:46 PM, Bron Gondwana wrote:

> On Thu, Jan 08, 2009 at 08:01:04AM -0800, Vincent Fox wrote:
>> (Summary of filesystem discussion)
>>
>> You left out ZFS.
>>
>> Sometimes Linux admins remind me of Windows admins.
>>
>> I have adminned a half-dozen UNIX variants professionally but
>> keep running into admins who only do ONE and for whom every
>> problem is solved with "how can I do this with one OS only?"
>
> We run one zfs machine.  I've seen it report issues on a scrub
> only to not have them on the second scrub.  While it looks shiny
> and great, it's also relatively new.

You'd be surprised how unreliable disks and the transport between the  
disk and host can be. This isn't a ZFS problem, but a statistical  
certainty as we're pushing a large amount of bits down the wire.

You can, with a large enough corpus, have on-disk data corruption, or  
data corruption that appeared en-flight to the disk, or in the  
controller, that your standard disk CRCs can't correct for. As we keep  
pushing the limits, data integrity checking at the filesystem layer --  
before the information is presented for your application to consume --  
has basically become a requirement.

BTW, the reason that the first scrub saw the error, and the second  
scrub didn't, is that the first scrub fixed it -- that's the job of a  
ZFS scrub.

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: choosing a file system

[Robert Banz]

>
> There's a significant upfront cost to learning a whole new system
> for one killer feature, especially if it comes along with signifiant
> regressions in lots of other features (like a non-sucky userland
> out of the box).

...

The "non-sucky" userland comment is simply a matter of preference, and  
bait for a religious war, which I'm not going to bite.

What I will say is that switching between Solaris, Linux, IRIX,  
Ultrix, FreeBSD, HP-UX, OSF/1 -- any *nix variant, should not be  
considered a stumbling block. Your comment shows the narrow-mindedness  
of the current Linux culture, many of us were brought up supporting  
and using a collection of these platforms at any one time.

(notice, didn't mention AIX. I've got my standards ;)

Patching is always an issue on any OS, and you do have the choice of  
running X applications remotely (booting an entire graphic  
environment!?), and many other tools available such as pca to help you  
patch on Solaris, which provide many of the features that you're used  
to.

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Implement Cyrus IMAPD in High Load Enviromment

[Robert Banz]

With my ever-growing experience with these things, I'm tending to  
think that application-level HA solutions are a much more robust way  
of dealing with the potential failure modes of hardware or software.

While this doesn't mean you shouldn't buy reasonably robust hardware  
(not the cheapest thing that falls off the truck), what it does is  
mean is that you should probably spend that extra money you would have  
spent on that redundant storage system and hot-plug-gable whatsit with  
replicating your server environment and allowing application-level  
replication such as Cyrus offers to do the rest.

This does mean that we need to lean on software "vendors", be it  
people we pay or ourselves to create applications that provide for  
application-layer high availability. However, in the long run it IS a  
much more robust solution, and covers a much larger number of failure  
domains for the time and money spent than throwing money and effort at  
covering the handful of failure-domains that comprise a hardware  
system -- since that really is an infinite money pit.

These "quality" enterprise software solutions that we're sold today  
that rely on "fault tolerant" hardware, instead of being "fault  
tolerant" themselves are not acceptable.

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Cyrus Aggregator

[Robert Banz]

Wow, did I miss something? It's not called "murder" anymore?



Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: OT: Enhanced IMAP protocol

[Robert Banz]

An extension or protocol enhancement is only good as the client  
implementations are -- and we know how successful that's been for  
other optional capabilities -- such as ACL management.

On Jan 14, 2010, at 9:42 AM, kael wrote:

> On 01/06/2010 08:47 AM, Rob Banz wrote:
>> I would argue that it's out of scope -- credential management should
>> be taken care of by your credential management system, be it  
>> through a
>> web interface or whatever. Even if it were to be an accepted spec,  
>> the
>> chances of all of the client-writers implementing it, and in a
>> reasonable way, are slim to none.
>
> Well, IMAP could support a credential management system as an  
> extension
> similar to the XMPP one : XEP-0077: In-Band Registration
> .
>
> -- 
> kael
>
> 
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Nginx configuration for imap

[Robert Banz]

memcached would certainly be fast, but what sort of authentication rate are
you talking about here. My bet is that you've got other bits of system, such
as the authentication validation with the target IMAP server, that will be
more of a dominant term when it comes to the performance of your system.

I deployed an nginix proxy to assist in my migration to Cyrus (once all my
users were on Cyrus, murder took over) -- just had it do lookups against our
LDAP directory to determine which IMAP provider to redirect to, it worked
perfectly fine.

-rob

On Wed, Mar 17, 2010 at 10:35 PM, Robert Mueller  wrote:

>
> > But I thought a memcache lookup will be much more inexpenisve than
> > connecting to a mysql db  to do lookup for every cyrus connection
>
> Probably slightly. But what happens if the value isn't in memcached?
> Where do you get the value from?
>
> Anyway, it's still WAY better than doing:
>
> > > > $user['user1'] = 10.1.1.1;
> > > > $user['user2'] = 10.1.1.2;
> > > > 
> > > > $user[user15000]=10.1.1.1;
>
> For every lookup.
>
> Rob
> 
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Nginx configuration for imap

[Robert Banz]

On Thu, Mar 18, 2010 at 3:49 AM, ram  wrote:

>
> On Wed, 2010-03-17 at 22:45 -0700, Robert Banz wrote:
> >
> >
> > memcached would certainly be fast, but what sort of authentication
> > rate are you talking about here. My bet is that you've got other bits
> > of system, such as the authentication validation with the target IMAP
> > server, that will be more of a dominant term when it comes to the
> > performance of your system.
> >
> >
> > I deployed an nginix proxy to assist in my migration to Cyrus (once
> > all my users were on Cyrus, murder took over) -- just had it do
> > lookups against our LDAP directory to determine which IMAP provider to
> > redirect to, it worked perfectly fine.
> >
> >
> > -rob
> >
> I dont think I am going to be able to connect to LDAP for every
> connection. I have atleast 3000 pop and 1000 concurrent imap connections
> on the server.
>

Um, I think you're confusing concurrent connections with connection rate.
You'll only be talking to LDAP for a quick query during session
initiation... Your connection numbers aren't really that big ;)


>
>
>
> One more issue with nginx 0.8.34  is that when auth-fails on the real
> server the nginx returns "BAD: internal server error"
> The email clients are not able to interpret this error.
>
> Can I configure nginx to pass on the actual message from my cyrus server
> "NO LOGIN failed"
>
> Thanks
> ram
>
>
>
>
>
>
>
>
> > On Wed, Mar 17, 2010 at 10:35 PM, Robert Mueller 
> > wrote:
> >
> > > But I thought a memcache lookup will be much more
> > inexpenisve than
> > > connecting to a mysql db  to do lookup for every cyrus
> > connection
> >
> >
> > Probably slightly. But what happens if the value isn't in
> > memcached?
> > Where do you get the value from?
> >
> > Anyway, it's still WAY better than doing:
> >
> > > > > $user['user1'] = 10.1.1.1;
> > > > > $user['user2'] = 10.1.1.2;
> > > > > 
> > > > > $user[user15000]=10.1.1.1;
> >
> >
> > For every lookup.
> >
> > Rob
> >
> > 
> > Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> > List Archives/Info:
> > http://asg.web.cmu.edu/cyrus/mailing-list.html
> >
> >
> >
>
>

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: tool for migration to cyrus ?

[Robert Banz]

I found the "mailutil" program, which is distributed in the UW imapd  
distribution, particularly well suited for this.  It allows you to  
transfer the contents of one c-client supported mail hierarchy to  
another, such as a local mail store to a cyrus-hosted IMAP space.   
The 'transfer' function will probably do what you're looking for.


-rob


On Jul 17, 2006, at 8:37 AM, RJ45 wrote:



Hello,
I Want to migrate mailbox from imapuw to a new server with cyrus  
2.3 installed.

Is there any tool or scritp avaliable which I an customize
to achieve my goal of migrate imap-uw mbox to cyrus mailboxes ?
thanks a lot

rj

Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html



Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Cyrus::SIEVE::managesieve documentation?

[Robert Banz]

On Jul 19, 2006, at 11:23, Chris St. Pierre wrote:


Ditto here.  I tried emailing the guy listed in the
Cyrus::SIEVE::managesieve man page, but the address no longer appears
to be active.  As of now, my options appear to be:

1.  Learn Ruby, and write the first (and probably last) Ruby script at
my site; or

2.  Guess with Perl.

Neither is a great alternative. :)


Guess?  You have the source, and that's the best documentation of all.

*ducks*

-rob

Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: hardware recommendations for MURDER?

[Robert Banz]

My personal leaning is towards Sun hardware with RHEL4 but I wanted  
to get
some fresh opinions. Thought this topic worth a rehash since 2004  
data is

useful but not current enough IMO.


(sun just announce a 3u dual proc 16G ram box with
24TB
of disk space for ~$70k for example)


I admit a fear of multi-terabyte filesystems and their long check  
times
when things go south.  Perhaps there is some configuration of this  
that is

safe?


Just curious, and not to start any religious wars, but if you're  
going to go so far as buying the Sun hardware (which is quite good),  
what's keeping you from running Solaris 10 x86?  We've actually been  
migrating most of our Linux services to Solaris for it's much more  
mature storage management capabilities, and from an enterprise  
administration point of view, more stable operating environment.


We haven't started our testing with Cyrus, but we're optimistic that  
Sun's ZFS, especially with it's on-disk compression, will for a great  
Cyrus mail storage backend.


-rob

Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Performance and cheap storage

[Robert Banz]

I'm not a postfix expert, however, if it's like any other MTA it's  
"queue" directory is usually what runs pretty hot.  The first thing I  
would do is isolate the MTA-related stuff to a very fast piece disk  
that is *not* the same storage that houses you Cyrus mailboxes &  
databases.


The second thing to consider is that the performance on modern SATA  
drives, if you're using a driver for the SATA interface that supports  
advanced features such as command queueing, are going to show you  
performance akin to SCSI drives -- even more so if you place them  
behind a 'quality' RAID subsystem that provides read/write caching.


But, if you're really getting your system to buckle under I/O load,  
the first thing you should consider doing is split out the  
functions.  The first thing I would do is migrate the MTA (postfix)  
duties to some other systems (plural, because you should never have  
just one of anything) and deliver to your Cyrus install via LMTP over  
TCP.  From there, you'll be able to scale each to those subsystems  
appropriately, without worry that one's I/O load profile is directly  
impacting the other.


-rob

Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Idea for filtered access to cyrus

[Robert Banz]

Cyrus gets this and slices off the +filter= and places the value "foo"
into a FILTER variable.

On the mail delivery side:  LMTP is changed to look for X-IMAP- 
FILTER headers

and to store the value of the header as an IMAP flag.  Assuming
X-IMAP-Filter: foo
then we add /filter=foo to the IMAP flags.  Do so for each X-IMAP- 
FILTERheader found.


Could this be more generically implemented by creating a sieve  
extension to manipulate IMAP flags when storing a message to a folder?


-rob

Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Cyrus, Solaris 10, ZFS? (and NIS?)

[Robert Banz]

On Oct 5, 2006, at 12:40, Elizabeth Schwartz wrote:

Is anyone happily running all of the above? All of the above except  
NIS? Any tuning hints?


I'm running Solaris 10 (06/06), cyrus 2.3.7 (Blastwave build) ,  
sendmail 8.13.8 (ditto), mailspool on a zfs filesystem, authenticating
via NIS. I've already solved one problem with VERY slow sendmail  
response, turned out to be a Solaris NIS bug  ( patch 123186-01)


Run procsystime (from the dtrace toolkit) on the cyrus imap  
processes.  I'm going to bet that they do a lot of fdsync's.


There's a bug in ZFS regarding performance problems when fsync'ing  
file descriptors -- there's apparently going to be a patch coming  
"real soon now" -- your options are:


1) Move your mail spool off of ZFS.
2) Remove all of the fsync() calls from cyrus.  (this may mean  
removing them from berkely db ;) )


-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Cyrus, Solaris 10, ZFS? (and NIS?)

[Robert Banz]

On Oct 5, 2006, at 13:59, Elizabeth Schwartz wrote:


>There's a bug in ZFS regarding performance problems when fsync'ing
>file descriptors -- there's apparently going to be a patch coming
>"real soon now" -- your options are:

Thanks!

Ugh, that would be bad news. Except, I think the delay is happening  
earlier. The delay is between the A001 login and A001 ok, and the  
only zfs file system is the mail spool (I should have spelled that  
out)


I've been playing with the dtrace toolkit this afternoon - still a  
bit stuck on how to get from my tcp port connection to the process  
number to run dtrace dtruss (lsof *should* be the right tool but  
I'm missing a step somewhere)


"topsyscall" is also really a good way to see what fun things are  
going on with the machine.  Damn I'm in love with dtrace ;)


Once you find out what your PID is after you connect, connect  
'dapptrace' to it.  It'll give you a run down of all of the function  
calls* that are going on (as they're going on), so you'll really get  
an idea of what's going on.


*yes, function calls.  Not just syscalls.   Don't ask how it works --  
it's magic.


-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Cyrus, Solaris 10, ZFS? (and NIS?)

[Robert Banz]

On Oct 5, 2006, at 4:46 PM, Chaskiel M Grundman wrote:




--On Thursday, October 05, 2006 04:13:18 PM -0400 Elizabeth  
Schwartz <[EMAIL PROTECTED]> wrote:



http://www.gsd.harvard.edu/users/betsys/dapptrace.timed


The interesting bit seems to be here:
 .  . -> mynewstate(0x165769, 0x40404040, 0x0)
 .  . -> mycanonifyid(0x165769, 0x0, 0x0)
 .  . -> libcyrus_config_getswitch(0x2, 0x11AF55, 0x5)
43  5 <- libcyrus_config_getswitch = 84
   135 20 <- mycanonifyid = 292
 .  . -> xmalloc(0x5C, 0x11AF5D, 0x5)
38 14 <- xmalloc = 28
 .  . -> libcyrus_config_getswitch(0x1, 0x0, 0x1647CB)
40  3 <- libcyrus_config_getswitch = 84
 .  . -> xrealloc(0x0, 0x4, 0xE488)
68 14 <- xrealloc = 64
 .  . -> xstrdup(0x16871C, 0x4, 0xE488)
 .  . -> xmalloc(0x9, 0x2A0031, 0x168724)
30  7 <- xmalloc = 28
94 16 <- xstrdup = 40
8235260 109820 <- mynewstate = 356

mynewstate is taking 8s to run, and very little of the time is  
taken up in local subroutines.
auth_unix.c:mynewstate calls getpwnam, and then iterates over all  
the groups using getgrent(),
checking to see what groups the user is in. The fact that imapd  
does this twice might be a bug, but even if it didn't do it twice,  
it would still be slow.


Is running "getent group" slow?


Oy yes.

The only "way" to find out what groups a user is in, of course, is to  
interate over the groups file (or map) and look at the whole list of  
users assigned to a group.  Ugly.


Now, if you can't think of any reasons you'd actually care about  
someone's group membership, it wouldn't be out of the question to  
remove said junk out of the auth_state function in auth_unix.c.   
Sendmail contains a nice option to turn off initgroups() like  
functionality, perhaps Cyrus could use one as well?


-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Cyrus, Solaris 10, ZFS? (and NIS?)

[Robert Banz]

On Oct 5, 2006, at 10:05 PM, Kjetil Torgrim Homme wrote:


On Thu, 2006-10-05 at 16:46 -0400, Chaskiel M Grundman wrote:
mynewstate is taking 8s to run, and very little of the time is  
taken up in

local subroutines.
auth_unix.c:mynewstate calls getpwnam, and then iterates over all the
groups using getgrent(),
checking to see what groups the user is in. The fact that imapd  
does this
twice might be a bug, but even if it didn't do it twice, it would  
still be

slow.


to speed up initgroups, make sure you have the netid.byname NIS  
map.  if

you're not familiar with it, the keys should be "[EMAIL PROTECTED]",
the values are "UID:GID,GID,GID,...".  an example for my account,  
which

is uid 1232 in domain "ifi":

key: "[EMAIL PROTECTED]"
value: "1232:0,6,15,7411,11232"

presto, no iteration through the complete group map needed!


Unfortunately, in the cyrus implementation (they don't call the  
"real" initgroups() specifically), they're iterating through the  
group map entry by entry...  The (somewhat mysterious) netid map  
isn't going to help :(


-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Cyrus, Solaris 10, ZFS? (and NIS?)

[Robert Banz]

On Oct 5, 2006, at 10:50 PM, Elizabeth Schwartz wrote:


On 10/5/06, Igor Brezac <[EMAIL PROTECTED]> wrote:
Already done.  man imapd.conf  :)

unix_group_enable: 0


Cool :)  I was looking at an older cyrus distribution that doesn't  
seem to have it...


-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: very slow syncing, any ideas?

[Robert Banz]

On Oct 20, 2006, at 04:09, Michael Menge wrote:


Hi,

i had much better performance with mailutil from UW-Imapd. It uses  
the IMAP-protocol like imapscyn but is not a scipt but a binary  
program and uses the imap APPEND command and does noe checks to see  
wich E-Mails are on the new server.


Michael Menge


I second the usefulness of mailutil.  What also makes it neat is that  
it can read (natively) any c-client supported mail spool -- be it  
imap, local filesystem mail folders, etc.  So, if you've got backend  
mail store that has a c-client driver, you can pull the data directly  
from the filesystem and bypass IMAP on the reading end altogether.


-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: ZFS fsync performance and cyrus (was Patches used at FastMail.FM)

[Robert Banz]

On Jan 8, 2007, at 21:08, Rob Mueller wrote:


We are using 2.3.7 on Debian Sarge.
We will maybe move to solaris because of the features of ZFS.


Does anyone know the status of ZFS + fsync performance problems?

http://www.irbs.net/internet/info-cyrus/0610/0058.html


Supposedly fixed in Solaris 10 11/06...

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Connection throttling POP3.

[Robert Banz]

On May 21, 2007, at 21:50, Daniel O'Connor wrote:


On Tuesday 22 May 2007 05:10, Matthew Schumacher wrote:

I'm getting some spammer trying to guess usernames and passwords:


I use the following to protect my SSH server (well not the SSH server
per se, just me reading logfiles the next day)

http://www.gsoft.com.au/~doconnor/brute-force-mitigation.html

Needs PF though.


I take the approach of having a perl script (yay! File::Tail) sit and  
watch the logs on each server looking for signs of ssh (could easily  
be used for other things like pop as well) brute force attacks.  A  
certain # of failed logins in a time window from a single IP will  
cause that IP to get blocked by ipfilter for an appropriate period of  
time, after which the block is removed.  This stops most of your  
brute-force guessers; after a few tries of having their packets end  
up on the floor, they go away.


-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Connection throttling POP3.

[Robert Banz]

On May 22, 2007, at 10:34, Philip H. O'Neill wrote:


We do the same but there is an issues.

One File::Tail delays polling the log for up to 30 seconds unless you
tell it otherwise. So it will allow a number of attempts before  
reading

the log. If you increase the polling you add load to the system. Not
much but some.

We like the idea of adding the timer to iptables along with logging so
the address can be tracked. If the address comes back then it can be
added to a permanent block.


We're not running this on linux (no iptables) but using Solaris'  
ipfilter.  The timer function seems nice; we just have the daemon  
keep a database of the 'bad' ips and release the block whenever one  
times out.


It's not, by any means, the "perfect" solution* -- there is no such  
thing.  However, it's quick, easy, and stops 99% of your problems.


*security people seem to obsess on "perfect" solutions.  It bothers me.

-rob


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Cyrus & ZFS performance

[Robert Banz]

On Jul 4, 2007, at 12:59 PM, Vincent Fox wrote:

> Dale Ghent wrote:
>> each with a zpool comprising of a mirror between two se3511s on our
>> SAN...
> Sun recommends against the 3511 in most literature I read, saying that
> the SATA drives
> are slower and not going to handle as much IOPS loading.  But they are
> working out okay
> for you?  Perhaps it's just vendor upsell to the more expensive  
> 3510FC

Of course they don't recommend it -- the margins on FC/SCSI/SAS  
drives are what their commission is made of ;)

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

xfer problem - unified murder

[Robert Banz]

Hey all,

I'm currently testing out a two-server + mupdate master unified  
murder cluster.

I was testing the mailbox xfer functionality -- I created a mail box  
(this one empty), and tried to move it between servers and got this:

Jul 24 16:09:57 ms4.mail.umbc.edu imap[15165]: [ID 813612  
local6.debug] wanting to dump /ms4/meta/B/user/joe/cyrus.header
Jul 24 16:09:57 ms4.mail.umbc.edu imap[15165]: [ID 691129  
local6.error] Sync Error: expected '+' got 'D'

What the heck does that mean?

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: rename bunch of folders [auf Viren überpr üft]

[Robert Banz]

On Aug 1, 2007, at 09:19, Hans Moser wrote:

> Hans Moser schrieb:
>
>> If I do this outside IMAPd (i.e. by shell's mv command), I have to  
>> run
>> reconstruct to repair mailboxes.db, right?
> First I remove the Spam dir in the file system.
> # rm -rf /var/imap/users/foo/Spam
>
> When I dump mailbox.db
> # /opt/mail/cyrus/ctl_mboxlist -d > file
> , edit dump file and delete the Spam folder row, restore mailbox.db
> # /opt/mail/cyrus/ctl_mboxlist -u < file
> and dump again
> # /opt/mail/cyrus/ctl_mboxlist -d
> , the row is still in.
>
> How do I get rid of the Spam folder?

If you're re-importing the entire mailboxes.db, you have to delete  
the database file itself and let it re-create it.

-u "updates" it, but doesn't remove things.

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Odd mailbox xfer result

[Robert Banz]

Hi,

I've got a 'traditional' Cyrus cluster, 2.3.8 running on Solaris  
x86.  Everything seems to be fine, and we're about to go "live" and  
in production... however...

Earlier today I was testing how well transferring my rather massive  
personal folder hierarchy from one backend server to another -- which  
sent apparently without a hitch.  Quite fast, I might add.  However,  
after the move I noticed my sieve rules weren't working -- and when I  
connected up with 'sieveshell' to have a look-see at my script, I was  
surprised to find my sieve script turned out to be an actual mail  
message!?

So, anyone seen anything like this before?  I did some looking around  
on bugzilla, and nothing jumped out at me.

-rob


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Odd mailbox xfer result

[Robert Banz]

>>
>> So, anyone seen anything like this before?  I did some looking around
>> on bugzilla, and nothing jumped out at me.
>
> Look for bug 2917 `xfer copies the last message instead of sieve
> scripts to the remote server'.  The patch is there too.

Wow, how did I miss that one?

Thanks!

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: SQUAT config

[Robert Banz]

>
> Last question: i'm really not sure i understand what the benefit of
> having a SQUAT index is (aside from avoiding seeing "SQUAT failed to
> open index file" in the log). I mean, i get that it makes searching
> faster. But does this have to do with, let's say, clicking on a  
> mailbox
> folder in a GUI client (eg. thunderbird)? Does that cause it to  
> make an
> IMAP SEARCH call? Or is this something else, entirely?

Do a full-body-text search of a 10k message folder and have it return  
instantly.  That's the benefit of the squat index file ;)

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: SQUAT config

[Robert Banz]

On Aug 14, 2007, at 9:32 PM, brian wrote:

> brian wrote:
>> I'd like to index my mailboxes using squatter and have a couple of
>> questions.
>>
>
> ~groan~ I had one more question. The squatter man page says:
>
> -s Skip mailboxes whose index file is older than their current  
> squat
> file (within a small time delta).
>
> The squat file isn't the index file? Confused.

Its a different index file.  A more comprehensive one.  The basic  
cyrus index file contains some header information -- enough for most  
mail clients to do what they need to do.  The squat index file  
contains a lot more.  My experience has found the squat files end up  
being about 1/10th the size of your total amount of mailbox data...

-rob


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: SQUAT config

[Robert Banz]

>>
>
> Yes, i see, that's to invoke it. But i'm also reading that the config
> must be in imapd.conf yet i've seen no example for the syntax to do so
> (nor what is required (or even suggested)).
>
> I'm also wondering how to have squatter run against all IMAP  
> mailboxes.
> Do i need to specify them all when i invoke it? Or can i get away with
> feeding it "/var/spool/imap" with the -r switch and hoping it will
> figure it out?

It pretty much does the right thing out of the box.

The only configuration we've done is use the "metapartition" option  
to place the squat data on a separate filesystem than the mail spool  
databases & files.   This way we're not snapshotting them or backing  
them up, since they're easily recreated.

While their size is only a fraction of a folders', they get  
completely regenerated any time a folder gets updated -- so they  
could end up making a significant impact on your daily incremental  
backups.

-rob 

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: SSL/TLS certificates with virtual domains

[Robert Banz]

>
> How much configuration similarity does there have to be between the
> different config files?  Can I change anything except for the
> tls_[*]_file directives?
>
> Thanks very much for the information!  I think this could work for us.

Make one master imapd.conf file with everything but the certificate  
definitions (or any other domain-specific settings), then @include it  
from the "stub" imapd.confs that you point your cyrus components to.   
Less to worry about ;)

-rob


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

sieve and date

[Robert Banz]

Following on to the sieve & vacation discussion -- anyone have any  
interest in the sieve "date" extension?  Sounds like it'd be awfully  
nice to have, for example, being able to wrap your vacation in a date- 
range conditional...

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: sieve and date

[Robert Banz]

Here's the extension I'm referring to:

http://tools.ietf.org/wg/sieve/draft-freed-sieve-date-index-06.txt

On Aug 29, 2007, at 09:38, Jan Schneider wrote:

> Zitat von Robert Banz <[EMAIL PROTECTED]>:
>
>>
>>
>> Following on to the sieve & vacation discussion -- anyone have any
>> interest in the sieve "date" extension?  Sounds like it'd be awfully
>> nice to have, for example, being able to wrap your vacation in a  
>> date-
>> range conditional...
>
> Definitely. We currently hack around this in Ingo with some wicked
> Received: header regexp that I really feel uncomfortable with. A date
> extension would be of great help.
>
> Jan.
>
> 
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

squatter problem

[Robert Banz]

Hidey ho everybody,

So, one of my backend mailservers in my cyrus cluster is doing  
something silly:

Sep  5 02:00:00 ms1.mail.umbc.edu master[29759]: [ID 392559  
local6.debug] about to exec /local/cyrus/bin/squatter
Sep  5 02:00:00 ms1.mail.umbc.edu squatter[29759]: [ID 621814  
local6.notice] indexing mailboxes
Sep  5 02:00:00 ms1.mail.umbc.edu squatter[29759]: [ID 454541  
local6.debug] skipping mailbox shared/test
Sep  5 02:00:00 ms1.mail.umbc.edu squatter[29759]: [ID 454541  
local6.debug] skipping mailbox user/a11
...
Sep  5 02:00:00 ms1.mail.umbc.edu squatter[29759]: [ID 454541  
local6.debug] skipping mailbox user/a28/Spam
Sep  5 02:00:00 ms1.mail.umbc.edu squatter[29759]: [ID 454541  
local6.debug] skipping mailbox user/a28/Trash
Sep  5 02:00:00 ms1.mail.umbc.edu squatter[29759]: [ID 811920  
local6.notice] done indexing mailboxes

Squatter seems to think its work is done rather quickly when running  
under the master's control, but if I run the same command line ( / 
local/cyrus/bin/squatter -s ) from the command line, it happily chugs  
along, indexing and indexing.  I'm not seeing this behavior on any of  
my other backend servers...  Its obviously not dying of some horrible  
segfault or bus error, since it gets to log the 'done indexing  
mailboxes' message -- but what could possibly give it the idea that  
its done?

-rob


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: squatter problem

[Robert Banz]

On Sep 5, 2007, at 14:49, Michael D. Sofka wrote:

> On Wednesday 05 September 2007 09:44:16 am Robert Banz wrote:
>
>> Sep  5 02:00:00 ms1.mail.umbc.edu squatter[29759]: [ID 454541
>> local6.debug] skipping mailbox user/a28/Spam
>> Sep  5 02:00:00 ms1.mail.umbc.edu squatter[29759]: [ID 454541
>> local6.debug] skipping mailbox user/a28/Trash
>> Sep  5 02:00:00 ms1.mail.umbc.edu squatter[29759]: [ID 811920
>> local6.notice] done indexing mailboxes
>>
>> Squatter seems to think its work is done rather quickly when running
>> under the master's control, but if I run the same command line ( /
>> local/cyrus/bin/squatter -s ) from the command line, it happily chugs
>> along, indexing and indexing.
>
> I've been seeing the same behavior on both our back-end servers.
> We're running Cyrus 2.2.12.

I'm up on 2.3.8 with a few of the fastmail.fm patches -- sorry,  
forgot to include that in the mix.

-rob


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: squatter problem

[Robert Banz]

>
> Are you running the command line squatter as root?  If so, then maybe
> there's a file early in the squatter run that's root owned and causing
> squatter to abort when run as the cyrus user - I'm afriad I don't know
> much about squatter, but that would be my first suspicion.

> Another theory - do you have a custom imapd.conf for the master
> process?  If so, the hand run squatter may be reading a different
> config file from the one run by master.
>
> Finally - have you tried changing the squatter command that master
> runs to do an strace dump to a file somewhere so you can see what
> it was trying?

I'm a good boy and, of course, don't run anything with cyrus under  
anyone other than cyrus ;)  And no funky multiple imapd.confs.

However, here's what I think may be going on...  I've got my  
cyr_expire process *and* squatter running at the same time.  I  
decided while I was sitting around this afternoon, to change sqatter  
start time to run, well, right then.  Squatter started running and  
seems to have not had a problem.

So, this begs my question:  Could there be an interaction between  
squatter and cyr_expire, where if they hit the same mailbox at the  
same time, could cause one (squatter) to decide that it doesn't have  
any more work to do?

-rob


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Murder works wonderfully but alarms users

[Robert Banz]

Its much more productive to eliminate the complaining users and  
replace them with something less alarmist.

On Sep 10, 2007, at 11:59, Gary Mills wrote:

> We have a Cyrus murder configuration with one proxy front-end and
> one storage back-end.  I'm very pleased with it.  However, users who
> happen to look at the full headers of their e-mail are often alarmed
> by the word `murder' that appears in the first `Received' header.
> It's even worse when the message is from daemon!
>
> Here are some typical headers.  Would it be possible to eliminate
> the offensive word or replace it with something more meaningful?
>
>   Received: from murder (electra.cc.umanitoba.ca [130.179.16.23])
>by mbox.cc.umanitoba.ca (Cyrus v2.3.8) with LMTPA;
>Sun, 09 Sep 2007 22:14:47 -0500
>   X-Sieve: CMU Sieve 2.3
>   Received: from electra.cc.umanitoba.ca ([unix socket])
>by cc.umanitoba.ca (Cyrus v2.3.8) with LMTPA;
>Sun, 09 Sep 2007 22:14:47 -0500
>   X-Sieve: CMU Sieve 2.3
>
> -- 
> -Gary Mills--Unix Support--U of M Academic Computing and  
> Networking-
> 
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Quick Utilities for getting into user mail file

[Robert Banz]

>
> Currently, I'm modifying a pinerc file and running PC-Pine to do  
> this if
> necessary.  Are there any other tools available for this kind of  
> quick and
> dirty mailbox manipulation?  Modifying config files on a per-user  
> basis is
> kind of a pain, and running webmail can be slow with huge mailboxes.

When I'm checking the results of reconstructs and such, I find it  
pretty easy to set the acls on a particular mailbox to allow my user  
to read it, then I can view it from my account with *insert mail  
client here* as just another folder.  I usually only give myself read  
access, but there's nothing stopping you from giving yourself full  
access and going to town...

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: delete messages from INBOX.Trash INBOX.Junk

[Robert Banz]

Going this same direction...  With our previous mail system, we used  
to chug through our users' Spam & Trash folders, and send them a  
weekly notification reminding them of the messages they had in these  
folders, and how many would be deleted in the next week.

Anyone done something similar with Cyrus?

-rob

On Sep 20, 2007, at 05:21, Paul Dekkers wrote:

> Hi,
>
> Vladi Lemurov wrote:
>> Hello!
>>  Is there any way to delete old messages e.g. older than 30 days from
>> INBOX.Trash and INBOX.Junk mailboxes of all users in a proper  
>> "cyrus-way"?
>>
>
> You can use ipurge, and run it periodically from cyrus.conf. We  
> have for
> instance:
>
>   purgetrashcmd="ipurge -d 14 -f user.%.Trash" at=0200
>
> (and a couple of others for the spam folder for instance) to clear the
> Trash folder of users for messages over two weeks old... people  
> tend to
> fill it up and never look at it again, this saves over 10G in our  
> case :-)
>
> You can run ipurge from the commandline too, as user cyrus. (E.g. sudo
> -u cyrus /path/to/ipurge)
>
> Paul
>
> 
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: delete messages from INBOX.Trash INBOX.Junk

[Robert Banz]

>
> I have an 'expire' set from inside 'cyradm' to automatically delete
> messages over a specific number of days old. I don't generate any
> emails to my users telling them what nor how many will be deleted...
> I just delete them nightly, or rather let cyrus do the deleting.

Our users need some notification.

They're the kind of folks that routinely ask for "important mail  
we've deleted from their Trash folder" to be restored.

Really.

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: Cyrus 2.3.9: Errors compiling outside source directory

[Robert Banz]

I have a repository copy of the cyrus source, and build in an 'lndir'- 
d tree.  Sadly, while building-outside-the-source tree should work  
fine, every once and awhile you run into gotchas like this which  
aren't work the time tracking down.  lndir symlink trees give you  
almost the same functionality, but its been awhile since I haven't  
seen something work right in that environment.


On Sep 23, 2007, at 00:52, Vincent Fox wrote:

>
> After hours of struggle, I am abandoning this and switching to
> simply having my script unpack the tarball into /tmp and
> compile it there.  Every time I found some fix for one
> piece there was another one that needed fixing.  Like in
> the sieve dir there is the xversion.sh that creates an xversion.h.
> All fixable one by one, but I got tired of chasing them.  It seems
> obvious this software is meant to be compiled in it's own source.
>
> 
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: duplicate messages

[Robert Banz]

>
>> it appears that after Cyrus gets the message it gets
>> duplicated. Anyone have any suggestions?
>
>
> Sieve rules?
>

I always think its because the users are staring at their mail  
clients cross-eyed. ;)



Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Nifty imapmagicplus usage

[Robert Banz]

So, I got an iPhone this week.  Totally rocks -- even if I can't hack  
it.

But, I was a bit frustrated by it showing *every* folder I had in its  
IMAP client.  I mean, really, I'm not going to want to read my  
managers "mathematica license information" folder while I'm sportin'  
the iPhone trying to look super cool, right?

Boom.  Finally, a really good use for imapmagicplus.  Remember,  
putting a "+" after your IMAP login name will restrict the folders  
returned to the client to only the ones you have a subscription for.   
Now I can sport my utter coolness without all that scrolling through  
a folder list!

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: LARGE single-system Cyrus installs?

[Robert Banz]

>
> One of the things Rob Banz recently did here was to move the data/
> config/proc directory from a "real" fs to tmpfs. This reduces the
> disk IO from Cyrus process creation/management.
>
> So the way we do stuff here is that each Cyrus backend has its own
> ZFS pool. That zpool is divided up into four file systems:
>
> /ms1/data
> /ms1/mail
> /ms1/meta
> /ms1/sieve
>

Just a side note on the partition lay-out - originally I was thinking  
of having the 'meta' partition be all of the databases/index files.   
Figured it'd be nice to be able to tune that zfs fs with a different  
recordsize if necessary.  However, I changed my mind, and our "meta"  
partition only contains the squat index files.

Why?  We're doing our backups via ZFS snapshots -- and while we DO  
want to snapshot the various meta-files, we're not too interested in  
saving all the squat indexes.

It seems to be working pretty well, and having a 7-day backlog of the  
data/mail/sieve directories has come in quite handy in recovering  
from stupid user tricks, such as wiping our their sieve rules, or  
folder deletes.*

* haven't added the delayed-folder-delete patch yet ;)

-rob

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: LARGE single-system Cyrus installs?

[Robert Banz]

On Oct 5, 2007, at 10:01, John Madden wrote:

>> I think that this is partly because ext3 does more aggressive read  
>> ahead
>> (which would be a mixed blessing under heavy load), partly because
>> reiserfs suffers from fragmentation. I imagine that there is  
>> probably a
>> tipping point under the sort of very heavy load that Fastmail see.
>
> I second that - reiserfs seems to be truly horrible in write-heavy
> situations.  Worse, a backup of our remaining reiserfs partition takes
> *days* to complete -- 165GB at ~500k/s.  And this is a 32-disk  
> stripe of
> fibre channel.

I think what truly scares me about reiser is those rather regular  
posts to various mailing lists I'm on saying "my reiser fs went poof  
and lost all my data, what should I do?"

-rob


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html