Re: v2.3.6 message delete causes signal 10
On Fri, May 26, 2006 at 12:11:05PM -0400, Ken Murchison wrote:
> >Cyrus IMAP v2.3.3 (with sasl v2.1.21) ran fine. Any ideas?
>
> Can you get a backtrace from a core dump?
I have a similar problem using 2.3.6 murder on CentOS 4.3 (very like
RHEL 4) on 32-bit x86.
In my case it is the front-end imap proxyd that crashes and the
signal is 11 (SIGSEGV) but the case seems very similar: it happens
when deleting messages and the traceback shows prot_printf() as the
active function.
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1208183104 (LWP 2)]
0x0809462c in prot_printf (s=0x8312250, fmt=0x80aa579 " %s ") at prot.c:960
960 prot.c: No such file or directory.
in prot.c
(gdb) where
#0 0x0809462c in prot_printf (s=0x8312250, fmt=0x80aa579 " %s ") at prot.c:960
#1 0x0805124b in cmd_store (tag=0x83126a8 "a0008", sequence=0x8312788 "8",
usinguid=1)
at imapd.c:4169
#2 0x0805f53b in cmdloop () at imapd.c:1640
#3 0x08060687 in service_main (argc=1, argv=0x8308008, envp=0xbff0b8ac) at
imapd.c:789
#4 0x0804c545 in main (argc=1, argv=0xbff0b8a4, envp=0xbff0b8ac) at
service.c:532
(gdb) up
#1 0x0805124b in cmd_store (tag=0x83126a8 "a0008", sequence=0x8312788 "8",
usinguid=1)
at imapd.c:4169
4169imapd.c: No such file or directory.
in imapd.c
(gdb) print tag
$1 = 0x83126a8 "a0008"
(gdb) print operation
$2 = {s = 0x0, len = 0, alloc = 0}
(gdb) print &operation
$3 = (struct buf *) 0x8135d20
(gdb) print tag
$4 = 0x83126a8 "a0008"
(gdb) print cmd
$5 = 0x80aa558 "UID Store"
(gdb) print sequence
$6 = 0x8312788 "8"
(gdb) print operation
$7 = {s = 0x0, len = 0, alloc = 0}
I think the problem is the last parameter to the prot_printf call:
in cmd_store() the call looks like this:
if (backend_current) {
/* remote mailbox */
prot_printf(backend_current->out, "%s %s %s %s ",
tag, cmd, sequence, operation);
pipe_command(backend_current, 65536);
return;
}
'operation' is declared as:
static struct buf operation, flagname;
but it is not assigned a value before prot_printf() is called so
when prot_printf() tries to do strlen() it gets SEGV.
I am not sure what is intended here, nor am I sure why we are doing
a STORE operation in the first place!
Andrew
--
---
| From Andrew Findlay, Skills 1st Ltd |
| Consultant in large-scale systems, networks, and directory services |
| http://www.skills-1st.co.uk/+44 1628 782565 |
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: v2.3.6 message delete causes signal 10
On Mon, Jul 03, 2006 at 09:24:15AM -0400, Ken Murchison wrote: > I was thinking too hard about this. The fix is trivial: > > https://bugzilla.andrew.cmu.edu/cgi-bin/cvsweb.cgi/src/cyrus/imap/imapd.c.diff?r1=1.443.2.83&r2=1.443.2.84&f=u That fixes it, thanks! Andrew -- --- | From Andrew Findlay, Skills 1st Ltd | | Consultant in large-scale systems, networks, and directory services | | http://www.skills-1st.co.uk/+44 1628 782565 | --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
2.3.6: cannot administer the murder
I am having great difficulty in getting cyradm to work properly
with a 2.3.6 murder. This is a 'standard' murder with separate
front-end, backends, and mupdate server. 'virtdomains: userid' is in
effect. Operations done by ordinary users are OK; I only have trouble
with admin ops.
Problem 1: creating top-level user mailboxes.
I connect to the front-end with cyradm:
/usr/local/cyrus/bin/cyradm -user zqvh_admin fe1.srv.tile
Things like listmailbox work OK. However, if I try to create
a new user on a specified backend server:
fe1.srv.tile> cm user/[EMAIL PROTECTED] ms1.srv.tile
createmailbox: Permission denied
(If I leave out the backend name, the mailbox gets created on
the frontend and then things get *really* confusing.)
This permission denied is very odd, as I can connect directly
to the mailstore ms1.srv.tile and authenticate as either the
main admin user or as the frontend proxy user and create the
mailbox with no trouble. I enable protocol logs on the
message store, and found this:
Create via front-end, as seen on the mailstore:
-- zqvh_admin Tue Jul 4 20:56:32 2006
<1152042992<6 CREATE {16+}
user/[EMAIL PROTECTED]
>1152042992>6 NO Permission denied
<11520430631152043063>* BYE LOGOUT received
Q01 OK Completed
Create directly on mailstore:
-- zqvh_admin Tue Jul 4 20:57:58 2006
<1152043078<4 RLIST "" ""
>1152043078>* LIST (\Noselect) "/" ""
4 OK Completed (0.000 secs 0 calls)
<1152043104<5 CREATE user/[EMAIL PROTECTED]
>1152043104>5 OK Completed
The only difference that I can see is that the failing operation is
using the IMAP 'literal' syntax, but operations done by users are the
same, and they work!
[ The literal syntax does look rather odd: I don't think the '+'
symbol should be there ]
I have several other problems, but I suspect they all have a similar
cause. I have turned up syslog logging to debug level and all I get
in the log is a note that the user logged in with DIGEST-MD5 auth.
Can anyone suggest what is wrong?
Thanks
Andrew
--
---
| From Andrew Findlay, Skills 1st Ltd |
| Consultant in large-scale systems, networks, and directory services |
| http://www.skills-1st.co.uk/+44 1628 782565 |
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: 2.3.6: cannot administer the murder
On Tue, Jul 04, 2006 at 08:28:54PM +0100, Andrew Findlay wrote: > I am having great difficulty in getting cyradm to work properly > with a 2.3.6 murder. This is a 'standard' murder with separate > front-end, backends, and mupdate server. 'virtdomains: userid' is in > effect. Operations done by ordinary users are OK; I only have trouble > with admin ops. > fe1.srv.tile> cm user/[EMAIL PROTECTED] ms1.srv.tile > createmailbox: Permission denied I am beginning to suspect that the problem is due to the frontend using IMAP AUTHENTICATE where the client code uses LOGIN. The code-paths are different, and it looks as if the imapd_userisadmin and imapd_userisproxyadmin variables in imapd.c are not getting set in the AUTHENTICATE case. This probably means that there is some SASL config that I need to do to define the admins as well as setting the admins item in imapd.conf - auth_mech perhaps? The frontends use DIGEST-MD5 when authenticating to the backends, but then they use exactly the same when authenticating to the mupdate master, and that does treat them as admins! Can anyone suggest where to look next? Thanks Andrew -- ----------- | From Andrew Findlay, Skills 1st Ltd | | Consultant in large-scale systems, networks, and directory services | | http://www.skills-1st.co.uk/+44 1628 782565 | --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Mailstore filesystem
On Wed, Jul 05, 2006 at 08:48:23AM -0400, John Madden wrote: > How big? ext3 STILL only supports 32000 directories within a directory. > That gets to be quite a problem on large installs. You can contain the fanout by enabling both fulldirhash and hashimapspool in imapd.conf (but you will need to stop the server and rehash everything if you do this on an existing system). The 32000 limit applies to subdirectories, not to files so it would only be a problem for the main spool directory of a server with a lot of users. Hashing should raise the supportable number of users by a factor of about 20. I suspect there are other things that would stop you putting 600,000 users on one server... Andrew -- --- | From Andrew Findlay, Skills 1st Ltd | | Consultant in large-scale systems, networks, and directory services | | http://www.skills-1st.co.uk/+44 1628 782565 | --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Murder without Kerberos?
Is anyone running a Cyrus Murder without using Kerberos? I am trying to build a Murder using DIGEST-MD5 authentication, but I am running into weird problems and would like to know if it has been done before. I am particularly keen to know what SASL options are needed in this environment. Thanks Andrew -- --- | From Andrew Findlay, Skills 1st Ltd | | Consultant in large-scale systems, networks, and directory services | | http://www.skills-1st.co.uk/+44 1628 782565 | --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Murder without Kerberos?
On Thu, Jul 06, 2006 at 11:43:50AM -0400, Patrick Radtke wrote: > We use PLAIN to authenticate between all the machines in our murder. That is very interesting. I found that I had to enable MD5 because the backends (and mupdate?) would not accept lower-strength authentication. PLAIN would be preferable for several reasons. > What version of Cyrus are you using? 2.3.6 > do you have a > sasl_mech_list: > line in your imapd.conf? That is commented out at the moment, to allow MD5. I started with PLAIN and LOGIN only. > Can you auth using imtest and DIGEST-MD5? Yes > Do you support other mechanisms for users? I would like to support PLAIN, LOGIN, and DIGEST-MD5, but the latter requires a plaintext password database so it will probably be judged too risky. Thanks Andrew -- --- | From Andrew Findlay, Skills 1st Ltd | | Consultant in large-scale systems, networks, and directory services | | http://www.skills-1st.co.uk/+44 1628 782565 | --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Murder without Kerberos?
On Thu, Jul 06, 2006 at 04:38:24PM -0400, Patrick Radtke wrote: > I haven't tried it with 2.3.6, but PLAIN should work. The result seems to be the same as with MD5: ordinary user connections work fine, but admin stuff that goes through to the backends fails. PLAIN would not work at all until I enabled TLS. > I would suggest starting with > sasl_mech_list: PLAIN > > in all your imapd.conf files (make sure it says only PLAIN). > > and make sure there is no > force_sasl_client_mech > lines anywhere. > > Then make sure you can use imtest (with -m PLAIN and -t "" (for > tls)) to connect to backends, and then see if the backends will > communicate correctly. Similar results: here, frontend is the proxy authentication ID and [EMAIL PROTECTED] is an ordinary user. ms1.srv.tile is a backend store: imtest -t '' -m plain -u [EMAIL PROTECTED] -a frontend ms1.srv.tile Gave the password for 'frontend'. Connects OK, the backend logs that [EMAIL PROTECTED] has logged in, and an IMAP LIST command shows fred's mailboxes. imtest -t '' -m plain -u admin -a frontend ms1.srv.tile Gave the password for 'frontend'. Connects and logs in OK. Backend logs that admin has logged in, but IMAP LIST does not show anything. imtest -t '' -m plain -u admin -a admin ms1.srv.tile Gave the password for 'admin'. Connects OK, backend logs that admin has logged in, and IMAP LIST shows all mailboxes on the server. So it looks as if the backend will not accept proxied admin accounts. I am still stuck! Andrew -- --- | From Andrew Findlay, Skills 1st Ltd | | Consultant in large-scale systems, networks, and directory services | | http://www.skills-1st.co.uk/+44 1628 782565 | --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Murder without Kerberos?
On Fri, Jul 07, 2006 at 04:53:59PM +0100, Andrew Findlay wrote: > So it looks as if the backend will not accept proxied admin accounts. This looks more and more like a 2.3.x issue. This comment is in cmd_getquotaroot in imapd.c : /* If they are an admin, they won't retain that privledge if we * proxy for them, so we need to refer them -- even if they haven't * told us they're able to handle it. */ so I suspect that all admin operations should be referred rather than proxied, and that is not happenning. Andrew -- --- | From Andrew Findlay, Skills 1st Ltd | | Consultant in large-scale systems, networks, and directory services | | http://www.skills-1st.co.uk/+44 1628 782565 | --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.7 Released
On Mon, Jul 10, 2006 at 09:19:16PM -0400, Ken Murchison wrote: > I am pleased to announce the release of Cyrus IMAPd 2.3.7. This release > should be considered late beta quality, reflecting that most large bugs > have been resolved, but some small buglets might still remain. Bug 2858 is still present. The diff in the Bugzilla entry still fixes the problem for me: https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=2858 This only affects Murder front-ends. Andrew -- --- | From Andrew Findlay, Skills 1st Ltd | | Consultant in large-scale systems, networks, and directory services | | http://www.skills-1st.co.uk/+44 1628 782565 | --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: performance issue (imap spool on san)
On Fri, Jul 28, 2006 at 12:18:12AM -0700, Nikola Milutinovic wrote: > So, perhaps we could state that the desired behavior of any IMAP client would > be to fetch only those message headers it nedds to and perhaps a bit more. In > case of TB, that would transalte to fetching only headers that would be > visible to the user and perhaps screenful of header up and down. It also helps if the clients ask for a limited set of headers from each message of interest. For example, Squirrelmail asks for this lot at mailbox opening time: (FLAGS UID RFC822.SIZE INTERNALDATE BODY.PEEK[HEADER.FIELDS (Date To Cc From Subject X-Priority Importance Priority Content-Type)]) Cyrus caches some headers in the index file, but unfortunately does not include Importance or Content-Type so it has to open and parse every message file in the mailbox to satisfy the request! This is defined in imapd/mailbox.c in struct mailbox_header_cache mailbox_cache_headers. I am considering expanding the list in that definition to improve efficiency: does anyone know of any problems this might cause? (This is for a new deployment, so there are no existing index files to worry about). Andrew -- --- | From Andrew Findlay, Skills 1st Ltd | | Consultant in large-scale systems, networks, and directory services | | http://www.skills-1st.co.uk/+44 1628 782565 | --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: performance issue (imap spool on san)
On Fri, Jul 28, 2006 at 10:37:43AM +0100, Alain Williams wrote: > Might it not be better to have Cyrus 'learn' what header lines are needed, > rather than just bloating the list with more headers. The set of headers > would needed to be dynamically changable. The points are: > > 1) different IMAP clients want different sets of headers. The same IMAP >client at different releases might change the set requested. > 2) most individual sites run only 2 or 3 different IMAP clients, why get >Cyrus to collect the headers that the IMAP clients at that site don't want. > 3) most system admins don't have the skills/inclination/... to optimise >the set of headers cached. The problem with the learning approach is that the index is no use at all unless it contains the same headers from every message, and it is built at message delivery time rather than when the message is read. I suppose some sort of 'lazy indexing' could add headers to the index on demand, but the code changes might be significant. Headers are not usually very large, so I would be more inclined to the idea that the index should store every header (perhaps with a blocklist to avoid things like Received:) Andrew -- ----------- | From Andrew Findlay, Skills 1st Ltd | | Consultant in large-scale systems, networks, and directory services | | http://www.skills-1st.co.uk/+44 1628 782565 | --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
