2.3.11 STARTTLS broken if tls_ca_file is defined
Hi! I always had tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt defined in my imapd.conf. Since I updated to 2.3.11 yesterday STARTTLS didn't work anymore because negotiation failed and timed out. $CLIENT was waiting for more packets from server AFAIS in a tcpdump, where $CLIENT is Thunderbird, gnutls-cli, apple-mail. IMAPS always worked...so I searched for differences in the code and found the "client cert verfication" code triggered by askcert == 1 in tls.c:738 Removing the tls_ca_file definition helped. I didn't find the exact cause yet since there are no changes in tls_init_serverengine() since 2.3.10. One thing I noticed was that it worked when connecting via the loopback interface. But connecting via network always failed while negotiating STARTTLS. Log always showed: 00:00 imap[8508]: accepted connection +02 imap[8508]: SSL_accept() incomplete -> wait <- here the client waits +23 imap[8508]: EOF in SSL_accept() -> fail <- here client sent FIN After the FIN from the client, the server sends lots of stuff on the dead connection and closes with "NO ssl negotiation failed". cyrus-imapd-2.3.11 was built from invoca.ch src.rpm on fc5 and rhel5.1. Both failed. Regards, Wolfgang Breyha -- Wolfgang Breyha <[EMAIL PROTECTED]> | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: 2.3.11 STARTTLS broken if tls_ca_file is defined
Jure Pečar wrote: > Interesting ... works for me with 2.3.11rc1 and sylpheed. Using imaps on > port 993 only. As I mentioned ... IMAPS works for me too. Using STARTTLS on port 143 doesn't work. Regards, Wolfgang -- Wolfgang Breyha <[EMAIL PROTECTED]> | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus 2.3.11: cyr_expire can't remove deleted mailboxes
Ernst Jeschek wrote, on 31.03.2008 10:24: > Hello list members, > > we have delete_mode and expunge_mode set to delayed on our cyrus > 2.3.11 backend. When cyr_expire tries to remove deleted mailboxes, it > fails with the following error: > > cyr_expire[24222]: can not connect to mupdate server for delete of > 'DELETED.DELETED.user...47EA470C.47EA773C' > cyr_expire[24222]: couldn't authenticate to backend server : SASL library > not initialized https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3034 http://www.invoca.ch/pub/packages/cyrus-imapd/cyrus-imapd-2.3.11-3.src.rpm and newer versions include the proposed fix. Best regards, Wolfgang -- Wolfgang Breyha <[EMAIL PROTECTED]> | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
murder frontend CAPABILITY check
Hi! I recognized that IDLE was not working as expected on our murder setup. We've two frontends, 3 backends and a dedicated mupdate server, all running 2.3.13. idled is active on the backends. I did some strace'ing on front- and backends and as far as I can tell, the frontends do not proxy the IDLE command to the backends, because the frontends think the backends are not CAPAable to do IDLE. I searched the source and it seems that there is a problem with the capability check the frontends do on the backends. On one hand cyrus announces CAPABILITY in the greeting banner. But CAPA_PREAUTH doesn't contain all of the caps the frontend wants to know. eg. IDLE, MULTIAPPEND, RIGHTS and LIST-SUBSCRIBED. On the other hand, the frontend is told by the protocol definition in imap_proxy.c:107, that it should exclusively check the banner in backend.c:400 for the CAPS, which doesn't contain the needed info in pre_auth state. Am I right, that either capa_response(CAPA_PREAUTH); should return the needed caps for the frontends, too, or the frontends should'nt believe blindly in the banner capabilites? Regards, Wolfgang Breyha -- Wolfgang Breyha <[EMAIL PROTECTED]> | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: murder frontend CAPABILITY check
Wolfgang Breyha wrote, on 25.11.2008 15:35: > Am I right, that either > capa_response(CAPA_PREAUTH); > should return the needed caps for the frontends, too I tried this, by adding CAPA_POSTAUTH in imapd.c:cmdloop() and my frontends proxy IDLE to the backends again. This confirms my previous findings and I filed a bug: https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3121 Regards, Wolfgang Breyha -- Wolfgang Breyha <[EMAIL PROTECTED]> | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: murder frontend CAPABILITY check
Duncan Gibb wrote, on 26.11.2008 17:36:
> Could you post your patch, please? Or attach it to the bug.
I'll post it here, since it's only a quick and dirty "fix" to have proof of
the bug.
Regards, Wolfgang
--
Wolfgang Breyha <[EMAIL PROTECTED]> | http://www.blafasel.at/
Vienna University Computer Center | Austria
diff -urN cyrus-imapd-2.3.13.orig/imap/imapd.c cyrus-imapd-2.3.13/imap/imapd.c
--- cyrus-imapd-2.3.13.orig/imap/imapd.c2008-11-15 23:40:36.0
+0100
+++ cyrus-imapd-2.3.13/imap/imapd.c 2008-11-26 11:22:03.0 +0100
@@ -987,7 +987,7 @@
const char *err;
prot_printf(imapd_out, "* OK [CAPABILITY ");
-capa_response(CAPA_PREAUTH);
+capa_response(CAPA_PREAUTH|CAPA_POSTAUTH);
prot_printf(imapd_out, "]");
if (config_serverinfo) prot_printf(imapd_out, " %s", config_servername);
if (config_serverinfo == IMAP_ENUM_SERVERINFO_ON) {
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
cyr_expire SIGSEGV
Hi!
I've a cyrus-imapd-2.3.7-7 built from the src.rpm at invoca.ch running on FC5.
I use the "delayed expunge" feature.
My cyrus.conf therefor has
delprune cmd="cyr_expire -E 3 -X 2" at=0400
Most of the time cyr_expire runs it crashes with SIGBUS or SIGSEGV.
It also crashes if I start it manually after it crashed during the night.
Doing a strace and "reconstruct -r"ing the user last accessed makes
cyr_expire happy again... at least for this users mailboxes.
I've activated coredumps and the backtrace always shows:
#0 0x0804e6f0 in cyrus_mutex_free ()
#1 0x08053213 in cyrus_mutex_free ()
#2 0x0804cb50 in expire ()
#3 0x0805a955 in mboxlist_findall ()
#4 0x0808ef0e in mboxlist_findall ()
#5 0x08057fc2 in mboxlist_findall ()
#6 0x0804cfa7 in expire ()
#7 0x004554e4 in __libc_start_main () from /lib/libc.so.6
#8 0x0804c6f1 in ?? ()
The last lines from strace shows write access to the quota file
stat64("/var/spool/imap/user/x/Trash/cyrus.expunge",
{st_mode=S_IFREG|0600, st_size=736, ...}) = 0
mmap2(NULL, 736, PROT_READ, MAP_SHARED, 9, 0) = 0xb66ba000
lseek(9, 736, SEEK_SET) = 736
open("/var/spool/imap/user/x/Trash/cyrus.expunge.NEW",
O_RDWR|O_CREAT|O_TRUNC, 0666) = 12
time(NULL) = 1169137631
write(10, "\0\0\0\357\0\0\0\0\0\0\0\t\0\0\0`\0\0\0P\0\0\0\0E\257?"..., 96) = 96
_llseek(10, 0, [0], SEEK_SET) = 0
read(10, "\0\0\0\357\0\0\0\0\0\0\0\t\0\0\0`\0\0\0P\0\0\0\0E\257?"..., 4096) = 96
_llseek(10, 96, [96], SEEK_SET) = 0
open("/var/lib/imap/quota/g/user.x", O_RDWR) = 13
fcntl64(13, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}) = 0
fstat64(13, {st_mode=S_IFREG|0600, st_size=16, ...}) = 0
stat64("/var/lib/imap/quota/g/user.x", {st_mode=S_IFREG|0600, st_size=16,
...}) = 0
fstat64(13, {st_mode=S_IFREG|0600, st_size=16, ...}) = 0
mmap2(NULL, 16, PROT_READ, MAP_SHARED, 13, 0) = 0xb66b9000
munmap(0xb66b9000, 16) = 0
unlink("/var/lib/imap/quota/g/user.x.NEW") = -1 ENOENT (No such file or
directory)
open("/var/lib/imap/quota/g/user.x.NEW", O_RDWR|O_CREAT|O_TRUNC, 0666) = 14
fcntl64(14, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}) = 0
lseek(14, 0, SEEK_SET) = 0
write(14, "30244719\n512000\n", 16) = 16
ftruncate(14, 16) = 0
fsync(14) = 0
fstat64(14, {st_mode=S_IFREG|0600, st_size=16, ...}) = 0
rename("/var/lib/imap/quota/g/user.x.NEW",
"/var/lib/imap/quota/g/user.x") = 0
fcntl64(14, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0
close(14) = 0
fcntl64(13, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0
close(13) = 0
fstat64(12, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb66b9000
time(NULL) = 1169137631
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV (core dumped) +++
Any ideas?
Regards, Wolfgang Breyha
--
Wolfgang Breyha <[EMAIL PROTECTED]> | http://www.blafasel.at/
Vienna University Computer Center | Austria
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyr_expire SIGSEGV
Hi again!
I compiled a cyr_expire with debug symbols... the backtrace looks like:
#0 process_records (mailbox=0xbfb1786c, newindex=0x9558f68,
index_base=0xb66f9000 , exists=65,
deleted=0x9559158, numdeleted=0xbfb14794, quotadeleted=0xbfb14768,
numansweredflag=0xbfb14790, numdeletedflag=0xbfb1478c,
numflaggedflag=0xbfb14788, newcache=0x9558e00,
new_cache_total_size=0xbfb14780, expunge_fd=-1, last_offset=0,
decideproc=0x804cc10 , deciderock=0xbfb184e4, expunge_flags=2)
at mailbox.c:1932
1932cacheitem = CACHE_ITEM_NEXT(cacheitem);
#1 0x080532d3 in mailbox_expunge (mailbox=0xbfb1786c, decideproc=0x804cc10
, deciderock=0xbfb184e4, flags=Variable "flags" is not
available.) at mailbox.c:2308
#2 0x0804cb50 in expire (name=0xbfb17d9d "user.mxx.Trash",
matchlen=22, maycreate=1, rock=0xbfb184e4) at cyr_expire.c:224
#3 0x0805aa15 in find_cb (rockp=0xbfb18070, key=0xb69881b4 , keylen=22, data=0xb69881d0 , datalen=34) at mboxlist.c:2035
#4 0x0808efce in myforeach (db=0x9558240, prefix=0xbfb180b2 "*", prefixlen=0,
goodp=0x805b480 , cb=0x805a880 , rock=0xbfb18070,
tid=0x0) at cyrusdb_skiplist.c:989
#5 0x08058082 in mboxlist_findall (namespace=0x0, pattern=0xbfb18510 "*",
isadmin=1, userid=0x0, auth_state=0x0, proc=0x804c7d0 ,
rock=0xbfb184e4) at mboxlist.c:2227
#6 0x0804cfa7 in main (argc=6, argv=Cannot access memory at address 0x4
It seems that cyr_expire only crashes on folders that where "touched" by
ipurge before.
I've
purgetrashcmd="ipurge -fX -d 31 user.*.Trash" at=0200
purgejunk cmd="ipurge -fX -d 60 user.*.Junk" at=0300
running before cyr_expire (at=0400)
Every time I look into a folder which causes cyr_expire to coredump I find eg
...
-rw--- 1 cyrus mail4 Jan 23 02:00 cyrus.cache
-rw--- 1 cyrus mail 124 Jan 23 13:31 cyrus.cache.NEW
... a cyrus.cache.NEW file that is larger then the old one.
Putting a debug printf in the loop
for (cache_ent = 0; cache_ent < NUM_CACHE_FIELDS; cache_ent++) {
cacheitem = CACHE_ITEM_NEXT(cacheitem);
}
... shows that cache_ent always is 0 if the crash occures, so it seems that
cacheitembegin = cacheitem = mailbox->cache_base + cache_offset;
is invalid at that moment.
Regards, Wolfgang Breyha
--
Wolfgang Breyha <[EMAIL PROTECTED]> | http://www.blafasel.at/
Vienna University Computer Center | Austria
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyr_expire SIGSEGV
Wolfgang Breyha wrote, on 23.01.2007 13:44: > It seems that cyr_expire only crashes on folders that where "touched" by > ipurge before. I was able to reproduce that. I deleted some messages via IMAP in Trash folder. I "EXPUNGE"d the folder. After that I had -rw--- 1 cyrus mail 1529752 24. Jan 14:21 cyrus.cache -rw--- 1 cyrus mail 576 24. Jan 14:23 cyrus.expunge -rw--- 1 cyrus mail 183 11. Jun 2006 cyrus.header -rw--- 1 cyrus mail 142096 24. Jan 14:23 cyrus.index Then a started "ipurge -fX -d 31 user..Trash". ipurge removed _no_ message. But the databases looked like -rw--- 1 cyrus mail 1525368 24. Jan 14:24 cyrus.cache -rw--- 1 cyrus mail 576 24. Jan 14:23 cyrus.expunge -rw--- 1 cyrus mail 183 11. Jun 2006 cyrus.header -rw--- 1 cyrus mail 142096 24. Jan 14:24 cyrus.index It seems that ipurge removes info from cyrus.cache needed by cyr_expire later, because after these steps cyr_expire crashes exactly when working on this folder. Regards, Wolfgang Breyha -- Wolfgang Breyha <[EMAIL PROTECTED]> | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
cyrus 2.3.x and thunderbird 2.x delete issue
Hi!
I had troubles with cyrus 2.3.x and thunderbird 2.x. Thunderbird checks the
ACLs now and issues a "myrights" and "getacl" command. Since thunderbird
only checks RFC 2086 flags it disables "DELETE" access since cyrus 2.3.x
reports the compatibility flags only with "myrights" but not with "getacl".
I think RFC 4314 requires both ...
"When any of the "delete" member rights is set in a list of rights,
the server MUST also include the "d" right when returning the list in
a MYRIGHTS or ACL response."
Sample response from cyrus for "MYRIGHTS"...
. myrights INBOX
* MYRIGHTS INBOX lrswipkxtecda
... "cd" included here,
but cyrus fails to return them on "GETACL" eg.:
. getacl INBOX
* ACL INBOX testuser lrswipkxtea
I've tried a quickfix...
--- cyrus-imapd-2.3.8/imap/imapd.c.orig 2007-04-19 22:43:37.0 +0200
+++ cyrus-imapd-2.3.8/imap/imapd.c 2007-04-19 22:41:36.0 +0200
@@ -5979,6 +5979,7 @@
int r, access;
char *acl;
char *rights, *nextid;
+char str[ACL_MAXSTR];
r = (*imapd_namespace.mboxname_tointernal)(&imapd_namespace, name,
imapd_userid, mailboxname);
@@ -6021,7 +6022,7 @@
prot_printf(imapd_out, " ");
printastring(acl);
prot_printf(imapd_out, " ");
- printastring(rights);
+ printastring(cyrus_acl_masktostr(cyrus_acl_strtomask(rights), str));
acl = nextid;
}
prot_printf(imapd_out, "\r\n");
Works as proof-of-concept with Thunderbird now.
Regards, Wolfgang
--
Wolfgang Breyha <[EMAIL PROTECTED]> | http://www.blafasel.at/
Vienna University Computer Center | Austria
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus 2.3.x and thunderbird 2.x delete issue
Sebastian Hagedorn wrote, on 20.04.2007 10:00: > Hm, are there any other conditions necessary? Because I just tried TB 2 > with our Cyrus 2.3.8 server and I was able to delete a message in my INBOX > just fine. I didn't actually check the protocol, so I can't see if TB did > "myrights" and "getacl" and - if so - what the replies were ... Check your cyrus.header files. Maybe you upgraded your installation from older cyrus and there are still "c" and "d" flags set on your folders. Mailboxes here only have RFC 4314 flags set. Regards, Wolfgang -- Wolfgang Breyha <[EMAIL PROTECTED]> | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus 2.3.x and thunderbird 2.x delete issue
Sebastian Hagedorn wrote, on 20.04.2007 11:00: > I'm not sure I understand why. Are you saying that a 2.3.8 installed from > scratch behaves differently than an upgraded one? I don't know;-) Most of the mailboxes here have "lrswipkxtea" set. And searching the source I found code in cmd_myrights, which adds "c" and "d" if they are missing. So I thought cmd_getacl should do that, too. But it seems there was a change from 2.3.7 to 2.3.8 in handling legacy flags. At least a diff on lib/acl.c makes me think so. In 2.3.7 the flags where added "on-the-fly" and in 2.3.8 they are added permanently. If I'm right that means that I've to update all my ACLs to fix that. Regards, Wolfgang -- Wolfgang Breyha <[EMAIL PROTECTED]> | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: OT: Cyrus IMAP Interacting with Thunderbird 2.0
Forrest Aldrich wrote, on 28.04.2007 18:41: > It's been suggested that there may be something different about Cyrus in > the IMAP command sequence - but that seems very vague. I don't see why > it shouldn't just work. I can try recompiling the FreeBSD port and see > if that resolves it. I'm sure my libraries are up-to-date, and > certainly if there were problems I would have seen it by now with other > functions. Look at the thread starting here... http://cyrusimap.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&msg=43590 and https://bugzilla.mozilla.org/show_bug.cgi?id=377900 ...it's not thunderbirds fault and it's "easily" fixed by resetting your ACLs. Regards, Wolfgang -- Wolfgang Breyha <[EMAIL PROTECTED]> | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: OT: Cyrus IMAP Interacting with Thunderbird 2.0
Jo Rhett wrote: > > On Apr 30, 2007, at 6:38 AM, Wolfgang Breyha wrote: >> ...it's not thunderbirds fault > > Um... I disagree. So do the thunderbird developers. No, they don't. See https://bugzilla.mozilla.org/show_bug.cgi?id=377900#c11 cyrus has to support 'c' and 'd' flag as required by RFC 4314. They made a quick fix to check the RFC 4314 't' flag, too. And they opened a second 'bug' to implement full RFC 4314 support. See https://bugzilla.mozilla.org/show_bug.cgi?id=310958 > Fixing the ACL is a backwards hack to support an obsolete IMAP > specification. Updating Thunderbird to properly read the modern flags > was the appropriate fix, which has already been committed to the tree > and may appear in the next update. Again, no. RFC 4314 requires the server to support 'c' and 'd' flag from RFC 2068. And cyrus does it. But as I meantioned before there was a change from cyrus 2.3.7 to 2.3.8. At least 2.3.7 added 'c' and 'd' on the fly while answering ACL requests if the appropriate RFC 4314 flags were set in the stored ACL. cyrus 2.3.8 now writes the 'c' and 'd' flag into the stored ACL. If you're upgrading from 2.3.7 to 2.3.8 some people have missing 'c' and 'd' flags in their ACLs and they are not added automatically anymore. This get's fixed by simply setting the same ACLs again since cyrus sets 'c' and 'd' accordingly now. Regards, Wolfgang -- Wolfgang Breyha <[EMAIL PROTECTED]> | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
2.4.7 complains about mails named '0.'
Hi! I upgraded from 2.3.16 to 2.4.7 yesterday. Doing a reconstruct on all mailboxes showed some messages like: reconstruct[32741]: user.xx.Folder odd file /var/spool/imap/user/xxx/Folder/0. reconstruct[32741]: run reconstruct with -O to remove odd files But this is a the first mail in this folder written with a older version of cyrus? Is this a bug in 2.4, or is there an easy way to fix that? Wouldn't hurt that much, because I got this message only about 10 times. Regards, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: 2.4.7 complains about mails named '0.'
Hi! On 2011-04-02 11:59, Bron Gondwana wrote: > Was there seriously a version of Cyrus that allows UID zero? That's > bogus. Surely IMAP UIDs start at one. I found some historic info;-) I used cyrus-imap 2.3.7 back then. I used "imapsync" to migrate mailboxes from courier to cyrus. It happend an exactly one mailbox, but on several folders there. > If UID zero is allowed, I'll have to fix that code - and probably > some other code which assumes UIDs are never zero... Ok, if that's the case I'll rename that files to the first available UID and reconstruct the folder. Regards, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: 2.4.7 complains about mails named '0.'
On 2011-04-02 14:19, Bron Gondwana wrote: > Ooh... just a guess - but did you have autocreate patches? I wonder if > autocreate caused uidnext to be zero rather than one due to doing the > append during the same lock as the create... Yes, I'm using Simons SRPMS as a base with minor patches for syslogging (like auditlog in 2.4 now). I think I really used autocreate back then. I disabled it after a short while. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: 2.4.7 complains about mails named '0.'
On 2011-04-02 14:48, Bron Gondwana wrote: > It seems more likely to me that your '0.' files are caused > by some bug somewhere along the way. Did you check them to > see if they're duplicates of another message in that folder? No, no duplicates so far. Since only one mailbox is affected I can't tell for sure what happened some years ago;-) Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
2.4.7 mailbox rename looses mail
Hi! If I do the following within Thunderbird 3.1.9: *) create a folder "test" *) create a subfolder within "test" called "test2" *) copy a message to test *) copy a message to test2 *) rename "test" to "testx" the result in thunderbird is: *) a folder "testx" with subfolder "test2" *) both are *empty* a reconstruct is needed to make them visible again: $ reconstruct -r user..testx user..testx uid 1 found - adding user..testx user..testx.test2 uid 1 found - adding user..testx.test2 Renaming a folder again before reconstructing it wipes the message files from disc. I filed a bug: http://bugzilla.cyrusimap.org/show_bug.cgi?id=3434 Regards, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: 2.4.7 mailbox rename looses mail
On 2011-04-05 22:27, Bron Gondwana wrote:
> I will test now with those configs, thanks Lucas.
It's an uninitialized return value! config doesn't matter IMO. It's pure
luck what r contains;-) If r != 0 the meta files are not copied and the
state of the mailbox is lost.
--
--- cyrus-imapd-2.4.7/imap/mailbox.c.orig 2011-04-06 01:20:09.0
+0200
+++ cyrus-imapd-2.4.7/imap/mailbox.c2011-04-06 01:17:01.0 +0200
@@ -2994,6 +2994,7 @@
cyrus_mkdir(path, 0755);
mkdir(path, 0755);
+r = 0;
/* Copy over meta files */
for (mf = meta_files; !r && mf->metaflag; mf++) {
struct stat sbuf;
---
Regards, Wolfgang
--
Wolfgang Breyha | http://www.blafasel.at/
Vienna University Computer Center | Austria
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: 2.4.7 mailbox rename looses mail
Hi again! It looks like these changes broke it: http://git.cyrusimap.org/cyrus-imapd/commit/?id=463f7571f8aeca00630a2bc26ebf24fadf844b4c Versions up to 2.4.6 should be fine. Only 2.4.7 and 2.4.8 prereleases are affected. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
expunge_days & cyr_expire best practice
Hi! I'm currently preparing our "murder" migration from 2.3 to 2.4. Since we already use delayed expunge and delete I try to fully understand the impact of expunge_days: setting. Currently we use -X 2 with cyr_expire. Since QRESYNC needs at least "expunge_days: 7" as stated in "man imapd.conf" I tried to figure out how to set it up right. If I read the source right only mailbox.c:mailbox_close() and cyr_expire do cleanup jobs on cyrus cache and unlink files. mailbox_close() waits until expunge_days+8 after the index record was expired. What are these 8 days for? Is this the minimum needed for QRESYNC mentioned in "man imapd.conf"? cyr_expire doesn't care about expunge_days at all. So I guess -X should be at least >= expunge_days, right? Is expunge_days: 7 and cyr_expire -X 7 ok so far? Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
cyrus-master dies/exits on startup
Hi!
I had this on 2.3.x already and today a brand new 2.4.11 cyrus-master died as
well
On my murder frontends sometimes (~1 out of 10) cyrus-master dies/exits(?!)
right after forking the initial set of processes defined in cyrus.conf.
This initial set of mupdated/idled/lmtpd/imapd/pop3d processes are left in the
system, but cyrus-master is gone afterwards.
After killing the leftovers and restarting cyrus-master he usually survives.
And he never died/exited if he survived the initial execs.
The 2.3.x frontends are RHEL5/i386 and the 2.4.11 are RHEL6/x86_64.
I thought it is the heavy initial load on our frontends causing this, but the
2.4.11 frontend has no load or connection at all currently.
cyrus-master is started with "-d -x /var/agentx/master -P 30"
The log of the master says nothing unusual:
> Sep 19 14:20:32 jarvis master[1738]: process started
> Sep 19 14:20:32 jarvis master[1747]: about to exec
> /usr/lib/cyrus-imapd/ctl_cyrusdb
> Sep 19 14:20:49 jarvis master[1786]: about to exec /usr/lib/cyrus-imapd/idled
> Sep 19 14:20:49 jarvis master[1738]: ready for work
> Sep 19 14:20:49 jarvis master[1789]: about to exec /usr/lib/cyrus-imapd/imapd
> Sep 19 14:20:49 jarvis master[1790]: about to exec /usr/lib/cyrus-imapd/imapd
> Sep 19 14:20:49 jarvis master[1791]: about to exec /usr/lib/cyrus-imapd/pop3d
and some more exec, but no note about exiting.
My cyrus.conf:
START {
recover cmd="ctl_cyrusdb -r"
idled cmd="idled"
}
SERVICES {
mupdate cmd="mupdate" listen=3905 prefork=1
Fimap cmd="imapd" listen="imap" prefork=5
Fimapscmd="imapd -s" listen="imaps" prefork=2
Fpop3 cmd="pop3d" listen="pop3" prefork=3
Fpop3scmd="pop3d -s" listen="pop3s" prefork=1
Fsieveold cmd="timsieved" listen="sieve-old" prefork=0
Fsievecmd="timsieved" listen="sieve" prefork=0
Flmtp cmd="lmtpd" listen="lmtp" prefork=1 maxchild=30
}
My 2.3.x backends run a similar config (except mupdate) and never died on
startup yet.
Has somebody else seen this behavior already or should I try to get more
details? As a first step I allowed coredumps.
Greetings, Wolfgang
--
Wolfgang Breyha | http://www.blafasel.at/
Vienna University Computer Center | Austria
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
mbpath output 2.4
Hi! On 2.3 if I ask a murder frontend server with "mbpath user.x" it displays backend.fqdn!partition_name On 2.4 I only get Remote Mailbos: user.x Is that intended behaviour? Is there another way to ask my frontend where a mailbox is located instead on 2.4? Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: mbpath output 2.4
Bron Gondwana wrote, on 21.09.2011 09:17: > Probably a bug. Can you please file a bug and I'll fix it. https://bugzilla.cyrusimap.org/show_bug.cgi?id=3556 Patch already attached;-) Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
singleinstancestore obsolete?
Hi! Is singleinstancestore obsolete/useless in 2.4.(12)? This option is read into "singleinstance" in lmtpd.c and nntpd.c, but never used afterwards? Or did I miss something searching the source? Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: singleinstancestore obsolete?
Bron Gondwana wrote, on 19.10.2011 12:27: > On Wednesday, October 19, 2011 12:18 PM, "Wolfgang Breyha" > wrote: >> Hi! >> >> Is singleinstancestore obsolete/useless in 2.4.(12)? >> >> This option is read into "singleinstance" in lmtpd.c and nntpd.c, but never >> used afterwards? Or did I miss something searching the source? > > Hmm... > > r = append_fromstage(&as, &content->body, stage, 0, > (const char **) flag, nflags, !singleinstance); > > Looks to me like you missed something. Obviously;-) Sorry for that. The main reason I was looking into these things was, that I'm searching for the best way to migrate a 2.3.16 backend with ~4TB spools and ~80k users to 2.4. Unlike my other backends this one profits pretty much from singleinstancestore (many massmailings) and I don't want to loose these savings if possible. Simply upgrading the backend is impossible. The index upgrades will kill this machine. Since replication is not a possible way from 2.3.16 to 2.4 I tried to figure out if a simple XFER takes care about that. But it doesn't as far as I can tell. Do you know of a possible way to do that? Even if it's not trivial in the first place. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: singleinstancestore obsolete?
Ramprasad wrote, on 19.10.2011 15:37: > I think , writing a standalone index upgrade utility , like the ipurge , > seems to be a reasonable thing to do > > > If there was a light enough index upgrade possible ( only for inboxes .. > not subfolders ) Then I could stop cyrus , fork probably around 100 > parallel upgrades take a 2-3 hour downtime and then start services again Currently I'm considering the following way: *) build a new backend with same partition count/names *) output something like "find ../partxx -type -f -links +2 -ls |sort" to a file for every partition on the old backend. -ls maybe replaced by special -printf. *) move mailboxes from 2.3 to 2.4 with rename keeping partitions the same *) write a script reading "find"-output and relink all the files with same inodes. This can be done at any time and with low impact. I think that should be pretty safe if the script has enough safty belts in place. Mails moved or deleted in the meantime are a special case. Don't know if it's worth to try hard to find them. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: singleinstancestore obsolete?
On 2011-10-19 17:42, Simon Matter wrote: > I think for the singleinstancestore, you can redo it after migration with > tools like hardlink or http://www.freedup.org/. IIRC I did this once and > it worked fine - I think I was using a simple bash script as you suggested > above. The only problem could be that you need the extra space while > migration is going on. My first find /var/spool/imap/partxx -type f -links +2 -printf "%10i,%3n,%9s,%C@,%c,%T@,%t,%u,%g,%#m,%p\n" finished and my current savings are 60GB on a 256GB(230 used) partition. So, extra space is in fact a topic;-) This backend holds all our student accounts. That's why the savings are that impressive. With the list find provides I am able to run my short perl script several times while migrating mailboxes to get space back very fast and with low IO impact. Running freedup(.org) is still a very good idea. Thanks a lot for the hint. I tried it in dry run mode on one of my other backends containing only XFERed mailboxes from our employees. It took about 2 hours to analyze a 256GB(165 used) partition with ~800k messages. Savings will be ~2,5GB. Not that much, but still worth the time. I used "freedup -n -v -a -T -o '-name "*."' -l /var/spool/..". Memory usage was ~300MB on x86_64. hardlink (at least the version provided with RHEL6) is not suitable since it hardlinks many cyrus.index files together because I've to compare on file contents only and there is no option to exclude them. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: singleinstancestore obsolete?
On 2011-10-19 22:35, Henrique de Moraes Holschuh wrote: > rsync -H will also work, but it can be painful. YMMV. Unfortunately rsync is not an option since I need to XFER the mailboxes to do the 2.3=>2.4 index upgrades. That's the only way to do it without long downtime on a backend that large. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: cyrus.index file
Eric Luyten wrote, on 07.11.2011 17:36: > and ... busy servers will probably produce huge audit trace files ... Who > is running production service with "auditlog: 1" in their imapd.conf ? I am. I patched cyrus <2.4 to get similar logs in custom format and use auditlog since 2.4 for all our >100k users. No problem at all. On the other hand not knowing which way a mail took until it was deleted by the user is not an option. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
lmtp session ids in murder env.
Hi! lmtpd returns a session ID with his "250 ok SESSIONID=". This includes the murders frontend sessionid. But this sessionid is never written to syslog by lmtpd on frontends. Only the backends log their sessionid when the frontend logs in. Even if the frontend logs its own session id, there wont be a easily grep'ed connection to the backends sessionid. And here we have the second drawback for murders. Frontends tend to keep lmtp sessions alive pretty long and backend sessionids never change until disconnected. So, currently these sessionids don't help as much as expected in murder environment. On a single host they're very helpful. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: lmtp session ids in murder env.
On 2011-11-11 23:43, Bron Gondwana wrote: > Do you want to write up a patch for what you think is best? My first > thought is for the frontend to log a backend to frontend linking > record... unless you can push the backend SESSIONID back in the > frontend response somehow. Sure, I'll try my best;-) Should be possible in the next weeks. Maybe a backend should generate a new session-id on RSET? But I'll take a look to the code first to check for possible solutions. Greetings, Wolfgang PS: May I point you to https://bugzilla.cyrusimap.org/show_bug.cgi?id=3563 ...this one really hurts and my fix works pretty well here. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: lmtp session ids in murder env.
Hi! On 2011-11-11 23:43, Bron Gondwana wrote: > Do you want to write up a patch for what you think is best? My first > thought is for the frontend to log a backend to frontend linking > record... unless you can push the backend SESSIONID back in the > frontend response somehow. I attached my proposed patch to https://bugzilla.cyrusimap.org/show_bug.cgi?id=3598 Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: mailbox referrals in 2.4.12
Andrew Morgan wrote, on 14.12.2011 01:15: > I worked up a simple patch against 2.4.12 which seems to work in my test > environment. See the attachment. Wouldn't it be cleaner to remove the whole option "proxyd_disable_mailbox_referrals" and use suppress_capabilities: MAILBOX-REFERRALS on the proxy servers instead? Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: mailbox referrals in 2.4.12
Andrew Morgan wrote, on 14.12.2011 19:38:
> In that case, I should keep the original behavior in cmd_list. Here is an
> updated patch that still honors disable_referrals when setting
> supports_referrals.
Your patch still removes the disable_referrals from section 2819ff.
And I still suggest using suppress_capabilites instead of waking the zombi...
patch attached.
Greetings, Wolfgang
--
Wolfgang Breyha | http://www.blafasel.at/
Vienna University Computer Center | Austria
diff -urN cyrus-imapd-2.4.12.orig/imap/imapd.c cyrus-imapd-2.4.12/imap/imapd.c
--- cyrus-imapd-2.4.12.orig/imap/imapd.c 2011-10-04 21:53:03.0 +0200
+++ cyrus-imapd-2.4.12/imap/imapd.c 2011-12-15 13:22:33.0 +0100
@@ -5979,7 +5979,9 @@
clock_t start = clock();
char mytime[100];
-if (listargs->sel & LIST_SEL_REMOTE) supports_referrals = !disable_referrals;
+if ((listargs->sel & LIST_SEL_REMOTE) &&
+ (!capa_is_disabled("MAILBOX-REFERRALS")))
+ supports_referrals = !disable_referrals;
list_callback_calls = 0;
diff -urN cyrus-imapd-2.4.12.orig/lib/imapoptions cyrus-imapd-2.4.12/lib/imapoptions
--- cyrus-imapd-2.4.12.orig/lib/imapoptions 2011-10-04 21:53:03.0 +0200
+++ cyrus-imapd-2.4.12/lib/imapoptions 2011-12-15 13:26:57.0 +0100
@@ -968,8 +968,9 @@
authentication load on the respective backend server. */
{ "proxyd_disable_mailbox_referrals", 0, SWITCH }
-/* Set to true to disable the use of mailbox-referrals on the
- proxy servers. */
+/* Option removed in 2.4.13. Use
+ suppress_capabilities: MAILBOX-REFERRALS
+ instead */
{ "proxyservers", NULL, STRING }
/* A list of users and groups that are allowed to proxy for other
diff -urN cyrus-imapd-2.4.12.orig/lib/imapopts.c cyrus-imapd-2.4.12/lib/imapopts.c
--- cyrus-imapd-2.4.12.orig/lib/imapopts.c 2011-10-04 22:03:49.0 +0200
+++ cyrus-imapd-2.4.12/lib/imapopts.c 2011-12-15 13:24:06.0 +0100
@@ -547,9 +547,6 @@
{ IMAPOPT_PROXYD_ALLOW_STATUS_REFERRAL, "proxyd_allow_status_referral", 0, OPT_SWITCH,
{(void*)0},
{ { NULL, IMAP_ENUM_ZERO } } },
- { IMAPOPT_PROXYD_DISABLE_MAILBOX_REFERRALS, "proxyd_disable_mailbox_referrals", 0, OPT_SWITCH,
-{(void*)0},
-{ { NULL, IMAP_ENUM_ZERO } } },
{ IMAPOPT_PROXYSERVERS, "proxyservers", 0, OPT_STRING,
{(void *)(NULL)},
{ { NULL, IMAP_ENUM_ZERO } } },
diff -urN cyrus-imapd-2.4.12.orig/lib/imapopts.h cyrus-imapd-2.4.12/lib/imapopts.h
--- cyrus-imapd-2.4.12.orig/lib/imapopts.h 2011-10-04 22:03:49.0 +0200
+++ cyrus-imapd-2.4.12/lib/imapopts.h 2011-12-15 13:23:34.0 +0100
@@ -173,7 +173,6 @@
IMAPOPT_PROXY_PASSWORD,
IMAPOPT_PROXY_REALM,
IMAPOPT_PROXYD_ALLOW_STATUS_REFERRAL,
- IMAPOPT_PROXYD_DISABLE_MAILBOX_REFERRALS,
IMAPOPT_PROXYSERVERS,
IMAPOPT_PTS_MODULE,
IMAPOPT_PTLOADER_SOCK,
diff -urN cyrus-imapd-2.4.12.orig/man/imapd.conf.5 cyrus-imapd-2.4.12/man/imapd.conf.5
--- cyrus-imapd-2.4.12.orig/man/imapd.conf.5 2011-10-04 22:03:49.0 +0200
+++ cyrus-imapd-2.4.12/man/imapd.conf.5 2011-12-15 13:27:43.0 +0100
@@ -820,8 +820,9 @@
connections that these referrals would cause, thus resulting in a higher
authentication load on the respective backend server.
.IP "\fBproxyd_disable_mailbox_referrals:\fR 0" 5
-Set to true to disable the use of mailbox-referrals on the
-proxy servers.
+Removed in 2.4.13. Use
+suppress_capabilities: MAILBOX-REFERRALS
+instead.
.IP "\fBproxyservers:\fR " 5
A list of users and groups that are allowed to proxy for other
users, separated by spaces. Any user listed in this will be
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: mailbox referrals in 2.4.12
Wolfgang Breyha wrote, on 15.12.2011 13:42: > Your patch still removes the disable_referrals from section 2819ff. > > And I still suggest using suppress_capabilites instead of waking the zombi... > patch attached. Filed a bug... https://bugzilla.cyrusimap.org/show_bug.cgi?id=3615 Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Cyrus 2.4.13beta1 Released
On 2011-12-24 01:15, Bron Gondwana wrote: > Please send any feedback to the cyrus-devel mailing list. > I'm hoping to have a stable 2.4.13 out by the end of the > year. Bron, please! https://bugzilla.cyrusimap.org/show_bug.cgi?id=3598 Merry Xmas, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Cyrus 2.4.13beta1 Released
On 2011-12-28 01:59, Andrew Morgan wrote: > A quick read of the code makes me think that this only crashes a backend > imapd process that is performing the mailbox move between partitions. Is > that correct? That is correct. > Does this have any effect on the mupdate master (running 2.4.12)? mupdate master wont crash if you mean that. But you end up with different mailbox entries on backend and mupdate. "ctl_mboxlist -m" is needed to get it in sync again. Even more repairs are required on the backend to fix the moved mailbox IIRC. And ACLs need to be checked. I'm not sure anymore what exactly had to be done to repair it completely. But it was PITA enough to fix it fast;-) Don't know why Bron missed my report/fix for 2.4.11 since it was already in bugzilla ready for 2.4.12 release. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Cyrus 2.4.13beta1 Released
On 2011-12-28 14:07, Bron Gondwana wrote: > Sorry - I have probably missed a few things along the way! > > Is there anything still outstanding to add now? Except the lmtp session-ids, no;-) > Are you happy with the fix that I'm proposing to run with (only push > the MUPDATE from the new backend after the DUMP, so that the new > backend chooses the servername)? Oh, sorry. I mixed bugs. I was referring to the SEGV on partition move on the same backend (#3563). Andrew referred to sync issues. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: migration and seen flag
Bron Gondwana wrote, on 06.01.2012 17:53: > Seen data is only stored in .seen files for shared or other user > mailboxes. The seen data for the mailbox owner is stored in the > cyrus.index. This has two benefits: Hmmm, I'm running 2.4.12/13 on my backends now. All mailboxes were migrated from 2.3.16 using murder rename. I've seenstate_db: skiplist in my imapd.conf and I see .seen skiplist files in $CONFIGPATH/user//$uid.seen as always. And this for mailboxes which are not shared. Only the user himself has access rights. Is it the seenstate_db setting overruling your general statement above? man page says that skiplist is default for seenstate_db? Or are these leftovers which get removed with the completely undocumented cyr_userseen tool? -d does it for real then looking at the source, right? Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Set quota "none" using Cyrus::IMAP::Admin
Bron Gondwana wrote, on 17.01.2012 11:36: > On Tue, Jan 17, 2012, at 03:57 PM, Ram wrote: >> http://www.manpagez.com/man/3/Cyrus::IMAP::Admin/ >> There is a setquota function which should accept "none" for removing >> quota. But that does not work >> >> How do I set unlimited quota using the setquota function > > Does '-1' work? Setting an "empty" quota works: use Cyrus::IMAP::Admin; print "setting some quota\n"; my $quotaroot = "user.mailboxid"; my @quotaargv = ($quotaroot, "STORAGE", 500); $res = $backend->setquota(@quotaargv); print "setting \"unlimited\"\n"; @quotaargv = ($quotaroot); $res = $backend->setquota(@quotaargv); Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Set quota "none" using Cyrus::IMAP::Admin
Ram wrote, on 17.01.2012 12:03: > That means we omit the word "STORAGE" , thanks .. will try that. Yes. I'm always setting "some" quota before to get it into a "defined" state to remove it afterwards. I use it for several years now and recently moved ~100k users without any troubles. Maybe you have to check/fix your quotaroots before, too. I had to do that in the past migrating from older cyrus releases. Maybe it's not necessary anymore. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: [SOLVED] Re: Set quota "none" using Cyrus::IMAP::Admin
Ram wrote, on 17.01.2012 12:17:
> I am using cyrus-imapd-2.4.12-2
> As Wolfgang suggested using
> $imap->setquota("user/$user") removed the quota.
Looking at the code of Cyrus::IMAP::Admin there is no code checking for "none"
or something like that. There are two ways to "craft" a proper IMAP command.
First the way I described setting no STORAGE argument at all resulting in:
SETQUOTA id ()
to remove quota. And second setting
->setquota(<-id->, "STORAGE", "-1")
resulting in
SETQUOTA id (STORAGE -1)
what Bron suggested. That should also work IMO. At least on 2.4.12+.
The man page of Cyrus::IMAP::Admin does not describe how to remove quota at
all. Neither "none" nor anything else. Nothing wrong in that part so far.
Greetings, Wolfgang
--
Wolfgang Breyha | http://www.blafasel.at/
Vienna University Computer Center | Austria
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: [SOLVED] Re: Set quota "none" using Cyrus::IMAP::Admin
On 2012-01-18 14:18, Greg Banks wrote: >> And second setting >> ->setquota(<-id->, "STORAGE", "-1") >> resulting in >> SETQUOTA id (STORAGE -1) >> what Bron suggested. That should also work IMO. At least on 2.4.12+. > > And that isn't correct according to RFC2087. It works only because the > server is deliberately not enforcing syntax correctly, to workaround a bug in > a > different part of the code. Yes, it's wanted behaviour;-) cyrus uses quota -1 if a quotaroot is set, but quota is unlimited. If a mailbox is xfer'ed cyrus uses it itself after some bugfixes when dumping the mailbox. eg. https://bugzilla.cyrusimap.org/show_bug.cgi?id=3188 and https://bugzilla.cyrusimap.org/show_bug.cgi?id=3559 In 3559 Bron said, that maybe 2.5 will handle that differently. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Cyrus 2.4.13 memory use
David Carter wrote, on 08.02.2012 17:19: > It looks like 3000 IMAP sessions are going to take around 8 GBytes of RAM > just to run, and we will need to buy additional RAM for buffer cache. This > isn't the end of the world: memory is cheap. I'm just curious if anyone > else saw a similar increase when upgrading from 2.3 to 2.4. Short answer. Yes, I did;-) 2.3 on RHEL5 32bit => 2.4 on RHEL6 64bit and memory usage is more then doubled. My two old frontends had 4GB (completely exhausted, some swap) each, now I'm using three VMs with 10 (6.5 used). Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: "body" sieve extension
On 2012-02-17 20:55, Fabio S. Schmidt wrote: > Hi ! > > Does Cyrus imap 2.4.13 support the "body" sieve extension? I'm trying to > use it but fails with this message: Sure, if you tell him to: man imapd.conf sieve_extensions: ... body is not supported by default. Greetings, Wolfgang Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Another 2.4 upgrade horror story
Sebastian Hagedorn wrote, on 25.09.2012 14:01: > I realize that some of our problems were caused by infrastructure that's > not up to current standards, but nonetheless I would really urge you to > never again use an upgrade mechanism like that. Give admins a chance to > upgrade indexes in the background and over time. There is such an upgrade path using a murder environment and moving mailboxes between backends. We used that for our 150k user infrastructure and had no IO headaches at all. It was a good moment to update distribution, filesystems, hardware, as well. I tested conversion speed from 2.3 to 2.4 on about 100 mailboxes and it was pretty obvious that touching all our mailboxes in one shot is clearly impossible without unreasonable downtime. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Another 2.4 upgrade horror story
Janne Peltonen wrote, on 25.09.2012 15:34: > Could you elaborate on that? I considered that option, but seeing as moving > even a couple dozen users from a backend to another using RENAME takes hours > and one backend contains thousands of users, I decided to just live with the > ~1 > day of unbearable slowness. Or do you know of a fast way? Time was not the limiting factor for us. Availability and safety of our mailboxes was. Nobody can guarantee you that the migration works out flawlessly. If moving one mailbox fails I have troubles with exactly one mailbox. If reconstruction of one mailbox fails while migrating hard from 2.3 to 2.4 your system is down even longer. And we had a couple of problematic mailboxes as partly documented on this mailinglist;-) We had 5 backend stores and have 6 now. Complete migration of our 150k users with ~22TB took about 3 or 4 month. Including building the machines, requesting storage, etc... But you can't have less headache then moving mailboxes from backend to backend. In detail I *) check for active sessions on the mailbox *) lock the mailbox by denying access via userdeny.db *) move the mailbox *) unlock it Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Another 2.4 upgrade horror story
On 2012-09-25 19:05, Simon Beale wrote: > The only gotcha I experienced was I forgot that cyrus was configured to > hardlink mail, which of course was no longer the case after each mailbox > was migrated, so my disk usage exploded. (But easily fixed/restored once > identified). What did you use for restoring the hardlinks? freedup as well? I'm asking because I found a bug in freedup causing dataloss. I already sent a patch fixing it to the author of freedup last november, but he didn't release a new version yet. In case cyr_expire is running while freedup tries to hardlink files it is possible to loose both the source freedup wants to link to and the copy freedup still removes on error. Running cyr_expire and freedup (up to 1.6-2) together is a really bad idea. If it's of interest I can provide my patch here, too. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Another 2.4 upgrade horror story
Simon Matter wrote, on 26.09.2012 06:58:
> I have not used freedup for restoring the hardlinks but I'm interested in
> the patch. If it's not big could you post it here?
Sure! attached.
I built a RPM based on
http://pkgs.repoforge.org/freedup/freedup-1.5.3-1.rf.src.rpm
and latest source.
But building based on
http://www.freedup.org/freedup-1.6-2.src.rpm
should work as well.
Greetings, Wolfgang
--
Wolfgang Breyha | http://www.blafasel.at/
Vienna University Computer Center | Austria
--- freedup-1.6/freedup.c.orig 2011-02-04 08:22:15.0 +0100
+++ freedup-1.6/freedup.c 2011-11-11 10:52:24.788733835 +0100
@@ -613,7 +613,7 @@
*/
if( mktemp(tmpfilename) == NULL )
{
- perror("There is no unique temporory file name.");
+ perror("There is no unique temporary file name.");
}
if( dirmtime!=0 )
{
@@ -628,7 +628,7 @@
if( lstat(tmpfilename,&tstat) != 0 )
{
/*
- * The errror needs not to be catched, since it is wanted
+ * The error needs not to be catched, since it is wanted
* that no file exists with the target name
*/
rename( bname, tmpfilename );
@@ -643,12 +643,23 @@
}
if( lnk( symaname, bname ) != 0 )
{
- perror("Linking failed.");
- }
- if( unlink( tmpfilename ) != 0 )
- {
- perror("Unlink failed.");
- }
+ // linking failed! try to move original in place again and
+ // log that fact
+ fprintf(stderr, "Linking failed. Trying roleback: \"%s\"", bname);
+ if ( rename( tmpfilename, bname ) != 0 )
+ {
+ // moving old file in place again failed.
+ // at least log that -v
+ fprintf(stderr, "unable to rename: \"%s\"", tmpfilename);
+ }
+ }
+ else
+ // unlink renamed original only of linking was successful
+ if( unlink( tmpfilename ) != 0 )
+ {
+ // unlinking failed! log that -v
+ fprintf(stderr, "Unlink failed: \"%s\"", tmpfilename);
+ }
if( (dirmtime!=0) & (gotdirtime!=0) )
{
utimecache.actime = dstat.st_atime;
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: How to get rid of reserved mailboxes?
Hi!
Frank Richter wrote, on 21.12.2012 12:00:
> In this process we've got one "reserved mailbox" in a backend's mailboxes.db:
> user.xyz.bla 2 j xyzlrswipkxtecda
I would try to stop cyrus on the backend, then look what
ctl_mboxlist -mw
says. If that looks sane and only talks about the mentioned mailbox then try
ctl_mboxlist -m
to fix it. If another "-mw" is silent then restart cyrus.
Do not try ctl_mboxlist while cyrus is running!
Usually "ctl_mboxlist -m" is executed in cyrus.conf on startup on a backend.
Do you have this entry?
START {
mupdatepush cmd="ctl_mboxlist -m"
}
Greetings, Wolfgang
--
Wolfgang Breyha | http://www.blafasel.at/
Vienna University Computer Center | Austria
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
last subscription entry lost after migration
Hi! We did our migration some time ago from 2.3.16 to 2.4.x (I think it was 2.4.12 back then). We recognized now that lots of users (~50k of our 100k userbase) lost their subscription to their last entry, which in most cases was "INBOX.Trash". The piled up junk in all these "unseen" but existing Trash-Folders was the main cause we recognized it at all. I'm currently checking the source of both versions to get an idea what happend while the mailboxes moved from one backend to the other, but have no clue yet. Our subscriptions_db format was flat on 2.3 and still is flat. Has somebody else seen such behaviour? Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: last subscription entry lost after migration
Wolfgang Breyha wrote, on 08.01.2013 15:53: > I'm currently checking the source of both versions to get an idea what happend > while the mailboxes moved from one backend to the other, but have no clue yet. > > Our subscriptions_db format was flat on 2.3 and still is flat. Looking at the list of affected users it seems that another step is involved. We already migrated from 2.2 to 2.3 some years ago. Most of the affected users had our standard subscriptions (Drafts, Sent, Templates, Trash) and never changed it. The *.sub files moved from 2.2 to 2.3 and never got changed. 2.4 didn't like the last line then as it seems. Maybe a missing newline from 2.2? Still checking... Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Disable client authentication with certificates
On 2013-12-07 10:08, Stefan Gofferje wrote:
> On 12/03/2013 09:28 PM, Stefan Gofferje wrote:
>> So why does Thunderbird ask me which certificate to use for
>> authentication? Does my Cyrus ask for a client certificate or does it
>> not? ^^
>
> Nobody a clue?
It depends. On IMAPS/POP3S ports cyrus never asks for client certs. On the
other hand it always asks for them doing STARTTLS. That's caused by the way
tls_init_serverengine() is called in imapd.c and pop3d.c.
You can either connect to ports 993/995 to prevent the use of client certs
or you can completely disable client certs with this patch (still works on
2.4.17):
# cat SOURCES/cyrus-imapd-2.3.14-disable_clientcerts.patch
diff -urN cyrus-imapd-2.3.14.orig/imap/imapd.c cyrus-imapd-2.3.14/imap/imapd.c
--- cyrus-imapd-2.3.14.orig/imap/imapd.c2009-05-06 14:05:17.0
+0200
+++ cyrus-imapd-2.3.14/imap/imapd.c 2009-05-06 14:09:48.0 +0200
@@ -6764,7 +6764,7 @@
result=tls_init_serverengine("imap",
5,/* depth to verify */
-!imaps, /* can client auth? */
+0,/* can client auth? */
!imaps); /* TLS only? */
if (result == -1) {
diff -urN cyrus-imapd-2.3.14.orig/imap/pop3d.c cyrus-imapd-2.3.14/imap/pop3d.c
--- cyrus-imapd-2.3.14.orig/imap/pop3d.c2009-05-06 14:05:17.0
+0200
+++ cyrus-imapd-2.3.14/imap/pop3d.c 2009-05-06 14:10:05.0 +0200
@@ -1077,7 +1077,7 @@
result=tls_init_serverengine("pop3",
5,/* depth to verify */
-!pop3s, /* can client auth? */
+0,/* can client auth? */
!pop3s); /* TLS only? */
if (result == -1) {
-------
Greetings, Wolfgang
--
Wolfgang Breyha | http://www.blafasel.at/
Vienna University Computer Center | Austria
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Disable client authentication with certificates
On 2013-12-09 16:59, Stefan Gofferje wrote: > That worked fine :). Thanks. Is it planned to integrate your patch into > Cyrus? It *is* kinda illogical to ask for a client cert when client cert > authentication is explicitly disabled ^^. My patch is not suitable for general use. IMO client cert requests should either depend on a new option or on the availability of configured CAs. Both is possible, but I'm not aware of the reason why client certs are requested historically. I would simply make it dependent of CA availability. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Disable client authentication with certificates
On 2013-12-09 18:10, Wolfgang Breyha wrote: > I would simply make it dependent of CA availability. proposed patch for that: https://bugzilla.cyrusimap.org/show_bug.cgi?id=3830 Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
bugzilla HTTPS broken
Hi! http://bugzilla.cyrusimap.org/ is redirected to HTTPS. That's good. But then the site is completely broken, since all links still lead to HTTP and firefox rejects to load CSS, JS, ... until explicitly unblocking unencrypted content. Please either allow HTTP again, or fix HTTPS. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Disable client authentication with certificates
Stefan Gofferje wrote, on 10.12.2013 08:17: > There are options? > > tls_require_cert: false > tls_imap_require_cert: false > tls_pop3_require_cert: false > tls_lmtp_require_cert: false > tls_sieve_require_cert: false > > Why ask for a cert when the config says it's not needed? Or do I see > this too naive? cyrus distinguishes between asking for a cert and requiring a cert. I don't know why, sorry. Sometimes it is practical to ask for a cert and only try to verify it without enforcing it. But asking for certs while incapable to verify them (without CAs) seems odd. That's why I decided to do it that way. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Disable client authentication with certificates
Stefan Gofferje wrote, on 10.12.2013 16:33: > Maybe the existing options could just be extended, like in the Postfix > setting for TLS, e.g. > > tls_imap_require_cert: no|ask|require Changing the way how existing options work and breaking compatibility to existing configurations is most likely not the best idea;-) > I think, having logical options which are clear to the admin are better > than some implicit consequences which are not not bilaterally logical. > I don't know if I express this right/understandable :). Adding a new option is easy. Done in 30 minutes. I can do it if a official dev says it makes sense and will be added... but I don't get any answers from Bron for very long time now. So I'm pushing the patches I use myself locally to bugzilla waiting for a response. > The background is that a bunch of TLS tutorials on the web include > configuring the CA but not explaining in detail why, so an inexperienced > admin could assume that he should put the CA certificate for the server > cert's CA there. TLS tutorials for cyrus-imapd including tls_ca_path/file by default? Most likely to get rid of the debug warnings. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Moving mailboxes in Cyrus Murder leaves mails behind on source server
Hi! Peter Bücker wrote, on 23/07/13 16:33: > Hi all, > > We're having an issue with our Cyrus Murder environment when moving > mailboxes from one backend to another backend. Some mails are left > behind on the source server while we think they should be removed. We > issue the rename by using "cyradm" on the source server with the > following command: > > oldserver> rename user.foo user.foo newserver!default I came across your pretty old, but unanswered posting because I searched for troubles with... > disconnect_on_vanished_mailbox: 1 ... which causes the same troubles as you see on your murder. We also see these troubles with 2.4.17 if a user or admin renames or "moves" a mailbox while keeping a lock to this mailbox in another connection. If I got it right mailbox.c:mailbox_delete() marks the mailbox as deleted in the mailboxes.db and "waits" until mailbox_close() is called with the last lock which closes and removes the mailbox then. This failes IMO in the case disconnect_on_vanished_mailbox == true because imapd errors out with a fatal() call without calling mailbox_close(). If this connection held the very last lock on this mailbox it ends up without being removed from filesystem. For all the leftovers I see in the filesystem I find a Mailbox user..xx has been (re)moved out from under client in the logs. Most likely it's recommended not to use disconnect_on_vanished_mailbox in the current implementation. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Patch for adding tls_honor_cipher_order
Kristian Kræmmer Nielsen wrote on 17/10/14 14:48: > "already" - I see you just added it ;-) No, they (and more) were there for month as https://bugzilla.cyrusimap.org/show_bug.cgi?id=3822 https://bugzilla.cyrusimap.org/show_bug.cgi?id=3823 https://bugzilla.cyrusimap.org/show_bug.cgi?id=3830 https://bugzilla.cyrusimap.org/show_bug.cgi?id=3861 Regards, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Patch for adding tls_honor_cipher_order
Kristian Kræmmer Nielsen wrote on 17/10/14 15:13:
> The more important part of my previous mail are that there are issues with
> the patches that now have been merged into git. E.g. compression is not
> merged correctly and it is recommended to do negative list and not
> positive lists of protocols.
Yes, you're right. The patches in master tree have broken logic...
Option documentation says:
tls_versions: ssl2 ssl3 tls1_0 tls1_1 tls1_2
Disable SSL/TLS protocols not in this list.
Code says:
+ if (strstr(tls_versions, "tls1_2") == NULL) {
+#if (OPENSSL_VERSION_NUMBER >= 0x1000105fL)
+ off |= SSL_OP_NO_TLSv1_2;
+#else
+ syslog(LOG_ERR, "ERROR: TLSv1.2 configured, OpenSSL < 1.0.1e insufficient");
+#endif
+ }
Setting the NO_TLSv1_2 option does the opposite of the expected/wanted
behavior. I also would prefer a negative list as most other daemons like
apache, exim, ... use. Maybe a more generic
tls_openssl_options: no_ssl2 no_ssl3 no_compression prefer_server_cipher_order
would be better?
And yes, you're also right with mentioning that functionality is missing.
tls_compression: 0
Enable TLS compression. Disabled by default.
tls_eccurve: prime256v1
Select the elliptic curve used for ECDHE.
description is there, but there is no code doing it actually. Support for ECDH
auto mode in Openssl 1.2+ as provided in
https://bugzilla.cyrusimap.org/attachment.cgi?id=1535
is missing in the documentation as well.
I think this should be fixed/enhanced for a alpha release of 2.5.
Greetings, Wolfgang
--
Wolfgang Breyha | http://www.blafasel.at/
Vienna University Computer Center | Austria
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cyrus 2.4.17 TLS woes
Patrick Goetz wrote on 15/01/15 13:34: > Does anyone have a secure, functional cipher list entry they'd like to > share? Maybe https://bettercrypto.org/ is of help. This document includes not only cyrus-imapd. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
2.5.0 make install fails with enable-http
Hi! I tried to build a new 2.5.0 release on CentOS 7 from source. I used: ./configure --with-extraident=xx-1 --with-syslogfacility=MAIL --disable-static --enable-idled --enable-murder --enable-http --enable-replication --with-snmp --without-bdb --disable-gssapi configure did not detect the missing glib2-devel package needed by vzic. make did at the last steps;-) make install fails with > Making install in tools/vzic/ > make[2]: Entering directory > `/opt/cyrus-imapd/src/cyrus-imapd-2.5.0/tools/vzic' > make[2]: *** No rule to make target `install'. Stop. > make[2]: Leaving directory `/opt/cyrus-imapd/src/cyrus-imapd-2.5.0/tools/vzic' > make[1]: *** [install-recursive] Error 1 > make[1]: Leaving directory `/opt/cyrus-imapd/src/cyrus-imapd-2.5.0' > make: *** [install] Error 2 Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
cyrus 2.5 with myroundcube calendar
Hi! I tried to access the cyrus 2.5 calendar with myroundcube calendar and failed. cyrus-imapd 2.5.1 is running and works as expected. Calendar is accessible with Thunderbird/Lightning following http://cyrusimap.org/docs/cyrus-imapd/2.5.0/install-http.php Nothing "unusual" in the configs. https, noplaintext auth, digest disabled for http. I can add/edit/remove events with lightning as expected using https:///dav/calendars/user//Default/ Then I tried adding myroundcube calendar to my existing and working roundcube installation. I added a caldav calandar with exactly same URL, but without trailing "/" to myrc cal. It works mostly, except that myrc calendar says: > May 4 01:38:28 xxx roundcube: <435cti4v> caldav_driver: Initialize sync > client for calendar 3 > May 4 01:38:28 xxx roundcube: <435cti4v> caldav_driver: Syncing calendar id > "3". > May 4 01:38:28 xxx roundcube: <435cti4v> caldav_driver: Found new event > /dav/calendars/user//Default/ > May 4 01:38:28 xxx roundcube: <435cti4v> caldav_driver: Found new event > /dav/calendars/user//Default/85eaa723-327a-4cf5-8e20-bb04aa705829.ics > May 4 01:38:28 xxx roundcube: <435cti4v> caldav_driver: Found new event > /dav/calendars/user//Default/B79D296A8C9CC4B7B7794FADD7B57217-C2177D4A10BA4D17.ics > May 4 01:38:28 xxx roundcube: <435cti4v> caldav_driver: Created 0 new > events, updated 0 event. > May 4 01:38:28 xxx roundcube: <435cti4v> caldav_driver: Successfully synced > calendar id "3". So, it actually finds the events, but doesn't add them to the rc calendar afterwards. Don't know why. Without trailing "/" in the caldav URI it even works to add an event, roundcube saves it, I get it in thunderbird, but roundcube itself doesn't display it. I'm not sure if this is roundcube related or if cyrus-httpd is the cause. I added unencrypted access for debugging purposes and tcpdumped the access. What I see is that rc cal. asks for all 3(!) "events", including /dav/calendars/user//Default/ and gets a "404 not found" in the multi-status response. That's the point were I don't know if cyrus or rc calendar is doing something wrong. Maybe it's caused by something completely differnt? Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: RFC6154 and upgrade to Cyrus 2.5
Marco wrote on 09/10/15 08:58:
> Hello,
>
> on Cyrus 2.4 I see extenstion for xlist folder flags. Folders and
> flags must be configured on imapd.conf. Great, it works! Flags are
> magically applied to all folders!
>
> If I upgrade to Cyrus 2.5 or upper, what does it happen? I see that
> Cyrus 2.5 also support "create special use", but I can't see no more
> xlist-* configs. It seems that starting from Cyrus>=2.5 only client
> can set flags during CREATE. But what does it happen to previously
> Cyrus2.4 folder-flag associations?
With a little help from Bron I came to the following solution...
2.5 does not have xlist- and the only way to "migrate" it is to set it as
annotation to every mailbox.
It is possible to set it by connecting to the backend/server as admin user
authorized as the individual user and push eg.
. SETANNOTATION INBOX.Trash "/specialuse" ("value.priv" "\Trash")
commands. Maybe I get this to work over my 2.4 frontends as well.
On 2.5 the new SETMETADATA is available as well...
. SETMETADATA INBOX.Sent ("/private/specialuse" "Sent")
Greetings, Wolfgang
--
Wolfgang Breyha | http://www.blafasel.at/
Vienna University Computer Center | Austria
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: plaintextloginpause affects all POP3 connections
On 24/09/15 22:57, Michael D. Sofka wrote:
> The plaintextloginpause affects all POP3 connections, even those that
> are encrypted via pop3s or STARTTLS. Not that this is necessarily a
> bad thing But is it expected behaviour?
True for 2.4.18 and 2.5.6. Looking at the code in pop3d.c it is expected
behaviour. Looking at imapd.c as well it seems it should not affect TLS
protected connections.
I patched it a while ago for my local RPMs:
> --- pop3d.c.orig 2014-09-15 15:33:35.993293154 +0200
> +++ pop3d.c 2015-07-06 16:32:15.281275940 +0200
> @@ -1497,7 +1508,8 @@
> popd_userid, popd_subfolder ? popd_subfolder : "",
> popd_starttls_done ? "+TLS" : "", "User logged in",
> session_id());
>
> - if ((plaintextloginpause = config_getint(IMAPOPT_PLAINTEXTLOGINPAUSE))
> + if ((!popd_starttls_done) &&
> +(plaintextloginpause =
> config_getint(IMAPOPT_PLAINTEXTLOGINPAUSE))
> != 0) {
> sleep(plaintextloginpause);
> }
Greetings, Wolfgang
--
Wolfgang Breyha | http://www.blafasel.at/
Vienna University Computer Center | Austria
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Problems with cyrus 2.5.10 after system update
Patrick Goetz wrote on 17/05/17 16:40: > SSL alert number 40 This error has nothing to do with SSLv3 or protocol version at all. Maybe your private key and certificate do not match on server side. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
cyr_dbtool repack
Hi! Is it safe to use cyr_dbtool (skiplist|twoskip) repack while cyrus is running? Or do I have to shut it down while repacking the databases? mycheckpoint() in both cyrus_skiplist.c and cyrus_twoskip.c seems to do locking and transaction checking as it seems. Still I'm not sure if it's safe to use while active. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cyr_dbtool repack
Wolfgang Breyha wrote on 17/10/17 12:22: > Hi! > > Is it safe to use > cyr_dbtool (skiplist|twoskip) repack > while cyrus is running? Or do I have to shut it down while repacking the > databases? mycheckpoint() in both cyrus_skiplist.c and cyrus_twoskip.c seems > to do locking and transaction checking as it seems. Still I'm not sure if it's > safe to use while active. Sorry, forgot to mention... I'm running 2.5.x. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
repairing a calendar
Hi! I'm currently debugging some oddities with a users default calendar after the mailbox got XFER'ed to a different murder backend. The whole setup uses 2.5.11 currently. After lightning started to fail the user removed the calendar and added it again. lightning now tries to fetch the whole calendar (about 100 entries). I see the REPORT request which gets a Multi-Status response with all ics URIs and 200 OK. After that I see the calendar-multiget request for all these URIs. But this request gets a HTTP/1.1 404 Not Found response. I assume that at least one of the URIs fails. strace does not give any clues which one. The session trace from cyrus doesn't help as well. dumping the .dav sqlite DB it seems that there are at least 2 entries (the latest ones after the XFER) which are not in the DB table and only in the filesystem. The timestamp of the .dav DB is exactly from the XFER and not from the newer entries found in the calendar folder. Currently I'm searching my logs if there is something odd after the XFER especially while the new entries got added. While doing that the general questions raised how to repair a calendar at all, since reconstruct does not do it as it seems? At least the .dav sqlite database is not rebuilt. Removing the .dav file does no good as well. I get an empty one after the next access to the calendar. But it is still not accessible/syncable. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: repairing a calendar
On 20/12/17 17:35, Ken Murchison wrote: > Try: > > dav_reconstruct Oh sorry, didn't find that one... but doing so removes the .dav file without generating a new one. So I assume that something happend XFERing that mailbox destroying important information about DAV mailboxes. strace shows that dav_reconstruct unlinks the DB and scans mailboxes.db afterwards. And then it ends without further steps. Looking at a ctl_mboxlist dump... $ ctl_mboxlist -d|grep user. user..#addressbooks 0 part18 lrswipkxtecdan user..#addressbooks.Default 0 part18 lrswipkxtecdan user..#calendars 0 part18 lrswipkxtecdan user..#calendars.Default 0 part18 lrswipkxtecdan user..#calendars.Inbox 0 part18 lrswipkxtecdan user..#calendars.Outbox 0 part18 lrswipkxtecdan compared to a working one... user..#addressbooks64 part06 lrswipkxtecdan user..#addressbooks.Default64 part06 lrswipkxtecdan user..#calendars 32 part06 lrswipkxtecdan user..#calendars.Default 32 part06 lrswipkxtecdan user..#calendars.Inbox 32 part06 lrswipkxtecdan user..#calendars.Outbox32 part06 lrswipkxtecdan I guess there are missing important flags on those DAV folders. Most likely "lost in translation^WXFER". Should I file a bug? BTW(ab)using dav_reconstruct with user. instead of causes a SIGSEGV. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: repairing a calendar
On 20/12/17 20:57, Ken Murchison wrote: > The mailbox types for the broken user's DAV mailboxes are wrong. Type 0 > (zero) says its a regular email mailbox. You'll have to change them to 32 > for calendars and 64 for addressbooks. Ok, fixed it with ctl_mboxlist, but $ dav_reconstruct Reconstructing DAV DB for ... Inserting DAV DB entries for user..#addressbooks... Inserting DAV DB entries for user..#addressbooks.Default... Inserting DAV DB entries for user..#calendars... Inserting DAV DB entries for user..#calendars.Default... Segmentation fault ...at this point I'll move this thread to github issue tracker. It seems both XFER and dav_reconstruct are broken an 2.5.11. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
INBOX.INBOX.
Hi! I'm not sure if this is a bug or intended... cyrus checks for mailbox names including "INBOX.INBOX." since 2.5 and prevents creation to stop buggy clients like Apple Mail from recursing. But it still allows eg. INBOX.INBOX.Trash since the check is done after conversion to the internal user format. INBOX.INBOX.(Trash|Sent|Drafts) is widely used by K9 (and others) as it seems. K9 fixed it "recently" ... https://github.com/k9mail/k-9/issues/701 ... but it seems existing configs still use this pattern. Does cyrus intentionally allow user..INBOX. and only prevents user..INBOX.INBOX or is this a bug and will be changed in future versions? With kind regards, Wolfgang Breyha -- Wolfgang Breyha | https://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Help! Cyrus 2.5.11 segmentation fault
Hi! Sorry for the late answer... most likely you already fixed your issue... On 16/07/18 10:43, Marco Chesi wrote: > Hello, > we have a Cyrus 2.5.11 on Debian 7 using murder (2 frontends, 3 backends, 1 > mupdate, about 5000 mailboxes) in production environment, hosted by a > VMware cluster 6.5. > > Suddenly, ALL mailboxes have becomed inaccessible. > > In the log, we found many messages like this: > > master[5965]: process type:SERVICE name:imaps path:/usr/cyrus/bin/proxyd > age:82.255s pid:5987 signaled to death by signal 11 (Segmentation fault) > > I tried many times to restart the cyrus service on all servers without > success. > > Users are authenticated correctly, error occurs when they try to access the > mailbox (trying with a manual connection using telnet, client dies on a > "SELECT INBOX" command after a successfull login) Actually that's the point when your frontend connects to your backend to access the mailbox. So, you've troubles with backend authentication. Most likely the proxy user is somehow broken. At this point you're hit by an quite old bug... 2.5.11 misses the patch for this issue which was committed (backported from 3.0 where it was applied in 2016) in November 2017. Don't know why no release was done for that yet. see: https://github.com/cyrusimap/cyrus-imapd/issues/40 At the bottom you'll find the two patches which fix this issue. If you're able to rebuild new packages include these patches. If not, you've to convince the debian people to provide an update with them. But most likely the issue goes away if you fix your backend authentication anyway. Greetings, Wolfgang -- Wolfgang Breyha | https://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cyrus 2.5 imap idle/stuck connections (DOS like)
Heiler Bemerguy via Info-cyrus wrote on 07/03/2019 15:39: > Yes I've read imapd.conf and cyrus.conf and found no options to limit > connections per source IP or "idleness".. > > It means anyone can open a lot of connections to any port (143, 25, 110 etc) > and render the server unusable?? You can try to set imapidletimeout: as documented in "man imapd.conf". To keep your server "RFC friendly" this value should not be lower than 30 minutes, because RFC 2177:3. recommends that clients show activity at least every 29 minutes. Greetings, Wolfgang -- Wolfgang Breyha | https://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cyrus 2.5 imap idle/stuck connections (DOS like)
Heiler Bemerguy via Info-cyrus wrote on 07/03/2019 16:30: > My "man imapd.conf" doesn't list this option at all. I think my Cyrus > 2.5.10 is older than it.. Yes. It appeared in 2.5.11 according to my build environment. But there already was the option timeout: 30 before. So every connection was closed by default after 30 minutes if there was no activity. "imapidletimeout" uses the value of "timeout" as upper limit default as well. > But I've added it to the .conf anyway. Won't hurt if it doesn't exist > yet, right? lol I currently don't remember if imapd ignores unknown options or fails on startup. But maybe the better option is to use tcp_keepalive: yes anyway. We had "dead" connections as well before imapidletimeout existed and I was able to get rid of them by activating keepalives to trigger the "timeout: 30" check. Greetings, Wolfgang -- Wolfgang Breyha | https://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus Murder Environment Upgrade
On 08/06/2020 17:37, Miguel Mucio Santos Moreira wrote: > Now we're in doubt about how is the best solution to replace the mupdate > master server for a new one. > Nowadays we have around 16K mailboxes. IIRC we simply replaced the mupdate server and did a "ctl_mboxlist -m" on all backends to fill it. Shouldn't take that long since we did it with 130k on 8 backends in under 20 minutes (or even shorter). Greetings, Wolfgang -- Wolfgang Breyha | https://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus Murder Environment Upgrade
Hi! On 09/06/2020 14:56, Miguel Mucio Santos Moreira wrote: > Dear Wolfgang, > > Firstly thanks for your answer, secondly I have one more doubt, during this > time where the new Mupdate Master is receiving mailboxes information from > backend servers, is necessary stopping comunications between frontend > servers and mupdate master or none action is necessary besides that one > you've mentioned before? > We're concerned if frontend servers will connect to Mupdate Master and > receive from it an information which there's no mailboxes anymore until the > backend push entirely mailboxes information and this situation to cause any > trouble. I recommend the follow steps: *) check that your currently running setup has no conflicts in mailboxes.db by running "ctl_mboxlist -mw". It is possible that you see output if somebody changes his mailboxes while you run the command, but it should not appear again if you run the command a second time. If everything is fine... *) shut down all running cyrus frontends first, then backends and mupdate last *) backup your mailboxes.db on mupdate server *) replace/update mupdate and start it with empty/removed mailboxes.db. *) backup your mailboxes.db on the backends *) do the ctl_mboxlist -m by hand on the backends (only one at the same time) you can check if everything went ok with "-mw" at any time afterwards. *) start cyrus on backends In our setup the frontends have mupdate running as well. I can't currently remember if this is mandatory. If this is true in your setup then: *) remove or rename the mailboxes.db database on the frontends and start them. They will fetch the database immediately from the mupdate server. This is visible in syslog as well and takes about 30 seconds in our setup (~250MB mailboxes.db). otherwise *) start the frontends At this point everything should be up and running again. Watching syslog output all the time usually helps. IIRC we updated the mupdate server to 2.5 first. Then we had a mix of 2.4 and 2.5 backends for some time. The 2.5 backends had some capabilities suppressed. Frontends had 2.4 until all backends had 2.5. Last but not least we upgraded to 2.5 on the frontends. If you need more info/details feel free to ask. Greetings, Wolfgang -- Wolfgang Breyha | https://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus Murder Environment Upgrade
On 10/06/2020 20:47, Miguel Mucio Santos Moreira wrote: > Wolfgang, > > If you don't mind, I'd like to know if when you upgraded backend servers > from 2.4 version to 2.5 version you have an increase, in metadata size.Here > we had 30% around of increase Yes, cyrus.* are larger. Not sure about .cache, but .index for sure. And the databases will grow as well if you change to twoskip. We had to go back to skiplist on some hosts after the database reached ~200MB due to long locking times. The frontends started to have troubles with long locks leading to timeouts on connect. Greetings, Wolfgang -- Wolfgang Breyha | https://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: drown/SSL issue
Hi! Tony Galecki via Info-cyrus wrote on 02/03/16 03:57: > I’m trying to figure out how to make my Cyrus install to not be susceptible to > the drown issue. > I have tried limiting the ciphers to TLSv1.2 but haven’t had much success. Limiting the cipher list does not deactive protocol support in OpenSSL. I don't know which patches Fedora backported from 2.4.18, but it seems not enough, because 2.4.18 disables SSLv2/v3 by default and you can set tls_versions: ... in your config. Setting these is the only way to get rid of the protocolls themself. On older cyrus versions you can set tlsonly: 1 but this can/will limit your protocoll support to TLSv1, with disabled v1.1 and v1.2, because TLSv1_server_method() was used. You do not need to rebuild OpenSSL. I would check the SPEC File of the CentOS 7 RPM which patches they included. If the TLS changes were not backported I would try to build one of the newer 2.4.18 SRPMs for Fedora (eg. 23) on CentOS 7. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: drown/SSL issue
On 02/03/16 12:02, Wolfgang Breyha via Info-cyrus wrote: > You do not need to rebuild OpenSSL. I would check the SPEC File of the CentOS > 7 RPM which patches they included. If the TLS changes were not backported I > would try to build one of the newer 2.4.18 SRPMs for Fedora (eg. 23) on > CentOS 7. As of today RHEL/CentOS ships openssl updates with deactivated SSLv2 at build time. It should be enough to update it and restart cyrus. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
@ in mailboxname: 2.4 vs. 2.5
Hi! If not using virtual domains it was always allowed to use "@" in mailbox names and our users did. Up to 2.4. 2.5 considers it as "invalid mailbox name" due to a change in mboxname.c This makes 2.5 highly incompatible with existing murder infrastructure. Why was this change introduced and not mentioned in the changelog? Is it save to remove this limitation from mboxname.c? Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: @ in mailboxname: 2.4 vs. 2.5
On 08/04/16 23:28, Wolfgang Breyha via Info-cyrus wrote: > Is it save to remove this limitation from mboxname.c? https://bugzilla.cyrusimap.org/show_bug.cgi?id=3693 This patch was reverted in "some" releases, but obviously not in 2.5. Please do so. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
2.5.7 caldav interoperability
Hi! In our efforts to move on to 2.5 we also try to support caldav/carddav for our employees. In our tests we tried to share the Default calendar through as many devices/platforms as possible (not only) including Thundbird, OSX and iOS. One problem we have is that iOS can access the Account and creates a new Kalender on first access, but it does not use/detect the Default calendar. It creates one with long UID name like 22xxx9EA-EFxx-4Exx-BAxx-Ax3D7xxx and gives it a localized "displayname" attribute. In english "Calendar". Calendars added by OSX are detected as well. Only Default is missing. OSX on the other hand detects the Default as well as the iOS "Calendar" and can access both. For a user it is impossible to "guess" that UID mailbox/calendar name to use it in clients without autodetection like lightning. Is this a know issue that iOS can't/wont detect the Default calendar? Maybe fixed in HEAD? Or is it something we did wrong in our configuration? Greetings, Wolfgang PS: carddav Default addressbook on the other hand works with iOS as well. -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: mailboxes.db invalid entries
Jan Kowalsky via Info-cyrus wrote on 22/04/16 01:28: > First I tried to dump the mailbox.db with ctl_mboxlist -d /tmp/mailboxes.txt > > After deleting the wrong entry manually I wanted to reload the mailbox > again with ctl_mboxlist -u /tmp/mailboxes.txt. All operation with > stopped cyrus. Have you renamed your mailboxes.db after using -d and before using -u? Otherwise ctl_mboxlist will import your dump into the existing mailboxes.db. And are this exactly the commands you used? I think ctl_mboxlist -d >/tmp/mailboxes.txt and ctl_mboxlist -f /tmp/mailboxes.db.new -u | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: 2.5.7 caldav interoperability
rsto--- via Info-cyrus wrote on 21/04/16 22:51: > P.S.: I am based in Vienna as well, so feel free to reach out to me :) Thanks for your offlist help. I think you pointed me in the right direction. 2.5.7 announces VPOLL for all except iOS/8 devices. HEAD doesn't announce VPOLL at all HEAD http_caldav.c:4520 /* Apple clients don't like VPOLL */ types &= ~CAL_COMP_VPOLL; vs. 2.5.7 http_caldav.c:3512 /* XXX iOS/8 doesn't like VPOLL */ if (hdr && strstr(hdr[0], "iOS/8")) types &= ~CAL_COMP_VPOLL; May I ask to backport this change to 2.5.x? I patched it locally and can confirm that our iOS/7 and 9 devices are able to access the Default Calendar now. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: disable starttls, ssl only
Gabriele Bulfon via Info-cyrus wrote on 26/04/16 12:25: > Hi, > > is it possible to disable completely the starttls option on cyrus imapd? > I want just connections on 993 over ssl, no starttls at all. > I noticed a number of "Fatal error: tls_start_servertls() failed" and noticed > a slow down of the entire machine during these errors. > I wouldn't like to have some kind of ddos attak using starttls on 993 or > something. Remove it from cyrus.conf? Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cannot connect with cyradm
On 06/05/16 04:24, Stuart Castergine via Info-cyrus wrote: > C: A01 AUTHENTICATE PLAIN Y3lydXMAY3lydXMAaGVsbC1oYXRoLW5vLWZ1cnk= I recommend changing the password from the "fury" thingy to something else. Maybe you want to strip base64 encoded credentials in the future. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
prefork and IPv6
Hi! I recently wondered why some of my preforked processes on my murder backends never get used. I detected them because some quite old lmtpd's were holding locks on an already deleted deliver.db. After some debugging I recognized that cyrus-master seems to fork the configured amount of "prefork" daemons twice. One half listening on IPv4 and the other half on IPv6. Since IPv6 is practically never used from our frontends they stay forever doing nothing on the backends. Is there some reasonable way to prevent this other than setting prefork=0? I'm only using SERVICE entries like: Bimap cmd="imapd" listen="imap" prefork=5 Only the port is used for listen= without interface/IP. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: idled multiplying and consuming the server
Eugene M. Zheganin via Info-cyrus wrote on 10/06/16 16:58:
> idled cmd="idled" listen="/var/imap/socket/idled" proto="udp" prefork=0
That looks like an entry in the SERVICE section. That's wrong for idled.
idled should be added at the end of the START section like
START {
idled cmd="idled"
}
Greetings, Wolfgang
--
Wolfgang Breyha | http://www.blafasel.at/
Vienna University Computer Center | Austria
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
squat broken after upgrade from 2.4 to 2.5?
Hi! I upgraded some backend hosts from 2.4.18 to 2.5.8. Currently they are all still running with index version 12. Some of these mailboxes have squatter activated as annotation and my cyrus.conf has squatter cmd="squatter -asir *" at=0500 I see that it runs at night and logs entries like: Jun 20 05:00:15 lauren squatter[27593]: indexing mailbox user. There are no errors in the log from squatter. But today I recognized on my own mailbox that squatter returns nothing while searching with thunderbird. The resulting imap request looked like: <1466414132<18 UID Search UNDELETED (OR (OR OR TO "search" HEADER CC "search" SUBJECT "search") BODY "search") <1466414132>* SEARCH 18 OK Completed (0 msgs in 0.010 secs) I then moved the current squat db and the same search returned 214 messages. Rebuiling the squat db and searching again resulted in the exact same 214 messages. It seems that something changed within the squat DB and 2.5 can't search in the old 2.4 DBs. That's bad in two ways. *) search doesn't fail with an error and simply returns no result. no fallback to searching without squat db. *) the squat db is not refreshed/rebuilt after upgrade. Is this a bug or is something broken on my side? Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: squat broken after upgrade from 2.4 to 2.5?
Bron Gondwana via Info-cyrus wrote on 21/06/16 01:52: > This could very well be a bug in the upgrade :( It would be great to have > a test case to check this. I can't do that today, but maybe ellie can do > something. > > Sadly our bug tracking situtation for Cyrus is a mess at the moment, but > I'm hoping to have that cleaned up soon. I filed https://bugzilla.cyrusimap.org/show_bug.cgi?id=3938 Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
2.5.9 build issues
After modifying my spec file from 2.5.8 to 2.5.9 I ended up with two new issues: *) jcal.c can't be compiled with jansson < 2.7 (eg. every RHEL) because json_boolean_value() is missing. I fixed it with: - --- cyrus-imapd-2.5.9/imap/jcal.h.orig 2016-08-20 16:20:52.049359274 +0200 +++ cyrus-imapd-2.5.9/imap/jcal.h 2016-08-20 16:21:03.719723892 +0200 @@ -50,6 +50,10 @@ #include "util.h" +#if (JANSSON_MINOR_VERSION < 7) +#define json_boolean_value json_is_true +#endif + extern char *icalcomponent_as_jcal_string(icalcomponent* comp); extern icalcomponent *jcal_string_as_icalcomponent(const char *str); extern const char *begin_jcal(struct buf *buf); *) htmlstrip.c gets installed as source to /usr/lib/cyrus-imapd. It seems it doesn't get compiled at all and the source gets installed instead. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Huge performance problems after updating from 2.4 to 2.5.9
Hi! A can add another story of that type, but with different setup: We already migrated to 2.5.7 on our ten backends some month ago step by step and upgraded to 2.5.9 lately. We never had any performance issues on them. All of them have done a full "reconstruct -V max" and special-use metadata is set to step in for static xlist missing in 2.5 frontends. Yesterday I did the final step by updating our mupdate server and the murder frontends. No problems on the mupdate server so far. The configs are mostly the same like for 2.4 (except for the changed names). The only key change done was from skiplist to twoskip for all databases: annotation_db: twoskip duplicate_db: twoskip ptscache_db: twoskip mboxlist_db: twoskip seenstate_db: twoskip statuscache_db: twoskip subscription_db: flat tls_sessions_db: twoskip userdeny_db: flat All of them are placed in /var/spool/imap/config except statuscache and tlscache which reside in /dev/shm. After migrating the frontends yesterday evening I already recognized a higher load and decided to keep an eye on it. Our 3 frontends are: CentOS 6 (vmware based) 16GB RAM 4 CPUs They usually have ~4000-4500 concurrent imap(s) connections at peak time and had a load ~1-3 with cyrus 2.4. swap was not used. Today in the morning we had the same connection count with load ~100-250 with cyrus 2.5.9 and twoskip. no swap used. top shows a huge amount of imap processes in running state. Another thing I recognized was that changes in mailboxes.db on backends reached the mupdate server, but didn't come through to the frontends anymore even after 15 minutes. I changed mailboxes.db and tlscache back to skiplist and load went down to 5-15. So, much better but still 5 times as high as with 2.4 and still causing headaches. Currently I have no idea what causes the load besides twoskip (which I can live without since skiplist never caused us any troubles). I compared "USAGE" loglines from two days (2.4) ago to 2.5 with skiplist and twoskip. 2.4 : user: 0.384322, sys: 0.029056 (23538 USAGE lines, 09:00-10:00) 2.4 : user: 0.353515, sys: 0.030814 (25790 USAGE lines, 13:00-14:00) 2.5 ts: user: 0.903491, sys: 0.028196 (23429 ..., 08:00-09:00) 2.5 ts: user: 1.130391, sys: 0.032894 (27709 ..., 09:00-10:00) 2.5 sl: user: 0.864270, sys: 0.026580 (29567 ..., 13:00-14:00) 2.5 sl: user: 1.077462, sys: 0.032952 (25879 ..., 14:00-15:00) Comparing all three I wonder why skiplist makes such a big difference. lmtpd shows now relevant difference for all three. Most likely I will go for horizontal scaling putting a 4th frontend in the line. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: command line deletion of files
Vladislav Kurz via Info-cyrus wrote on 29/09/16 17:04: > ipurge is nice but I would really appreciate if it had a --dry-run > option. I'm never sure what it will really delete. It got one in 2.5.9. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Front End not forwarding requests to the backend
Michael Menge via Info-cyrus wrote on 07/11/16 10:30: > Quoting Alberto Cardenas via Info-cyrus : > >> I'm trying to setup Cyrus Murder for testing and have three virtual >> machines for this purpose (frontend, mupdate and backend). The three >> machines are running Centos 6.5, SASL 2.1.23 and Cyrus 2.3.16 >> > > you might want to consider using a more recent cyrus Version. Cyrus 2.3.16 > is about 7 years old. 2.5.10 is the latest stable, 3.0 will be coming out > soon. I don't think it has something to do with your problems but, there > have been many bugfixes and improvements since 2.3. And if you set up a > new system it would make things easier in the future. For murder I recommend the latest 2.4.x release. 2.5 has still several issues open with murder and I can not recommend using it without in deep knowledge how to handle them (eg. maintain your own patches). 2.4 simply works compared to 2.5. Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Bugs in cyrus murder 2.5
Michael Menge via Info-cyrus wrote on 15/11/16 17:08: > Thanks for the info, we are also still running cyrus 2.4 on our > production environment. > But mostly because that was the stable version at the time we migrated > from stand alone > to murder setup and the "never touch a running system" mentality. I > was not aware that > there was decreased stability for murder setup in the cyrus 2.5 version. For 2.5.10 it's primarily not stability. It's functionality. > Do you have some specific bugs in mind? The major ones are: If you use XFERs between backends... they are still completely broken: https://github.com/cyrusimap/cyrus-imapd/issues/46 This one may still hurt if you use shared mailboxes (on different backends): https://github.com/cyrusimap/cyrus-imapd/issues/41 And last but not least .. performance issues on frontends (compared to 2.4): https://github.com/cyrusimap/cyrus-imapd/issues/43 Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: mailboxes.db locking problem after updating from 2.4 to 2.5.9
On 17/11/16 14:00, Deniss via Info-cyrus wrote:
> Hello,
>
> I trying to migrate one big cyrus imap server from 2.4 to 2.5.9.
>
> I updated binaries, fix db backend in imapd.conf and converted
> mailboxes.db with ctl_mboxlist -d & -u to twoskip.
>
> cyrus ran fine until morning when a count of simultanious sessions
> started to rise.
Sounds familiar;-)
> I converted mailboxes.db back to skiplist and got a bunch (10-20 per
> sec) of following logs:
I did the same. twoskip was and is useless on my frontends. With skiplist my
loads dropped at least down to 2-5. After doubling our vCPUs it runs quite well.
Do you use roundcube by any chance? If yes update to 2.5.10 and do not use
$config['imap_disabled_caps'] = array('LIST-EXTENDED');
Otherwise roundcube sends two 'LIST "" "*"' on each login and triggers two
full table scans on your mailboxes.db
> 180M /var/imap/mailboxes.db.skiplist.24
> 215M /var/imap/mailboxes.db.skiplist.25
> 262M /var/imap/mailboxes.db.twoskip.25
Wow, larger then ours. We've ~140MB on skiplist.25.
> Any ideas or suggestion for investigation ?
I already filed a bug
https://github.com/cyrusimap/cyrus-imapd/issues/43
but no response so far. I directly asked Bron, but no response as well.
Greetings, Wolfgang
--
Wolfgang Breyha | http://www.blafasel.at/
Vienna University Computer Center | Austria
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
