Re: Multi-thread cyrus delayed forking of imapd processes after connecting to master listener

2017-05-23 Thread Michael Hieb
I set maxforkrate=100 to make to raise it well above a default. 100 a second is quite a lot and logs show I am no where near that many connections. I am not sure if I can define the maxforkrate parameter so that it is unlimited (for testing) so I just set it at a number well above what I see

Re: Multi-thread cyrus delayed forking of imapd processes after connecting to master listener

2017-05-23 Thread Michael Hieb
ot; period=60 cleanup cmd="ipurge -f -d 30 user/%/Trash*@domain4.com" period=60 cleanup cmd="ipurge -f -d 30 user/%/Trash*@domain5.com" period=60 cleanup cmd="ipurge -f -d 30 user/%/Trash*@domain6.com" period=60 cleanup cmd="ipurge -f -d 60 user/%/Sent*@domain1.com" period=60 cleanup cmd="ipurge -f -d 60 user/%/Sent*@domain2.com" period=60 cleanup cmd="ipurge -f -d 60 user/%/Sent*@domain3.com" period=60 cleanup cmd="ipurge -f -d 60 user/%/Sent*@domain4.com" period=60 cleanup cmd="ipurge -f -d 60 user/%/Sent*@domain5.com" period=60 cleanup cmd="ipurge -f -d 60 user/%/Sent*@domain6.com" period=60 # Create search indexes regularly squatter cmd="squatter -s -i" at=0530 # running sa-learn sa-learn cmd="/usr/local/bin/cyrus-salearn.pl" period=60 sa-update cmd="/usr/bin/sa-update -v" at= } MAILSERVER:~ # cat /etc/imapd.domain1.com.conf configdirectory: /var/lib/imap partition-default: /var/spool/imap sievedir: /var/lib/sieve annotation_definitions: /etc/imapd.annotations.conf # admins: cy...@domain1.net allowanonymouslogin: no autocreatequota: 1 reject8bit: no quotawarn: 90 timeout: 30 poptimeout: 10 dracinterval: 0 drachost: localhost sasl_pwcheck_method: saslauthd lmtp_overquota_perm_failure: no #lmtp_catchall_mailbox: admin lmtp_downcase_rcpt: yes lmtp_fuzzy_mailbox_match: yes expunge_mode: delayed deletedprefix: DELETED delete_mode: delayed # added by Michael Hieb Jun 22 2014 allowplaintext: yes unixhierarchysep: yes allowplainwithouttls: no altnamespace: no virtdomains: userid servername: imap.domain1.net # # if you want TLS, you have to generate certificates and keys # tls_cert_file: /etc/letsencrypt/live/imap.domain1.net/cert.pem tls_key_file: /etc/letsencrypt/live/imap.domain1.net/privkey.pem tls_ca_file: /etc/letsencrypt/live/imap.domain1.net/chain.pem tls_ca_path: /etc/ssl/certs tls_versions: tls1_0 tls1_1 tls1_2 # added by Michael Hieb 20-Jan-2017 for lmtp on port (not socket) lmtp_admins: lmtpuser MAILSERVER:~ # Note: before I discovered the problem with forking could be triggered by switching between single threaded and multi-threaded cyrus server, I thought problem could be related to resource constraints. This is why I set all processes with maxchild=-1 maxforkrate=100. I also set the following in /etc/systemd/system/cyrus.service MAILSERVER:~ # cat /etc/systemd/system/cyrus.service [deleted...] LimitRTPRIO=5 LimitNOFILE=5 LimitNPROC=5 MAILSERVER:~ # None of these resource settings made any difference and the problem goes away as soon as cyrus is single threaded - so I do not believe it is a resource constraint (at least one I understand at this point). Any help would be much appreciated. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- next part -- A non-text attachment was scrubbed... Name: boutilpj.vcf Type: text/x-vcard Size: 286 bytes Desc: not available URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20170523/c8f3d992/attachment.vcf> Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Info-cyrus Digest, Vol 142, Issue 30

2017-05-23 Thread Michael Hieb
ot; period=60 cleanup cmd="ipurge -f -d 30 user/%/Trash*@domain4.com" period=60 cleanup cmd="ipurge -f -d 30 user/%/Trash*@domain5.com" period=60 cleanup cmd="ipurge -f -d 30 user/%/Trash*@domain6.com" period=60 cleanup cmd="ipurge -f -d 60 user/%/Sent*@domain1.com" period=60 cleanup cmd="ipurge -f -d 60 user/%/Sent*@domain2.com" period=60 cleanup cmd="ipurge -f -d 60 user/%/Sent*@domain3.com" period=60 cleanup cmd="ipurge -f -d 60 user/%/Sent*@domain4.com" period=60 cleanup cmd="ipurge -f -d 60 user/%/Sent*@domain5.com" period=60 cleanup cmd="ipurge -f -d 60 user/%/Sent*@domain6.com" period=60 # Create search indexes regularly squatter cmd="squatter -s -i" at=0530 # running sa-learn sa-learn cmd="/usr/local/bin/cyrus-salearn.pl" period=60 sa-update cmd="/usr/bin/sa-update -v" at= } MAILSERVER:~ # cat /etc/imapd.domain1.com.conf configdirectory: /var/lib/imap partition-default: /var/spool/imap sievedir: /var/lib/sieve annotation_definitions: /etc/imapd.annotations.conf # admins: cy...@domain1.net allowanonymouslogin: no autocreatequota: 1 reject8bit: no quotawarn: 90 timeout: 30 poptimeout: 10 dracinterval: 0 drachost: localhost sasl_pwcheck_method: saslauthd lmtp_overquota_perm_failure: no #lmtp_catchall_mailbox: admin lmtp_downcase_rcpt: yes lmtp_fuzzy_mailbox_match: yes expunge_mode: delayed deletedprefix: DELETED delete_mode: delayed # added by Michael Hieb Jun 22 2014 allowplaintext: yes unixhierarchysep: yes allowplainwithouttls: no altnamespace: no virtdomains: userid servername: imap.domain1.net # # if you want TLS, you have to generate certificates and keys # tls_cert_file: /etc/letsencrypt/live/imap.domain1.net/cert.pem tls_key_file: /etc/letsencrypt/live/imap.domain1.net/privkey.pem tls_ca_file: /etc/letsencrypt/live/imap.domain1.net/chain.pem tls_ca_path: /etc/ssl/certs tls_versions: tls1_0 tls1_1 tls1_2 # added by Michael Hieb 20-Jan-2017 for lmtp on port (not socket) lmtp_admins: lmtpuser MAILSERVER:~ # Note: before I discovered the problem with forking could be triggered by switching between single threaded and multi-threaded cyrus server, I thought problem could be related to resource constraints. This is why I set all processes with maxchild=-1 maxforkrate=100. I also set the following in /etc/systemd/system/cyrus.service MAILSERVER:~ # cat /etc/systemd/system/cyrus.service [deleted...] LimitRTPRIO=5 LimitNOFILE=5 LimitNPROC=5 MAILSERVER:~ # None of these resource settings made any difference and the problem goes away as soon as cyrus is single threaded - so I do not believe it is a resource constraint (at least one I understand at this point). Any help would be much appreciated. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- next part -- A non-text attachment was scrubbed... Name: boutilpj.vcf Type: text/x-vcard Size: 286 bytes Desc: not available URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20170523/c8f3d992/attachment.vcf> Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Multi-thread cyrus delayed forking of imapd processes after connecting to master listener

2017-05-23 Thread Michael Hieb
I have had good success connecting to the right process and can replicate the result. Here is what I found; MAILSERVER:~ # ps auwwx|grep imapd cyrus 1738 0.0 0.0 119572 10140 ?S03:32 0:00 imapd -C /etc/imapd.domain1.com.conf -s cyrus 1745 0.0 0.0 119580 9812 ?

Re: lmtpd segfaults after update to 2.4.19, seems it wants to act as proxy

2017-05-23 Thread ellie timoney
On Mon, May 22, 2017, at 03:55 PM, Simon Matter wrote: > Hi Ellie, > > Thanks for looking into it. > > Unfortunately with your new version of the patch I only get this: > > lmtpunix[23046]: FATAL: Internal error: assertion failed: append.c: 368: > stage != NULL && stage->parts[0] != '\0' > > Re

Re: Multi-thread cyrus delayed forking of imapd processes after connecting to master listener

2017-05-23 Thread Blake Hudson
Michael Hieb wrote on 5/19/2017 4:48 AM: Apologies for long post, most of it is configuration details. ... # UNIX sockets start with a slash and are put into /var/lib/imap/socket SERVICES { # add or remove based on preferences #imap cmd="imapd" listen="imap" maxchild=-1 maxforkrate=100 imap cm

Re: BDB despite apparent lack of BDB databases

2017-05-23 Thread Boylan, Ross
In partial answer to my original question, cyrusdb.c contains (in 2.4) struct cyrusdb_backend *cyrusdb_backends[] = { #ifdef HAVE_BDB &cyrusdb_berkeley, &cyrusdb_berkeley_nosync, &cyrusdb_berkeley_hash, &cyrusdb_berkeley_hash_nosync, #endif &cyrusdb_flat, &cyrusdb_skiplist,

Re: Multi-thread cyrus delayed forking of imapd processes after connecting to master listener

2017-05-23 Thread Scott Lambert
I may be completely missing the point and I don't know your environment. It looks like you may have designed in some delays if the server is pretty busy, especially during restart. On Fri, May 19, 2017 at 05:55:32PM +0800, Michael Hieb wrote: > > Here are the configuration details: > > > > imap cm

Re: Multi-thread cyrus delayed forking of imapd processes after connecting to master listener

2017-05-23 Thread Patrick Boutilier
I have done something like this before, define the command in /etc/cyrus.conf as the strace command. Define it on a test IP so you are the only one hitting it. Something like: imap cmd="/usr/bin/strace -q -v -s200 -f -F -o /home/cyrus/pop3 /full/path/to/imapd -C /etc/imapd.domain4.com.conf "

Re: Multi-thread cyrus delayed forking of imapd processes after connecting to master listener

2017-05-23 Thread Michael Ulitskiy
you can use 'strace -p ', specifying the pid of process you want to trace. you will have to "catch" the process that blocks. it may not be easy, but I have no other suggestions. On Tuesday, May 23, 2017 06:19:17 PM Michael Hieb wrote: > The SASL is cyrus-sasl 2.1.26-9.5. I believe it does use /de

Re: Multi-thread cyrus delayed forking of imapd processes after connecting to master listener

2017-05-23 Thread Michael Hieb
The SASL is cyrus-sasl 2.1.26-9.5. I believe it does use /dev/urandom. I am not sure how to run strace on the process that the Master listener forks. Here is the strace run on the imapd process at a command prompt (which is not quite the same as the master listener forking it to a port where a