It doesn't seem to be linear. Here's how I tested that theory. First I did 4 tests from gmail with TLS disabled for the google IP's: 100KB file, 1.1MB file, five 1.1MB files in one email, and ten 1.1MB files in one email. Then I did those same 4 tests with TLS enabled for the google ip's.
100KB NO TLS received DATA size: 142.81 kByte - sent DATA size: 143.44 kByte processing time 2 seconds 1.1MB NO TLS received DATA size: 1.50 MByte - sent DATA size: 1.50 MByte processing time 6 seconds 5.5MB NO TLS received DATA size: 7.49 MByte - sent DATA size: 7.49 MByte processing time 13 seconds 11MB NO TLS received DATA size: 14.97 MByte - sent DATA size: 14.97 MByte processing time 19 seconds Same files attached, now with TLS ON for the google ip addresses 100KB With TLS received DATA size: 142.87 kByte - sent DATA size: 143.54 kByte processing time 3 seconds (1 second longer, but still totally acceptable) 1.1MB With TLS received DATA size: 1.50 MByte - sent DATA size: 1.50 MByte processing time 27 seconds about *4.5x *loger than without TLS, only 27 seconds, but that's a pretty long time for a 1.5mb email 5.5MB TLS received DATA size: 7.49 MByte - sent DATA size: 7.49 MByte processing time 318 seconds about *24x *longer than without TLS and nearly 1/3 the speed of the 1MB tls version 11.0MB received DATA size: 14.97 MByte - sent DATA size: 14.97 MByte processing time 772 seconds about *40x *longer than without TLS almost 13 minutes instead of just 19 seconds about 2.5x the time of the 5.5MB with tls, expected 2x I can't test larger emails with google, Google will timeout after 15 minutes. I had debugging on for the gmail address I was sending from and got a huge debug log as expected. However, there's nothing useful in there. I don't see anything about speed, SSL renegotiation, or anything. For reference, sending that same 11.0MB email from a test *Outlook.com* account (whihch uses TLS) gets me: received DATA size: 14.98 MByte - sent DATA size: 14.98 MByte processing time 76 seconds (reasonable in my book for a TLS session) I also watched other traffic after the tests were done. I happened to see messages 5MB, 12MB, 17MB all came through quickly from non-Google sources with TLS on, but other gmail emails with attachments were slow slow. I haven't seen any mails be slow over TLS except for google, but that doesn't mean that there aren't others. Whatever the case, Gmail is too big of a player in this game to ignore the problem IMO. THANKS SO MUCH On Thu, Sep 22, 2016 at 5:58 AM, Thomas Eckardt <[email protected]> wrote: > Ken, please check the following. > > Investigate a relatve small (eg 100KB), a middle size (1MB) and one mail > that takes very long. > > Is the processing time in a nearly linear relation to the message size? > > like: > > 100KB - six seconds > 1MB - one minute > 2MB - two minutes > 3MB - three minutes > .... > > Or grows the time required for one MB, if the message size grows? > > Thomas > > > > > > Von: K Post <[email protected]> > An: ASSP development mailing list <[email protected]> > Datum: 03.08.2016 03:37 > Betreff: Re: [Assp-test] Inbound TLS from gmail.com addresses / > servers > > > > Thanks Thomas, but what OpenSSL should I be using? I really don't think > this is the problem, but I might as well eliminate it. I've got > activestate's perl 5.20 installed and net::ssleay from the activestate > ppm. However,the OpenSSL binaries that I have (I'm talking about the FULL > openssl installation in c:\openssl) not the dll files that net::ssleay > >might< have, is 1.0.2h from Shiining LIght ( > slproweb.com/products/Win32OpenSSL.html) > > ASSP says net::ssleay is OpenSSL 1.0.2g - apparently it hasn't been > compiled using 1.0.2h yet. That the readme from net::ssleay talks > specifically about shining light and that it's best to roll your own > worries me. > > And Bob, > Thanks for testing this out. 3MB in 25 seconds is about what I'm > generally > seeing now that I've tweaked the performance settings of ASSP, but without > TLS, we can receive a 10mb attachment in just a few seconds thanks to a > fast line. Curious, if you disable TLS temporarily and send yourself that > same 3mb attachment from gmail, how long does it take? > > > > On Tue, Aug 2, 2016 at 2:04 PM, Thomas Eckardt > <[email protected]> > wrote: > > > >Having looked through the Net:SSLEAY readme, there's a bunch that > > suggests > > >that it's best to compile your own net:ssleay and OpenSSL on the same > > >machine with the same settings. > > > > This will be the case, if you use the PPM from ActiveState. Perl and all > > modules are compiled with the same compiler and header files. > Net::SSLeay > > is compiled static, means it contains all required openssl code. > > > > >I'd love to find the time to give this a go, > > You'll find something better to do, than to compile this module on > > windows. > > > > > > Thomas > > > > > > > > > > Von: K Post <[email protected]> > > An: ASSP development mailing list <[email protected]> > > Datum: 02.08.2016 19:42 > > Betreff: Re: [Assp-test] Inbound TLS from gmail.com addresses / > > servers > > > > > > > > Having looked through the Net:SSLEAY readme, there's a bunch that > suggests > > that it's best to compile your own net:ssleay and OpenSSL on the same > > machine with the same settings. I've not done that, and never have (nor > do > > I have the skillset to do much more than run a simple make command). I'd > > love to find the time to give this a go, but what do you all think - > could > > this be it? Why would gmail.com always be bad, but others not (that > I've > > seen)? > > > > On Tue, Aug 2, 2016 at 1:22 PM, Thomas Eckardt > > <[email protected]> > > wrote: > > > > > >How do you know the type of encryption that gmail is using? > > > > > > You'll find it in the Received header line written by assp. > > > > > > >I have SSLDebug set to level 3, > > > > > > This helps not much. Most of the SSL-debug output goes to NUL. > > > But if there were errors in SSL - you would see them in the maillog. > > > > > > >I changed EnableHighPerformace to "very high," > > > I don't recommend to do this. This cuts the cycle time (poll/select > wait > > > time) in the workers to a minmum. Even if assp is idle - if this is > set, > > > it will permanently poll the sockets and will produce unwanted CPU > > > workload. I know 'EnableHighPerformace' sounds magic, but it is more > > > implemented to tweak exceptional environments. > > > How ever, if your host accepts this workload - it is fine. > > > > > > >Anything else I should try tweaking? > > > > > > Don't try to much. Tweak (if) one by one step. Use the > > > 'notes/confighistory.txt' - the old and new values are recoded there. > > > > > > I have an idea about the gmail problem. It may be the case, that they > > > request SSL rehandshakes more or less often depending on the used > > > certificate and/or cipher to raise the security of the connection. > Such > > a > > > behavior would slow down the SSL speed - BUT, now the bad news, this > is > > a > > > client request (made my gmail). Perl's Net::SSLeay has no easy way to > > > ignore these requests. The only way would be to pipe all SSL packest > > > through an assp routine (this is possible), which would drop the > > > renegotiation requests. Such a code will slow down ALL SSL traffic > > > dramaticaly, if written in pure perl. > > > > > > >We are using a 2048bit certificate. It's a wildcard > (*.ourcharity.org) > > > >cert, but I don't think that has anything to do with it. > > > > > > Who knows? But to exclude this, you may use an innocent selfcert > > > certificate and key - create it with openssl - for a while. > > > BTW. assp will create such certificate and keys, if the 'assp/certs' > > > folder is empty at startup. :):) > > > > > > Thomas > > > > > > > > > > > > > > > Von: K Post <[email protected]> > > > An: ASSP development mailing list > <[email protected]> > > > Datum: 02.08.2016 18:34 > > > Betreff: Re: [Assp-test] Inbound TLS from gmail.com addresses / > > > servers > > > > > > > > > > > > Thanks for chiming in Thomas with such a detailed response. > > > > > > First, when Google gives up, it gives a message like: > > > > > > Technical details of temporary failure: > > > > > > Missed upload deadline (899.97s) (state SENT_MESSAGE) > > > > > > So it's 15 minutes that it'll try to send a file for. At under 2mb a > > > minute, anything over about 25megs (considering overhead) will > > ultimately > > > fail. No good since lots of gmail users send us large files. > > > > > > > > > We're on a 100mbit line, both directions, but I'd happily take a 9.1 > mb > > > attachment sent over TLS taking 2 minutes. I suspect when i find out > > what > > > the problem is that it'll be MUCh faster than that. > > > > > > We are using a 2048bit certificate. It's a wildcard > (*.ourcharity.org) > > > cert, but I don't think that has anything to do with it. > > > > > > We're using local storage on the Hypver-V host, RAID 10 with 4 7200rpm > > SAS > > > drives. It's not the fasted disk array, but it seems fine. I can't > see > > > slow disks impacting TLS like this if non-TLS connections fly. > > > > > > The hyper-v host is a dual processor, 2.6ghz, 6 core each, 12mb cache. > > > I've got a total of 10 cores assigned to the ASSP guest. > > > > > > I have SSLDebug set to level 3, but I don't see anything in the > maillog. > > > How do you know the type of encryption that gmail is using? It would > > be > > > nice to compare how gmail is connecting vs outlook.com which seems > much > > > faster (though not super fast) > > > > > > I've got SSL_Version set to > > > SSLv23:!SSLv3:!SSLv2 > > > > > > and > > > SSL_Cipher_List set to > > > > > > > > > > > kEECDH+ECDSA:kEECDH:kEDH:HIGH:+SHA:+RC4:RC4:!aNULL:!eNULL:! > LOW:!3DES:!MD5:!EXP:!DSS:!PSK:!SRP:!kECDH:!CAMELLIA128:!IDEA:!SEED > > > > > > my unscientific test of changing the cipher list to the default > doesn't > > > seem to make a difference. > > > > > > MinPollTime is 1, I think it always has been. > > > I changed EnableHighPerformace to "very high," changed thread cycle > time > > > to > > > 1000, maintenance thread cycle time to 2000, and > rebuildthreadcycletime > > to > > > 15. That definitely made a difference in the rebuild time, almost > > halving > > > it (not that I really care about that though). > > > > > > Anything else I should try tweaking? I don't care if there's high CPU > > > usage, we have reasonable processing power to spare. > > > > > > Thank you > > > > > > On Tue, Aug 2, 2016 at 12:02 PM, Thomas Eckardt > > > <[email protected]> > > > wrote: > > > > > > > I just made simlar tests with my gmail account. I can't reproduce > this > > > > behavior related to gmail.com. > > > > > > > > I've sent a 9.1MB attachment in 133 seconds. Gmail used > SMTPS(TLSv1_2 > > > > ECDHE-RSA-AES256-GCM-SHA384)- which is commonly used by many > > > > clients/servers. > > > > Sender was mail-qt0-f181.google.com ([209.85.216.181] > > > > helo=mail-qt0-f181.google.com) > > > > My line speed is 16MB/s inbound and 4MB/s outbound. > > > > > > > > I saw many faster SMTPS connections but also many slower - this may > > > depend > > > > on the usage of my ISP connection. > > > > > > > > 133 seconds for such a mail is acceptable (I think). > > > > > > > > SSLv2/3:!SSLv3:!SSLv2 > > > > DEFAULT:!aNULL:!RC4:!MD5 > > > > > > > > are my SSL settings - not very strong - I know :):) > > > > > > > > the privat key used is 2048 Bit long > > > > > > > > In front of assp is the ISP-router and a pfsense 2.3.2 with snort > > > 3.2.9.1 > > > > . Snort is configured the very hard way, except the SMTP rules are a > > bit > > > > more weak, because I need some spam. > > > > ASSP is running on a 4 Core 6GB W2K3 enterprise with an absolute > > > uptodate > > > > ActivePerl 5.16.3 - using all Plugins, features and a replicated > MySQL > > > > 5.6. > > > > Domain based mail routing (in- and out-bound) is done by hmailserver > > > > 5.6.4-B2283. > > > > All components are configured to use SSL/TLS when ever this is > > possible. > > > > For testing purposes I use a FreeBSD 10.2 with Perl 5.20 and ASSP - > it > > > > runs the same way stable like the production system. > > > > > > > > You see - nothing magic, but maintenained (except the nice old W2K3 > - > > > but > > > > it works like a swiss made watch with an ETA 7750). > > > > > > > > I really don't know what I can do to fix up the SSL/TLS problems. > > > > > > > > Only to be complete: > > > > Backend for the mail environment and LDAP stuff is a Domino > 9.0.1FP6. > > > > All the stuff above (and very much more) is running on a single > VMWare > > > > vSphere 5.5 ( 8x 2.66GHz 48GB / x3650M2). > > > > Backups are done with EMC-Networker + EBR + DataDomain-VE, stored at > a > > > > QNAP 419P+ > > > > > > > > Thomas > > > > > > > > > > > > > > > > > > > > Von: K Post <[email protected]> > > > > An: ASSP development mailing list > > <[email protected]> > > > > Datum: 02.08.2016 00:07 > > > > Betreff: [Assp-test] Inbound TLS from gmail.com addresses / > > > servers > > > > > > > > > > > > > > > > I originally thought that we had a problem with all TLS inbound > email. > > > As > > > > it turns out, my conclusion appears to have been wrong. > > > > > > > > > > > > - There are some SLOW servers outside that are just plain slow > > > (nothing > > > > I can do there), > > > > > > > > - TLS seems to work reasonably fast with most inbound mail, > though > > > > significantly slower than without TLS (5 seconds for an 11mb > file > > > > without > > > > tls, vs 45 seconds with TLS on) > > > > > > > > - GMAIL.com inbound TLS emails are SLOW, no matter what settings > I > > > > tweak > > > > > > > > > > > > With inbound gmail.com message. if I have TLS off, an 11mb > attachment > > is > > > > delivered through ASSP in under 5 seconds. With TLS on it takes > close > > > to > > > > 10 minutes, which gets close to gmail's limit. > > > > > > > > I've tested with Outlook.com and that same 11mb attachment comes in > > > > through > > > > ASSP with TLS on in about 45 seconds. > > > > > > > > Sending a 30mb attachment from gmail FAILS because it takes too > long. > > > > gmail > > > > will try for I believe 10 minutes to send a message, then it quits > and > > > > retries. After a couple tries, it sends an NDR. > > > > > > > > This is a Windows 2012 R2 server, latest ASSP dev, OpenSSL 1.0.2h > > > > installed > > > > from slproweb.com/products/Win32OpenSSL.html (though I've also tried > > > with > > > > the OpenSSL I downloaded a while back from the ASSP sourceforge > site. > > > > net::ssleay 1.74 (openssl 1.0.2g). I'm almost certain that the > > OpenSSL > > > > installation is not used by ASSP, but I've not been able to get > > > > confirmation of that here. > > > > > > > > Just updated IO::Socket::SSL to 2.033. > > > > Net::SMTP:SSL 1.02. > > > > > > > > CPU usage as reported by assp is 4.78%. It's not on the fastest > > machine > > > > in > > > > the world (it's a hypver-v guest on a decent machine), but it seems > > > speedy > > > > enough. 24gb ram. We've got similar physical hosts running > Exchange > > as > > > a > > > > guest without any speed issues whatsoever. > > > > > > > > Any other info I can provide to help figure this out? > > > > > > > > Disabling TLS for any gmail inbound mail isn't a feasible option, > plus > > I > > > > don't know if it really is just google, or just the way that google > > > > connects which others might too... > > > > > > > > Thank you all. > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------ > ------------------ > > > > _______________________________________________ > > > > Assp-test mailing list > > > > [email protected] > > > > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > > > > > > > > > > > > > > > > DISCLAIMER: > > > > ******************************************************* > > > > This email and any files transmitted with it may be confidential, > > > legally > > > > privileged and protected in law and are intended solely for the use > of > > > the > > > > > > > > individual to whom it is addressed. > > > > This email was multiple times scanned for viruses. There should be > no > > > > known virus in this email! > > > > ******************************************************* > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------ > ------------------ > > > > > > > > _______________________________________________ > > > > Assp-test mailing list > > > > [email protected] > > > > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------ > ------------------ > > > _______________________________________________ > > > Assp-test mailing list > > > [email protected] > > > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > > > > > > > > > > > DISCLAIMER: > > > ******************************************************* > > > This email and any files transmitted with it may be confidential, > > legally > > > privileged and protected in law and are intended solely for the use of > > the > > > > > > individual to whom it is addressed. > > > This email was multiple times scanned for viruses. There should be no > > > known virus in this email! > > > ******************************************************* > > > > > > > > > > > > > > > > > ------------------------------------------------------------ > ------------------ > > > > > > _______________________________________________ > > > Assp-test mailing list > > > [email protected] > > > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > > > > > > > ------------------------------------------------------------ > ------------------ > > _______________________________________________ > > Assp-test mailing list > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > > > > > > DISCLAIMER: > > ******************************************************* > > This email and any files transmitted with it may be confidential, > legally > > privileged and protected in law and are intended solely for the use of > the > > > > individual to whom it is addressed. > > This email was multiple times scanned for viruses. There should be no > > known virus in this email! > > ******************************************************* > > > > > > > > > ------------------------------------------------------------ > ------------------ > > > > _______________________________________________ > > Assp-test mailing list > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > ------------------------------------------------------------ > ------------------ > _______________________________________________ > Assp-test mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > DISCLAIMER: > ******************************************************* > This email and any files transmitted with it may be confidential, legally > privileged and protected in law and are intended solely for the use of the > > individual to whom it is addressed. > This email was multiple times scanned for viruses. There should be no > known virus in this email! > ******************************************************* > > > ------------------------------------------------------------ > ------------------ > > _______________________________________________ > Assp-test mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/assp-test > >
------------------------------------------------------------------------------
_______________________________________________ Assp-test mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/assp-test
