On July 16, 2019 7:20:28 AM UTC, "Rémy Maucherat" <r...@apache.org> wrote: >On Mon, Jul 15, 2019 at 11:30 PM Manuel Dominguez Sarmiento ><m...@renxo.com> >wrote: > >> Hi, we had been running Tomcat 9.0.17 for quite some time on our >high-load >> production servers, using the attached server.xml configuration. >> >> Upon upgrading to 9.0.21 we started experiencing many random >deadlocks. We >> run performance advertising campaigns, and our conversion rates >dropped to >> below half of what they usually are, which was an obvious consequence >of >> our servers randomly locking up. Plus, it was very easy to reproduce >the >> deadlocks, which seemed to "magically unlock" when opening a second >> tab/window and opening the same URL that was locked on the other >> window/tab. Doing this unlocked both windows/tabs at once, >immediately. >> >> We found this was only happening on HTTPS, but NOT on HTTP. >Furthermore, >> we found this was only happening when the browser negotiated an >upgrade to >> HTTPS/2.0 >> Once we found this, we temporarily removed the <UpgradeProtocol >> className="org.apache.coyote.http2.Http2Protocol" /> configuration, >and all >> was back to normal. >> However, we need HTTP/2, so we continued to look for a proper >solution. >> >> Looking at the Tomcat changelog, we found there have been many >changes >> since 9.0.17 related to useAsyncIO and HTTP/2. One particular change >for >> 9.0.22 caught our attention: >> >> >> *"Remove a source of potential deadlocks when using HTTP/2 when the >> Connector is configured with useAsyncIO as true. (markt)" *We also >found >> the following discussion thread, which describes issues similar to >what we >> were experiencing: >> >> >http://mail-archives.apache.org/mod_mbox/tomcat-dev/201906.mbox/%3c20190606204631.bab6c8a...@gitbox.apache.org%3e >> >> So we upgraded to 9.0.22 thinking that the deadlock would be gone. >But >> alas, it was not. The deadlocks remained. >> We found that 9.0.20 changed the default for useAsyncIO from "false" >to >> "true". So we changed useAsyncIO back to what it was when we were >running >> 9.0.17 (false) and all is back to normal on 9.0.22 >> >> So the conclusion is: there are still deadlock bugs on the NIO >connector >> with useAsyncIO="true" and upgrades to HTTP/2.0 >> Besides fixing them, we believe that the useAsyncIO default should be >> reverted to "false". >> >> Looking forward to the team's feedback. Thanks, >> > >You should investigate on the user list, or if you already have details >on >how to reproduce the deadlock and/or stack traces that show where it >occurs, you can create a BZ.
+1 a thread dump when the deadlock occurs with blocked thread(s) identified should be enough to figure out where things are going wrong. Mark --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
