Hi Mark, thanks, I just tried this. It does cause the server to insert a 20 second delay on shutdown and I get this message in the log:
18-Nov-2022 18:16:29.777 INFO [main] org.apache.coyote.AbstractProtocol.awaitConnectionsClose Waiting [20,000] milliseconds for existing connections to ["http-nio-8080"] to complete and close. The server keeps responding to further requests on the established connections, but it also sends back headers as if in non-shutdown mode: Keep-Alive: timeout=20 Connection: keep-alive Thereby informing the client of the full 20 sec. keepalive timeout, making it think it can keep using the connection - so the client ends up getting the same error just with a 20 second delay... In fact, the server even accepts completely new connections during the graceful shutdown period period. Br, M. Thiim Den fre. 18. nov. 2022 kl. 14.34 skrev Mark Thomas <ma...@apache.org>: > On 17/11/2022 19:39, M. Thiim wrote: > > Hi, > > > > We have observed that Tomcat doesn't gracefully close > > keep-alive connections. Tomcat waits for already started requests to > > complete, but once those are done, Tomcat will close all connections > > immediately, irrespective of any configured keepAliveTimeout. This causes > > problems for some HTTP clients, especially in Kubernetes-like > environments > > when scaling down pods. Here, it can only work gracefully if the HTTP > > client who falls victim to an unexpectedly closed connection retries on a > > fresh connection, and it is not all clients that do this. > > > > I would think that an entirely graceful shutdown sequence, in the > presence > > of keep-alive connections, would work like the following: > > > > 1) Server receives shutdown request > > 2) Server immediately stops accepting new connections (already happens) > > 3) Server completes all requests already in (already happens) > > 4) New behavior: If new requests come in on already established > keep-alive > > connections those are processed, but a "Connection: close" is returned so > > the client knows this connection can no longer be used. So at most one > more > > request can be processed on each of those existing connections. > > 5) New behavior: When all keep-alive connections are gone, shutdown > > proceeds. If there are still connections left after the keepAliveTimeout > > has passed, this means no requests can have been received on those during > > the shutdown period (otherwise they would have been closed in #4). And > > since Tomcat returned the keep-alive timeout value to the client when the > > connection was setup, the client should know that the connection is no > > longer usable. Therefore it is from this point safe for Tomcat to close > > those remaining connections. > > 6) Rest of server shutdown continues > > Seems a reasonable addition. > > It looks like extending the behaviour when gracefulStopAwaitMillis is > set on the Service would work. > > gracefulStopAwaitMillis would need to be greater than or equal to the > keep-alive timeout but we can document that as part of the patch. > > Mark > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
