https://bz.apache.org/bugzilla/show_bug.cgi?id=62332
Bug ID: 62332
Summary: Possible memory leak in
org.apache.coyote.AbstractProtocol$ConnectionHandler
when there's a protocol error
Product: Tomcat 8
Version: 8.5.x-trunk
Hardware: PC
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: WebSocket
Assignee: [email protected]
Reporter: [email protected]
Target Milestone: ----
We have an application that, after 12-24 hours, we need to restart it because
of heavy memory usage.
We detected, using JProfiler, that
org.apache.coyote.AbstractProtocol$ConnectionHandler is the responsible for the
leak, because it's storing several closed socket connections that should've
been removed, but for some reason they weren't.
Our application has heavy usage of Websockets and we have some clients that can
open over 20,000 connections/day because of bad 3G/4G reception.
Upon debugging and looking at debug log levels in production, we noticed that
we have a high load of exceptions in WsFrameBase.processInitialHeader that
throws WsIoException with CloseReason.PROTOCOL_ERROR (probably due to packet
loss given the bad connection). This exception is caught in
WsHttpUpgradeHandler.upgradeDispatch() and closing the channel, but not
returning a SocketState.CLOSED in this specific case (this issue is very
similar to https://bz.apache.org/bugzilla/show_bug.cgi?id=62024 - but for a
different CloseReason).
Unfortunately, we couldn't reproduce the bug locally using a script, so we
forced WsFrameBase.processInitialHeader() to throw the WsIoException passing a
PROTOCOL_ERROR(1002) as CloseReason.
We added a condition in the exception treatment in WsHttpUpgradeHandler
"if (cr.getCloseCode() == CloseCodes.CLOSED_ABNORMALLY || cr.getCloseCode() ==
CloseCodes.PROTOCOL_ERROR) {
return SocketState.CLOSED;
}
and used this modification in production for our application. After 24 hours,
we no longer observed a leak and
org.apache.coyote.AbstractProtocol$ConnectionHandler has a "healthy" amount of
cached websocket connections.
I hope this helps and I'm available to answer any questions.
Best Regards,
Leonardo.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
