Hello,

We are experiencing a memory leak in Jetty 9.4.6.

I am seeking for help in determining if we are not using Jetty properly or
if there is indeed a limitation in Jetty.

This behavior is observed under high load in a simulated environment using
gatling and bench-rest, where QueuedTheadPool is saturated and rejects jobs
<https://github.com/eclipse/jetty.project/blob/jetty-9.4.6.v20170531/jetty-util/src/main/java/org/eclipse/jetty/util/thread/QueuedThreadPool.java#L384>
.

We are getting the following WARN log from QueuedThreadPool :

org.eclipse.jetty.util.thread.QueuedThreadPool: dw{STARTED,4<=4<=4,i=0,q=4}
rejected org.eclipse.jetty.io.*ManagedSelector$$Lambda*
$72/1495355211@413f1bdf

The Lambda is defined in ManagedSelector.destroyEndPoint
<https://github.com/eclipse/jetty.project/blob/jetty-9.4.6.v20170531/jetty-io/src/main/java/org/eclipse/jetty/io/ManagedSelector.java#L431>


Since the Lambda is not called, AbstractConnector.onEndPointClosed
<https://github.com/eclipse/jetty.project/blob/jetty-9.4.6.v20170531/jetty-server/src/main/java/org/eclipse/jetty/server/AbstractConnector.java#L676>
is never called.

Therefore, EndPoint objects remain in AbstractConnector._endpoints
<https://github.com/eclipse/jetty.project/blob/jetty-9.4.6.v20170531/jetty-server/src/main/java/org/eclipse/jetty/server/AbstractConnector.java#L147>
and
lead to out of memory errors.

A screenshot of a memory dump showing 624159 endpoint objects :
https://imagebin.ca/v/3VTm2n97Ntki
A screenshot of how we validated the behavior using a "logging breakpoint"
in Intellij : https://imagebin.ca/v/3VTnTSgJDnXx

Please tell me if you need any additional information

Julien
_______________________________________________
jetty-users mailing list
[email protected]
To change your delivery options, retrieve your password, or unsubscribe from 
this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users

Reply via email to