Hey all, Some background: we have an app that consists of webservices that talk to each other. So the pattern of usage is that we have a few components that host a webserver and at the same time have a http client to talk to the others. We use Jetty webserver and Jetty httpclient for communication (9.4.7.v20170914).
The problem that we face is that we regularly see IOExceptions exceptions occurring in the communication between these components. See stacktrace below. These IOExceptions always contain the earlyEOF. A long time ago I already posted a similar message on this forum (see https://dev.eclipse.org/mhonarc/lists/jetty-users/msg07965.html). I followed the advice mentioned, upgraded to the latest version and explicitly set a different idleTimeout on both the server and the client. However, that didn't help. We still see the occasional earlyEOF. What's worse, we now have another app with more webservices and more rest-communication. In this app, the earlyEOF happens frequently. And this during mild load tests (not that heavy, so nothing is overloaded), so it's clear to me that nothing is idle for a second. To get around the problem, as a test, we tried to replace the JettyClient with Apache HTTP client. There the problem is less of an issue, but we still see that REST calls failing. Our conclusion is that for some reason the server is closing the connection, but this while the client is still reading from the inputstream that contains the response body. Does anyone have an idea how we can further diagnose and fix this? Here's a typical stacktrace: Caused by: java.io.EOFException: HttpConnectionOverHTTP@516b4fbd(l:/10.58.234.140:34446<mailto:HttpConnectionOverHTTP@516b4fbd(l:/10.58.234.140:34446> <-> r:be-pom-node-04.clear2pay.com/10.58.234.175:28080,closed=false)=>HttpChannelOverHTTP@125fa92a(exchange=HttpExchange@71e96cb2 req=TERMINATED/null@null res=PENDING/null@null)[send=HttpSenderOverHTTP@71a5f017(req=QUEUED,snd=COMPLETED,failure=null)[HttpGenerator@7a384f39{s=START}],recv=HttpReceiverOverHTTP@15c1bfad(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0 of -1}]]<-SocketChannelEndPoint@2f77cd5b{be-pom-node-04.clear2pay.com/10.58.234.175:28080<-<mailto:-SocketChannelEndPoint@2f77cd5b%7bbe-pom-node-04.clear2pay.com/10.58.234.175:28080%3c->>/10.58.234.140:34446,ISHUT,fill=-,flush=-,to=1/0}{io=1/0,kio=1,kro=1}->HttpConnectionOverHTTP@516b4fbd(l:/10.58.234.140:34446<mailto:HttpConnectionOverHTTP@516b4fbd(l:/10.58.234.140:34446> <-> r:be-pom-node-04.clear2pay.com/10.58.234.175:28080,closed=false)=>HttpChannelOverHTTP@125fa92a(exchange=HttpExchange@71e96cb2 req=TERMINATED/null@null res=PENDING/null@null)[send=HttpSenderOverHTTP@71a5f017(req=QUEUED,snd=COMPLETED,failure=null)[HttpGenerator@7a384f39{s=START}],recv=HttpReceiverOverHTTP@15c1bfad(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0 of -1}]] at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.earlyEOF(HttpReceiverOverHTTP.java:320) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1419) at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.shutdown(HttpReceiverOverHTTP.java:196) at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:143) at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:70) at org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:130) at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:116) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:104) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:243) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:679) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:597) thanks, Bert The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.
_______________________________________________ jetty-users mailing list [email protected] To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/jetty-users
