Looks like something reset the connection between the Solr server you are
talking to and the one hosting the other shard (
http://192.168.120.202:8080/solr)
I would check the logs of the _other_ server and see if something is there.
Otherwise, I would look into something like a firewall in between two Solr
servers and perhaps a response taking so long that the firewall kills the
connection in between.
Regards,
Alex.
On 21/11/2014 6:02 am, "馬 飛虎" <[email protected]> wrote:
>
> Hi
>
> We used Solr4.6 for search and there was a exception occured randomly.
> The exception message in application(use SolrJ) was :
>
> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
> org.apache.solr.client.solrj.SolrServerException: IOException occured
> when talking to server at: http://192.168.xxx.xxx:8080/solr
> at
>
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:495)
> at
>
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
> at
>
org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
> at
org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:310)
> at search.Searcher.search(Searcher.java:379)
> at search.Searcher.doSearchFull(Searcher.java:105)
> at
>
com.c1s.common.search.YuukyuuSearcher.doSearchFull(YuukyuuSearcher.java:128)
> at
>
com.c1s.common.dispPartInfoList.DispPartInfoListUtils.getNewPatSyubetu(DispPartInfoListUtils.java:802)
> at
>
com.c1s.common.dispPartInfoList.DispPartInfoListUtils.getDetailPartInfo(DispPartInfoListUtils.java:1459)
> at
>
userweb.action.dispDetail.DispDetailAction.doExecute(DispDetailAction.java:411)
> at com.c1s.action.BaseAction.execute(BaseAction.java:193)
> at
>
org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)
> at
>
org.seasar.struts.processor.S2RequestProcessor.processActionPerform(S2RequestProcessor.java:145)
> at
org.seasar.struts.processor.AcceptorImpl.process(AcceptorImpl.java:133)
> at
>
org.seasar.struts.processor.S2RequestProcessor.process(S2RequestProcessor.java:74)
> at
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)
> at
org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:414)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:718)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
> at
>
org.apache.catalina.core.ApplicationFilterChain.servletService(ApplicationFilterChain.java:411)
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:317)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:198)
> at
>
com.c1s.action.RequestParameterFilter.doFilter(RequestParameterFilter.java:84)
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:198)
> at
>
org.seasar.extension.filter.EncodingFilter.doFilter(EncodingFilter.java:62)
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:198)
> at
org.seasar.struts.filter.S2StrutsFilter.doFilter(S2StrutsFilter.java:50)
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:198)
> at
>
org.seasar.framework.container.filter.S2ContainerFilter.doFilter(S2ContainerFilter.java:60)
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:198)
> at com.c1s.action.SessionFilter.doFilter(SessionFilter.java:99)
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:198)
> at com.c1s.action.UrlFilter.doFilter(UrlFilter.java:354)
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:198)
> at
>
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:288)
> at
>
org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:271)
> at
>
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:202)
> at
>
org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:632)
> at
>
org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:577)
> at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:94)
> at
>
com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:96)
> at
>
com.sun.enterprise.ee.web.sessmgmt.SessionLockingStandardPipeline.invoke(SessionLockingStandardPipeline.java:120)
> at
>
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:206)
> at
>
org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:632)
> at
>
org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:577)
> at
>
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:571)
> at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1080)
> at
>
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:150)
> at
>
org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:632)
> at
>
org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:577)
> at
>
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:571)
> at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1080)
> at
org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:272)
> at
org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
> at
org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
> at
org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
> at
>
org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
> at
>
org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
> at
>
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:666)
> at java.lang.Thread.run(Thread.java:619)
>
>
>
> And the message in solr server is :
> org.apache.solr.common.SolrException:
> org.apache.solr.client.solrj.SolrServerException: IOException occured
> when talking to server at: http://192.168.120.202:8080/solr
> at
>
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:302)
> at
>
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
> at
>
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:721)
> at
>
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:417)
> at
>
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:201)
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
>
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
> at
>
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
> at
>
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
> at
>
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
> at
>
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> at
>
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> at
>
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
> at
>
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
> at
>
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
> at
>
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.solr.client.solrj.SolrServerException: IOException
> occured when talking to server at: http://192.168.120.202:8080/solr
> at
>
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:507)
> at
>
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
> at
>
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:156)
> at
>
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:118)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ... 3 more
> Caused by: java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:196)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at
>
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
> at
>
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
> at
>
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
> at
>
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
> at
>
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:61)
> at
>
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
> at
>
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
> at
>
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
> at
>
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
> at
>
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
> at
>
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
> at
>
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:715)
> at
>
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:520)
> at
>
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> at
>
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> at
>
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> at
>
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:395)
> ... 9 more
>
> We thought this occured because of the solrServer has too many
> connections at 1 second,
> But we used Jmeter to simulate a very heavy load on the solrSever,and
> the similar Excepion was not hanppened.
>
> We used SolrJ in the application,and the SolrServer class was
> HttpSolrServer class with the default HttpClient.
> The solr was run on Tomcat,and the Tomcat config was the default config.
> And now we still receive this Exception a few times a day.
>
> Any ideas on what the root cause of this issue may be and how we can
> remediate it.
>
> Thanks.
