In case anyone has the same issue, it looks like switching Solr over
from Jetty to Tomcat fixed the problem.
I am using Tomcat v. 6.0.18.
Regards,
Anoop Bhatti
--
Committed to open source technology.
On Mon, Dec 1, 2008 at 10:51 AM, Anoop Bhatti <[EMAIL PROTECTED]> wrote:
> I increased this param to 10MB and still got the same exception. I
> doubt my HTTP requests are exceeding 10MB. I can rerun everything
> again and log the sizes of the requests, just to be 100% sure, but
> this will take some time. The stacktrace appears when
> the Lucene index is around 30 GB. Could the size of the indexes be
> the cause of the problem? Any other ideas?
>
>
> Thanks again,
>
> Anoop Bhatti
> --
> Committed to open source technology.
>
>
>
>
> On Mon, Nov 24, 2008 at 11:44 PM, Yonik Seeley <[EMAIL PROTECTED]> wrote:
>> I thought the Jetty maxFormContentSize was only for form data (not for
>> the POST body).
>> Does increasing this param help?
>>
>> -Yonik
>>
>>
>> On Mon, Nov 24, 2008 at 2:45 PM, Anoop Bhatti <[EMAIL PROTECTED]> wrote:
>>> Hello Solr Community,
>>>
>>> I'm getting the stracktrace below when adding docs using the
>>> CommonsHttpSolrServer.add(Collection<org.apache.solr.common.SolrInputDocument>
>>> docs)
>>> method. The server doesn't seem to be able to recover from this error.
>>> We are adding a collection with 1,000 SolrInputDocument's at a time.
>>> I'm using Solr 1.3.0 and Java 1.6.0_07.
>>>
>>> It seems that this problem occurs in Jetty when the TCP connection is
>>> broken while the stream (from the add(...) method) is being read. The
>>> XML read from the broken stream is not valid. Is this a correct
>>> diagnosis?
>>>
>>> Could this stacktrace be occurring when the max POST size has been
>>> exceeded? I'm referring to the example/etc/jetty.xml file, which has
>>> the setting:
>>> <!-- Increase the maximum POST size to 1 MB to be able to handle
>>> large shard requests -->
>>> <Call class="java.lang.System" name="setProperty">
>>> <Arg>org.mortbay.jetty.Request.maxFormContentSize</Arg>
>>> <Arg>1000000</Arg>
>>> </Call>
>>> Right now maxFormContentSize is set to the default 1 MB on my server.
>>>
>>> Also, in some cases I have two clients, could Jetty be blocking one
>>> client and causing it to finally timeout?
>>>
>>> This stacktrace doesn't happen right away, is occurs once the Lucene
>>> indexes are about 30 GB.
>>> Could the periodic merging of segments be the culprit?
>>>
>>> I was also thinking that the problem could be with writing back the
>>> response (the UpdateRequest).
>>>
>>> Here's the gist of my Java client code:
>>>
>>> CommonsHttpSolrServer solrServer = new CommonsHttpSolrServer(solrServerURL);
>>> solrServer.setConnectionTimeout(100);
>>> solrServer.setDefaultMaxConnectionsPerHost(100);
>>> solrServer.setMaxTotalConnections(100);
>>> solrServer.add(solrDocs); //the collection of docs
>>> solrServer.commit();
>>>
>>> And here's the stacktrace:
>>>
>>> Nov 20, 2008 5:25:33 PM org.apache.solr.core.SolrCore execute
>>> INFO: [] webapp=/solr path=/update params={wt=javabin&version=2.2}
>>> status=0 QTime=469
>>> Nov 20, 2008 5:25:37 PM org.apache.solr.common.SolrException log
>>> SEVERE: com.ctc.wstx.exc.WstxIOException: null
>>> at com.ctc.wstx.sr.StreamScanner.throwFromIOE(StreamScanner.java:708)
>>> at
>>> com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1086)
>>> at
>>> org.apache.solr.handler.XmlUpdateRequestHandler.readDoc(XmlUpdateRequestHandler.java:321)
>>> at
>>> org.apache.solr.handler.XmlUpdateRequestHandler.processUpdate(XmlUpdateRequestHandler.java:195)
>>> at
>>> org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateRequestHandler.java:123)
>>> at
>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204)
>>> at
>>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
>>> at
>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
>>> at
>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
>>> at
>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
>>> at
>>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>> at
>>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
>>> at
>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
>>> at
>>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
>>> at
>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
>>> at
>>> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
>>> at
>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
>>> at org.mortbay.jetty.Server.handle(Server.java:285)
>>> at
>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
>>> at
>>> org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:835)
>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641)
>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:202)
>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
>>> at
>>> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
>>> at
>>> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
>>> Caused by: org.mortbay.jetty.EofException
>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:302)
>>> at
>>> org.mortbay.jetty.HttpParser$Input.blockForContent(HttpParser.java:919)
>>> at org.mortbay.jetty.HttpParser$Input.read(HttpParser.java:897)
>>> at
>>> sun.nio.cs.StreamDecoder$CharsetSD.readBytes(StreamDecoder.java:411)
>>> at
>>> sun.nio.cs.StreamDecoder$CharsetSD.implRead(StreamDecoder.java:453)
>>> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:183)
>>> at java.io.InputStreamReader.read(InputStreamReader.java:167)
>>> at com.ctc.wstx.io.MergedReader.read(MergedReader.java:101)
>>> at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:84)
>>> at
>>> com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.java:57)
>>> at
>>> com.ctc.wstx.sr.StreamScanner.loadMoreFromCurrent(StreamScanner.java:1046)
>>> at
>>> com.ctc.wstx.sr.StreamScanner.loadMoreFromCurrent(StreamScanner.java:1053)
>>> at
>>> com.ctc.wstx.sr.StreamScanner.getNextCharFromCurrent(StreamScanner.java:811)
>>> at
>>> com.ctc.wstx.sr.BasicStreamReader.readEndElem(BasicStreamReader.java:3206)
>>> at
>>> com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2832)
>>> at
>>> com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1019)
>>> ... 24 more
>>>
>>> Nov 20, 2008 5:25:37 PM org.apache.solr.core.SolrCore execute
>>> INFO: [] webapp=/solr path=/update params={wt=javabin&version=2.2}
>>> status=500 QTime=3215
>>> Nov 20, 2008 5:25:37 PM org.apache.solr.common.SolrException log
>>> SEVERE: com.ctc.wstx.exc.WstxIOException: null
>>> at com.ctc.wstx.sr.StreamScanner.throwFromIOE(StreamScanner.java:708)
>>> at
>>> com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1086)
>>> at
>>> org.apache.solr.handler.XmlUpdateRequestHandler.readDoc(XmlUpdateRequestHandler.java:321)
>>> at
>>> org.apache.solr.handler.XmlUpdateRequestHandler.processUpdate(XmlUpdateRequestHandler.java:195)
>>> at
>>> org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateRequestHandler.java:123)
>>> at
>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204)
>>> at
>>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
>>> at
>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
>>> at
>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
>>> at
>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
>>> at
>>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>> at
>>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
>>> at
>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
>>> at
>>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
>>> at
>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
>>> at
>>> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
>>> at
>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
>>> at org.mortbay.jetty.Server.handle(Server.java:285)
>>> at
>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
>>> at
>>> org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:835)
>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641)
>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:202)
>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
>>> at
>>> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
>>> at
>>> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
>>> Caused by: org.mortbay.jetty.EofException
>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:302)
>>> at
>>> org.mortbay.jetty.HttpParser$Input.blockForContent(HttpParser.java:919)
>>> at org.mortbay.jetty.HttpParser$Input.read(HttpParser.java:897)
>>> at
>>> sun.nio.cs.StreamDecoder$CharsetSD.readBytes(StreamDecoder.java:411)
>>> at
>>> sun.nio.cs.StreamDecoder$CharsetSD.implRead(StreamDecoder.java:453)
>>> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:183)
>>> at java.io.InputStreamReader.read(InputStreamReader.java:167)
>>> at com.ctc.wstx.io.MergedReader.read(MergedReader.java:101)
>>> at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:84)
>>> at
>>> com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.java:57)
>>> at
>>> com.ctc.wstx.sr.StreamScanner.loadMoreFromCurrent(StreamScanner.java:1046)
>>> at
>>> com.ctc.wstx.sr.StreamScanner.loadMoreFromCurrent(StreamScanner.java:1053)
>>> at
>>> com.ctc.wstx.sr.StreamScanner.getNextCharFromCurrent(StreamScanner.java:811)
>>> at
>>> com.ctc.wstx.sr.BasicStreamReader.readEndElem(BasicStreamReader.java:3206)
>>> at
>>> com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2832)
>>> at
>>> com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1019)
>>> ... 24 more
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Anoop Bhatti
>>> --
>>> Committed to open source technology.
>>>
>>
>
