I thought the Jetty maxFormContentSize was only for form data (not for
the POST body).
Does increasing this param help?
-Yonik
On Mon, Nov 24, 2008 at 2:45 PM, Anoop Bhatti <[EMAIL PROTECTED]> wrote:
> Hello Solr Community,
>
> I'm getting the stracktrace below when adding docs using the
> CommonsHttpSolrServer.add(Collection<org.apache.solr.common.SolrInputDocument>
> docs)
> method. The server doesn't seem to be able to recover from this error.
> We are adding a collection with 1,000 SolrInputDocument's at a time.
> I'm using Solr 1.3.0 and Java 1.6.0_07.
>
> It seems that this problem occurs in Jetty when the TCP connection is
> broken while the stream (from the add(...) method) is being read. The
> XML read from the broken stream is not valid. Is this a correct
> diagnosis?
>
> Could this stacktrace be occurring when the max POST size has been
> exceeded? I'm referring to the example/etc/jetty.xml file, which has
> the setting:
> <!-- Increase the maximum POST size to 1 MB to be able to handle
> large shard requests -->
> <Call class="java.lang.System" name="setProperty">
> <Arg>org.mortbay.jetty.Request.maxFormContentSize</Arg>
> <Arg>1000000</Arg>
> </Call>
> Right now maxFormContentSize is set to the default 1 MB on my server.
>
> Also, in some cases I have two clients, could Jetty be blocking one
> client and causing it to finally timeout?
>
> This stacktrace doesn't happen right away, is occurs once the Lucene
> indexes are about 30 GB.
> Could the periodic merging of segments be the culprit?
>
> I was also thinking that the problem could be with writing back the
> response (the UpdateRequest).
>
> Here's the gist of my Java client code:
>
> CommonsHttpSolrServer solrServer = new CommonsHttpSolrServer(solrServerURL);
> solrServer.setConnectionTimeout(100);
> solrServer.setDefaultMaxConnectionsPerHost(100);
> solrServer.setMaxTotalConnections(100);
> solrServer.add(solrDocs); //the collection of docs
> solrServer.commit();
>
> And here's the stacktrace:
>
> Nov 20, 2008 5:25:33 PM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=javabin&version=2.2}
> status=0 QTime=469
> Nov 20, 2008 5:25:37 PM org.apache.solr.common.SolrException log
> SEVERE: com.ctc.wstx.exc.WstxIOException: null
> at com.ctc.wstx.sr.StreamScanner.throwFromIOE(StreamScanner.java:708)
> at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1086)
> at
> org.apache.solr.handler.XmlUpdateRequestHandler.readDoc(XmlUpdateRequestHandler.java:321)
> at
> org.apache.solr.handler.XmlUpdateRequestHandler.processUpdate(XmlUpdateRequestHandler.java:195)
> at
> org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateRequestHandler.java:123)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
> at
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
> at
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
> at org.mortbay.jetty.Server.handle(Server.java:285)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:835)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:202)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
> at
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
> at
> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
> Caused by: org.mortbay.jetty.EofException
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:302)
> at
> org.mortbay.jetty.HttpParser$Input.blockForContent(HttpParser.java:919)
> at org.mortbay.jetty.HttpParser$Input.read(HttpParser.java:897)
> at sun.nio.cs.StreamDecoder$CharsetSD.readBytes(StreamDecoder.java:411)
> at sun.nio.cs.StreamDecoder$CharsetSD.implRead(StreamDecoder.java:453)
> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:183)
> at java.io.InputStreamReader.read(InputStreamReader.java:167)
> at com.ctc.wstx.io.MergedReader.read(MergedReader.java:101)
> at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:84)
> at
> com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.java:57)
> at
> com.ctc.wstx.sr.StreamScanner.loadMoreFromCurrent(StreamScanner.java:1046)
> at
> com.ctc.wstx.sr.StreamScanner.loadMoreFromCurrent(StreamScanner.java:1053)
> at
> com.ctc.wstx.sr.StreamScanner.getNextCharFromCurrent(StreamScanner.java:811)
> at
> com.ctc.wstx.sr.BasicStreamReader.readEndElem(BasicStreamReader.java:3206)
> at
> com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2832)
> at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1019)
> ... 24 more
>
> Nov 20, 2008 5:25:37 PM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=javabin&version=2.2}
> status=500 QTime=3215
> Nov 20, 2008 5:25:37 PM org.apache.solr.common.SolrException log
> SEVERE: com.ctc.wstx.exc.WstxIOException: null
> at com.ctc.wstx.sr.StreamScanner.throwFromIOE(StreamScanner.java:708)
> at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1086)
> at
> org.apache.solr.handler.XmlUpdateRequestHandler.readDoc(XmlUpdateRequestHandler.java:321)
> at
> org.apache.solr.handler.XmlUpdateRequestHandler.processUpdate(XmlUpdateRequestHandler.java:195)
> at
> org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateRequestHandler.java:123)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
> at
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
> at
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
> at org.mortbay.jetty.Server.handle(Server.java:285)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:835)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:202)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
> at
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
> at
> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
> Caused by: org.mortbay.jetty.EofException
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:302)
> at
> org.mortbay.jetty.HttpParser$Input.blockForContent(HttpParser.java:919)
> at org.mortbay.jetty.HttpParser$Input.read(HttpParser.java:897)
> at sun.nio.cs.StreamDecoder$CharsetSD.readBytes(StreamDecoder.java:411)
> at sun.nio.cs.StreamDecoder$CharsetSD.implRead(StreamDecoder.java:453)
> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:183)
> at java.io.InputStreamReader.read(InputStreamReader.java:167)
> at com.ctc.wstx.io.MergedReader.read(MergedReader.java:101)
> at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:84)
> at
> com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.java:57)
> at
> com.ctc.wstx.sr.StreamScanner.loadMoreFromCurrent(StreamScanner.java:1046)
> at
> com.ctc.wstx.sr.StreamScanner.loadMoreFromCurrent(StreamScanner.java:1053)
> at
> com.ctc.wstx.sr.StreamScanner.getNextCharFromCurrent(StreamScanner.java:811)
> at
> com.ctc.wstx.sr.BasicStreamReader.readEndElem(BasicStreamReader.java:3206)
> at
> com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2832)
> at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1019)
> ... 24 more
>
>
>
> Thanks,
>
> Anoop Bhatti
> --
> Committed to open source technology.
>
