I've run into this also while trying to index ~4 million documents
using CommonsHttpSolrServer.add(Iterator<SolrInputDocument>
docIterator) to stream all the documents. It worked great with
working with only ~2 million documents but eventually it would take ~5
tries for a full indexing job to complete without running into the
"Unbuffered entity enclosing request can not be repeated" error. I'd
say about 25% of the errors were within the first 1,000 documents and
the rest were usually when indexing was 80-90% complete.
I tried switching to the StreamingUpdateSolrServer but ran into issues
with it locking up (which I think have been fixed since then). Now I
use CommonsHttpSolrServer.add(Collection<SolrInputDocument> docs) to
batch up 250-500 documents per call with wrapped exception handling to
retry on errors.
-Tim
On Tue, May 11, 2010 at 8:57 AM, Satish Kumar
<satish.kumar.just.d...@gmail.com> wrote:
> I upload only 50 documents per call. We have about 200K documents to index,
> and we index every night. Any suggestions on how to handle this? (I can
> catch this exception and do a retry.)
>
> On Mon, May 10, 2010 at 8:33 PM, Lance Norskog <goks...@gmail.com> wrote:
>
>> Yes, these occasionally happen with long indexing jobs. You might try
>> limiting the number of documents per upload call.
>>
>> On Sun, May 9, 2010 at 9:16 PM, Satish Kumar
>> <satish.kumar.just.d...@gmail.com> wrote:
>> > Found these errors in Tomcat's log file:
>> >
>> > May 9, 2010 10:57:24 PM org.apache.solr.common.SolrException log
>> > SEVERE: java.lang.RuntimeException: [was class
>> > java.net.SocketTimeoutException] Read timed out
>> > at
>> >
>> com.ctc.wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18)
>> > at
>> > com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731)
>> > at
>> >
>> com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3657)
>> > at
>> > com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809)
>> > at org.apache.solr.handler.XMLLoader.readDoc(XMLLoader.java:279)
>> > at
>> > org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:138)
>> > at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:69)
>> > at
>> >
>> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54)
>> >
>> > at
>> >
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>> >
>> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
>> >
>> >
>> > at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
>> >
>> > at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
>> >
>> > at
>> >
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>> >
>> > at
>> >
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>> >
>> > at
>> >
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>> >
>> >
>> >
>> >
>> > May 9, 2010 10:57:24 PM org.apache.solr.core.SolrCore execute
>> >
>> >
>> > INFO: [] webapp=/solr path=/update params={wt=javabin&version=1}
>> status=500
>> > QTime=25938
>> >
>> > May 9, 2010 10:57:24 PM org.apache.solr.common.SolrException log
>> >
>> >
>> > SEVERE: java.lang.RuntimeException: [was class
>> > java.net.SocketTimeoutException] Read timed out
>> >
>> > at
>> >
>> com.ctc.wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18)
>> >
>> > at
>> > com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731)
>> >
>> > at
>> >
>> com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3657)
>> >
>> > at
>> > com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809)
>> >
>> > at org.apache.solr.handler.XMLLoader.readDoc(XMLLoader.java:279)
>> >
>> >
>> > at
>> > org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:138)
>> >
>> > at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:69)
>> >
>> >
>> > at
>> >
>> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54)
>> >
>> > at
>> >
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>> >
>> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
>> >
>> >
>> > at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
>> >
>> > at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
>> >
>> >
>> >
>> >
>> > May 9, 2010 10:57:33 PM
>> org.apache.solr.update.processor.LogUpdateProcessor
>> > finish
>> >
>> > INFO: {} 0 2
>> >
>> >
>> > May 9, 2010 10:57:33 PM org.apache.solr.common.SolrException log
>> >
>> >
>> > SEVERE: org.apache.solr.common.SolrException: Invalid chunk header
>> >
>> >
>> > at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:72)
>> >
>> >
>> > at
>> >
>> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54)
>> >
>> > at
>> >
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>> >
>> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
>> >
>> >
>> > at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
>> >
>> > at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
>> >
>> > at
>> >
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>> >
>> > :
>> >
>> >
>> > On Mon, May 10, 2010 at 12:10 AM, Satish Kumar <
>> > satish.kumar.just.d...@gmail.com> wrote:
>> >
>> >> Hi,
>> >>
>> >> I am getting the following error when I run the index process once in a
>> >> while. I'm using Solr 1.4. Any suggestions on how to resolve this error?
>> >>
>> >> Caused by: org.apache.solr.client.solrj.SolrServerException:
>> >> org.apache.commons.httpclient.ProtocolException: Unbuffered entity
>> enclosing
>> >> request can not be repeated.
>> >> at
>> >>
>> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:469)
>> >> at
>> >>
>> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:243)
>> >> at
>> >>
>> org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105)
>> >> at
>> org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:49)
>> >>
>> >> ... 3 more
>> >> Caused by: org.apache.commons.httpclient.ProtocolException: Unbuffered
>> >> entity enclosing request can not be repeated.
>> >> at
>> >>
>> org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:487)
>> >> at
>> >>
>> org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2114)
>> >> at
>> >>
>> org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
>> >> at
>> >>
>> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
>> >> at
>> >>
>> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
>> >> at
>> >>
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
>> >> at
>> >>
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
>> >> at
>> >>
>> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:416)
>> >> ... 7 more
>> >>
>> >>
>> >>
>> >> Thanks,
>> >> Satish
>> >>
>> >
>>
>>
>>
>> --
>> Lance Norskog
>> goks...@gmail.com
>>
>
