Re: "background merge hit exception"

Wed, 03 Sep 2008 03:15:43 -0700 

OK I opened:
    https://issues.apache.org/jira/browse/LUCENE-1374

Mike

Chris Harris wrote:


I've made some changes to my Solr setup, and now I'm getting the
"background merge hit exception" pasted at the end of this message.
The most notable changes I've made are:

Update to r690989 (Lucene r688745)

Change a few things in my schema. In particular, I was previously
storing my main document text and the metadata fields in a single
"body" field, like this:

*****

  <!-- This is what we use for the default text search and for
highlighting : -->
  <field name="body" type="shingleText" indexed="true" stored="true"
omitNorms="false" multiValued="true" />

  <copyField source="companyname" dest="body"/>
  <copyField source="date" dest="body"/>

*****

whereas I'm now using "body" as a sort of alias that just gets
redirected to other fields, like this:

*****
<!-- This is only an alias used to populate other fields using copyField. --> <field name="body" type="shingleText" indexed="false" stored="false" /> 
  <!-- This is what we use for the default text search: -->
  <field name="everything" type="shingleText" indexed="true"
stored="false" omitNorms="false" multiValued="true" />

  <!-- Stored text for use with highlighting -->
  <field name="kwic" type="shingleText" indexed="false" stored="true"
compressed="true" multiValued="false" />

   <!-- Note: The maxLength parameter is for the copyField max length
patch (SOLR-538). At first I thought
                     that patch was causing my difficulties, but then
I had the same trouble when I built without that patch -->
  <copyField source="body" dest="kwic" maxLength="100000" />

  <copyField source="body" dest="everything"/>
  <copyField source="companyname" dest="everything"/>
  <copyField source="date" dest="everything"/>

*****

When I was indexing with this new setup, things were initially fine,
and segments seemed to be merging fine. I ran into trouble when I sent
an <optimize/>, though. I think in an earlier run I also got a very
similar exception just from document adds, without an explicit
<optimize/>.

I'm also running with a shingle-related patch
(https://issues.apache.org/jira/browse/LUCENE-1370 /
https://issues.apache.org/jira/browse/SOLR-744) and the rich document
handler patch, though I've used these before without trouble.

Is it possible that my schema change is illegitimate? Am I not allowed
to have non-indexed, non-stored fields, for example?

Anyway, here is my stack trace:

*****

background merge hit exception: _1h:C2552 _1i:C210->_1i _1j:C266->_1i
_1k:C214->_1i _1l:C329->_1i _1m:C231->_1i _1n:C379->_1i _1o:C447
_1p:C453->_1p _1q:C485->_1p into _1r [optimize]

java.io.IOException: background merge hit exception: _1h:C2552
_1i:C210->_1i _1j:C266->_1i _1k:C214->_1i _1l:C329->_1i _1m:C231->_1i
_1n:C379->_1i _1o:C447 _1p:C453->_1p _1q:C485->_1p into _1r [optimize]
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java: 2303) at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java: 2233) at org .apache .solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java: 355) at org .apache .solr .update .processor .RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:77) at org .apache .solr .handler .XmlUpdateRequestHandler.processUpdate(XmlUpdateRequestHandler.java: 228) at org .apache .solr .handler .XmlUpdateRequestHandler .handleRequestBody(XmlUpdateRequestHandler.java:125) at org .apache .solr .handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) 
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1156)
at org .apache .solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:341) at org .apache .solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:272) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1089) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java: 365) at org .mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java: 216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java: 181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java: 712) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java: 405) at org .mortbay .jetty .handler .ContextHandlerCollection.handle(ContextHandlerCollection.java:211) at org .mortbay .jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java: 139) 
        at org.mortbay.jetty.Server.handle(Server.java:285)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java: 502) at org.mortbay.jetty.HttpConnection $RequestHandler.content(HttpConnection.java:835) 
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
at org.mortbay.jetty.bio.SocketConnector $Connection.run(SocketConnector.java:226) at org.mortbay.thread.BoundedThreadPool $PoolThread.run(BoundedThreadPool.java:442) 
Caused by: java.lang.NullPointerException
        at java.lang.System.arraycopy(Native Method)
at org .apache .lucene .store.BufferedIndexOutput.writeBytes(BufferedIndexOutput.java:49) at org.apache.lucene.index.FieldsWriter.writeField(FieldsWriter.java:215) at org.apache.lucene.index.FieldsWriter.addDocument(FieldsWriter.java: 268) at org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java: 359) at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java: 138) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java: 3988) 
        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3637)
at org .apache .lucene .index .ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:214) at org.apache.lucene.index.ConcurrentMergeScheduler $MergeThread.run(ConcurrentMergeScheduler.java:269) 





















					Reply via email to