Thanks for your answers Michael! I was using a pre-1.3 Solr build, but I've
now upgraded to the 1.3 release, run the new CheckIndex shipped as part of
the Lucene 2.4 dev build and I'm still getting the CorruptIndexException:
docs out of order exceptions I'm afraid.
Upon a fresh start, on newly Checked indices, I actually get a lot of
Exceptions like:
SEVERE: java.lang.RuntimeException: [was class
org.mortbay.jetty.EofException] null
at
com.ctc.wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18)
at
com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731)
at
com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3657)
at
com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809)
at
org.apache.solr.handler.XmlUpdateRequestHandler.readDoc(XmlUpdateRequestHandler.java:327)
at
org.apache.solr.handler.XmlUpdateRequestHandler.processUpdate(XmlUpdateRequestHandler.java:195)
at
org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateRequestHandler.java:123)
Before any CorruptIndexExceptions - could that be the root cause?
Unfortunately the indices are large and contain confidential information; is
there anything else I can do to identify where the problem is and why
CheckIndex isn't catching it?
Thanks
James
2009/2/23 Michael McCandless <luc...@mikemccandless.com>
>
> Actually, even in 2.3.1, CheckIndex checks for docs-out-of-order both
> within and across segments, so now I'm at a loss as to why it's not catching
> your case. Any of these indexes small enough to post somewhere i could
> access?
>
> Mike
>
>
> James Brady wrote:
>
> Hi,My indices sometime become corrupted - normally when Solr has to be
>> KILLed - these are not normally too much of a problem, as
>> Lucene's CheckIndex tool can normally detect missing / broken segments and
>> fix them.
>>
>> However, I now have a few indices throwing errors like this:
>>
>> INFO: [core4] webapp=/solr path=/update params={} status=0 QTime=2
>> Exception in thread "Thread-75"
>> org.apache.lucene.index.MergePolicy$MergeException:
>> org.apache.lucene.index.CorruptIndexException: docs out of order (1124 <=
>> 1138 )
>> at
>>
>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:271)
>> Caused by: org.apache.lucene.index.CorruptIndexException: docs out of
>> order
>> (1124 <= 1138 )
>> at
>>
>> org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:502)
>> at
>>
>> org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:456)
>> at
>>
>> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:425)
>> at
>> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:389)
>> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:134)
>> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3109)
>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
>> at
>>
>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:240)
>>
>> and
>>
>> INFO: [core7] webapp=/solr path=/update params={} status=500 QTime=5457
>> Feb 22, 2009 12:14:07 PM org.apache.solr.common.SolrException log
>> SEVERE: org.apache.lucene.index.CorruptIndexException: docs out of order
>> (242 <= 248 )
>> at
>>
>> org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:502)
>> at
>>
>> org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:456)
>> at
>>
>> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:425)
>> at
>> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:389)
>> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:134)
>> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3109)
>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
>> at
>>
>> org.apache.lucene.index.ConcurrentMergeScheduler.merge(ConcurrentMergeScheduler.java:193)
>> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1800)
>> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1795)
>> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1791)
>> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:2398)
>> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1465)
>> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1424)
>> at
>>
>> org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:278)
>>
>>
>> CheckIndex reports these cores as being completely healthy, and yet I
>> can't
>> commit new documents in to them.
>>
>> Rebuilding indices isn't an option for me: is there any other way to fix
>> this? If not, any ideas on what I can do to prevent it in the future?
>>
>> Many thanks,
>> James
>>
>
>
