One more question: when you run CheckIndex, are you enabling asserts?
(java -ea:org.apache.lucene)?
Mike
James Brady wrote:
Thanks for your answers Michael! I was using a pre-1.3 Solr build,
but I've
now upgraded to the 1.3 release, run the new CheckIndex shipped as
part of
the Lucene 2.4 dev build and I'm still getting the
CorruptIndexException:
docs out of order exceptions I'm afraid.
Upon a fresh start, on newly Checked indices, I actually get a lot of
Exceptions like:
SEVERE: java.lang.RuntimeException: [was class
org.mortbay.jetty.EofException] null
at
com
.ctc
.wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18)
at
com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731)
at
com
.ctc
.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:
3657)
at
com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809)
at
org
.apache
.solr
.handler
.XmlUpdateRequestHandler.readDoc(XmlUpdateRequestHandler.java:327)
at
org
.apache
.solr
.handler
.XmlUpdateRequestHandler.processUpdate(XmlUpdateRequestHandler.java:
195)
at
org
.apache
.solr
.handler
.XmlUpdateRequestHandler
.handleRequestBody(XmlUpdateRequestHandler.java:123)
Before any CorruptIndexExceptions - could that be the root cause?
Unfortunately the indices are large and contain confidential
information; is
there anything else I can do to identify where the problem is and why
CheckIndex isn't catching it?
Thanks
James
2009/2/23 Michael McCandless <luc...@mikemccandless.com>
Actually, even in 2.3.1, CheckIndex checks for docs-out-of-order both
within and across segments, so now I'm at a loss as to why it's not
catching
your case. Any of these indexes small enough to post somewhere i
could
access?
Mike
James Brady wrote:
Hi,My indices sometime become corrupted - normally when Solr has to
be
KILLed - these are not normally too much of a problem, as
Lucene's CheckIndex tool can normally detect missing / broken
segments and
fix them.
However, I now have a few indices throwing errors like this:
INFO: [core4] webapp=/solr path=/update params={} status=0 QTime=2
Exception in thread "Thread-75"
org.apache.lucene.index.MergePolicy$MergeException:
org.apache.lucene.index.CorruptIndexException: docs out of order
(1124 <=
1138 )
at
org.apache.lucene.index.ConcurrentMergeScheduler
$MergeThread.run(ConcurrentMergeScheduler.java:271)
Caused by: org.apache.lucene.index.CorruptIndexException: docs out
of
order
(1124 <= 1138 )
at
org
.apache
.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:502)
at
org
.apache
.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:456)
at
org
.apache
.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:425)
at
org
.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:
389)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:
134)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:
3109)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
at
org.apache.lucene.index.ConcurrentMergeScheduler
$MergeThread.run(ConcurrentMergeScheduler.java:240)
and
INFO: [core7] webapp=/solr path=/update params={} status=500
QTime=5457
Feb 22, 2009 12:14:07 PM org.apache.solr.common.SolrException log
SEVERE: org.apache.lucene.index.CorruptIndexException: docs out of
order
(242 <= 248 )
at
org
.apache
.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:502)
at
org
.apache
.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:456)
at
org
.apache
.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:425)
at
org
.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:
389)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:
134)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:
3109)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
at
org
.apache
.lucene
.index
.ConcurrentMergeScheduler.merge(ConcurrentMergeScheduler.java:193)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:
1800)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:
1795)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:
1791)
at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:2398)
at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:
1465)
at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:
1424)
at
org
.apache
.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:
278)
CheckIndex reports these cores as being completely healthy, and
yet I
can't
commit new documents in to them.
Rebuilding indices isn't an option for me: is there any other way
to fix
this? If not, any ideas on what I can do to prevent it in the
future?
Many thanks,
James
