Re: "Auto commit error" and java.io.FileNotFoundException

Michael McCandless Mon, 18 Aug 2008 03:06:04 -0700

The output from CheckIndex shows quite a few missing files! Is thereany possibility that two instances of Solr were somehow sharing thesame index directory?

It looks like you are using the 2.3 version of the Lucene jar (not thetrunk version). Which version of Solr are you using?

Since it seems reproducible, could you turn on debugging output(IndexWriter.setInfoStream(...)), get the FileNotFoundException tohappen again, and post the resulting output?


Mike

Chris Harris wrote:

On Sat, Aug 16, 2008 at 4:33 AM, Grant Ingersoll<[EMAIL PROTECTED]> wrote:

Can you try Lucene's CheckIndex tool on it and report what it says?

On Aug 15, 2008, at 1:35 PM, Chris Harris wrote:

I have an index (different from the ones mentioned yesterday) thatwas
working fine with 3M docs or so, but when I added a bunch more docs,
bringing it closer to 4M docs, the index seemed to get corrupted. In
particular, now when I start Solr up, or when when my indexingprocess
tries add a document, I get a complaint about missing index files.

The error on startup looks like this:

[...]


So I've run the Lucene CheckIndex tool twice.

First I ran it on my Windows machine, on the original compound format
index, which I then moved over to Linux and added to. It checked out
ok:


 NOTE: testing will be more thorough if you run java with
'-ea:org.apache.lucene', so assertions are enabled

 Opening index @ E:\solr-9999\solr\exhibitcore\data\index

 Segments file=segments_2 numSegments=1
version=FORMAT_SHARED_DOC_STORE [Lucene 2.3]
   1 of 1: name=_0 docCount=2829254
     compound=true
     numFiles=1
     size (MB)=30,423.298
     no deletions
     test: open reader.........OK
     test: fields, norms.......OK [21 fields]
     test: terms, freq, prox...OK [20208545 terms; 1092415125
terms/docs pairs; 6041972577 tokens]
     test: stored fields.......OK [82628579 total field count; avg
29.205 fields per doc]
     test: term vectors........OK [0 total vector count; avg 0
term/freq vector fields per doc]

 No problems were detected with this index.


Then I ran it on Linux, against the index in its problematic state.
(Note: This is the end of my message. Everything that follows is
output from the tool):


 Opening index @
/home/guy/corrupt-solr-9999-from-ssd/solr/exhibitcore/data/index/

 Segments file=segments_10tu numSegments=48
version=FORMAT_SHARED_DOC_STORE [Lucene 2.3]
   1 of 48: name=_0 docCount=2829254
     compound=true
     numFiles=1
     size (MB)=30,423.298
     no deletions
     test: open reader.........OK
     test: fields, norms.......OK [21 fields]
     test: terms, freq, prox...OK [20208545 terms; 1092415125
terms/docs pairs; 6041972577 tokens]
     test: stored fields.......OK [82628579 total field count; avg
29.205 fields per doc]
     test: term vectors........OK [0 total vector count; avg 0
term/freq vector fields per doc]

   2 of 48: name=_uz docCount=1567952
     compound=false
     numFiles=8
     size (MB)=17,238.136
     no deletions
     test: open reader.........OK
     test: fields, norms.......OK [21 fields]
     test: terms, freq, prox...OK [11529959 terms; 613302128
terms/docs pairs; 3503689207 tokens]
     test: stored fields.......OK [45879263 total field count; avg
29.261 fields per doc]
     test: term vectors........OK [0 total vector count; avg 0
term/freq vector fields per doc]

   3 of 48: name=_v7 docCount=24507
     compound=false
     numFiles=0
     size (MB)=0
     no deletions
     test: open reader.........FAILED
     WARNING: would remove reference to this segment (-fix was not
specified); full exception:
 java.io.FileNotFoundException:

/home/guy/corrupt-solr-9999-from-ssd/solr/exhibitcore/data/index/_v7.fnm

(No such file or directory)
        at java.io.RandomAccessFile.open(Native Method)
        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)

at org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:539)at org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:569)at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:478)at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:473)

        at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:57)

atorg.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:300)

        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:264)
        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:199)
        at org.apache.lucene.index.CheckIndex.check(CheckIndex.java:178)
        at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:433)

   4 of 48: name=_v6 docCount=1
     compound=false
     numFiles=0
     size (MB)=0
     docStoreOffset=6095
     docStoreSegment=_v4
     docStoreIsCompoundFile=false
     no deletions
     test: open reader.........FAILED
     WARNING: would remove reference to this segment (-fix was not
specified); full exception:
 java.io.FileNotFoundException:

/home/guy/corrupt-solr-9999-from-ssd/solr/exhibitcore/data/index/_v6.fnm

(No such file or directory)
        at java.io.RandomAccessFile.open(Native Method)
        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)

        at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:57)

atorg.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:300)

        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:264)
        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:199)
        at org.apache.lucene.index.CheckIndex.check(CheckIndex.java:178)
        at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:433)

   5 of 48: name=_v8 docCount=1714
     compound=false
     numFiles=0
     size (MB)=0
     docStoreOffset=0
     docStoreSegment=_v8
     docStoreIsCompoundFile=false
     no deletions
     test: open reader.........FAILED
     WARNING: would remove reference to this segment (-fix was not
specified); full exception:
 java.io.FileNotFoundException:

/home/guy/corrupt-solr-9999-from-ssd/solr/exhibitcore/data/index/_v8.fnm

(No such file or directory)
        at java.io.RandomAccessFile.open(Native Method)
        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)

        at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:57)

atorg.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:300)

        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:264)
        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:199)
        at org.apache.lucene.index.CheckIndex.check(CheckIndex.java:178)
        at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:433)

   6 of 48: name=_v9 docCount=2807
     compound=false
     numFiles=6
     size (MB)=7.41
     docStoreOffset=1714
     docStoreSegment=_v8
     docStoreIsCompoundFile=false
     no deletions
     test: open reader.........FAILED
     WARNING: would remove reference to this segment (-fix was not
specified); full exception:
 java.io.FileNotFoundException:

/home/guy/corrupt-solr-9999-from-ssd/solr/exhibitcore/data/index/_v8.fdt

(No such file or directory)
        at java.io.RandomAccessFile.open(Native Method)
        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)

        at org.apache.lucene.index.FieldsReader.<init>(FieldsReader.java:77)