If you need to fix the index and maybe lose some data (in bad segments), check.... Lucene's CheckIndex (cmd-line app)
Otis -- Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch ----- Original Message ---- > From: Jake Brownell <ja...@benetech.org> > To: "solr-user@lucene.apache.org" <solr-user@lucene.apache.org> > Sent: Thu, January 7, 2010 3:08:55 PM > Subject: Corrupted Index > > Hi all, > > Our application uses solrj to communicate with our solr servers. We started a > fresh index yesterday after upping the maxFieldLength setting in solrconfig. > Our > task indexes content in batches and all appeared to be well until noonish > today, > when after 40k docs, I started seeing errors. I've placed three stack traces > below, the first occurred once and was the initial error, the second occurred > a > few times before the third started occurring on each request. I'd really > appreciate any insight into what could have caused this, a missing file and > then > a corrupt index. If you know we'll have to nuke the entire index and start > over > I'd like to know that too-oddly enough searches against the index appear to > be > working. > > Thanks! > Jake > > #1 > > January 7, 2010 12:10:06 PM CST Caught error; TaskWrapper block 1 > January 7, 2010 12:10:07 PM CST solr-home/core0/data/index/_fsk_1uj.del (No > such > file or directory) > > solr-home/core0/data/index/_fsk_1uj.del (No such file or directory) > > request: /core0/update solr-home/core0/data/index/_fsk_1uj.del (No such file > or > directory) > > solr-home/core0/data/index/_fsk_1uj.del (No such file or directory) > > request: /core0/update > January 7, 2010 12:10:07 PM CST solr-home/core0/data/index/_fsk_1uj.del (No > such > file or directory) > > solr-home/core0/data/index/_fsk_1uj.del (No such file or directory) > > request: /core0/update solr-home/core0/data/index/_fsk_1uj.del (No such file > or > directory) > > solr-home/core0/data/index/_fsk_1uj.del (No such file or directory) > > request: /core0/update > org.benetech.exception.WrappedException > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer#request(424) > > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer#request(243) > > org.apache.solr.client.solrj.request.AbstractUpdateRequest#process(105) > org.apache.solr.client.solrj.SolrServer#commit(86) > org.apache.solr.client.solrj.SolrServer#commit(75) > org.bookshare.search.solr.SolrSearchServerWrapper#add(63) > org.bookshare.search.solr.SolrSearchEngine#index(232) > > org.bookshare.service.task.SearchEngineIndexingTask#initialInstanceLoad(95) > org.bookshare.service.task.SearchEngineIndexingTask#run(53) > org.bookshare.service.scheduler.TaskWrapper#run(233) > java.util.TimerThread#mainLoop(512) > java.util.TimerThread#run(462) > Caused by: > solr-home/core0/data/index/_fsk_1uj.del (No such file or directory) > > solr-home/core0/data/index/_fsk_1uj.del (No such file or directory) > > request: /core0/update > org.apache.solr.common.SolrException > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer#request(424) > > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer#request(243) > > org.apache.solr.client.solrj.request.AbstractUpdateRequest#process(105) > org.apache.solr.client.solrj.SolrServer#commit(86) > org.apache.solr.client.solrj.SolrServer#commit(75) > org.bookshare.search.solr.SolrSearchServerWrapper#add(63) > org.bookshare.search.solr.SolrSearchEngine#index(232) > > org.bookshare.service.task.SearchEngineIndexingTask#initialInstanceLoad(95) > org.bookshare.service.task.SearchEngineIndexingTask#run(53) > org.bookshare.service.scheduler.TaskWrapper#run(233) > java.util.TimerThread#mainLoop(512) > java.util.TimerThread#run(462) > > #2 > > January 7, 2010 12:10:10 PM CST Caught error; TaskWrapper block 1 > January 7, 2010 12:10:10 PM CST > org.apache.lucene.index.CorruptIndexException: > doc counts differ for segment _hug: fieldsReader shows 8 but segmentInfo > shows 2 > > org.apache.lucene.index.CorruptIndexException: doc counts differ for segment > _hug: fieldsReader shows 8 but segmentInfo shows 2 > > request: /core0/update org.apache.lucene.index.CorruptIndexException: doc > counts > differ for segment _hug: fieldsReader shows 8 but segmentInfo shows 2 > > org.apache.lucene.index.CorruptIndexException: doc counts differ for segment > _hug: fieldsReader shows 8 but segmentInfo shows 2 > > request: /core0/update > January 7, 2010 12:10:10 PM CST > org.apache.lucene.index.CorruptIndexException: > doc counts differ for segment _hug: fieldsReader shows 8 but segmentInfo > shows 2 > > org.apache.lucene.index.CorruptIndexException: doc counts differ for segment > _hug: fieldsReader shows 8 but segmentInfo shows 2 > > request: /core0/update org.apache.lucene.index.CorruptIndexException: doc > counts > differ for segment _hug: fieldsReader shows 8 but segmentInfo shows 2 > > org.apache.lucene.index.CorruptIndexException: doc counts differ for segment > _hug: fieldsReader shows 8 but segmentInfo shows 2 > > request: /core0/update > org.benetech.exception.WrappedException > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer#request(424) > > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer#request(243) > > org.apache.solr.client.solrj.request.AbstractUpdateRequest#process(105) > org.apache.solr.client.solrj.SolrServer#commit(86) > org.apache.solr.client.solrj.SolrServer#commit(75) > org.bookshare.search.solr.SolrSearchServerWrapper#add(63) > org.bookshare.search.solr.SolrSearchEngine#index(232) > > org.bookshare.service.task.SearchEngineIndexingTask#initialInstanceLoad(95) > org.bookshare.service.task.SearchEngineIndexingTask#run(53) > org.bookshare.service.scheduler.TaskWrapper#run(233) > java.util.TimerThread#mainLoop(512) > java.util.TimerThread#run(462) > Caused by: > org.apache.lucene.index.CorruptIndexException: doc counts differ for segment > _hug: fieldsReader shows 8 but segmentInfo shows 2 > > org.apache.lucene.index.CorruptIndexException: doc counts differ for segment > _hug: fieldsReader shows 8 but segmentInfo shows 2 > > request: /core0/update > org.apache.solr.common.SolrException > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer#request(424) > > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer#request(243) > > org.apache.solr.client.solrj.request.AbstractUpdateRequest#process(105) > org.apache.solr.client.solrj.SolrServer#commit(86) > org.apache.solr.client.solrj.SolrServer#commit(75) > org.bookshare.search.solr.SolrSearchServerWrapper#add(63) > org.bookshare.search.solr.SolrSearchEngine#index(232) > > org.bookshare.service.task.SearchEngineIndexingTask#initialInstanceLoad(95) > org.bookshare.service.task.SearchEngineIndexingTask#run(53) > org.bookshare.service.scheduler.TaskWrapper#run(233) > java.util.TimerThread#mainLoop(512) > java.util.TimerThread#run(462) > > > #3 > > > January 7, 2010 12:10:21 PM CST Caught error; TaskWrapper block 1 > January 7, 2010 12:10:21 PM CST Array index out of range: 7 > > Array index out of range: 7 > > request: /core0/update Array index out of range: 7 > > Array index out of range: 7 > > request: /core0/update > January 7, 2010 12:10:21 PM CST Array index out of range: 7 > > Array index out of range: 7 > > request: /core0/update Array index out of range: 7 > > Array index out of range: 7 > > request: /core0/update > org.benetech.exception.WrappedException > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer#request(424) > > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer#request(243) > > org.apache.solr.client.solrj.request.AbstractUpdateRequest#process(105) > org.apache.solr.client.solrj.SolrServer#commit(86) > org.apache.solr.client.solrj.SolrServer#commit(75) > org.bookshare.search.solr.SolrSearchServerWrapper#add(63) > org.bookshare.search.solr.SolrSearchEngine#index(232) > > org.bookshare.service.task.SearchEngineIndexingTask#initialInstanceLoad(95) > org.bookshare.service.task.SearchEngineIndexingTask#run(53) > org.bookshare.service.scheduler.TaskWrapper#run(233) > java.util.TimerThread#mainLoop(512) > java.util.TimerThread#run(462) > Caused by: > Array index out of range: 7 > > Array index out of range: 7 > > request: /core0/update > org.apache.solr.common.SolrException > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer#request(424) > > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer#request(243) > > org.apache.solr.client.solrj.request.AbstractUpdateRequest#process(105) > org.apache.solr.client.solrj.SolrServer#commit(86) > org.apache.solr.client.solrj.SolrServer#commit(75) > org.bookshare.search.solr.SolrSearchServerWrapper#add(63) > org.bookshare.search.solr.SolrSearchEngine#index(232) > > org.bookshare.service.task.SearchEngineIndexingTask#initialInstanceLoad(95) > org.bookshare.service.task.SearchEngineIndexingTask#run(53) > org.bookshare.service.scheduler.TaskWrapper#run(233) > java.util.TimerThread#mainLoop(512) > java.util.TimerThread#run(462)
