OK thanks Paul. I just committed another (hopefully, last!) fix, so if you get a chance can you try that (just "svn up")? Thanks, and, sorry,
Mike On Tue, Aug 3, 2010 at 12:40 PM, Paul Dlug <paul.d...@gmail.com> wrote: > Just reporting back, no issues on the latest branch3x build with your > revert of the optimization. > > > --Paul > > On Tue, Aug 3, 2010 at 9:22 AM, Paul Dlug <paul.d...@gmail.com> wrote: >> Sure, I'm reindexing now, I'll let you know how it goes. >> >> >> --Paul >> >> On Tue, Aug 3, 2010 at 9:05 AM, Michael McCandless >> <luc...@mikemccandless.com> wrote: >>> Ugh... I think there may still be a bug lurking. >>> >>> Karl is also still having problems, much further into his indexing process. >>> >>> I'm hunting it now!! >>> >>> For the time being, I just disabled (committed to trunk & 3x) the >>> optimization that's causing the bug. Can you update to 3x head (or >>> trunk head), remove your current index, and try again? >>> >>> Mike >>> >>> On Tue, Aug 3, 2010 at 8:52 AM, Paul Dlug <paul.d...@gmail.com> wrote: >>>> Thanks, I updated to the latest version with the fix but I'm now >>>> getting another error when optimizing the index (or when searching >>>> certain fields). It mentions "unknown compression method" but I'm not >>>> using compressed fields at all. >>>> >>>> SEVERE: java.io.IOException: background merge hit exception: >>>> _a:C248670/19645 _l:C206701/14563 _m:C12186/100 _n:C11356 _o:C9945 >>>> _p:C9000 _q:C5704 _r:C2214 _s:C2000 _t:C1264 into _u [optimize] >>>> [mergeDocStores] >>>> at >>>> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2392) >>>> at >>>> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2320) >>>> at >>>> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:403) >>>> at >>>> org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:85) >>>> at >>>> org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:169) >>>> at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:69) >>>> at >>>> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54) >>>> at >>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) >>>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1322) >>>> at >>>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:341) >>>> at >>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:244) >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >>>> at >>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) >>>> at >>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) >>>> at >>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) >>>> at >>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) >>>> at >>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) >>>> at >>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) >>>> at >>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857) >>>> at >>>> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) >>>> at >>>> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) >>>> at java.lang.Thread.run(Thread.java:619) >>>> Caused by: org.apache.lucene.index.CorruptIndexException: field data >>>> are in wrong format: java.util.zip.DataFormatException: unknown >>>> compression method >>>> at >>>> org.apache.lucene.index.FieldsReader.uncompress(FieldsReader.java:585) >>>> at >>>> org.apache.lucene.index.FieldsReader.addField(FieldsReader.java:357) >>>> at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:239) >>>> at >>>> org.apache.lucene.index.SegmentReader.document(SegmentReader.java:894) >>>> at >>>> org.apache.lucene.index.IndexReader.document(IndexReader.java:684) >>>> at >>>> org.apache.lucene.index.SegmentMerger.copyFieldsWithDeletions(SegmentMerger.java:410) >>>> at >>>> org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:338) >>>> at >>>> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:159) >>>> at >>>> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4053) >>>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3647) >>>> at >>>> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:339) >>>> at >>>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:407) >>>> Caused by: java.util.zip.DataFormatException: unknown compression method >>>> at java.util.zip.Inflater.inflateBytes(Native Method) >>>> at java.util.zip.Inflater.inflate(Inflater.java:238) >>>> at java.util.zip.Inflater.inflate(Inflater.java:256) >>>> at >>>> org.apache.lucene.document.CompressionTools.decompress(CompressionTools.java:106) >>>> at >>>> org.apache.lucene.index.FieldsReader.uncompress(FieldsReader.java:582) >>>> ... 11 more >>>> >>>> >>>> >>>> On Mon, Aug 2, 2010 at 6:04 PM, Michael McCandless >>>> <luc...@mikemccandless.com> wrote: >>>>> This looks like the index corruption caused by a commit on Friday. >>>>> >>>>> See the thread I sent earlier with subject "heads up -- index >>>>> corruption on Solr/Lucene trunk/3.x branch". >>>>> >>>>> Mike >>>>> >>>>> On Mon, Aug 2, 2010 at 6:00 PM, Paul Dlug <paul.d...@gmail.com> wrote: >>>>>> I'm running a recent build of branch3x (r981609), queries with >>>>>> multiple wildcards (e.g. a*b*c*) are failing with the exception below >>>>>> in the log. These queries worked fine for me with solr 1.4, known bug? >>>>>> >>>>>> >>>>>> SEVERE: java.lang.IndexOutOfBoundsException: Index: 114, Size: 39 >>>>>> at java.util.ArrayList.RangeCheck(ArrayList.java:547) >>>>>> at java.util.ArrayList.get(ArrayList.java:322) >>>>>> at >>>>>> org.apache.lucene.index.FieldInfos.fieldInfo(FieldInfos.java:285) >>>>>> at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:225) >>>>>> at >>>>>> org.apache.lucene.index.SegmentReader.document(SegmentReader.java:894) >>>>>> at >>>>>> org.apache.lucene.index.DirectoryReader.document(DirectoryReader.java:527) >>>>>> at >>>>>> org.apache.lucene.index.IndexReader.document(IndexReader.java:684) >>>>>> at >>>>>> org.apache.solr.search.SolrIndexReader.document(SolrIndexReader.java:444) >>>>>> at >>>>>> org.apache.solr.search.SolrIndexSearcher.doc(SolrIndexSearcher.java:428) >>>>>> at >>>>>> org.apache.solr.util.SolrPluginUtils.optimizePreFetchDocs(SolrPluginUtils.java:267) >>>>>> at >>>>>> org.apache.solr.handler.component.QueryComponent.doPrefetch(QueryComponent.java:279) >>>>>> at >>>>>> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:186) >>>>>> at >>>>>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195) >>>>>> at >>>>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) >>>>>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1322) >>>>>> at >>>>>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:341) >>>>>> at >>>>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:244) >>>>>> at >>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) >>>>>> at >>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >>>>>> at >>>>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) >>>>>> at >>>>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) >>>>>> at >>>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) >>>>>> at >>>>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) >>>>>> at >>>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) >>>>>> at >>>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) >>>>>> at >>>>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857) >>>>>> at >>>>>> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) >>>>>> at >>>>>> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) >>>>>> at java.lang.Thread.run(Thread.java:619) >>>>>> >>>>> >>>> >>> >> >
