benwtrent opened a new issue, #14429:
URL: https://github.com/apache/lucene/issues/14429

   ### Description
   
   Found this in the wild. I haven't been able to replicate :(
   
   I don't even know what it means to hit this `fst.outputs.merge` branch and 
under what conditions is it valid/invalid. Any pointers here would be useful.
   
   
   We ran into a strange postings merge error in production.
   
   The FST compiler reaches the "merge" line when merging some segments:
   ```    if (lastInput.length() == input.length && prefixLenPlus1 == 1 + 
input.length) {
         // same input more than 1 time in a row, mapping to
         // multiple outputs
         lastNode.output = fst.outputs.merge(lastNode.output, output);```
   
   However, the "outputs" provided by `Lucene90BlockTreeTermsWriter` is 
`ByteSequenceOutputs`, which does not override merge, and thus throws an 
unsupported operation exception.
   
   Given this, it seems like it should be "impossible" to reach the 
"Outputs.merge" path when merging with the `Lucene90BlockTreeTermsWriter`, but 
somehow it did.
   
   Any ideas on where I should look?
   
   ```Caused by: 
org.elasticsearch.common.io.stream.NotSerializableExceptionWrapper: 
unsupported_operation_exception: null
   at org.apache.lucene.util.fst.Outputs.merge(Outputs.java:95) 
~[lucene-core-9.11.1.jar:?]
   at org.apache.lucene.util.fst.FSTCompiler.add(FSTCompiler.java:936) 
~[lucene-core-9.11.1.jar:?]
   at 
org.apache.lucene.codecs.lucene90.blocktree.Lucene90BlockTreeTermsWriter$PendingBlock.append(Lucene90BlockTreeTermsWriter.java:593)
 ~[lucene-core-9.11.1.jar:?]
   at 
org.apache.lucene.codecs.lucene90.blocktree.Lucene90BlockTreeTermsWriter$PendingBlock.compileIndex(Lucene90BlockTreeTermsWriter.java:562)
 ~[lucene-core-9.11.1.jar:?]
   at 
org.apache.lucene.codecs.lucene90.blocktree.Lucene90BlockTreeTermsWriter$TermsWriter.writeBlocks(Lucene90BlockTreeTermsWriter.java:776)
 ~[lucene-core-9.11.1.jar:?]
   at 
org.apache.lucene.codecs.lucene90.blocktree.Lucene90BlockTreeTermsWriter$TermsWriter.finish(Lucene90BlockTreeTermsWriter.java:1163)
 ~[lucene-core-9.11.1.jar:?]
   at 
org.apache.lucene.codecs.lucene90.blocktree.Lucene90BlockTreeTermsWriter.write(Lucene90BlockTreeTermsWriter.java:402)
 ~[lucene-core-9.11.1.jar:?]
   at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:95) 
~[lucene-core-9.11.1.jar:?]
   at 
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.merge(PerFieldPostingsFormat.java:204)
 ~[lucene-core-9.11.1.jar:?]
   at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:211) 
~[lucene-core-9.11.1.jar:?]
   at 
org.apache.lucene.index.SegmentMerger.mergeWithLogging(SegmentMerger.java:300) 
~[lucene-core-9.11.1.jar:?]
   at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:139) 
~[lucene-core-9.11.1.jar:?]
   at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:5293) 
~[lucene-core-9.11.1.jar:?]
   at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4761) 
~[lucene-core-9.11.1.jar:?]
   at 
org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:6582)
 ~[lucene-core-9.11.1.jar:?]
   at 
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:660)
 ~[lucene-core-9.11.1.jar:?]
   at 
org.elasticsearch.index.engine.ElasticsearchConcurrentMergeScheduler.doMerge(ElasticsearchConcurrentMergeScheduler.java:134)
 ~[elasticsearch-8.15.0.jar:?]
   at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:721)
 ~[lucene-core-9.11.1.jar:?]```
   
   ### Version and environment details
   
   Lucene 9.11.1
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

Reply via email to