[
https://issues.apache.org/jira/browse/LUCENE-9751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17283419#comment-17283419
]
Michael McCandless commented on LUCENE-9751:
--------------------------------------------
{quote}It's definitely not a single huge document, Mike.
{quote}
OK, hrmph.
{quote}I've tried reproducing last night (on a different machine but with the
same heap/ threads setup) but no luck - it finished successfully.
{quote}
Also hrmph.
{quote}I guess this means the problem is gone?... :)
{quote}
I wish! Non ignorance is non bliss!
> Assertion error (int overflow) in ByteSliceReader
> -------------------------------------------------
>
> Key: LUCENE-9751
> URL: https://issues.apache.org/jira/browse/LUCENE-9751
> Project: Lucene - Core
> Issue Type: Bug
> Affects Versions: 8.7
> Reporter: Dawid Weiss
> Priority: Major
>
> New computers come with insane amounts of ram and heaps can get pretty big.
> If you adjust per-thread buffers to larger values strange things start
> happening. This happened to us today:
> {code}
> Caused by: java.lang.AssertionError
> at
> org.apache.lucene.index.ByteSliceReader.init(ByteSliceReader.java:44)
> ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 -
> atrisharma - 2020-10-29 19:35:28]
> at
> org.apache.lucene.index.TermsHashPerField.initReader(TermsHashPerField.java:88)
> ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 -
> atrisharma - 2020-10-29 19:35:28]
> at
> org.apache.lucene.index.FreqProxFields$FreqProxPostingsEnum.reset(FreqProxFields.java:430)
> ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 -
> atrisharma - 2020-10-29 19:35:28]
> at
> org.apache.lucene.index.FreqProxFields$FreqProxTermsEnum.postings(FreqProxFields.java:247)
> ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 -
> atrisharma - 2020-10-29 19:35:28]
> at
> org.apache.lucene.codecs.PushPostingsWriterBase.writeTerm(PushPostingsWriterBase.java:127)
> ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 -
> atrisharma - 2020-10-29 19:35:28]
> at
> org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter$TermsWriter.write(BlockTreeTermsWriter.java:907)
> ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 -
> atrisharma - 2020-10-29 19:35:28]
> at
> org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter.write(BlockTreeTermsWriter.java:318)
> ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 -
> atrisharma - 2020-10-29 19:35:28]
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:170)
> ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 -
> atrisharma - 2020-10-29 19:35:28]
> at
> org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:120)
> ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 -
> atrisharma - 2020-10-29 19:35:28]
> at
> org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:264)
> ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 -
> atrisharma - 2020-10-29 19:35:28]
> at
> org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:350)
> ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 -
> atrisharma - 2020-10-29 19:35:28]
> at
> org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:480)
> ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 -
> atrisharma - 2020-10-29 19:35:28]
> at
> org.apache.lucene.index.DocumentsWriter.postUpdate(DocumentsWriter.java:394)
> ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 -
> atrisharma - 2020-10-29 19:35:28]
> at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:440)
> ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 -
> atrisharma - 2020-10-29 19:35:28]
> at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1471)
> ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 -
> atrisharma - 2020-10-29 19:35:28]
> ... 7 more
> {code}
> Likely an int overflow in TermsHashPerField:
> {code}
> reader.init(bytePool,
>
> postingsArray.byteStarts[termID]+stream*ByteBlockPool.FIRST_LEVEL_SIZE,
> streamAddressBuffer[offsetInAddressBuffer+stream]);
> {code}
> Don't know if this can be prevented somehow.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
