rmuir commented on PR #11867:
URL: https://github.com/apache/lucene/pull/11867#issuecomment-1286335299
With current test i hit the exception on the 9.4 tag: BUILD FAILED in 2h 24m
45s:
2GB heap. Never saw any significant time (e.g. 0.1%) in GC or other jvm
threads when inspecting the running test:
The initial indexing takes about an hour and then the forcemerge takes an
eternity (over an hour), but it works:
```
org.apache.lucene.document.TestManyKnnVectors > testLargeSegment FAILED
java.lang.IllegalStateException: Vector data length 3072000000 not
matching size=1000000 * dim=768 * byteSize=4 = -1222967296
at
__randomizedtesting.SeedInfo.seed([CF186B7BCEFCCF79:EBD7012A6CACC57]:0)
at
org.apache.lucene.codecs.lucene94.Lucene94HnswVectorsReader.validateFieldEntry(Lucene94HnswVectorsReader.java:185)
at
org.apache.lucene.codecs.lucene94.Lucene94HnswVectorsReader.readFields(Lucene94HnswVectorsReader.java:156)
at
org.apache.lucene.codecs.lucene94.Lucene94HnswVectorsReader.readMetadata(Lucene94HnswVectorsReader.java:103)
at
org.apache.lucene.codecs.lucene94.Lucene94HnswVectorsReader.<init>(Lucene94HnswVectorsReader.java:64)
at
org.apache.lucene.codecs.lucene94.Lucene94HnswVectorsFormat.fieldsReader(Lucene94HnswVectorsFormat.java:157)
at
org.apache.lucene.codecs.perfield.PerFieldKnnVectorsFormat$FieldsReader.<init>(PerFieldKnnVectorsFormat.java:219)
at
org.apache.lucene.codecs.perfield.PerFieldKnnVectorsFormat.fieldsReader(PerFieldKnnVectorsFormat.java:81)
at
org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:157)
at
org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:91)
at
org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:179)
at
org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:221)
at
org.apache.lucene.index.IndexWriter.lambda$getReader$0(IndexWriter.java:536)
at
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:138)
at
org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:598)
at
org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:112)
at
org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:91)
at
org.apache.lucene.document.TestManyKnnVectors.testLargeSegment(TestManyKnnVectors.java:94)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
