kaivalnp commented on PR #14843:
URL: https://github.com/apache/lucene/pull/14843#issuecomment-3043135788
@mikemccand it succeeded a few times, but I ran into a failure after
multiple runs:
Command (I'm on JDK21):
```sh
gradlew :lucene:sandbox:test --tests
"org.apache.lucene.sandbox.codecs.faiss.TestFaissKnnVectorsFormat.testLargeVectorData"
-Ptests.heapsize=16g -Ptests.jvms=12 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1
-XX:+UseParallelGC -XX:ActiveProcessorCount=1" -Ptests.seed=6FE1439CE22DB5DA
-Ptests.useSecurityManager=true -Ptests.monster=true -Ptests.gui=false
-Ptests.file.encoding=UTF-8 -Ptests.vectorsize=256
-Ptests.forceintegervectors=true -Ptests.faiss.run=true
```
Error:
```
TestFaissKnnVectorsFormat > testLargeVectorData FAILED
org.apache.lucene.index.CorruptIndexException: misplaced codec footer
(file extended?): remaining=2147483666, expected=16, fp=78
(resource=BufferedChecksumIndexInput(MockIndexInputWrapper((clone of)
ByteBuffersIndexInput (file=_0_FaissKnnVectorsFormat_0.vec,
buffers=2,147,483,744 bytes, block size: 33,554,432, blocks: 65, position: 0))))
at
__randomizedtesting.SeedInfo.seed([6FE1439CE22DB5DA:1BB7D7D4A612D229]:0)
at
app/org.apache.lucene.core@10.3.0-SNAPSHOT/org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:579)
at
app/org.apache.lucene.core@10.3.0-SNAPSHOT/org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:433)
at
app/org.apache.lucene.core@10.3.0-SNAPSHOT/org.apache.lucene.codecs.lucene90.Lucene90CompoundFormat.writeCompoundFile(Lucene90CompoundFormat.java:145)
at
app/org.apache.lucene.core@10.3.0-SNAPSHOT/org.apache.lucene.codecs.lucene90.Lucene90CompoundFormat.write(Lucene90CompoundFormat.java:99)
at
app/org.apache.lucene.core@10.3.0-SNAPSHOT/org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:5900)
```
The codec footer check seems to fail on the `.vec` file related to
[`Lucene99FlatVectorsFormat`](https://github.com/apache/lucene/blob/b4f34d0f9f83c5e71c380cd5beb57b9073d602a7/lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99FlatVectorsFormat.java#L68C20-L68C45)
(which we use to store raw vectors _outside_ Faiss) -- to verify this I
replaced the format
[here](https://github.com/apache/lucene/blob/b4f34d0f9f83c5e71c380cd5beb57b9073d602a7/lucene/sandbox/src/test/org/apache/lucene/sandbox/codecs/faiss/TestFaissKnnVectorsFormat.java#L121-L124)
to use only the flat format (and _not_ the new Faiss format):
```java
KnnVectorsFormat format =
new
Lucene99FlatVectorsFormat(FlatVectorScorerUtil.getLucene99FlatVectorsScorer());
```
and the test still fails, so it seems unrelated / pre-existing?
FYI I see the same failure on `main` after multiple runs..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
