Michael Sokolov created LUCENE-9652:
---------------------------------------
Summary: DataInput.readFloats to be used by Lucene90VectorReader
Key: LUCENE-9652
URL: https://issues.apache.org/jira/browse/LUCENE-9652
Project: Lucene - Core
Issue Type: Improvement
Reporter: Michael Sokolov
Benchmarking shows a substantial performance gain can be realized by avoiding
the additional memory copy we must do today when converting from `byte[]` read
using IndexInput into `float[]` returned by `Lucene90VectorReader`. We have a
model for how to handle the various alignments, and buffer underflow when a
value spans buffers, in `readLELongs`.
I think we should only support little-endian floats from the beginning here.
We're planning to move towards switching the whole IndexInput to that
endianness, right?
Lucene90VectorWriter relies on {VectorValues.binaryValue()} to return bytes in
the format expected by the reader, and its javadocs don't currently specify
their endianness. In fact the order has been the default supplied by
{ByteBuffer.allocate(int)}, which I now realize is big-endian, so this issue
also proposes to change the index format. That would mean a
backwards-incompatible index change, but I think if we're still unreleased and
in an experimental class that should be OK?
Also, we don't need a corresponding {DataOutput.writeFloats} to support the
current usage for vectors, since there we rely on {VectorValues} to do the
conversion, so I don't plan to implement that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]