[
https://issues.apache.org/jira/browse/LUCENE-9652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Sokolov resolved LUCENE-9652.
-------------------------------------
Resolution: Fixed
> DataInput.readFloats to be used by Lucene90VectorReader
> -------------------------------------------------------
>
> Key: LUCENE-9652
> URL: https://issues.apache.org/jira/browse/LUCENE-9652
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Michael Sokolov
> Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Benchmarking shows a substantial performance gain can be realized by avoiding
> the additional memory copy we must do today when converting from {{byte[]}}
> read using {{IndexInput}} into {{float[]}} returned by
> {{Lucene90VectorReader}}. We have a model for how to handle the various
> alignments, and buffer underflow when a value spans buffers, in
> {{readLELongs}}.
> I think we should only support little-endian floats from the beginning here.
> We're planning to move towards switching the whole IndexInput to that
> endianness, right?
> Lucene90VectorWriter relies on {{VectorValues.binaryValue()}} to return bytes
> in the format expected by the reader, and its javadocs don't currently
> specify their endianness. In fact the order has been the default supplied by
> {{ByteBuffer.allocate(int)}}, which I now realize is big-endian, so this
> issue also proposes to change the index format. That would mean a
> backwards-incompatible index change, but I think if we're still unreleased
> and in an experimental class that should be OK?
> Also, we don't need a corresponding {{DataOutput.writeFloats}} to support the
> current usage for vectors, since there we rely on {{VectorValues}} to do the
> conversion, so I don't plan to implement that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
