iverase commented on PR #13948: URL: https://github.com/apache/lucene/pull/13948#issuecomment-2439897160
I have a second iteration and I realise that this change can be easily done if we introduce a new abstraction. I have called it `RandomAccessInputRef` and it is equivalent to BytesRef but backed by a RandomAccessInput instead of a byte array. I am thinking of it s the off-heap version of BytesRef. Using this, it makes the changes almost mechanical. The big difference between BytesRef and RandomAccessInputRef is that the latter cannot implement efficient the Compare interface so it cannot be use for comparisons and equality. In that case we need to read bytes on-heap. With this new abstraction, we just need to add three new methods: - BytesRefBuilder#copyBytes(RandomAccessInputRef in) throws IOException - BytesRefArray#append(RandomAccessInputRef bytes) throws IOException - ByteBlockPool#append(final RandomAccessInput bytes, long offset, int length) throws IOException: for supporting the method above) - And two helper classes: - ByteArrayRandomAccessInput: To represent and byte array as a RandomAccessInput - RandomAccessInputDataInput: to wrap a RandomAccessInput as a DataInput so we can use DataOutput#copyBytes(DataInput input, long numBytes) There are few places that still needs to be adapted to take full advantage of the new API but they can be done in a follow up as they require a bit of work : - MatchingFacetSetsCounts - BaseShapeDocValuesQuery - SerializedDVStrategy - MonitorQuerySerializer And there are two places that we will need to put the data on-heap as they are using equality: - TermValComparator - BinaryDocValueSelector -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
