john-mlika opened a new pull request, #16619:
URL: https://github.com/apache/lucene/pull/16619
Creating a slice of a `DirectIOIndexInput` currently costs a device read.
`slice()` builds the new
input and then calls `seekInternal(0L)`, which fills the buffer from the
slice's first block. With
`O_DIRECT` that is a round-trip to the device, every time, whether or not
anything is ever read.
Every other `IndexInput` makes slicing free and reads on first use.
That matters for callers that create a view and throw it away.
`KnnFloatVectorQuery#approximateSearch`
asks for the float vector values only to null-check them and read `size()`;
obtaining them slices
the raw vector data, so each query pays one wasted read per segment whenever
a `DirectIODirectory`
subclass serves searches through `useDirectIO`.
## How this was noticed
On an Elasticsearch node serving kNN queries from an index whose raw vectors
are read through a
`DirectIODirectory` subclass, on a cloud disk with a provisioned IOPS
ceiling. Every query issued
small O_DIRECT reads that nothing consumed: this one, once per segment, next
to a prefetch of the
same kind in the `IndexedDISI` path (addressed separately). Because O_DIRECT
bypasses the page
cache, each of them is an IOPS every time, not a cache hit, and together
they exceeded the disk's
allowance. Past that ceiling the device holds throughput flat and queues —
on that disk class an
8 KiB read goes from 1.5 ms at queue depth 8 to 26 ms at depth 128 — so
every query, including the
ones that needed no I/O at all, waited behind reads that were then thrown
away. A JFR file-read
profile placed every one of this PR's reads at the slice construction inside
`Lucene99FlatVectorsReader#getFloatVectorValues`, one per query per segment.
With this change they
are gone: on a single-segment test index, 40 queries went from 40 reads of
the vector file to none.
## The change
Instead of reading at `slice()`, the slice remembers where it starts and
reads on first use.
Concretely, the constructor records the offset within the first block in a
new field,
`pendingDelta`, and the first `refill()` uses it to position the buffer.
Three details keep the
behaviour identical to today, quirks included:
- `seekInternal` clears the remembered offset before it repositions, so an
explicit seek checks
end-of-file against its own target rather than against the slice start.
- The first fill does exactly what the construction-time fill used to do. In
the one case where that
leaves nothing to read — a zero-length slice at the very end of the file —
it falls through to the
caller's own read, which throws `EOFException` as before rather than
`BufferUnderflowException`.
- `seek()` on a slice that has not been read yet fills first and only then
checks whether the target
is inside the buffer, so a seek past the slice's length that still lands
in that first block
repositions silently, exactly as it does today.
A caller that does read the slice pays the same single read, just at first
use.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
