goankur commented on PR #16145:
URL: https://github.com/apache/lucene/pull/16145#issuecomment-5806186207
Data point in favor of this, from a full-precision rerank workload over an
index much larger than
RAM (details in #16705): 25M Cohere-v3 1024-d vectors, 1-bit BBQ + fp32
rerank, ~101 GB of vectors in a 10 GB cgroup on local NVMe, single stream,
recall 0.967, kernel 6.18.48.
**The backoff costs 17x on p99: 194.44 ms with it, 11.56 ms without** (p50
44.81 vs 6.23 ms). On the
same box on kernel 6.1 it was 221.0 vs 17.09 ms, 13x.
The counters show the mechanism. With the backoff in place the process
issues **111 `madvise` calls
per query instead of 228**, and the suppressed half turns into **10x the
major faults — 543 per
query versus 53**. Each of those faults reads a readahead-shaped **17.23
KB** rather than the
**4.75 KB** actually wanted, and `aqu-sz` falls from 10.65 to 5.68. So the
backoff does not merely
skip a hint: it converts an issued prefetch into a synchronous fault that
also reads 3.6x more than needed.
Worth noting why this does not show up in a microbenchmark: in #16279 (4 KB
random reads, 100 GB file, cold) `mmapRandomBatchedPrefetch` measures 3.394
ops/ms with the backoff disabled versus
3.389 with it enabled — no difference, because every miss resets the
counter. The backoff only
bites when the access pattern mixes hits and misses, which is exactly what
a rerank shortlist does
against a partially cached index. That makes it invisible to the cold and
fully-cached cases most
benchmarks cover, and expensive in the case that actually matters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
