jpountz commented on PR #14156: URL: https://github.com/apache/lucene/pull/14156#issuecomment-2605738891
> An outright madvise call should be about as expensive as the isLoaded check when things are already in the page cache The PR where `consecutivePrefetchHitCount` was introduced had a benchmark that demonstrated an improvement with the current logic: https://github.com/apache/lucene/pull/13381#issuecomment-2118319218. That said, this benchmark had a different access pattern compared to this one, which I guess is terms dictionary lookups performed by `TermQuery`, so I can believe that it's not always better. I wonder if you are able to create a benchmark that shows an improvement with this change. > The counter guarding the madvise callback is not all that effective in preventing madvise calls on slices since it's not shared with the original input Thinking out loud, this `isLoaded()` check only makes sense when `prefetch()` is called multiple times on the same `IndexInput`, which is never the case for terms dictionary lookups (as you pointed out). So another way of improving things here may be to disable the `isLoaded()` check on the first (or first few) calls to `prefetch()`. > Additionally, and orthogonally, I'm less sure of the checks anyway, since they don't count the against the actual memory range but rather against the whole segment. These checks make an assumption that if many ranges of bytes on which `prefetch()` has been called were already loaded in memory, then there are good chances that most of the segment is loaded in memory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
