mikemccand opened a new issue, #15286: URL: https://github.com/apache/lucene/issues/15286
### Description In the cold case (KNN vectors don't all fit in RAM), it should help to add prefetching to our HNSW graph crawl? When we pop the best node out of the work queue, we could ask for all of its neighbor node's vectors to be prefetched before we loop through them evaluating their distance and inserting them into the work queue. We could maybe go further and do so for the top N nodes in the queue. Modern SSDs are typically highly concurrent (maybe 1000s of concurrent requests before they saturate?) so we should use that. Lucene's prefetch API is very similar to async IO, but maybe not quite as efficient (CPU core will sometimes be more idle with prefetch since Lucene isn't executing immediately as each `WILL_NEED` IO request finishes), but it should still be a big win with HNSW versus not prefetching? Maybe with virtual threads we could eventually build a "real" async IO solution, but that's a bigger thing. Or maybe we are already prefetching today in our latest `KnnVectorsFormat` Codec default and I missed it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
