michaeljmarshall commented on PR #16145: URL: https://github.com/apache/lucene/pull/16145#issuecomment-6278465631
Thank you @goankur for the benchmarks and @neoremind, @gsmiller for the reviews/suggestions! I support getting to a good enough spot to then merge and iterate. On the `N - 1` point, that's what I was thinking when I wrote this, though in re-reading it, my wording could have been clearer: > Alternatively, we could consider a middle solution where we probe for fewer than 1024 calls but more than 3 calls (the current default). 1024 comes from the cost of a major page fault vs an unnecessary `isLoaded` call. It might be worth considering the `N - 3` case, but I don't think it's necessary. > In addition, throwing out an idea for a possible follow-up: the NoReuseHint from #16682 might be a useful signal? Files flagged no-reuse (stored fields, term vectors, raw rescore vectors) are large relative to the index, have scattered page access and their pages are likely to be evicted once loaded, so they may have a stricter policy, something like a bigger N and a smaller SKIP. Others like postings, vector graph could keep what's in the PR. This is a great point and is worth exploration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
