uschindler commented on PR #16684: URL: https://github.com/apache/lucene/pull/16684#issuecomment-5830793543
Hi, no problem. I was involved in the original design of the how to differentiate between merges and searches. As you noted above, I am in favor of the following approach: If a segment is already open for searches and it needs to do merging, I originally said: "chage the read advice" for the already open IndexInputs. When mergin is done, revert. I still think this is the best approach, because the file system cache is a shared resource. So when a merge is done, the whole file will load into FS cache anyways, and the goal here is to optimize that. If there are searches in parallel Basically your approach will do exactly the same: It opens a second instance on the same file, gets a fully new virtual memory slice and then uses it with a different read advice. But as it is the same file and some of the pages are shared between searcher and merger, the last one wins. When the merger is closed the changed read advices stay alive. Therefor I'd like to have some method in indexinput to "temporarily override" read advice globally till the merge is done. After that it reverts to the original read advice. I have not yet looked into the PR here, but as stated in #13920 - and repeated here - I'd like to investiagte this "temporary override" and not open the file multiple times. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
