jimczi commented on issue #13920: URL: https://github.com/apache/lucene/issues/13920#issuecomment-5832988994
Coming back to this one after a while, since what was decided here still shapes the current code. The conclusion was to change the read advice on the already open `IndexInput` for the duration of a merge, and that is what vectors do today in `getMergeInstance`. In #16684 we tried the other route for stored fields: the merge opens the file a second time and advises that open sequential, so the mapping searches are using is never touched. The question that came up was whether two mappings of the same file interfere. They do not. Advice is per VMA, `MADV_RANDOM` and `MADV_SEQUENTIAL` set `VM_RAND_READ` / `VM_SEQ_READ` on the mapping you pass, the page cache is shared between the two, and nothing is left on the file when one goes away. Measured on 6.1 with `read_ahead_kb` 128: a touch through the random mapping faults 1 page, through the sequential one 32, and after unmapping the sequential one the random mapping still faults 1. Two things follow for what was discussed here. There is nothing to revert once the merge ends, so the open question about whether and when to restore the advice goes away. And a separate open is the only way the `Directory` gets a say, since `updateIOContext` only re-advises a mapping that already exists, so `DirectIODirectory` never sees a merge that borrows a search reader. If the stored fields change lands, the same would follow for vectors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
