uschindler opened a new pull request, #13196: URL: https://github.com/apache/lucene/pull/13196
This is a first idea how we can use Panama Foreign to pass `madvise()` hints to the kernel when mapping memory segments. The code looks up the function pointer from stdlib (libc) on Linux and Macos (untested, but should work) and then invokes `madvise()` for all MemorySegments we have mmapped when the following is true: - IOContext#readOnce is set - The chunk size is large enough (at least 8192 means `chunkSizePower>=13`) - this prevents TestMultiMMap from failing because for very small mappings (as done by this test), the `FileChannel#map` call will produce unaligned memory segments (it uses some tricks and maps larger segments and returns slices - which are no longer pageSize aligned) - There is a noop implementation doing nothing which is choosen if you disable native access Interestingly it works without any extra parameters to command line (at least in Java 21). This is a draft only to do some performance tests and extend the IOContext interpretation to try out more possibilities. The current "readOnce => MADV_SEQUENTIAL" is just an example as this is the main issue: We merge segments and don't want the soon to be trashed segments be sticky in RAM. MADV_SEQUENTIAL instructs kernel to forget about the mappings and also do readahead which helps during merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
