ChrisHegarty commented on issue #13194: URL: https://github.com/apache/lucene/issues/13194#issuecomment-2011662051
> @ChrisHegarty: For MMapDirectory there's another way for madvise to work (at least for reading index files): When the user has enabled native Panama access, we can also mmap our MemorySegments with our own code without using FileChannel, because we have to deal with file descriptor anyways and native access was already enabled. In fact we close the FileChannel after mapping anyways, so the FileChannel is completely useless for MMapDirectory code. We would need code for Posix and Windows. Yes, that is exactly what I've been thinking too. We're leaning more and more into FFI with Elasticsearch and it is proving to both behave and perform very well. Abstracting the functionality at a sightly higher-level may allow to have a Posix implementation and a fallback to an implementation based FileChannel. The latter would cover Windows - possibly ignoring the "hints" for random access. This could be a start, allowing to add more platform support as needed. > For writing files it is more complicated, so setting fadvise there also involves replicating the whole (File-)OutputStream used for writing index files. Ugh!! not great. We need either the memory address or the file descriptor. Not to mention that we map in 16GB chunks, so may require a little more plumbing (but nothing too hard). Is the mapping in chunks still needed? Did you @uschindler encounter issues when mapping larger chunks? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
