uschindler opened a new pull request #203: URL: https://github.com/apache/lucene/pull/203
While discussing about MMapDirectory and fast access to file contents through MMap #177 and previous versions of this draft, also), I figured out that for most Lucene files, the data inside is not aligned at all. We can't fix this easily and it's also not always important, but some files should really have a CPU fieldly alignment from beginning! This is escpecially important when we use slices(). I got many tests with aligned VarHandles to pass, but it broke instantly, if the file was inside a Compound CFS file. CompoundFormat.write() just appends all data to the IndexOutput and writes the offset to the entries file. The fix to make at least file starts aligned is to just write some null-bytes between the files, so startOffset is aligned to multiples of 8 bytes. At a later stage we could also think of aligning to LBA blocks/sectors/whatever to make OS paging work better. But for performance of index access, slices of compound files when memory mapped should at least align to 8 bytes. More info: https://issues.apache.org/jira/browse/LUCENE-10019 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
