On 2022-07-05 10:52 PM, Shawn Heisey wrote: ...
That is an interesting question. One of the reasons Lucene queries so fast when there is plenty of memory is because it accesses files on disk directly with MMAP, so there is no need to copy the really massive data structures into the heap at all.
mmap() doesn't side-step disk access though, dep. on the number of of mmap'ed chunks and chunk size, it can be slow. Especially if your "disk" is an iSCSI volume on a gigabit link to a slow underprovisioned NAS.
The flip side is IIRC when mmap'ing to RAM disk, linux kernel is supposedly smart enough to realize it's already in RAM and just flip a pointer. I.e. if you could have a huge RAM disk...
Dima
