>>> "Vitaly Zuevsky" <[email protected]> schrieb am 24.05.2021 um 22:06 >>> in Nachricht <[email protected]>: >> Vitaly Zuevsky wrote: >> > Hi >> > >> > MAP_LOCK flag to mmap() on LMDB readers should populate page cache >> > from the file being mapped and preclude those pages from eviction. I >> > was wondering if that was ever tested, especially with frequent writes >> > to the underlying file. And is there a standard way (a compile option) >> > to include that flag without modifying source code directly? I see >> > quite a few major faults with perf stat, and I am looking if they could be >> avoided. >> > >> > I am new to LMDB and I guess this discussion may have taken place >> > before, if you could point me out to the right place.. >> >> No. In general it's a bad idea to interfere with the OS's paging decisions. > In >> particular, on a machine with plenty of RAM, pageouts aren't an issue >> anyway. >> On a system where memory is tight, page faults will be unavoidable, so the >> question is what other memory are you going to lose if you force the LMDB >> pages to stay locked in? > > Memory pressure isn't my problem. I see 2-15 major faults a second in the > event loop of the reader thread, which can effectively stall the loop for > anything between 1-400ms (I measure iowait jiffies of the pid). I understand > the faults result from the writer thread, actively modifying the file mmaped > by the reader. I assume writes to the file invalidate corresponding pages of > the reader, and those invalid pages only get updated (via major faults) on > first access by the reader. If not locking, perhaps madvise/willneed > combination could help, what you think?
I don't know the internals of LMDB, but writing to a file that has mapped region seems to be a poor idea. Why not write the region? AFAIR HP-UX really didn't like that. Regards, Ulrich
