On Sat, Dec 2, 2017 at 8:59 PM, S G <sg.online.em...@gmail.com> wrote: > I am a bit curious on the docValues implementation. > I understand that docValues do not use JVM memory and > they make use of OS cache - that is why they are more performant. > > But to return any response from the docValues, the values in the > docValues' column-oriented-structures would need to be brought > into the JVM's memory. And that will then increase the pressure > on the JVM's memory anyways. So how do docValues actually > help from memory perspective?
docValues are not more performant than the on-heap fieldCache once a fieldCache entry has been populated. docValues do help with memory in a number of ways: 1) off-heap memory use degrades much more gracefully... heap memory will just cause an OOM exception when heap size is exceeded 2) off-heap memory (memory used by OS cache) can be dynamically shared with other processes running on the system (the OS allocates as needed) 3) easier to configure (as opposed to max-heap size) since the OS will just automatically use all free memory to cache relevant parts of memory mapped disk files 4) off-heap memory does not need to be garbage collected (helps with people that have huge GC pauses) -Yonik
