Re: [I] Examine the affects of MADV_RANDOM when MGLRU is enabled in Linux kernel [lucene]

2025-04-05 Thread via GitHub
jimczi commented on issue #14408: URL: https://github.com/apache/lucene/issues/14408#issuecomment-2755375551 I believe the question is whether we need to reconsider our assumptions when defaulting to random read advice in the current code. With the linked change, using `MADV_RANDOM` will ex

Re: [I] Examine the affects of MADV_RANDOM when MGLRU is enabled in Linux kernel [lucene]

2025-03-28 Thread via GitHub
rmuir commented on issue #14408: URL: https://github.com/apache/lucene/issues/14408#issuecomment-2755385165 A better default would also be possible, I think, if individual codecs such as vectors, did not set this directly, but instead allowed the Directory class to be in control. Use

Re: [I] Examine the affects of MADV_RANDOM when MGLRU is enabled in Linux kernel [lucene]

2025-03-26 Thread via GitHub
rmuir commented on issue #14408: URL: https://github.com/apache/lucene/issues/14408#issuecomment-2755662166 > The Linux change targets both MGLRU and normal LRU. The impact is more pronounced in MGLRU, as page reclamation is more aggressive there. However, the semantic change for this advic

Re: [I] Examine the affects of MADV_RANDOM when MGLRU is enabled in Linux kernel [lucene]

2025-03-26 Thread via GitHub
jimczi commented on issue #14408: URL: https://github.com/apache/lucene/issues/14408#issuecomment-2755462640 > Let the defaults be as smart as they need. Maybe check /sys/kernel/mm/lru_gen/enabled as part of the decision-making! But IMO let the user have the final say, in an easy way.

[I] Examine the affects of MADV_RANDOM when MGLRU is enabled in Linux kernel [lucene]

2025-03-26 Thread via GitHub
ChrisHegarty opened a new issue, #14408: URL: https://github.com/apache/lucene/issues/14408 With the relatively recent capability to call `madvise` in Lucene, we've started to use `MADV_RANDOM` in several places where it makes conceptual sense, e.g. for accessing vector data when navigating