uschindler commented on PR #15824: URL: https://github.com/apache/lucene/pull/15824#issuecomment-4068005128
The othe rproblem where this may come from: Are you seeing this with the shared arenas enabled or disabled? The check can only be elided, if you NOT reopen segments all the time. In Opensearch there were serious problems with your "index liveness checks", remember the issues where you run out of file handles/mappings, see https://github.com/apache/lucene/issues/15054. The problem is whenever you close a file with a shared arena it causes a safepoint in the JVM, killing the optimizations in the top frame of all threads. So please make sure to use READ_ONCE for accessing metadata files in OpenSerach. In our benchmarks we do not see those issues, because the checks are correctly elided, unless you close files and not using the default arena grouping behavious of Lucene. Elasticserach does not see the problem and also not our own benchmarks. Can you possibly rerun a benchmark of your index with refresh of index disabled temporarily (so disabling NRT)? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
