uschindler commented on issue #13325: URL: https://github.com/apache/lucene/issues/13325#issuecomment-2228749316
We also have another PR doing the same than @magibney's: https://github.com/apache/lucene/pull/13570 We also got some updates on the JDK front. Next to the fixes here (use confined segments for READONCE and grouping shared arenas) there are a few suggestions: I have the feeling that Elasticsearch does not see the issue like Solr does under high load because the concurrency is different. @ChrisHegarty Is it true that Elasticsearch refreshes indexes in a sepate thread, so all close() calls are from a few threads only? In large Solr the installations the problem is mainly that every core looks like having its own threads that refresh indexes. Maybe it would be a good idea in Apache Solr to somehow have a maintenance thread which is responsible for refreshing indexes. The original patch by @magibney already showed that a static singleton lock around `Arena#close` looks like it helps. This in fact makes all closes sequential. @dsmiley Maybe that's another solution to prevent this issue from happening: Introduce a global "maintenance task" in Solr which is responsible to reopen indexes. We could include that also in the SearcherManager tool? (cc @mikemccand). I think with all those fixes here we have fixed most of the overhead. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
