uschindler commented on PR #13146: URL: https://github.com/apache/lucene/pull/13146#issuecomment-2158221471
Hi @dsmiley, Thanks for the quick talk on Berlinbuzzwords. Actually this looks like the same issue we have seen in the dacapobench. When back at home I will try to write a JMH benchmark without Lucene code to reproduce the issue and mesaure how lare the issue is (depending on how often you close an IndexInput and how many parallel threads are using other MemorySegments (also those not tied to same Indexinput). My idea is: - Have many threads working on "hot" MemorySegments allocated with different shared Arenas. - Have a few threads closing those Arenas. Theoretically, the benchmark should not slow if the few threads close Arenas which are not bound to and hot workers. What we have seen in dacapo bench is that it seems to affect all MemorySegments negatively, not only those which are affected by the close. of Arena. Once I have the benchmark I will open issue at OpenJDK. In the meantime, we could "preserve" the old Lucene 9.1 (the version without MemorySegments) in the "misc" module of Lucene as LegacyMMapDirectory. This would allow people to use it, but it is strinly discouraged. It won't work anymore with Java 24 (or around that time). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
