uschindler commented on issue #13325: URL: https://github.com/apache/lucene/issues/13325#issuecomment-2217896710
> > Possibly group multiple files into one arena. One Idea that came into my mind: MMapDirectory groups files belonging to the same segment together and uses a single Arena for them. Behind the scenes it keeps a refcount for each group of files. So whenever a file is opened the refcount for that group increases and when it is closed it decreases. When the refcount goes to 0 the Arena is closed. > > This seems really promising as well. I may try to pick this up unless anyone else is already looking into it? This seems like the right thing to do semantically even, in addition to the potential performance benefits. I had no time to look into this. It should be doable with a instance ConcurrentHashMap, not sure where to place this instance (as it is also using those providers). So the problematic part is where to place this (as it cannot be in MMapDirectory where it should belong as it uses Java 21 preview apis). You can give it a try, but i have the feeling it gets tricky on the actual implementation. We may need to add some abstraction into MMapDirectory instances (the ConcurrentHashMap) with the refcounter and a pointer to some "hidden" payload (the areana) which is picked by the private MemorySegmentIndexInputProvider. We can work together. If you have a better idea, tell me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
