jpountz commented on issue #13194: URL: https://github.com/apache/lucene/issues/13194#issuecomment-2009851177
> If Elasticsearch or Solr wants to do this, ok. But lower level lucene should have all options. That works for me. I'm not worried about Elastisearch and Solr, they have people who know all the Lucene hidden features, I was more thinking of people who use Lucene directly who may not know about all our semi-hidden features. Now that you mentioned `NRTCachingDirectory`, `HardlinkCopyDirectoryWrapper` comes to mind as well as another useful optimization that may be worth folding into `FSDirectory`. If enabling some of these things by default is controversial, making them easier to use still sounds like a good step forward. > If you query the new segment it's cold...! On the other hand, why should the merger not make use of already cached segments? These are trade-offs indeed. The bias I have is that if you ask me if it's better for Lucene to optimize for the case when the whole index fits into RAM vs. the index is larger than the page cache, I'd probably prefer to optimize for making Lucene better at handling large indexes out-of-the-box. Like you said fadvise/madvise are likely more appropriate than direct I/O for merging. > I am planning to have some optional panama foreign in Mmapdir in future, so we can based on IOcontext give some hints to kernel. This would be great! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
