uschindler commented on issue #15054: URL: https://github.com/apache/lucene/issues/15054#issuecomment-3187226652
> Thanks [@uschindler](https://github.com/uschindler) , [@navneet1v](https://github.com/navneet1v) , [@onestardao](https://github.com/onestardao) , [@msfroh](https://github.com/msfroh) > > [@uschindler](https://github.com/uschindler) , what is the reason behind using `READONCE` ? Is there some issue with DEFAULT ? We have take heap dumps and verified that IndexInput does get closed . If there is a leak , there would be a problem even with `READONCE` as well , right ? > > > If you read/copy the stuff multi-threaded access is totally wrong (unless you have clones of indexinputs that do random access or similar in parallel). I'd strongly recommend to refactor your own code to use READONCE in a way like it is designed for: "like an InputStream and only from one thread". So I'd recommend to have the code possibly multi-threaded but open, read and copy and close the file in one thread > > [@msfroh](https://github.com/msfroh) is right about it . For multipart upload, we need all threads to read the file concurrently . Can you please share the reasoning of why multi-threaded access is totally wrong ? Is there a limitation which is undocumented ? It is wrong if you do this on the same IndexInput with sequential access. If you want to slice your IndexInput and use multiple threads then it is no longer (strictly speaking) "read once". Because the kernel can't do read-ahead correctly. For copying files over the network, MMapDirectory is a bad choice, sorry. And read-once only works correct if you have a single stream and copy it. > > To me this really looks like a leak in closing IndexInputs. > > We have take heap dumps and verified that IndexInput does get closed . There is a lot of information missing to us. We can't figure out what you are doing without an reproducer. Do you have index files open on the same directory at the same time? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
