msfroh commented on issue #15054: URL: https://github.com/apache/lucene/issues/15054#issuecomment-3185688231
> If you read/copy the stuff multi-threaded access is totally wrong (unless you have clones of indexinputs that do random access or similar in parallel). For the record, that's usually what you want to do when uploading a large file to an object store with parallel multipart upload. You split the file into chunks and each thread uploads a chunk at a time. (The object store accepts chunks in whatever order and stitches them together server-side.) I could be mistaken, but I think the best way to do that is for each part of the multipart upload to open an IndexInput with `READONCE`, seek to the part's start offset, upload to the part's end offset, then close the IndexInput. Of course, it may be easier to ignore the Lucene APIs altogether and just upload files, since the major object store clients all seem to make it pretty easy to upload *a file* with parallel multipart uploads. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
