vigyasharma commented on pull request #633: URL: https://github.com/apache/lucene/pull/633#issuecomment-1036989788
The `addIndexes(CodecReaders[])` API is transactional in nature - either all readers get added or all of them fail. Retaining this behavior with concurrent background merges needs some work. My plan is to defer adding the newly created merged segments to IndexWriter until all merges finish. MergeScheduler threads will do all the work for merge right upto creating segment files, and return. At addIndexes() API, we'll wait for all merges kicked off by the API to complete, and add them to `IndexWriter.segmentInfos` in a sync block only when all of them succeed. For partial failures, we'll delete the interim files created. Added code for this change to get some feedback on this approach. Tests are failing right now and I'm working on fixing them and adding new ones. Have put `nocommit` comments meanwhile. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
