zacharymorn commented on pull request #128: URL: https://github.com/apache/lucene/pull/128#issuecomment-851028668
I've gone ahead and reverted the changes to parallelize within segment, and then added the code that used many of the same ideas to parallelize across segments - with 11 threads the total runtime has been cut down to 130+ seconds, around 65% reduction! I also removed the restriction of using up to 11 threads, as large index may well contain more than 11 segments, so idle cores can be utilized as well! Please let me know how this looks to you. @mikemccand @rmuir @dweiss > I'm gonna throw out the crazy idea to make -fast the new default. The previous -slow could be moved to -slower and the previous current behavior could be activated by -slow. > I think the tool's defaults are unnecessarily slow just for historical reasons? (not having checksums originally) This also makes sense and seems to be an easy change to switch the default? Is there anything I need to add specifically so that users can be made aware of this change when they upgrade lucene version (e.g. extra log to indicate the switch) ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
