mikemccand commented on pull request #128:
URL: https://github.com/apache/lucene/pull/128#issuecomment-851538612
> To get the best speedup, even at -slow, we should do concurrency both
ways, and then sort those tasks by decreasing expected cost.
This fine-grained concurrency (one task for "part X segment") would then
mean we can get the postings check of the big segments all going, early,
concurrently across all segments, and likely the biggest segments postings
check would be the long pole, and {{CheckIndex}} would finish in (about) that
much time.
Versus "thread per segment" concurrency, where the long pole will be one
thread checking the biggest segment.
The really fun part of concurrency across *and* within segments will be
figure out the rough heuristic of how to assign estimated cost to each task :)
But this all can come later! "thread per segment" is already a big step
forward!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
