mikemccand commented on PR #240: URL: https://github.com/apache/lucene/pull/240#issuecomment-1801638506
> We are still a ways away For example: query concurrency is still tied to segment geometry, which is insane. An "optimized" (`forceMerge to 1 segment`) index loses all of its concurrency! Hideously, at my team (Amazon product search) we had to create a neutered merge policy that intentionally avoids "accidentally" making a lopsided index (where too many docs are in a single segment) because it harms our long pole latencies, due to this silly Lucene limitation. We have a longstanding issue open for this: https://github.com/apache/lucene/issues/9721, and it ought not be so difficult because Lucene already has strong support for one thread to search a "slice" of a big segment, i.e. we can make the thread work units slices of segments instead of whole segments or collections of segments (what IndexSearcher now calls slices, confusingly). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
