jimczi commented on a change in pull request #32:
URL: https://github.com/apache/lucene/pull/32#discussion_r599954418
##########
File path: lucene/core/src/java/org/apache/lucene/index/DirectoryReader.java
##########
@@ -56,7 +57,24 @@
* @throws IOException if there is a low-level IO error
*/
public static DirectoryReader open(final Directory directory) throws
IOException {
- return StandardDirectoryReader.open(directory, null);
+ return StandardDirectoryReader.open(directory, null, null);
+ }
+
+ /**
+ * Returns a IndexReader for the the index in the given Directory
+ *
+ * @param directory the index directory
+ * @param leafSorter a comparator for sorting leaf readers. Providing
leafSorter is useful for
+ * indices on which it is expected to run many queries with particular
sort criteria (e.g. for
+ * time-based indices this is usually a descending sort on timestamp).
In this case {@code
+ * leafSorter} should sort leaves according to this sort criteria.
Providing leafSorter allows
+ * to speed up this particular type of sort queries by early terminating
while iterating
+ * through segments and segments' documents.
Review comment:
It seems to work well in practice with the tiered merge policy and
natural merging. The natural order of insertion seems to be preserved nicely in
the levels. Things get complicated when sorting on descending order though. The
big segments tend to have the data in ascending order. So in this scenario,
it's much slower to retrieve the latest documents if the index was force merged
to 1 segment or few big ones. In Elasticsearch we shuffle the order of the
segments on force merge to alleviate this bias.
In this PR we opted for a single sort per index so I like the idea of having
a merge policy that is optimized for this use case. I think it's a good
compromise for cases where index sorting is too costly at the document level.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]