javanna commented on code in PR #13542:
URL: https://github.com/apache/lucene/pull/13542#discussion_r1750500023
##########
lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java:
##########
@@ -363,7 +374,88 @@ public static LeafSlice[] slices(
LeafSlice[] slices = new LeafSlice[groupedLeaves.size()];
int upto = 0;
for (List<LeafReaderContext> currentLeaf : groupedLeaves) {
- slices[upto] = new LeafSlice(currentLeaf);
+ slices[upto] =
+ new LeafSlice(
+ new ArrayList<>(
+ currentLeaf.stream()
+ .map(LeafReaderContextPartition::createForEntireSegment)
+ .toList()));
+ ++upto;
+ }
+
+ return slices;
+ }
+
+ /**
+ * Creates leaf slices that leverage intra-segment concurrency by splitting
segments into multiple
+ * partitions according to the provided max number of documents per slice
and maximum number of
+ * segments per slice. If a segment holds more documents than the provided
max per slice, it gets
+ * split into equal size partitions that each gets its own slice assigned.
Review Comment:
yes it can be a mix of entire segments and partitions of different segments.
I am only enforcing that different partitions of the same segment need to go to
different slices, one each. That is because it makes little sense to partition
and then use a single thread. Also, supporting that scenario would require to
further clarify that the getLeafCollector semantics need handling within each
Collector implementation, as opposed to only across multiple collectors
returned by the collector manager.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
