FrankChen021 commented on code in PR #20587:
URL: https://github.com/apache/druid/pull/20587#discussion_r4270842249
##########
multi-stage-query/src/main/java/org/apache/druid/msq/querykit/scan/ScanQueryFrameProcessor.java:
##########
@@ -275,28 +280,26 @@ protected ReturnOrAwait<SegmentsInputSlice>
runWithDataServerQuery(final DataSer
return cursorFromIterable.lhs;
}).collect(Collectors.toList());
- final Yielder<Cursor> cursorYielder =
Yielders.each(Sequences.simple(cursors));
+ dataServerCursors = cursors.iterator();
+ }
- if (cursorYielder.isDone()) {
- // No cursors!
- cursorYielder.close();
- return ReturnOrAwait.returnObject(handedOffSegments);
- } else {
- final long rowsFlushed = setNextCursor(cursorYielder.get(),
Collections.emptyList(), null, null);
- closer.register(cursorYielder);
+ if (cursor == null || cursor.isDone()) {
+ if (dataServerCursors.hasNext()) {
+ final long rowsFlushed = setNextCursor(dataServerCursors.next(),
Collections.emptyList(), null, null);
Review Comment:
[P2] Global limit is not carried across result yielders
**Finding:** For a finite LIMIT without ORDER BY, ScanQueryStageProcessor
supplies a shared row-limit counter while setNextCursor resets cursorRowsRead
for each cursor. When this new branch advances after a yielder leaves a
partially filled frame, setNextCursor flushes that frame without adding
rowsFlushed to sharedRunningCountForLimit, so the next yielder starts with a
fresh cursorPushDownLimit while the shared count omits rows already emitted.
Multiple retry yielders can therefore make the stage scan and write up to the
limit per yielder before the downstream offset/limit stage trims the result.
**Suggestion:** Preserve one aggregate remaining-row budget across yielders
by accounting for rows flushed during cursor advancement and applying that
remaining budget to the next cursor; add a finite-limit multi-yielder
regression test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
