dwsmith1983 commented on code in PR #7126:
URL: https://github.com/apache/datafusion-comet/pull/7126#discussion_r4332340136
##########
spark/src/main/scala/org/apache/spark/sql/comet/operators.scala:
##########
@@ -2952,9 +2954,17 @@ trait CometHashJoin {
// check `supportedSortType` makes for a sort. Nothing here can tell
whether an operator above
// relies on the order, so a join whose first key Comet cannot express
stays in Spark.
val (streamed, _) = streamedSide(join.buildSide, join.left, join.right)
+ val keepsStreamedOrder =
+ runsAsRightOuterJoin(join.joinType, join.buildSide) &&
join.outputOrdering.nonEmpty
+ if (keepsStreamedOrder && mayReportFalseOrder(streamed, countInPlan =
false)) {
+ withFallbackReason(
+ join,
+ "the streamed side's sort order, which this outer join has to keep,
cannot be " +
+ "checked, so it is not declared natively")
+ return None
Review Comment:
Changed in 01fdf012a. The reason now says "cannot be checked, so the join
stays in Spark", and it lives in one constant,
`CometHashJoin.uncheckedStreamedOrderReason`, which the four tests use.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
