dwsmith1983 commented on PR #6785: URL: https://github.com/apache/datafusion-comet/pull/6785#issuecomment-6282703286
> Could the new test also be shown to fail on this branch without the #6835 declaration, so we know it covers the forced join? Yes. Main, with #6835, is merged in 153a343f4, which also adds `forceShuffledHashJoin keeps the order of a sorted cache through an outer join` to `CometJoinSuite`. It caches a table sorted within its partitions, runs a left outer join built on the right and a right outer join built on the left, once forced with no hint and once with `SHUFFLE_HASH`, and then asks for `sortWithinPartitions(k, v)`. Each case checks that the plan is a Comet hash join on the expected side with no sort above it, that the join declares the streamed order, that the rows match Spark, and that every partition stays in `(k, v)` order. With only `.addAllStreamedSortOrders(streamedSortOrders.asJava)` removed from `CometHashJoin.doConvert`, the forced left outer case fails. Before the test asserted the declaration, it failed on the order check (`left_outer with hint None is out of order`) over `CometHashJoin LeftOuter, BuildRight` with no sort above it. Now it stops at the declaration check first. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
