andygrove commented on PR #6716: URL: https://github.com/apache/datafusion-comet/pull/6716#issuecomment-6126476825
Good catch, thanks. `LazilyGeneratedOrdering` evaluates each sort key on its own with no subexpression elimination, and the projection only sees the rows the sort keeps, so one set for both made Comet raise where Spark doesn't. In 0e49f1c03 the top-K plans use an empty set and the projection uses `projectList`, as you suggested. Both of your queries are in `array_null_short_circuit.sql` now, adapted to the fixture's table, and return the rows as Spark does, with `expect_native` on `array_contains` and `array_position`. They fail at 3090d859a, and an empty set for the projection fails the shared-projection query instead, so each set is pinned by a test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
