geoffreyclaude commented on code in PR #26983:
URL: https://github.com/apache/datafusion/pull/26983#discussion_r4435022023
##########
datafusion/core/tests/fuzz_cases/sort_query_fuzz.rs:
##########
@@ -210,9 +211,10 @@ impl SortQueryFuzzer {
if expected_results.is_none() {
expected_results = Some(results);
} else if let Some(ref expected) = expected_results {
- // Every projected column is an ORDER BY key, so tied
- // rows have identical output and can be compared in
order.
- assert_eq!(expected, &results);
+ // Check logical equivalence.
+ // `assert_eq!` may report false failures when
logically equivalent
+ // batches have different physical encodings.
+ check_equality_of_batches(expected, &results).unwrap();
Review Comment:
> @geoffreyclaude Thanks for the review!
>
> I think the long term goal is a util for 'Logical eq for SQL semantics',
the issues are
> - `not_distinct` is not exactly it, it might give us some surprises on
float/nested type, see
https://arrow.apache.org/rust/arrow/compute/kernels/cmp/fn.not_distinct.html
> - and I agree formatting specific issues are also valid concern
>
> So I think we can build some `sql_eq` util, likely a wrapper on arrow
`not_distinct` in the future, now I prefer to keep it simple and keep using
formatting.
Fair enough, let’s keep the formatting comparison here. It fixes the
reported failure and preserves the ordering check, which is what really matters.
Thanks for addressing this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
