zhuqi-lucas commented on code in PR #26215:
URL: https://github.com/apache/datafusion/pull/26215#discussion_r4254332680
##########
datafusion/physical-plan/src/sorts/cursor.rs:
##########
@@ -643,6 +649,11 @@ impl<T: CursorValues> CursorValues for ArrayValues<T> {
#[inline(always)]
fn eq(l: &Self, l_idx: usize, r: &Self, r_idx: usize) -> bool {
+ // Fast path: neither array has nulls, so the match below would take
+ // the (false, false) arm every time.
+ if !l.has_nulls && !r.has_nulls {
Review Comment:
Good point. Prototyped it and it is worth another ~6%, but it needs a new
cursor type and a nullability dispatch in `StreamingMergeBuilder`, so I opened
#26252 rather than growing this PR.
Thanks @alamb
##########
datafusion/physical-plan/src/sorts/cursor.rs:
##########
@@ -946,4 +968,28 @@ mod tests {
b.advance();
assert_eq!(a.cmp(&b), Ordering::Less);
}
+
+ /// Verifies the `ArrayValues::has_nulls` fast path is actually taken
+ /// (all null-free cursors) and produces the same ordering as the
+ /// null-aware slow path.
+ #[test]
+ fn test_array_values_has_nulls_fast_path_agrees_with_slow_path() {
+ let options = SortOptions {
+ descending: false,
+ nulls_first: false,
+ };
+
+ // Two null-free cursors -- compare takes the has_nulls fast path.
+ let a = new_primitive(options, ScalarBuffer::from(vec![1i32, 2, 3,
4]), 0);
+ let b = new_primitive(options, ScalarBuffer::from(vec![2i32, 3, 4,
5]), 0);
+ assert!(!a.values.has_nulls);
+ assert!(!b.values.has_nulls);
+ assert_eq!(a.cmp(&b), Ordering::Less);
+
+ // A null-containing cursor forces the slow (null-aware) path, and the
+ // two paths must agree.
+ let c = new_primitive(options, ScalarBuffer::from(vec![i32::MAX, 3i32,
4]), 1);
+ assert!(c.values.has_nulls);
+ assert_eq!(a.cmp(&c), Ordering::Less);
Review Comment:
You are both right, the old test proved nothing. Rewrote it to compare the
same values through both paths, across all index pairs, both sort directions
and both null placements, and it now covers `eq` and `eq_to_previous` too.
Verified by mutation: ignoring `descending`, inverting it, and `eq` /
`eq_to_previous` returning `true` each fail it.
Thanks @alamb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
