neilconway opened a new pull request, #21532:
URL: https://github.com/apache/datafusion/pull/21532
## Which issue does this PR close?
- Closes #21531.
## Rationale for this change
`array_remove` computes the result NULL bitmap incrementally (row-by-row).
This is inefficient; it is faster to compute the result NULL bitmap via the
bitwise AND of the input NULL bitmaps.
Benchmarks (Arm64):
```
- array_remove_binary/remove/10: 6.8ms → 6.6ms (-2.9%)
- array_remove_binary/remove/100: 13.0ms → 12.7ms (-2.3%)
- array_remove_binary/remove/500: 75.2ms → 76.1ms (+1.2%)
- array_remove_boolean/remove/10: 4.5ms → 4.5ms (-0.6%)
- array_remove_boolean/remove/100: 8.0ms → 7.8ms (-2.5%)
- array_remove_boolean/remove/500: 21.0ms → 21.3ms (+1.4%)
- array_remove_decimal64/remove/10: 5.6ms → 5.7ms (+1.8%)
- array_remove_decimal64/remove/100: 10.0ms → 10.2ms (+2.0%)
- array_remove_decimal64/remove/500: 58.6ms → 56.2ms (-4.1%)
- array_remove_f64/remove/10: 5.4ms → 5.4ms (0.0%)
- array_remove_f64/remove/100: 7.7ms → 7.8ms (+1.3%)
- array_remove_f64/remove/500: 36.5ms → 36.8ms (+0.8%)
- array_remove_fixed_size_binary/remove/10: 6.1ms → 5.4ms (-11.5%)
- array_remove_fixed_size_binary/remove/100: 12.4ms → 11.6ms (-6.5%)
- array_remove_fixed_size_binary/remove/500: 66.7ms → 64.4ms (-3.4%)
- array_remove_int64/remove/10: 5.6ms → 5.7ms (+1.8%)
- array_remove_int64/remove/100: 7.8ms → 7.8ms (-1.0%)
- array_remove_int64/remove/500: 35.5ms → 36.3ms (+2.3%)
- array_remove_strings/remove/10: 6.6ms → 6.8ms (+3.0%)
- array_remove_strings/remove/100: 18.0ms → 18.3ms (+1.7%)
- array_remove_strings/remove/500: 78.0ms → 79.7ms (+2.2%)
```
Not a massive win, but I think still worth making the change; the resulting
code is also more idiomatic.
## What changes are included in this PR?
* Implement optimization
* Tweak `array_remove` benchmark for f64: the previous code resulted in
never actually removing anything from the array, which was inconsistent with
the other benchmarks and probably unintentional.
## Are these changes tested?
Yes.
## Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
