geoffreyclaude opened a new pull request, #23311: URL: https://github.com/apache/datafusion/pull/23311
## Which issue does this PR close? - Part of #19241. - Based on #23299. - Next in stack: #23014, after it is rebased onto this PR. ## Rationale for this change #23299 extends the bitmap `IN` filter to the signed 1-byte and 2-byte integer types by handling each logical Arrow type directly. `Float16` is the remaining 2-byte primitive type that can use the same compact bitmap idea: it has 65,536 possible bit patterns, so an 8 KiB bitmap can represent every possible value. This PR follows the same direct typed shape as #23299. It does not reinterpret whole arrays as `UInt16`; instead, the `Float16` bitmap filter maps each value to its IEEE-754 half-precision bit pattern with `to_bits()`. That keeps the logical array type intact while preserving bit-pattern equality semantics, including distinct NaN payloads and `+0.0` versus `-0.0`. ## What changes are included in this PR? - Adds `Float16Type` support to the existing `BitmapFilter`. - Routes `DataType::Float16` constant-list filtering to that bitmap path. - Extends the existing type-combination coverage to include `Float16`. - Adds focused coverage for slices, nulls, `NOT IN`, `+0.0` / `-0.0`, and NaN payload bit patterns. ## Are these changes tested? Yes. - `cargo fmt --all -- --check` - `cargo test -p datafusion-physical-expr bitmap_filter_f16 --lib` - `cargo test -p datafusion-physical-expr test_in_list_from_array_type_combinations --lib` ## Are there any user-facing changes? No. This is an internal performance optimization only. ## Benchmark note No local benchmark snapshot is included because the current `in_list_strategy` benchmark does not include direct `Float16` rows. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
