lyne7-sc commented on code in PR #22905:
URL: https://github.com/apache/datafusion/pull/22905#discussion_r3527073873
##########
datafusion/expr/src/type_coercion/functions.rs:
##########
@@ -1831,6 +1869,57 @@ mod tests {
Ok(())
}
+ #[test]
+ fn test_coercible_dictionary_preserves_encoding() -> Result<()> {
Review Comment:
It looks like this discrepancy already exists today. Typed non-Native
TypeSignatureClass values can pass `Dictionary` through, while Native coercions
materialize to the target type. But this pr makes that contrast more visible by
adding an explicit preservation API for the Native path.
If we want `EncodingPreservation` to be the general mechanism for deciding
whether Dictionary is preserved, then I agree we should also think about typed
non-Native TypeSignatureClass values. I took a quick pass over the affected
built-ins. The good news is that I did not find an obvious core DataFusion
function in the affected typed non-Native TypeSignatureClass path that
intentionally depends on receiving `Dictionary(_, T)`🙂. Most seem to dispatch
on `T` directly, and some would likely reject `Dictionary(_, T)` today after
type coercion has accepted it. From that angle, aligning the default behavior
may actually fix cases where dictionary inputs currently pass signature
matching but fail later in the function implementation.
I did find a couple of Spark compatibility functions, such as `spark hex`
and `bitmap_count`, that have dictionary paths and would likely need explicit
opt-in if we align the default behavior.
From my side, the set of built-ins that may need changes looks bounded, so
aligning the behavior seems manageable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
