Toby1009 commented on PR #25575: URL: https://github.com/apache/datafusion/pull/25575#issuecomment-5765234394
While working on this PR, I explored whether preserving output types could be handled centrally in `ScalarFunctionExpr`. It already stores the resolved `return_field`. When a UDF’s `evaluate_bounds` returns the default unbounded `Null` interval, we could use that resolved type to construct a typed unbounded interval, while preserving any bounds the UDF provides. In a local prototype, I removed the `date_bin` and `from_unixtime` overrides and added this fallback. The tested timezone-free queries still removed the redundant sort, while explicit and session timezone cases retained it. The Goose Bay regression query also returned the expected ordering. I additionally observed redundant sort removal for `CAST(signum(c) AS REAL)` and `CAST(trunc(c) AS REAL)` over sorted inputs. This could reduce the need for individual UDFs to implement `evaluate_bounds` solely to preserve type information. However, it would affect property inference for all scalar UDFs, so broader regression testing and checking the behavior for unsupported interval types would be needed. I also tried storing `DataType` separately in `ExprProperties`. That worked for the datetime cases, but requires migrating consumers and addressing public API and FFI compatibility, so it seems better suited to a separate design discussion. Would a central fallback be a useful follow-up? My preference is to keep this PR focused and explore that separately if the direction makes sense. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
