geoffreyclaude commented on code in PR #25187:
URL: https://github.com/apache/datafusion/pull/25187#discussion_r4337287903
##########
datafusion/optimizer/src/simplify_expressions/inlist_simplifier.rs:
##########
@@ -19,24 +19,47 @@
use super::THRESHOLD_INLINE_INLIST;
-use datafusion_common::Result;
+use arrow::datatypes::DataType;
use datafusion_common::tree_node::{Transformed, TreeNodeRewriter};
-use datafusion_expr::Expr;
+use datafusion_common::{DFSchema, Result};
use datafusion_expr::expr::InList;
+use datafusion_expr::{Expr, ExprSchemable};
-pub(super) struct ShortenInListSimplifier {}
+pub(super) struct ShortenInListSimplifier<'a> {
+ schema: &'a DFSchema,
+}
+
+impl<'a> ShortenInListSimplifier<'a> {
+ pub(super) fn new(schema: &'a DFSchema) -> Self {
+ Self { schema }
+ }
-impl ShortenInListSimplifier {
- pub(super) fn new() -> Self {
- Self {}
+ /// The paired `in_list_rewrite` benchmark favors membership for these
+ /// representations.
+ fn should_retain_short_inlist(&self, expr: &Expr, list: &[Expr]) -> bool {
+ let Ok(data_type) = expr.get_type(self.schema) else {
+ return false;
+ };
+ matches!(
+ data_type,
+ DataType::Int32
+ | DataType::Int64
+ | DataType::UInt64
+ | DataType::Float32
+ | DataType::Float64
+ | DataType::FixedSizeBinary(1 | 2 | 4 | 8 | 16)
+ ) && list.iter().all(|expr| {
+ expr.as_literal()
Review Comment:
DataFusion normally converts the column and the values in an `IN` list to
the same type before this code runs. They can differ if someone builds an
expression directly without that conversion, but changing `IN ` to `OR` would
not fix it. I've removed that extra check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
