Ruchirtripathi opened a new pull request, #24715:
URL: https://github.com/apache/datafusion/pull/24715
## Which issue does this PR close?
• Closes #24701
## Rationale for this change
When a query renames a column using a DataFusion reserved name
(specifically names starting with __common_expr, which are used internally by
the Common
Subexpression Elimination optimizer pass), it currently fails during
execution with obscure Arrow errors (e.g., Arrow error: Invalid argument error:
Invalid comparison operation: Float64 > Boolean). This happens because the
user-provided alias collides with the internal optimizer logic.
This change ensures we fail earlier and return a clear, descriptive
planning error to the user indicating that the column name is reserved.
## What changes are included in this PR?
• Added a check_plan validation step inside
SqlToRel::sql_statement_to_plan_with_context in datafusion/sql/src/statement.rs.
• The unoptimized LogicalPlan is traversed, and if any field in any node's
schema starts with the reserved prefix __common_expr, we immediately throw a
plan_err!.
• Doing this validation on the raw unoptimized plan ensures that we
correctly flag user-provided reserved aliases without incorrectly flagging the
__common_expr aliases legitimately added later by the CSE optimizer pass.
## Are these changes tested?
Yes, they have been verified against the reproducer provided in the issue
to return the correct error (__common_expr_2 is a reserved DataFusion column
name, please use another name). It was also verified that this does not
break existing logic where CSE natively inserts these aliases (e.g.,
list_view_agg_test passes successfully).
## Are there any user-facing changes?
Yes. Users who alias their columns with the __common_expr prefix will now
receive a clear, actionable planning error instead of a confusing Arrow
execution error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
