Ruchirtripathi opened a new pull request, #24363:
URL: https://github.com/apache/datafusion/pull/24363
## Which issue does this PR close?
- Closes #24139
## Rationale for this change
The `scalar_subquery_to_join` optimizer rule was failing with a `Schema
error: No field named __always_true` when a CTE was used inside a correlated
scalar subquery.
This occurred because the `SubqueryAlias` logical plan node wasn't
refreshing its schema after the `__always_true` column was injected by the
correlated expression
pull-up logic (`PullUpCorrelatedExpr`). Because the schema became stale,
the parent `Projection` node would attempt to reference the newly injected
`__always_true`
column and fail with a schema error.
## What changes are included in this PR?
- Updated `PullUpCorrelatedExpr::f_up` logic for
`LogicalPlan::SubqueryAlias` in `datafusion/optimizer/src/decorrelate.rs`.
- We now check if the `SubqueryAlias`'s input schema has a different number
of fields than its own recorded schema. If so, we recreate the `SubqueryAlias`
using
`LogicalPlanBuilder` so that its schema correctly reflects any newly added
columns (such as `__always_true`).
## Are these changes tested?
Yes, they are covered by the existing sqllogictests and optimizer
integration tests. Queries with CTEs inside correlated scalar subqueries that
previously crashed
the optimizer will now correctly optimize and execute.
## Are there any user-facing changes?
No, this is strictly an internal optimizer bug fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
