discord9 opened a new pull request, #24686: URL: https://github.com/apache/datafusion/pull/24686
## Which issue does this PR close? - No issue has been filed yet. This is a draft while the fix is reviewed. ## Rationale for this change Nested projections that reuse the same output name can return wrong results. For example, applying `i + 1 AS i` in three nested subqueries to the values `3`, `4`, and `5` should return `6`, `7`, and `8`, but DataFusion returns `5`, `6`, and `7`. The logical `OptimizeProjections` rule currently merges an entire projection chain in one rule invocation. After one merge rebuilds a projection, merging that result again can lose a same-named alias boundary and drop one expression from the query semantics. ## What changes are included in this PR? - Merge only one adjacent logical projection level per optimizer rule invocation. Subsequent optimizer passes may continue simplifying the chain. - Add an end-to-end SQL execution regression test for three nested `i + 1 AS i` projections. - Update the optimizer integration expectation for the projection that can remain after a single-level merge. This intentionally changes only the logical whole-chain merge introduced in #22389. It retains that PR's physical `ProjectionExec` chain collapse and the physical-expression equal-column short circuit. A very deep logical projection chain may require multiple optimizer passes and can retain semantically valid projections if the configured pass limit is reached. ## Are these changes tested? Yes. The regression was first run on current `main` and failed with `5`, `6`, and `7`. With this change it returns `6`, `7`, and `8`. Verified with: ```text cargo fmt --all --check cargo clippy --all-targets --all-features -- -D warnings cargo test -p datafusion --test core_integration sql::select::nested_projection_runtime_regression -- --exact cargo test -p datafusion-optimizer merge_ cargo test -p datafusion-optimizer --test optimizer_integration extension_node_does_not_block_projection_pruning -- --exact ``` ## Are there any user-facing changes? Yes. Nested projections that reuse an output name now preserve every projection expression and return the correct result. There are no public API changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
