akshayakp97 commented on issue #9268: URL: https://github.com/apache/iceberg/issues/9268#issuecomment-1850797730
After `ColumnPruning` adds the new `Project [cs_warehouse_sk#54840, cs_order_number#54843L]`, when `V2ScanRelationPushDown` rule triggers, it doesn't match the [`ScanOperation`](https://github.com/apache/spark/blob/7db85642600b1e3b39ca11e41d4e3e0bf1c8962b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala#L352-L353) pattern, because, the scan operator for `catalog_sales` in the logical plan seems to have been updated in the previous iterations of `V2ScanRelationPushDown` rule - which resulted in the conversion of `ScanBuilderHolder` to `DataSourceV2ScanRelation`. To summarize, in this case, it looks like a new `Project` was added after the creation of `DataSourceV2ScanRelation` in the logical plan, causing it not prune columns. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
