huaxingao commented on PR #6252: URL: https://github.com/apache/iceberg/pull/6252#issuecomment-1756724970
If you have filters on the aggregated columns, e.g. SELECT MAX(col) FROM table WHERE col > 1 && col < 10, then push down is not supported. On Tue, Oct 10, 2023 at 9:52 AM Atif ***@***.***> wrote: > @huaxingao <https://github.com/huaxingao> I was executing max/count query > on iceberg table version 1.3.0 and Spark3.3.1 but unable to see aggregate > pushdown i.e. LocalTableScan > > Cc: @RussellSpitzer <https://github.com/RussellSpitzer> > > spark.sql(f""" select max(page_view_dtm) from schema.table1where > page_view_dtm between '2020-01-01 00:00:00' and '2021-12-31 23:59:59' > """).explain() > > and explain plan generated is > > == Physical Plan == > AdaptiveSparkPlan isFinalPlan=false > +- HashAggregate(keys=[], functions=[max(page_view_dtm#139)]) > +- Exchange SinglePartition, ENSURE_REQUIREMENTS, [plan_id=62] > +- HashAggregate(keys=[], functions=[partial_max(page_view_dtm#139)]) > +- Filter ((page_view_dtm#139 >= 2020-01-01 00:00:00) AND (page_view_dtm#139 <= 2021-12-31 23:59:59)) > +- BatchScan[page_view_dtm#139] spark_catalog.schema.table1(branch=null) [filters=page_view_dtm IS NOT NULL, page_view_dtm >= 1577854800000000, page_view_dtm <= 1641013199000000, groupedBy=] RuntimeFilters: [] > > — > Reply to this email directly, view it on GitHub > <https://github.com/apache/iceberg/pull/6252#issuecomment-1755857764>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/ADHWNQUWPCK4YSAYQQLO2TLX6V4NHAVCNFSM6AAAAAASITOUFCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJVHA2TONZWGQ> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
