Re: [I] Spark aggreation by partition could use metadata files [iceberg]

2024-11-07 Thread via GitHub
RussellSpitzer commented on issue #11394: URL: https://github.com/apache/iceberg/issues/11394#issuecomment-2462637697 Spark does have pushdowns for this. I don't know what Spark version you are using but aggregate pushdown is implemented by @huaxingao -- This is an automated message from

[I] Spark aggreation by partition could use metadata files [iceberg]

2024-10-25 Thread via GitHub
lrpt opened a new issue, #11394: URL: https://github.com/apache/iceberg/issues/11394 Hello everybody, I have a apache iceberg table in aws glue, this table is partitioned by string year-month. When I do a spark.sql("select count(1),partition_field from table group by partition_field")