Re: [PR] Add partition stats in snapshot summary [iceberg-python]

2024-03-18 Thread via GitHub
Fokko commented on PR #521: URL: https://github.com/apache/iceberg-python/pull/521#issuecomment-2003194763 This is great, thanks for working on this @jqin61 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] Add partition stats in snapshot summary [iceberg-python]

2024-03-18 Thread via GitHub
Fokko merged PR #521: URL: https://github.com/apache/iceberg-python/pull/521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Add partition stats in snapshot summary [iceberg-python]

2024-03-14 Thread via GitHub
Fokko commented on code in PR #521: URL: https://github.com/apache/iceberg-python/pull/521#discussion_r1524719927 ## pyiceberg/table/snapshots.py: ## @@ -172,100 +260,57 @@ class SnapshotLogEntry(IcebergBaseModel): class SnapshotSummaryCollector: -added_file_size: int -

Re: [PR] Add partition stats in snapshot summary [iceberg-python]

2024-03-12 Thread via GitHub
Fokko commented on PR #521: URL: https://github.com/apache/iceberg-python/pull/521#issuecomment-1992323876 > @Fokko As we were discussing last time, it looks like the trustPartitionMetrics flag is always set to true in spark, so I ignored the flag. Please let me know the concerns if there i

[PR] Add partition stats in snapshot summary [iceberg-python]

2024-03-12 Thread via GitHub
jqin61 opened a new pull request, #521: URL: https://github.com/apache/iceberg-python/pull/521 Changes include: - add write.summary.partition-limit to table properties - add changed-partition-count to snapshot summary - add partition stats. these are key value pairs whose key is