aokolnychyi commented on code in PR #12098: URL: https://github.com/apache/iceberg/pull/12098#discussion_r1951754710
########## format/spec.md: ########## @@ -927,20 +927,21 @@ These rows must be sorted (in ascending manner with NULL FIRST) by `partition` f The schema of the partition statistics file is as follows: -| v1 | v2 | Field id, name | Type | Description | -|----|----|----------------|------|-------------| -| _required_ | _required_ | **`1 partition`** | `struct<..>` | Partition data tuple, schema based on the unified partition type considering all specs in a table | -| _required_ | _required_ | **`2 spec_id`** | `int` | Partition spec id | -| _required_ | _required_ | **`3 data_record_count`** | `long` | Count of records in data files | -| _required_ | _required_ | **`4 data_file_count`** | `int` | Count of data files | -| _required_ | _required_ | **`5 total_data_file_size_in_bytes`** | `long` | Total size of data files in bytes | -| _optional_ | _optional_ | **`6 position_delete_record_count`** | `long` | Count of records in position delete files | -| _optional_ | _optional_ | **`7 position_delete_file_count`** | `int` | Count of position delete files | -| _optional_ | _optional_ | **`8 equality_delete_record_count`** | `long` | Count of records in equality delete files | -| _optional_ | _optional_ | **`9 equality_delete_file_count`** | `int` | Count of equality delete files | -| _optional_ | _optional_ | **`10 total_record_count`** | `long` | Accurate count of records in a partition after applying the delete files if any | -| _optional_ | _optional_ | **`11 last_updated_at`** | `long` | Timestamp in milliseconds from the unix epoch when the partition was last updated | -| _optional_ | _optional_ | **`12 last_updated_snapshot_id`** | `long` | ID of snapshot that last updated this partition | +| v1 | v2 | v3 | Field id, name | Type | Description | +|----|----|----|----------------|------|-------------| +| _required_ | _required_ | _required_ | **`1 partition`** | `struct<..>` | Partition data tuple, schema based on the unified partition type considering all specs in a table | +| _required_ | _required_ | _required_ | **`2 spec_id`** | `int` | Partition spec id | +| _required_ | _required_ | _required_ | **`3 data_record_count`** | `long` | Count of records in data files | +| _required_ | _required_ | _required_ | **`4 data_file_count`** | `int` | Count of data files | +| _required_ | _required_ | _required_ | **`5 total_data_file_size_in_bytes`** | `long` | Total size of data files in bytes | +| _optional_ | _optional_ | _required_ | **`6 position_delete_record_count`** | `long` | Count of position deletes across position delete files and deletion vectors | +| _optional_ | _optional_ | _required_ | **`7 position_delete_file_count`** | `int` | Count of position delete files ignoring deletion vectors | +| | | _required_ | **`13 dv_count`** | `int` | Count of deletion vectors | Review Comment: I spent a bit more time thinking about this and would still opt in for not introducing a new field to be consistent with how we handle these counts elsewhere (e.g. `SnapshotSummary`). DVs store position deletes so reusing the existing count is appropriate. My gut tells me it is unlikely the record count in DVs vs position delete files would matter based on our discussions around table maintenance. @stevenzwu, what do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
