anuragmantri commented on code in PR #16425: URL: https://github.com/apache/iceberg/pull/16425#discussion_r4567025226
########## format/spec.md: ########## @@ -88,7 +88,7 @@ This table format tracks individual data files in a table instead of directories Table state is maintained in metadata files. All changes to table state create a new metadata file and replace the old metadata with an atomic swap. The table metadata file tracks the table schema, partitioning config, custom properties, and snapshots of the table contents. A snapshot represents the state of a table at some time and is used to access the complete set of data files in the table. -Data files in snapshots are tracked by one or more manifest files that contain a row for each data file in the table, the file's partition data, an optional colocated deletion vector (v4), and its metrics. The data in a snapshot is the union of all live files in its manifests; each live file may only appear once (see [Content file uniqueness](#content-file-uniqueness)). Manifest files are reused across snapshots to avoid rewriting metadata that is slow-changing. Manifests can track data files with any subset of a table and are not associated with partitions. +Data files in snapshots are tracked by one or more manifest files that contain a row for each data file in the table, the file's partition data, an optional colocated deletion vector and optional column files (v4), and its metrics. The data in a snapshot is the union of all live files in its manifests; each live file may only appear once (see [Content file uniqueness](#content-file-uniqueness)). Manifest files are reused across snapshots to avoid rewriting metadata that is slow-changing. Manifests can track data files with any subset of a table and are not associated with partitions. Review Comment: Done. I moved the v4 items to the end of the list, so it reads "…its metrics, and, starting in v4, an optional colocated deletion vector and optional column files." That avoids two "and"s in one list. ########## format/spec.md: ########## @@ -1203,13 +1206,13 @@ Values for `snapshot_id`, `sequence_number`, and `file_sequence_number` are inhe Manifests track the sequence number when a data or delete file was added to the table. -The `sequence_number` field represents the data sequence number. It is set when a file is added to the dataset and must not change unless a column file is added. The data sequence number represents a relative age of the file content and should be used for planning which delete files apply to a data file. +The `sequence_number` field represents the data sequence number. It is set when a file is added to the dataset and must not change unless a column file is added to a data file. The data sequence number represents a relative age of the file content and should be used for planning which delete files apply to a data file. Review Comment: It's still tracked, it just doesn't change when column files are added to a leaf manifest. The sentence says "a file", which covers both data files and manifests. A leaf manifest's entries inherit null sequence numbers from it, so raising it would change the data sequence number of data files the update didn't touch. Did you mean we should call out manifests explicitly? ########## format/spec.md: ########## @@ -1748,7 +1751,7 @@ The rows in the delete file must be sorted by `file_path` then `pos` to optimize Equality delete files identify deleted rows in a collection of data files by one or more column values, and may optionally contain additional columns of the deleted row. -Equality delete files must not be added to v4 tables. Equality deletes cannot be added as an entry to a v4 manifest. Equality delete files referenced by v2 or v3 delete manifests must still be applied in a v4 table. +Equality delete files must not be added to v4 tables. Equality deletes cannot be added as an entry to a v4 manifest. Equality delete files referenced by v2 or v3 delete manifests must still be applied in a v4 table. Adding column files to a data file sets its data sequence number to that snapshot's sequence number, so existing equality delete files no longer apply to it (see [Writing Column Files](#writing-column-files)). Review Comment: Right, as Russell said. The `deletion_vector` bullet in Writing Column Files requires every delete that applied to the file, including equality deletes, to be written into its deletion vector before the column files are added, so no deletes are lost. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
