anuragmantri commented on code in PR #16425:
URL: https://github.com/apache/iceberg/pull/16425#discussion_r4567025226


##########
format/spec.md:
##########
@@ -88,7 +88,7 @@ This table format tracks individual data files in a table 
instead of directories
 
 Table state is maintained in metadata files. All changes to table state create 
a new metadata file and replace the old metadata with an atomic swap. The table 
metadata file tracks the table schema, partitioning config, custom properties, 
and snapshots of the table contents. A snapshot represents the state of a table 
at some time and is used to access the complete set of data files in the table.
 
-Data files in snapshots are tracked by one or more manifest files that contain 
a row for each data file in the table, the file's partition data, an optional 
colocated deletion vector (v4), and its metrics. The data in a snapshot is the 
union of all live files in its manifests; each live file may only appear once 
(see [Content file uniqueness](#content-file-uniqueness)). Manifest files are 
reused across snapshots to avoid rewriting metadata that is slow-changing. 
Manifests can track data files with any subset of a table and are not 
associated with partitions.
+Data files in snapshots are tracked by one or more manifest files that contain 
a row for each data file in the table, the file's partition data, an optional 
colocated deletion vector and optional column files (v4), and its metrics. The 
data in a snapshot is the union of all live files in its manifests; each live 
file may only appear once (see [Content file 
uniqueness](#content-file-uniqueness)). Manifest files are reused across 
snapshots to avoid rewriting metadata that is slow-changing. Manifests can 
track data files with any subset of a table and are not associated with 
partitions.

Review Comment:
   Done. I moved the v4 items to the end of the list, so it reads "…its 
metrics, and, starting in v4, an optional colocated deletion vector and 
optional column files." That avoids two "and"s in one list.



##########
format/spec.md:
##########
@@ -1203,13 +1206,13 @@ Values for `snapshot_id`, `sequence_number`, and 
`file_sequence_number` are inhe
 
 Manifests track the sequence number when a data or delete file was added to 
the table.
 
-The `sequence_number` field represents the data sequence number. It is set 
when a file is added to the dataset and must not change unless a column file is 
added. The data sequence number represents a relative age of the file content 
and should be used for planning which delete files apply to a data file.
+The `sequence_number` field represents the data sequence number. It is set 
when a file is added to the dataset and must not change unless a column file is 
added to a data file. The data sequence number represents a relative age of the 
file content and should be used for planning which delete files apply to a data 
file.

Review Comment:
   It's still tracked, it just doesn't change when column files are added to a 
leaf manifest. The sentence says "a file", which covers both data files and 
manifests. A leaf manifest's entries inherit null sequence numbers from it, so 
raising it would change the data sequence number of data files the update 
didn't touch. Did you mean we should call out manifests explicitly?



##########
format/spec.md:
##########
@@ -1748,7 +1751,7 @@ The rows in the delete file must be sorted by `file_path` 
then `pos` to optimize
 
 Equality delete files identify deleted rows in a collection of data files by 
one or more column values, and may optionally contain additional columns of the 
deleted row.
 
-Equality delete files must not be added to v4 tables. Equality deletes cannot 
be added as an entry to a v4 manifest. Equality delete files referenced by v2 
or v3 delete manifests must still be applied in a v4 table.
+Equality delete files must not be added to v4 tables. Equality deletes cannot 
be added as an entry to a v4 manifest. Equality delete files referenced by v2 
or v3 delete manifests must still be applied in a v4 table. Adding column files 
to a data file sets its data sequence number to that snapshot's sequence 
number, so existing equality delete files no longer apply to it (see [Writing 
Column Files](#writing-column-files)).

Review Comment:
   Right, as Russell said. The `deletion_vector` bullet in Writing Column Files 
requires every delete that applied to the file, including equality deletes, to 
be written into its deletion vector before the column files are added, so no 
deletes are lost.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to