rdblue commented on code in PR #16025:
URL: https://github.com/apache/iceberg/pull/16025#discussion_r4150373645
##########
format/spec.md:
##########
@@ -676,13 +700,19 @@ A manifest file must store the partition spec and other
metadata as properties i
| _optional_ | _required_ | `format-version` | Table format version
number of the manifest as a string
|
| | _required_ | `content` | Type of content files
tracked by the manifest: "data" or "deletes"
|
+=== "v4"
+ | Requirement | Key | Value
|
+
|-------------|---------------------|---------------------------------------------------------------------------------------------------------------------------------------------|
+ | _optional_ | `schema-id` | ID of the schema used to write the
manifest as a string
|
+ | _optional_ | `format-version` | Table format version number of the
manifest as a string
|
+
#### Content file uniqueness
Within a snapshot, each content file must be referenced by at most one live
manifest entry across all manifests; otherwise, the snapshot has undefined
behavior. Writers should not produce multiple manifest entries for the same
content file in a snapshot (for example, both ADDED and DELETED entries for the
same file). Writers are not required to validate uniqueness at commit time.
-#### Manifest Entry Fields
+#### Entries in Manifests
-The schema of a manifest file is defined by the `manifest_entry` struct, which
consists of the following fields:
+In v1-v3, manifest entries are described by the `manifest_entry` struct. In
v4, entries are called tracked files and are described by the `tracked_file`
struct. In v4, `data_file` struct fields are flattened directly into the
tracked file, and tracking fields are grouped into a nested `tracking` struct.
An entry is **live** in a snapshot if its `status` is ADDED, EXISTING, or
MODIFIED and its position is not set in the containing manifest's
[`manifest_info.dv`](#manifest-deletion-vectors).
Review Comment:
```suggestion
In v4, manifests store `tracked_file` records that describe data or manifest
files and contain `tracking` metadata, like `status`. The relationship between
a file and its tracking metadata was reversed in v1-v3 manifests that store
`manifest_entry` records that contain tracking information and data or delete
file struct.
The contents of a file is part of a snapshot if its tracking `status` is
_live_: ADDED, EXISTING, or MODIFIED. Changes in a snapshot are tracked using
the additional DELETED and REPLACED statuses.
```
For MDV handling, I think we want to state that the effect of MDVs in the
section on them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
