RussellSpitzer commented on code in PR #16425: URL: https://github.com/apache/iceberg/pull/16425#discussion_r4418373691
########## format/spec.md: ########## @@ -1801,6 +1804,40 @@ If a delete column in an equality delete file is later dropped from the table, i Manifests hold the same statistics for delete files and data files. For delete files, the metrics describe the values that were deleted. +### Column Files + +Starting in v4, a data file may store fields in column files listed in [`column_files`](#column-file-fields). A column file is an Iceberg data file and must have the same row count as its data file. + +A column file stores the entire value of each field in its `field_ids`. Each ID must identify a field in the table schema, or a row lineage field required by [Writing Column Files](#writing-column-files). A schema field must be top-level or nested in structs, must not be nested within a list or map, and must not be, or contain, a source field of the data file's partition spec. + +#### Writing Column Files + +In this subsection, the **previous entry** is the live entry of the data file before column files are added, an **updated field** is a table schema field whose ID is in the `field_ids` of an added column file, and an **updated row** is a row whose updated-field values were set by the update. Other rows keep their current values. Review Comment: I'm not sure this is helpful, especially the "other rows keep their current values" ########## format/spec.md: ########## @@ -1801,6 +1804,40 @@ If a delete column in an equality delete file is later dropped from the table, i Manifests hold the same statistics for delete files and data files. For delete files, the metrics describe the values that were deleted. +### Column Files + +Starting in v4, a data file may store fields in column files listed in [`column_files`](#column-file-fields). A column file is an Iceberg data file and must have the same row count as its data file. + +A column file stores the entire value of each field in its `field_ids`. Each ID must identify a field in the table schema, or a row lineage field required by [Writing Column Files](#writing-column-files). A schema field must be top-level or nested in structs, must not be nested within a list or map, and must not be, or contain, a source field of the data file's partition spec. + +#### Writing Column Files + +In this subsection, the **previous entry** is the live entry of the data file before column files are added, an **updated field** is a table schema field whose ID is in the `field_ids` of an added column file, and an **updated row** is a row whose updated-field values were set by the update. Other rows keep their current values. Review Comment: I'd probably just drop this line -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
