This is an automated email from the ASF dual-hosted git repository.
huaxingao pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/iceberg.git
The following commit(s) were added to refs/heads/main by this push:
new d303e7e880 Spec: Clarify row-level delete sections after v4
equality-delete prohibition (#18301)
d303e7e880 is described below
commit d303e7e8804385bf5e25813c3547911a219e4c80
Author: Huaxin Gao <[email protected]>
AuthorDate: Wed Nov 4 08:58:41 2026 -0800
Spec: Clarify row-level delete sections after v4 equality-delete
prohibition (#18301)
* Spec: Clarify row-level delete sections for v4
* address comments
---
format/spec.md | 21 +++++++++++++++------
1 file changed, 15 insertions(+), 6 deletions(-)
diff --git a/format/spec.md b/format/spec.md
index e1b89a1740..2a1801e9a9 100644
--- a/format/spec.md
+++ b/format/spec.md
@@ -114,11 +114,13 @@ Inheriting the sequence number from manifest metadata
allows writing a new manif
Row-level deletes are stored in delete files.
-There are two types of row-level deletes:
+The current representation is [_deletion vectors_](#deletion-vectors) (v3 or
above), which encode deleted positions of a single file in a bitmap.
-* **Position deletes** -- Mark a row deleted by data file path and the row
position in the data file. Position deletes are encoded in a [_position delete
file_](#position-delete-files) (V2) or [_deletion vector_](#deletion-vectors)
(V3 or above).
+Legacy representations:
-* **Equality deletes** -- Mark a row deleted by one or more column values,
like id = 5. Equality deletes are encoded in [_equality delete
file_](#equality-delete-files) (may be created in v2 and v3 tables only).
+* **Position deletes** -- Mark a row deleted by data file path and the row
position in the data file. Position deletes are encoded in a [_position delete
file_](#position-delete-files), which may be created in v2 tables only.
+
+* **Equality deletes** -- Mark a row deleted by one or more column values,
like id = 5. Equality deletes are encoded in an [_equality delete
file_](#equality-delete-files), which may be created in v2 and v3 tables only.
Like data files, delete files are tracked by partition. In general, a delete
file must be applied to older data files with the same partition; see [Scan
Planning](#scan-planning) for details. Column metrics can be used to determine
whether a delete file's rows overlap the contents of a data file or a scan
range.
@@ -1692,7 +1694,14 @@ Notes:
### Delete Formats
-This section details how to encode row-level deletes in Iceberg delete files.
Row-level deletes are added by v2 and are not supported in v1. Deletion vectors
are added in v3 and are not supported in v2 or earlier. Position delete files
must not be added to v3 tables, but existing position delete files are valid.
Equality delete files must not be added to v4 tables, but existing equality
delete files are valid.
+This section details how to encode row-level deletes in Iceberg delete files.
+
+Row-level deletes have changed across Iceberg versions:
+
+* v1 does not support row-level deletes
+* v2 allows position delete files and equality delete files stored as entries
in delete manifests
+* v3 forbids new position delete files and allows deletion vectors as a
replacement
+* v4 forbids new equality deletes and delete manifests. Deletion vector
metadata is now part of the manifest entries of the data files they modify
There are different formats for encoding row-level deletes:
@@ -2291,8 +2300,8 @@ Snapshot timestamp changes:
Equality deletes are prohibited in v4.
* Writers must not add equality delete files to v4 tables; equality deletes
cannot be added as an entry to a v4 manifest
-* Upgrading a v2 or v3 table to v4 does not require rewriting data or delete
files
-* Readers must continue to apply equality deletes for v2 and v3 tables and for
equality deletes carried over into upgraded v4 tables
+* In a table upgraded to v4, existing delete manifests and the delete files
they track remain valid and must be applied at read time
+* Readers must continue to apply equality deletes for v2 and v3 tables
Constraints are added in v4:
