amogh-jahagirdar commented on code in PR #16025:
URL: https://github.com/apache/iceberg/pull/16025#discussion_r4273199510
##########
format/spec.md:
##########
@@ -1133,6 +1280,14 @@ When expiring snapshots, retention policies in table and
snapshot references are
2. The snapshot is not one of the first `min-snapshots-to-keep` in the
branch (including the branch's referenced snapshot)
5. Expire any snapshot not in the set of snapshots to retain.
+#### Deleting Files
+
+A file may be deleted from the file system when the snapshot in which it was
deleted is garbage collected, assuming that older snapshots have also been
garbage collected [1].
+
+Notes:
+
+1. Technically, data files can be deleted when the last snapshot that contains
the file as "live" data is garbage collected. But this is harder to detect and
requires finding the diff of multiple snapshots. It is easier to track what
files are deleted in a snapshot and delete them when that snapshot expires. It
is not recommended to add a deleted file back to a table. Adding a deleted file
can lead to edge cases where incremental deletes can break table snapshots.
Review Comment:
replied
[here](https://github.com/apache/iceberg/pull/16025#discussion_r4273154240),
it's existing spec wording before my change but I do see how it reads as "you
should do incremental cleanup where possible" without really defining the
"where possible" (no other branches/tags). I'll tweak this in a minimal way,
though I think the first and the last sentences are true.
I would also argue that "It is easier to track..." is still true because it
is just a recommendation of what's easier but the issue is sometimes it's not
correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
