gaborkaszab commented on code in PR #16285:
URL: https://github.com/apache/iceberg/pull/16285#discussion_r4080178773
##########
core/src/main/java/org/apache/iceberg/TrackingBuilder.java:
##########
@@ -115,6 +118,19 @@ TrackingBuilder dvUpdated() {
return this;
}
+ /** Indicates that the column files list has been updated for the new
Tracking. */
+ TrackingBuilder columnFilesUpdated() {
+ this.latestColumnFileSnapshotId = newSnapshotId;
+ if (status == EntryStatus.EXISTING) {
+ this.status = EntryStatus.MODIFIED;
+ }
+ // Reset to null to inherit from the new snapshot sequence number. It is
safe to bump up the
+ // dataSequenceNumber as writers are required to rewrite v2 equality and
position deletes to DVs
+ // when applying column update.
+ this.dataSequenceNumber = null;
Review Comment:
The spec will contain that rewriting the deletes into DVs is a requirement
for the writers when adding a column file. Here, I think the comment makes
sense, it reflects the spec (well future spec, as still WiP), and adds clarity
why we are resetting data seq num here.
I'm not entirely sure we can actually enforce it here: We could check that
when columnFilesUpdated() is called, we have to call dvUpdated() too, but that
would be incorrect in the case when there are no deletes to be rewritten. Some
enforcement would the API to add column files: `ddColumnFiles(DataFile
baseFile, List<ColumnFile> column files, DeletionVector dv)`, but that can't
verify either if the deletes are rewritten, it enforces writers to provide the
new DV, though.
I think this is correct, as we don't really have control on what data is
actually used and rewritten when committing stuff, we have to rely on the
writer following the spec.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
