JeonDaehong commented on code in PR #18981:
URL: https://github.com/apache/iceberg/pull/18981#discussion_r4599962690


##########
core/src/main/java/org/apache/iceberg/BaseRowDelta.java:
##########
@@ -48,7 +48,7 @@ protected BaseRowDelta self() {
 
   @Override
   protected String operation() {
-    if (addsDataFiles() && !addsDeleteFiles() && !deletesDataFiles()) {
+    if (addsDataFiles() && !addsDeleteFiles() && !deletesDataFiles() && 
!deletesDeleteFiles()) {

Review Comment:
   `deletesDeleteFiles()` is `deleteFilterManager.containsDeletes()`, which is 
only true for explicitly removed delete files (paths, files, a delete 
expression or dropped partitions), so plain `addRows` commits still come out as 
`append`.



##########
core/src/main/java/org/apache/iceberg/BaseOverwriteFiles.java:
##########
@@ -52,7 +52,7 @@ protected String operation() {
       return DataOperations.DELETE;
     }
 
-    if (addsDataFiles() && !deletesDataFiles()) {
+    if (addsDataFiles() && !deletesDataFiles() && !deletesDeleteFiles()) {

Review Comment:
   One effect worth noting: incremental append scans, which the Flink source 
and Spark streaming reads use, only consume `append` snapshots. Rows added in a 
commit that also removes delete files will now be skipped there, or fail a 
Spark streaming read unless overwrites are skipped. That's what the spec asks 
for, but it may deserve a line in the release notes.



##########
core/src/test/java/org/apache/iceberg/TestOverwrite.java:
##########
@@ -169,6 +172,32 @@ public void addFilesProducesAppendOperation() {
     assertThat(latestSnapshot(table, 
branch).operation()).isEqualTo(DataOperations.APPEND);
   }
 
+  @TestTemplate
+  public void addFileAndRemoveDeleteFileProducesOverwriteOperation() {
+    assumeThat(formatVersion).isEqualTo(2);

Review Comment:
   This runs only on v2 since v3 rejects position delete files. A v3 variant 
with a DV would cover the same `BaseOverwriteFiles` path there.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to