bk-mz commented on PR #9945:
URL: https://github.com/apache/iceberg/pull/9945#issuecomment-2165151682
This looks plain wrong.
If I we set shuffle.partititions to 1 this will affect the rewrite
procedure, it will collapse all delete files into one, meaning we'd be having a
situation
bk-mz closed pull request #9945: Fix `rewrite_position_delete_files` result
file set
URL: https://github.com/apache/iceberg/pull/9945
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific com
bk-mz commented on PR #9945:
URL: https://github.com/apache/iceberg/pull/9945#issuecomment-2036840825
@nastra hey, yes, you are correct. sorry for that.
unfortunately I can't setup proper infrastructure to ensure this test
requirements.
I've found this bug in prod during produ
nastra commented on PR #9945:
URL: https://github.com/apache/iceberg/pull/9945#issuecomment-2034532286
@bk-mz I don't think the test actually reproduces the underlying issue. See
my comment in https://github.com/apache/iceberg/pull/9945#discussion_r1526415663
--
This is an automated messa
nastra commented on code in PR #9945:
URL: https://github.com/apache/iceberg/pull/9945#discussion_r1526415663
##
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/SparkBinPackPositionDeletesRewriter.java:
##
@@ -64,6 +64,8 @@ class SparkBinPackPositionDeletesRewrit
bk-mz commented on PR #9945:
URL: https://github.com/apache/iceberg/pull/9945#issuecomment-1998464433
@nastra done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsub
nastra commented on code in PR #9945:
URL: https://github.com/apache/iceberg/pull/9945#discussion_r1524645127
##
spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRewritePositionDeleteFilesProcedure.java:
##
@@ -119,11 +121,22 @@ public void testE
bk-mz commented on PR #9945:
URL: https://github.com/apache/iceberg/pull/9945#issuecomment-1996688216
Done:
```
./gradlew :iceberg-spark:iceberg-spark-extensions-3.5_2.12:test --tests
TestRewritePositionDeleteFilesProcedure
BUILD SUCCESSFUL in 30s
41 actionable tasks: 2 execu
nastra commented on code in PR #9945:
URL: https://github.com/apache/iceberg/pull/9945#discussion_r1523241967
##
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/SparkBinPackPositionDeletesRewriter.java:
##
@@ -64,6 +64,8 @@ class SparkBinPackPositionDeletesRewrit
