dramaticlly commented on code in PR #8289:
URL: https://github.com/apache/iceberg/pull/8289#discussion_r1295215503
##########
spark/v3.4/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRewritePositionDeleteFilesProcedure.java:
##########
@@ -130,6 +147,46 @@ public void testExpireDeleteFilesNoOption() throws
Exception {
output);
}
+ @Test
+ public void testExpireDeleteFilesFilter() throws Exception {
+ createTable(true);
+
+ sql("DELETE FROM %s WHERE id = 1 and data='a'", tableName);
+ sql("DELETE FROM %s WHERE id = 1 and data='b'", tableName);
+ sql("DELETE FROM %s WHERE id = 2 and data='d'", tableName);
+ sql("DELETE FROM %s WHERE id = 2 and data='e'", tableName);
+ sql("DELETE FROM %s WHERE id = 3 and data='g'", tableName);
+ sql("DELETE FROM %s WHERE id = 3 and data='h'", tableName);
+
+
+ Table table = validationCatalog.loadTable(tableIdent);
+ Assert.assertEquals(6, TestHelpers.deleteFiles(table).size());
+
+ List<Object[]> output =
+ sql(
+ "CALL %s.system.rewrite_position_delete_files("
+ + "table => '%s',"
+ // data filter is ignored as it cannot be applied to position
deletes
+ + "where => 'id IN (1, 2) AND data=\"bar\"',"
+ + "options => map("
+ + "'rewrite-all','true'))",
+ catalogName, tableIdent);
+ table.refresh();
+
+ Map<String, String> snapshotSummary = snapshotSummary();
+ assertEquals(
+ "Should delete 4 delete files and add 2",
+ ImmutableList.of(
+ row(
+ 4,
+ 2,
+ Long.valueOf(snapshotSummary.get(REMOVED_FILE_SIZE_PROP)),
Review Comment:
I think this shall be `REMOVED_POS_DELETE_FILES_PROP`
and next one shall be `ADD_POS_DELETE_FILES_PROP`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
