szehon-ho commented on code in PR #8289:
URL: https://github.com/apache/iceberg/pull/8289#discussion_r1295413625
##########
spark/v3.4/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRewritePositionDeleteFilesProcedure.java:
##########
@@ -130,6 +147,46 @@ public void testExpireDeleteFilesNoOption() throws
Exception {
output);
}
+ @Test
+ public void testExpireDeleteFilesFilter() throws Exception {
+ createTable(true);
+
+ sql("DELETE FROM %s WHERE id = 1 and data='a'", tableName);
+ sql("DELETE FROM %s WHERE id = 1 and data='b'", tableName);
+ sql("DELETE FROM %s WHERE id = 2 and data='d'", tableName);
+ sql("DELETE FROM %s WHERE id = 2 and data='e'", tableName);
+ sql("DELETE FROM %s WHERE id = 3 and data='g'", tableName);
+ sql("DELETE FROM %s WHERE id = 3 and data='h'", tableName);
+
+
+ Table table = validationCatalog.loadTable(tableIdent);
+ Assert.assertEquals(6, TestHelpers.deleteFiles(table).size());
+
+ List<Object[]> output =
+ sql(
+ "CALL %s.system.rewrite_position_delete_files("
+ + "table => '%s',"
+ // data filter is ignored as it cannot be applied to position
deletes
+ + "where => 'id IN (1, 2) AND data=\"bar\"',"
+ + "options => map("
+ + "'rewrite-all','true'))",
+ catalogName, tableIdent);
+ table.refresh();
+
+ Map<String, String> snapshotSummary = snapshotSummary();
+ assertEquals(
+ "Should delete 4 delete files and add 2",
+ ImmutableList.of(
+ row(
+ 4,
+ 2,
+ Long.valueOf(snapshotSummary.get(REMOVED_FILE_SIZE_PROP)),
Review Comment:
Hm I could do that as well, but was following example of other tests.
For expected value, I am put number of files (4 and 2), but use the snapshot
prop fo size to avoid hardcoding in test which will cause trouble (like now as
we are changing compression..). TestRewriteDataFilesProcedure for instance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
