dramaticlly commented on code in PR #7976:
URL: https://github.com/apache/iceberg/pull/7976#discussion_r1287762780
##########
core/src/main/java/org/apache/iceberg/io/ResolvingFileIO.java:
##########
@@ -83,6 +86,29 @@ public void deleteFile(String location) {
io(location).deleteFile(location);
}
+ @Override
+ public void deleteFiles(Iterable<String> pathsToDelete) throws
BulkDeletionFailureException {
+ Map<FileIO, List<String>> pathByFileIO =
+ StreamSupport.stream(pathsToDelete.spliterator(), false)
Review Comment:
I think we can do batch but I realized we ready have a batch size as
properties defined on concrete fileIO, here's one example for S3:
https://github.com/apache/iceberg/blob/29eebdd89efdcb9fb32e53325dfd2b29b9a3377b/aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIOProperties.java#L209-L217.
Shall we introduce a new set of configuration to batch the request or provide
some fixed batch size and allow for customization later on?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
