tanmayrauth commented on code in PR #916:
URL: https://github.com/apache/iceberg-go/pull/916#discussion_r3101819750
##########
io/io.go:
##########
@@ -90,6 +90,31 @@ type WriteFileIO interface {
WriteFile(name string, p []byte) error
}
+// BulkRemovableIO is an optional interface for IO implementations that
Review Comment:
Removed all caller wiring. The interface definition stays (so #917 has a
target to implement against), but no code in this PR references BulkRemovableIO
at call sites.
##########
io/io.go:
##########
@@ -90,6 +90,31 @@ type WriteFileIO interface {
WriteFile(name string, p []byte) error
}
+// BulkRemovableIO is an optional interface for IO implementations that
+// support deleting multiple files in a single batch operation.
+// Cloud object stores (S3 DeleteObjects, GCS batch, Azure batch) can
+// implement this for significantly better throughput than single-file Remove.
+type BulkRemovableIO interface {
Review Comment:
Added. Signature is now DeleteFiles(ctx context.Context, paths []string)
(deleted []string, err error). Cloud batch deletes are network calls — they
should be cancellable and carry deadlines from the start.
##########
table/updates.go:
##########
@@ -534,6 +535,15 @@ func (u *removeSnapshotsUpdate) PostCommit(ctx
context.Context, preTable *Table,
delete(filesToDelete, psf.StatisticsPath)
}
+ if bulk, ok := prefs.(io.BulkRemovableIO); ok {
Review Comment:
All three issues are moot in this PR — the bulk path in PostCommit has been
removed entirely. When it's re-added in #917, all three will be addressed:
1. ctx propagated to DeleteFiles
2. Failed bulk falls through to per-file Remove loop
3. len(paths) == 0 guard before calling DeleteFiles
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
