RussellSpitzer commented on code in PR #18477:
URL: https://github.com/apache/iceberg/pull/18477#discussion_r4515321483


##########
spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/actions/ExpireSnapshotsSparkAction.java:
##########
@@ -222,10 +230,24 @@ private String jobDesc() {
   }
 
   private ExpireSnapshots.Result doExecute() {
-    if (streamResults()) {
-      return deleteFiles(expireFiles().toLocalIterator());
-    } else {
-      return deleteFiles(expireFiles().collectAsList().iterator());
+    Dataset<FileInfo> expiredFiles = expireFiles();
+    Iterator<FileInfo> filesToDelete =
+        streamResults() ? expiredFiles.toLocalIterator() : 
expiredFiles.collectAsList().iterator();
+    ExpireSnapshots.Result result = deleteFiles(filesToDelete);
+    failIfMissingMetadata();
+    return result;
+  }
+
+  private void failIfMissingMetadata() {
+    Set<String> missingFiles = Sets.newTreeSet(missingManifestListLocations);
+    missingFiles.addAll(missingManifests.value());
+
+    if (!missingFiles.isEmpty()) {
+      throw new NotFoundException(

Review Comment:
   This could be a real big problem if I have say a million missing manifests. 
We should avoid having an unbounded collection in a single string especially 
with how many logging systems would deal with this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to