RussellSpitzer commented on code in PR #17868:
URL: https://github.com/apache/iceberg/pull/17868#discussion_r3883945312
##########
spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteDataFilesSparkAction.java:
##########
@@ -70,6 +70,23 @@ public class RewriteDataFilesSparkAction
extends BaseSnapshotUpdateSparkAction<RewriteDataFilesSparkAction>
implements RewriteDataFiles {
private static final Logger LOG =
LoggerFactory.getLogger(RewriteDataFilesSparkAction.class);
+
+ /**
+ * Use the executor cache for delete files while rewriting.
+ *
+ * <p>Enable this when the same delete file applies to many data files,
which is common with
+ * equality deletes.
+ *
+ * <p>This option sets {@link
SparkSQLProperties#EXECUTOR_CACHE_DELETE_FILES_ENABLED} for the
+ * rewrite, so any value configured for that property in the session is
ignored.
+ *
+ * <p>Defaults to false.
+ */
+ public static final String EXECUTOR_CACHE_DELETE_FILES_ENABLED =
Review Comment:
As I mentioned above, lets just drop the other parts here, I think just
CACHE_DELETE_FILES is fine
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
