shangxinli commented on code in PR #14435:
URL: https://github.com/apache/iceberg/pull/14435#discussion_r2530375006
##########
api/src/main/java/org/apache/iceberg/actions/RewriteDataFiles.java:
##########
@@ -147,6 +147,30 @@ public interface RewriteDataFiles
*/
String OUTPUT_SPEC_ID = "output-spec-id";
+ /**
+ * Use Parquet row-group level merging during rewrite operations when
applicable.
+ *
+ * <p>When enabled, Parquet files will be merged at the row-group level
using zero-copy operations
+ * instead of full read-rewrite. This provides significant performance
improvements (up to 13x
+ * faster) for compatible Parquet files.
+ *
+ * <p>Requirements for row-group merging:
+ *
+ * <ul>
+ * <li>All files must be in Parquet format
+ * <li>Files must have compatible schemas
+ * <li>Files must not be encrypted
+ * </ul>
+ *
+ * <p>If requirements are not met, the rewrite will automatically fall back
to the standard
+ * read-rewrite approach with a warning logged.
+ *
+ * <p>Defaults to false.
+ */
+ String USE_PARQUET_ROW_GROUP_MERGE = "use-parquet-row-group-merge";
Review Comment:
Good catch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
