rodmeneses commented on code in PR #15566:
URL: https://github.com/apache/iceberg/pull/15566#discussion_r2926132902
##########
docs/docs/flink-maintenance.md:
##########
@@ -302,19 +302,50 @@ public class TableMaintenanceJob {
Apache Iceberg Sink V2 for Flink allows automatic execution of maintenance
tasks after data is committed to the table, using the
addPostCommitTopology(...) method.
+#### DataStream API
+
+##### Builder
+
+```java
+IcebergSink.forRowData(dataStream)
+ .table(table)
+ .tableLoader(tableLoader)
+ .rewriteDataFiles(Map.of(
+ RewriteDataFilesConfig.MAX_BYTES, "1073741824"))
+ .expireSnapshots(Map.of(
+ ExpireSnapshotsConfig.RETAIN_LAST, "5",
+ ExpireSnapshotsConfig.MAX_SNAPSHOT_AGE_SECONDS, "604800"))
+ .deleteOrphanFiles(Map.of(
+ DeleteOrphanFilesConfig.MIN_AGE_SECONDS, "259200"))
+ .append();
+```
+
+##### Config
+
+All maintenance tasks are configured through string properties:
+
```java
Map<String, String> flinkConf = new HashMap<>();
-// Enable compaction and maintenance features
-flinkConf.put(FlinkWriteOptions.COMPACTION_ENABLE.key(), "true");
+// Enable maintenance tasks
+flinkConf.put("flink-maintenance.rewrite.enabled", "true");
+flinkConf.put("flink-maintenance.expire-snapshots.enabled", "true");
+flinkConf.put("flink-maintenance.delete-orphan-files.enabled", "true");
-// Configure JDBC lock settings
-flinkConf.put(LockConfig.LOCK_TYPE_OPTION.key(),
LockConfig.JdbcLockConfig.JDBC);
-flinkConf.put(LockConfig.JdbcLockConfig.JDBC_URI_OPTION.key(),
"jdbc:postgresql://localhost:5432/iceberg");
-flinkConf.put(LockConfig.LOCK_ID_OPTION.key(), "catalog.db.table");
+// Configure rewrite data files
+flinkConf.put("flink-maintenance.rewrite.max-bytes", "1073741824");
-// Add any other maintenance-related options here as needed
-// ...
+// Configure expire snapshots
+flinkConf.put("flink-maintenance.expire-snapshots.retain-last", "5");
+flinkConf.put("flink-maintenance.expire-snapshots.max-snapshot-age-seconds",
"604800");
Review Comment:
I see we're offering
```java
flinkConf.put("flink-maintenance.expire-snapshots.enabled", "true");
```
and we are also offering specific expire-snapshot configs like:
```java
flinkConf.put("flink-maintenance.expire-snapshots.retain-last", "5");
flinkConf.put("flink-maintenance.expire-snapshots.max-snapshot-age-seconds",
"604800")
```
Does `flink-maintenance.expire-snapshots.retain-last` require
`flink-maintenance.expire-snapshots.enabled` as true? (same question for the
other config, and this comment in general for all the other maintenance tasks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
