pvary commented on code in PR #12366:
URL: https://github.com/apache/iceberg/pull/12366#discussion_r1967759281
##########
data/src/main/java/org/apache/iceberg/data/GenericAppenderFactory.java:
##########
@@ -44,41 +46,85 @@
/** Factory to create a new {@link FileAppender} to write {@link Record}s. */
public class GenericAppenderFactory implements FileAppenderFactory<Record> {
-
+ private final Table table;
private final Schema schema;
private final PartitionSpec spec;
private final int[] equalityFieldIds;
private final Schema eqDeleteRowSchema;
private final Schema posDeleteRowSchema;
- private final Map<String, String> config = Maps.newHashMap();
+ private final Map<String, String> config;
+
+ private static final String WRITE_METRICS_PREFIX = "write.metadata.metrics.";
+ @Deprecated
public GenericAppenderFactory(Schema schema) {
- this(schema, PartitionSpec.unpartitioned(), null, null, null);
+ this(schema, PartitionSpec.unpartitioned());
}
+ @Deprecated
public GenericAppenderFactory(Schema schema, PartitionSpec spec) {
this(schema, spec, null, null, null);
}
+ @Deprecated
public GenericAppenderFactory(
Schema schema,
PartitionSpec spec,
int[] equalityFieldIds,
Schema eqDeleteRowSchema,
Schema posDeleteRowSchema) {
- this.schema = schema;
- this.spec = spec;
+ this(null, schema, spec, null, equalityFieldIds, eqDeleteRowSchema,
posDeleteRowSchema);
+ }
+
+ public GenericAppenderFactory(Table table) {
+ this(table, null, null, null, null, null, null);
+ }
+
+ public GenericAppenderFactory(
+ Table table,
+ Schema schema,
+ PartitionSpec spec,
+ Map<String, String> config,
+ int[] equalityFieldIds,
+ Schema eqDeleteRowSchema,
+ Schema posDeleteRowSchema) {
+ this.table = table;
+ if (table != null && schema == null) {
+ this.schema = table.schema();
+ } else {
+ this.schema = schema;
+ }
+
+ if (table != null && spec == null) {
+ this.spec = table.spec();
+ } else {
+ this.spec = spec;
+ }
+
+ this.config = config == null ? Maps.newHashMap() : config;
Review Comment:
I don't like that the handling of the config is different in the constructor
and the `set`, `setAll` method.
I don't think we have a well defined behavior here.
Please help me think through what is the expected behavior:
- Before the PR
- After the PR, but before removing the deprecated methods
- After removing the deprecated methods
My main questions:
- Do we still want to allow the users to create data files without providing
a table? (I think it would be good)
- How does the user configure the metrics in case the table has not been
created yet?
- How does the user migrate to the new behavior, and how they will detect
the behavior change?
Thanks,
Peter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
