aiborodin commented on code in PR #13394: URL: https://github.com/apache/iceberg/pull/13394#discussion_r2176752200
########## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicIcebergSink.java: ########## @@ -191,16 +187,11 @@ public static class Builder<T> { Builder() {} - public Builder<T> forInput(DataStream<T> inputStream) { + public Builder forInput(DataStream<DynamicRecord> inputStream) { Review Comment: With no object reuse, the dynamic sink performance test never completes with changes in this PR and spams with: ``` WARN org.apache.iceberg.flink.sink.dynamic.TableMetadataCache - Performance degraded as records with different schema is generated for the same table. Likely the DynamicRecord.schema is not reused. Reuse the same instance if the record schema is the same to improve performance WARN org.apache.iceberg.flink.sink.dynamic.TableMetadataCache - Performance degraded as records with different schema is generated for the same table. Likely the DynamicRecord.schema is not reused. Reuse the same instance if the record schema is the same to improve performance WARN org.apache.iceberg.flink.sink.dynamic.TableMetadataCache - Performance degraded as records with different schema is generated for the same table. Likely the DynamicRecord.schema is not reused. Reuse the same instance if the record schema is the same to improve performance ... ``` I guess this is because schemas are copied across stages. We may indeed need a custom serialiser to fix this or leave things as status quo. Maybe the current API is not too bad given it forces object reuse. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org