aiborodin commented on code in PR #13394:
URL: https://github.com/apache/iceberg/pull/13394#discussion_r2176752200


##########
flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicIcebergSink.java:
##########
@@ -191,16 +187,11 @@ public static class Builder<T> {
 
     Builder() {}
 
-    public Builder<T> forInput(DataStream<T> inputStream) {
+    public Builder forInput(DataStream<DynamicRecord> inputStream) {

Review Comment:
   With no object reuse, the dynamic sink performance test never completes with 
changes in this PR and spams with:
   ```
   WARN org.apache.iceberg.flink.sink.dynamic.TableMetadataCache - Performance 
degraded as records with different schema is generated for the same table. 
Likely the DynamicRecord.schema is not reused. Reuse the same instance if the 
record schema is the same to improve performance
   WARN org.apache.iceberg.flink.sink.dynamic.TableMetadataCache - Performance 
degraded as records with different schema is generated for the same table. 
Likely the DynamicRecord.schema is not reused. Reuse the same instance if the 
record schema is the same to improve performance
   WARN org.apache.iceberg.flink.sink.dynamic.TableMetadataCache - Performance 
degraded as records with different schema is generated for the same table. 
Likely the DynamicRecord.schema is not reused. Reuse the same instance if the 
record schema is the same to improve performance
   ...
   ```
   I guess this is because schemas are copied across stages. We may indeed need 
a custom serialiser to fix this or leave things as status quo. Maybe the 
current API is not too bad given it forces object reuse.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org

Reply via email to