swapna267 commented on code in PR #18525:
URL: https://github.com/apache/iceberg/pull/18525#discussion_r4296904875
##########
flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordInternalSerializer.java:
##########
@@ -94,21 +94,9 @@ public void serialize(DynamicRecordInternal toSerialize,
DataOutputView dataOutp
}
dataOutputView.writeInt(toSerialize.writerKey());
- final RowDataSerializer rowDataSerializer;
- if (writeSchemaAndSpec) {
- rowDataSerializer =
- serializerCache.serializer(
- toSerialize.tableName(), toSerialize.schema(),
toSerialize.spec());
- } else {
- // Check that the schema id can be resolved. Not strictly necessary for
serialization.
- Tuple3<RowDataSerializer, Schema, PartitionSpec> serializer =
- serializerCache.serializerWithSchemaAndSpec(
- toSerialize.tableName(),
- toSerialize.schema().schemaId(),
- toSerialize.spec().specId());
- rowDataSerializer = serializer.f0;
- }
-
+ RowDataSerializer rowDataSerializer =
+ serializerCache.serializer(
+ toSerialize.tableName(), toSerialize.schema(), toSerialize.spec());
Review Comment:
Yeah Peter, we are storing a new serializer for every schema,
https://github.com/apache/iceberg/blob/main/flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/TableSerializerCache.java#L98-L100
```
RowDataSerializer serializer =
info.serializers.computeIfAbsent(
schema, s -> new RowDataSerializer(FlinkSchemaUtil.convert(s)));
```
And this schema reference is either from DynamicRecord (when not found in
TableMetadataCache) or Schema reference from TableMetadataCache. So we should
still serve the serializer from cache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
