mxm commented on code in PR #11662:
URL: https://github.com/apache/iceberg/pull/11662#discussion_r1861944660
##########
flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySerializer.java:
##########
@@ -124,6 +124,14 @@ public void serialize(SortKey record, DataOutputView
target) throws IOException
for (int i = 0; i < size; ++i) {
int fieldId = transformedFields[i].fieldId();
Type.TypeID typeId = transformedFields[i].type().typeId();
+ Object value = record.get(i, Object.class);
Review Comment:
Nulls are not allowed everywhere in Flink. Primitive types generally don't
support null. In other places, Flink uses boolean flags to encode nulls, e.g.
in PojoSerializer:
https://github.com/apache/flink/blob/b9c92371dba07b6bfe4368d7b6d7f7c575b4c603/flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/PojoSerializer.java#L392
KeyBy operations are not allowed on null values.
+1 to Steven's suggestion to version the serializer. We already have
[SortKeySerializerSnapshot](https://github.com/apache/iceberg/blob/163e2068f96f139632488f36928bf443c9be326f/flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySerializer.java#L278).
We just need to add a version check for the new boolean serializer because
this is an incompatible change which requires serializer migration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
