twthorn commented on code in PR #16826:
URL: https://github.com/apache/iceberg/pull/16826#discussion_r3444144441
##########
kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/RecordConverter.java:
##########
@@ -268,6 +276,74 @@ private GenericRecord convertToStruct(
return result;
}
+ /**
+ * This method evolves schemas when the value is null. Note: logic should be
kept consistent with
+ * equivalent method used when value is not null: {@link
#convertToStruct(Struct, StructType, int,
+ * SchemaUpdate.Consumer)}
+ */
+ private void evolveSchemaFromConnectSchema(
+ org.apache.kafka.connect.data.Schema recordSchema,
+ Type tableType,
+ int tableFieldId,
+ SchemaUpdate.Consumer schemaUpdateConsumer) {
+ if (recordSchema == null) {
+ return;
+ }
+ switch (recordSchema.type()) {
+ case STRUCT:
Review Comment:
Agreed, added a small helper for logging since it's the same message/format
in each, and each branch calls that to keep the code DRY.
##########
kafka-connect/kafka-connect/src/test/java/org/apache/iceberg/connect/data/TestRecordConverter.java:
##########
@@ -859,6 +859,378 @@ private void assertTypesAddedFromStruct(Function<String,
Type> fn) {
assertThat(fn.apply("ma")).isInstanceOf(MapType.class);
}
+ @Test
+ public void testNestedSchemaEvolutionStructWithNullValue() {
+ org.apache.iceberg.Schema nestedStructSchema =
+ new org.apache.iceberg.Schema(
Review Comment:
Nice catch, I added the test for that null value no evolution case.
##########
kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/RecordConverter.java:
##########
@@ -268,6 +276,74 @@ private GenericRecord convertToStruct(
return result;
}
+ /**
+ * This method evolves schemas when the value is null. Note: logic should be
kept consistent with
Review Comment:
Good point. There are only certain parts of the method that mirror the other
function, but overall behaves fundamentally different. I have updated the java
doc to clarify this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
