Adam Strickland created KAFKA-16858:
---------------------------------------
Summary: Flatten SMT throws NPE
Key: KAFKA-16858
URL: https://issues.apache.org/jira/browse/KAFKA-16858
Project: Kafka
Issue Type: Bug
Components: connect
Affects Versions: 3.6.0
Environment: Kafka 3.6 by way of CP 7.6.0
Reporter: Adam Strickland
Attachments: FlattenTest.java
{{ConnectSchema.expectedClassesFor}} sometimes will throw an NPE as part of a
call to an SMT chain. Stack trace snippet:
{{at
com.github.momenttechnology.kafka.connect.transforms.MomentFlatten.apply(MomentFlatten.java:84)}}
{{at
com.github.momenttechnology.kafka.connect.transforms.MomentFlatten.applyWithSchema(MomentFlatten.java:173)}}
{{at
com.github.momenttechnology.kafka.connect.transforms.MomentFlatten.buildWithSchema(MomentFlatten.java:280)}}
{{at
com.github.momenttechnology.kafka.connect.transforms.MomentFlatten.buildWithSchema(MomentFlatten.java:280)}}
{{at
com.github.momenttechnology.kafka.connect.transforms.MomentFlatten.buildWithSchema(MomentFlatten.java:280)}}
{{at
com.github.momenttechnology.kafka.connect.transforms.MomentFlatten.buildWithSchema(MomentFlatten.java:274)}}
{{at org.apache.kafka.connect.data.Struct.put(Struct.java:203)}}
{{at org.apache.kafka.connect.data.Struct.put(Struct.java:216)}}
{{at
org.apache.kafka.connect.data.ConnectSchema.validateValue(ConnectSchema.java:255)}}
{{at
org.apache.kafka.connect.data.ConnectSchema.validateValue(ConnectSchema.java:213)}}
{{at
org.apache.kafka.connect.data.ConnectSchema.validateValue(ConnectSchema.java:224)}}
{{at
org.apache.kafka.connect.data.ConnectSchema.expectedClassesFor(ConnectSchema.java:268)}}
(the above transform is a sub-class of
{{{}o.a.k.connect.transforms.Flatten{}}}; have confirmed that the error occurs
regardless).
The field being transformed is an array of structs. If the call to
{{Schema#valueSchema()}} (o.a.k.connect.data.ConnectSchema.java:255) returns
{{{}null{}}}, the subsequent call to {{Schema#name()}} at
o.a.k.connect.data.ConnectSchema:268 throws an NPE.
The strange thing that we have observed is that this doesn't always happen;
*sometimes* the struct's schema is found and sometimes it is not. We have been
unable to determine the root cause, but have constructed a test that replicates
the problem as observed (see attachment).
In our case we have worked around the issue with the aforementioned sub-class
of {{{}Flatten{}}}, catching and logging the NPE on that specific use-case.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
