mxm commented on issue #13406: URL: https://github.com/apache/iceberg/issues/13406#issuecomment-3101927662
>Yes. To achieve that, we need to go through the entire list to collect all attributes. It gets much more complex when there are multiple nested lists. Of course, it's possible—but more complicated—and will impact performance. Got it. So you need to traverse the list, which is part of the input data, to figure out the correct type for the schema you provide. >We actually encountered issues with newly generated IDs due to changes in the event structure, which conflicted with the IDs already defined in the table schema. In some cases, the Dynamic Sink attempted to add a field to a non-struct/object field, which caused problems. If that's the case, we need to fix the code. The Dynamic Sink should never use field ids of the schema you provide. Instead, it uses fully-qualified name matching to match fields of the provided schema with the table schemas. Could you perhaps provide an example to reproduce this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org