djakupovic opened a new issue, #14530: URL: https://github.com/apache/iceberg/issues/14530
### Feature Request / Improvement Hey, my use case: debezium cdc into kafka + iceberg kafka connect sink iceberg kafka connect config: topics.regex: '^dev\.(.*)' --> fetch every topic starting with dev so only one kafka iceberg sink connector which is great idea: sink into iceberg.table.namespace + topic name therefore $1 from the matcher of the regex. but currently that's not possible, this would be great if this could be handled. improvement: the dynamic feature is somehow great to sink data into the landing zone. However, if we use: iceberg.tables.dynamic-enabled: "true" iceberg.tables.route-field: "source_table" --> filed in the record called source_table it does not respect the iceberg.table.namespace therefore not the schema e.g. "landing" and tries to write without any schema. So only the route-field as table name is being used instead namespace.route-field. Therefore we need to create a field in the record with the schema in order to route but that's not a good idea to be honest. This feature should be somehow rewritten at least with iceberg.table.namespace being respected or even better with a regex pattern to fetch from the incoming topic name. the record itself should be independent from the target. I do not see any possibility to do a 1:1 dynamic sink with the kafka connect iceberg sink at the current feature set. ### Query engine None ### Willingness to contribute - [ ] I can contribute this improvement/feature independently - [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community - [ ] I cannot contribute this improvement/feature at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
