tharvey5 opened a new issue, #12944: URL: https://github.com/apache/iceberg/issues/12944
### Feature Request / Improvement When using `table.exec.iceberg.use-flip27-source=true` we noticed that `IcebergSource` does not support inferParallelism like `FlinkSource` does for the legacy Flink Source. Further inspection of the code shows that this is not supported elsewhere for the new Source API. This is supported for HiveTableSource via HiveParallelismInference for the new Flink SourceAPI. Currently, it seems the only way to control the Iceberg Source parallelism for Flip27 is to use adaptive batch scheduler configs. https://flink.apache.org/2022/06/17/adaptive-batch-scheduler-automatically-decide-parallelism-of-flink-batch-jobs/ We are currently using Flink 1.18.1 and Iceberg 1.5.2. Are there any discussions around supporting this or design decision behind dropping this support in the new Source API? ### Query engine Flink ### Willingness to contribute - [ ] I can contribute this improvement/feature independently - [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community - [ ] I cannot contribute this improvement/feature at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
