kumarpritam863 opened a new pull request, #18894: URL: https://github.com/apache/iceberg/pull/18894
The dynamic sink's writer, committer, write result aggregator, record processor and table update operator load a catalog from the `CatalogLoader` but never close it, so every task restart in the same TaskManager leaks the HTTP clients, threads and connections of `Closeable` catalogs such as REST, Hive and Glue. Each `loadCatalog()` call yields a separate instance owned by one operator for its whole lifetime, so `close()` now closes it when it is `Closeable`, also when closing the other resources of the operator fails. `TableSerializerCache` is left alone because it intentionally keeps its catalog open. This is Flink 2.3 only; backports follow. Test plan: one test per operator (`TestDynamicWriter`, `TestDynamicCommitter`, `TestDynamicWriteResultAggregator`, `TestDynamicRecordProcessor`, `TestDynamicTableUpdateOperator`) asserts that a catalog counting its `close()` calls is closed exactly once, plus a `TestDynamicWriter` test that it is still closed when a task writer fails to close. All six fail without the change, and the `sink.dynamic` tests pass. --- **AI Disclosure** - Model: Claude Sonnet 5.5 - Platform/Tool: Claude Code -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
