pvary commented on PR #6614: URL: https://github.com/apache/iceberg/pull/6614#issuecomment-1399896611
> The problem for ContinuousSplitPlannerImpl of FLIP-27 IcebergSource is that it uses the Table as non read-only, as it needs to refresh the table. `Table` API assumes that you can have a refrence for the table "forever" and you can refresh the table data whenever you want. There should be no restrictions. This means that the table needs access to an open connection pool until the table is closed. I think that closing the JDBC pool before closing the table is the mistake here. I think it is not by chance that we do not have a close method on the general Catalog interface. As a general rule we expect the Catalog to be an easy static wrapper around the resources needed to access the table snapshot pointer. This could be strange and I had to push back several times on closing the HMS Connection pool in HiveCatalog when someone wanted to "fix" this issue. The JDBC and maybe some other Catalog implementations did not do this pushback, and we are in the situation where the different Catalog implementations behave differently. We should standardize the behavior (who is responsible for closing the connection pools). Hive has its own PoolCache to close unused pools, JDBC doesn't have this (if I understand correctly) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
