mgmarino commented on code in PR #12868: URL: https://github.com/apache/iceberg/pull/12868#discussion_r2079823809
########## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SerializableTableWithSize.java: ########## @@ -33,8 +33,9 @@ * * <p>This class also implements AutoCloseable to avoid leaking resources upon broadcasting. * Broadcast variables are destroyed and cleaned up on the driver and executors once they are - * garbage collected on the driver. The implementation ensures only resources used by copies of the - * main table are released. + * garbage collected on the driver. The implementation should avoid closing deserialized copies of + * shared resources like FileIO, as they may use a shared connection pool. Shutting down the pool Review Comment: SerializedTable (which is broadcast to the different instances) can get cleaned up/persisted to disk and this leads to the connection pool getting closed, which might still be in use. See also: - https://github.com/apache/iceberg/issues/12046#issuecomment-2621663034 - https://github.com/apache/iceberg/pull/12129#issue-2818297163 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
