findepi commented on code in PR #8315:
URL: https://github.com/apache/iceberg/pull/8315#discussion_r1650952000
##########
aws/src/main/java/org/apache/iceberg/aws/glue/GlueCatalog.java:
##########
@@ -619,4 +635,17 @@ public void setConf(Configuration conf) {
protected Map<String, String> properties() {
return catalogProperties == null ? ImmutableMap.of() : catalogProperties;
}
+
+ private Cache<TableOperations, FileIO> newFileIOCloser() {
+ return Caffeine.newBuilder()
+ .weakKeys()
+ .removalListener(
+ (RemovalListener<TableOperations, FileIO>)
+ (ops, fileIO, cause) -> {
+ if (null != fileIO) {
+ fileIO.close();
Review Comment:
This is not a regular cache, this could be better served with
`java.lang.ref.Cleaner`.
However, none of these addresses the main problem that we use Java GC as a
proxy to discover and close resource leaks. I understand that fileIO instances
can hold on to resources that require closing (like file descriptors or sockets
inside AWS S3 client). If i have abundance of memory, the weakly held instances
may not get collected, so the app can run out of descriptors. Can you please
help me understand why this isn't a concern?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
