xiaoxuandev commented on issue #15898: URL: https://github.com/apache/iceberg/issues/15898#issuecomment-4292834368
hi @boaz-gold, thanks for reporting the issue. CachingCatalog doesn't own FileIO and shouldn't manage its lifecycle. GlueCatalog and RESTSessionCatalog already handle this correctly with FileIOTracker at the catalog level — it uses weak keys tied to TableOperations, so FileIO gets closed automatically when the TableOperations is GC'd. One thing CachingCatalog could do to help: the current softValues() on the Caffeine cache keeps evicted Table objects alive (and their FileIO with them) until memory pressure triggers GC. Switching to weakValues() would let GC reclaim evicted tables promptly, allowing FileIOTracker to close the FileIO much sooner. Which catalog are you using? That would help identify whether the leak is a missing FileIOTracker in that catalog, or something else. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
