boaz-gold commented on issue #15898:
URL: https://github.com/apache/iceberg/issues/15898#issuecomment-4279356253
@manuzhang - I added production stats to the PR.
Responding on @anoopj two alternatives:
The explicit cleanup method won't help for long-running Thrift servers —
there's no point in the lifecycle where a cleanup hook is called between table
evictions. The leak still accumulates.
Reference counting inside CachingCatalog sounds like the right approach.
The catalog already owns the cache, so it can map each FileIO instance to a
ref count, increment on load, decrement on eviction, and call close() only when
the count hits zero.
That correctly handles the shared FileIO case without leaking anything.
wdyt ?
Happy to test any implementation against a live cluster that reproduces the
crash deterministically within 24 hours.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
