pvary commented on PR #6605: URL: https://github.com/apache/iceberg/pull/6605#issuecomment-1385069134
Funny thing that I have found the same issue when running the spark tests against my PR: #6570. #6570 would fix this too by recreating the `hiveLockHeartbeatExecutorService` for every Lock (the same way that we were doing for the HiveTableOperations before #6570). I have some concerns about the proposed solution: - The size of the threadpool is depends on the configuration of the first executor to call the `initHiveLockHeartbeatScheduledExecutorService` - this would create hard to reproduce situations and it would be hard to understand why the new `iceberg.hive.lock-heartbeat-threadpool-size` is "not used" in some cases - This creates a JVM wide threadpool, and if the pool is exhausted then the heartbeating of the Lock will not happen, and it is done silently, which can cause issues later On the plus side, we do not recreate the pool every time when there is a new HiveTableOperations/Lock -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
