pvary commented on code in PR #6570: URL: https://github.com/apache/iceberg/pull/6570#discussion_r1071920474
########## docs/configuration.md: ########## @@ -175,8 +175,13 @@ The HMS table locking is a 2-step process: | iceberg.hive.lock-heartbeat-interval-ms | 240000 (4 min) | The heartbeat interval for the HMS locks. | | iceberg.hive.metadata-refresh-max-retries | 2 | Maximum number of retries when the metadata file is missing | | iceberg.hive.table-level-lock-evict-ms | 600000 (10 min) | The timeout for the JVM table lock is | +| iceberg.lock.hive.enabled | true | If enabled HMS locks will be used to ensure of the atomicity of the commits | Note: `iceberg.hive.lock-check-max-wait-ms` and `iceberg.hive.lock-heartbeat-interval-ms` should be less than the [transaction timeout](https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.txn.timeout) of the Hive Metastore (`hive.txn.timeout` or `metastore.txn.timeout` in the newer versions). Otherwise, the heartbeats on the lock (which happens during the lock checks) would end up expiring in the Hive Metastore before the lock is retried from Iceberg. +Note: `iceberg.lock.hive.enabled` should only be set to `false` if [HIVE-26882](https://issues.apache.org/jira/browse/HIVE-26882) Review Comment: On HMS side HIVE-26882 is needed on both HMS nodes. This config value is on the Iceberg side (client side from the HMS point of view). But the question is good, because it highlights an important thing: - All of the Iceberg writers should have the same configuration This is the exact reason why I have added the table level configuration, so we can change the behaviour of every clients "atomically" by changing the table property. So the upgrade process would look like this: - Upgrade all of the clients to have the new code - Change the table property to turn off the locking -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
