szehon-ho commented on code in PR #6451:
URL: https://github.com/apache/iceberg/pull/6451#discussion_r1063985982
##########
docs/configuration.md:
##########
@@ -160,14 +160,20 @@ Here are the catalog properties related to locking. They
are used by some catalo
The following properties from the Hadoop configuration are used by the Hive
Metastore connector.
-| Property | Default | Description
|
-| ------------------------------------- | ---------------- |
----------------------------------------------------------------------------------
|
-| iceberg.hive.client-pool-size | 5 | The size of the
Hive client pool when tracking tables in HMS |
-| iceberg.hive.lock-timeout-ms | 180000 (3 min) | Maximum time in
milliseconds to acquire a lock |
-| iceberg.hive.lock-check-min-wait-ms | 50 | Minimum time in
milliseconds to check back on the status of lock acquisition |
-| iceberg.hive.lock-check-max-wait-ms | 5000 | Maximum time in
milliseconds to check back on the status of lock acquisition |
-
-Note: `iceberg.hive.lock-check-max-wait-ms` should be less than the
[transaction
timeout](https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.txn.timeout)
+| Property | Default | Description
|
+|-------------------------------------------|-----------------|------------------------------------------------------------------------------|
+| iceberg.hive.client-pool-size | 5 | The size of
the Hive client pool when tracking tables in HMS |
+| iceberg.hive.lock-timeout-ms | 180000 (3 min) | Maximum time
in milliseconds to acquire a lock |
Review Comment:
I see, just saw its from unrelated changelist. But it still doesnt fit very
well with the two phases (creation/acquisition) you put in the first part of
the doc, should we just say simply:
The HMS table locking is a 2-step process:
1. Lock create: Create lock in HMS and queue for acquisition
2. Lock check: Check if lock successfully acquired
This way, they are more relevant to the property name. Also wasnt sure
about putting the details of 'should wait until every previously created Lock
is released', is this the Hive internals? As I feel it will just confuse more.
##########
docs/configuration.md:
##########
@@ -159,15 +159,24 @@ Here are the catalog properties related to locking. They
are used by some catalo
## Hadoop configuration
The following properties from the Hadoop configuration are used by the Hive
Metastore connector.
-
-| Property | Default | Description
|
-| ------------------------------------- | ---------------- |
----------------------------------------------------------------------------------
|
-| iceberg.hive.client-pool-size | 5 | The size of the
Hive client pool when tracking tables in HMS |
-| iceberg.hive.lock-timeout-ms | 180000 (3 min) | Maximum time in
milliseconds to acquire a lock |
-| iceberg.hive.lock-check-min-wait-ms | 50 | Minimum time in
milliseconds to check back on the status of lock acquisition |
-| iceberg.hive.lock-check-max-wait-ms | 5000 | Maximum time in
milliseconds to check back on the status of lock acquisition |
-
-Note: `iceberg.hive.lock-check-max-wait-ms` should be less than the
[transaction
timeout](https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.txn.timeout)
+The HMS table locking is a 2-step process:
+- Lock creation - the Lock itself is created and queued
+- Lock acquisition - the queued Lock should wait until every previously
created Lock is released
+
+| Property | Default | Description
|
+|-------------------------------------------|-----------------|------------------------------------------------------------------------------|
+| iceberg.hive.client-pool-size | 5 | The size of
the Hive client pool when tracking tables in HMS |
+| iceberg.hive.lock-creation-timeout-ms | 180000 (3 min) | Maximum time
in milliseconds to create a lock in the HMS |
+| iceberg.hive.lock-creation-min-wait-ms | 50 | Minimum time
in milliseconds to check the lock creation in the HMS |
Review Comment:
How about, max/min time between retries of creating lock. (Using the word
'check' here confuses with the actual check.x properties)
##########
docs/configuration.md:
##########
@@ -159,15 +159,24 @@ Here are the catalog properties related to locking. They
are used by some catalo
## Hadoop configuration
The following properties from the Hadoop configuration are used by the Hive
Metastore connector.
-
-| Property | Default | Description
|
-| ------------------------------------- | ---------------- |
----------------------------------------------------------------------------------
|
-| iceberg.hive.client-pool-size | 5 | The size of the
Hive client pool when tracking tables in HMS |
-| iceberg.hive.lock-timeout-ms | 180000 (3 min) | Maximum time in
milliseconds to acquire a lock |
-| iceberg.hive.lock-check-min-wait-ms | 50 | Minimum time in
milliseconds to check back on the status of lock acquisition |
-| iceberg.hive.lock-check-max-wait-ms | 5000 | Maximum time in
milliseconds to check back on the status of lock acquisition |
-
-Note: `iceberg.hive.lock-check-max-wait-ms` should be less than the
[transaction
timeout](https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.txn.timeout)
+The HMS table locking is a 2-step process:
+- Lock creation - the Lock itself is created and queued
+- Lock acquisition - the queued Lock should wait until every previously
created Lock is released
+
+| Property | Default | Description
|
+|-------------------------------------------|-----------------|------------------------------------------------------------------------------|
+| iceberg.hive.client-pool-size | 5 | The size of
the Hive client pool when tracking tables in HMS |
+| iceberg.hive.lock-creation-timeout-ms | 180000 (3 min) | Maximum time
in milliseconds to create a lock in the HMS |
+| iceberg.hive.lock-creation-min-wait-ms | 50 | Minimum time
in milliseconds to check the lock creation in the HMS |
+| iceberg.hive.lock-creation-max-wait-ms | 5000 | Maximum time
in milliseconds to check the lock creation in the HMS |
+| iceberg.hive.lock-timeout-ms | 180000 (3 min) | Maximum time
in milliseconds to acquire a lock |
+| iceberg.hive.lock-check-min-wait-ms | 50 | Minimum time
in milliseconds to check back on the status of lock acquisition |
+| iceberg.hive.lock-check-max-wait-ms | 5000 | Maximum time
in milliseconds to check back on the status of lock acquisition |
Review Comment:
Same , how about, min/max time to retry checking acquisition status of lock
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
