HonahX commented on code in PR #701: URL: https://github.com/apache/iceberg-python/pull/701#discussion_r1591768312
########## pyiceberg/catalog/hive.py: ########## @@ -356,11 +371,46 @@ def _create_lock_request(self, database_name: str, table_name: str) -> LockReque return lock_request - def _commit_table(self, table_request: CommitTableRequest) -> CommitTableResponse: + def _wait_for_lock( + self, database_name: str, table_name: str, lockid: int, open_client: Client, properties: Properties + ) -> LockResponse: + acquire_lock_timeout = ( + properties.get(TableProperties.HIVE_ACQUIRE_LOCK_TIMEOUT_MS, TableProperties.HIVE_ACQUIRE_LOCK_TIMEOUT_MS_DEFAULT) + ) / 1000.0 + lock_check_min_wait_time = ( + properties.get(TableProperties.HIVE_LOCK_CHECK_MIN_WAIT_MS, TableProperties.HIVE_LOCK_CHECK_MIN_WAIT_MS_DEFAULT) + ) / 1000.0 + lock_check_max_wait_time = ( + properties.get(TableProperties.HIVE_LOCK_CHECK_MAX_WAIT_MS, TableProperties.HIVE_LOCK_CHECK_MAX_WAIT_MS_DEFAULT) + ) / 1000.0 Review Comment: We could use [`PropertyUtil.property_as_int`](https://github.com/apache/iceberg-python/blob/d872245f143b919b25fda90a7b7c7fb0729a402a/pyiceberg/table/__init__.py#L243-L258) to get the values. This utl methods throws error message when failing to parse the property. BTW: You would properly need `#type: ignore` at the end of each `PropertyUti.property_as_int` to pass the linter. ########## pyiceberg/catalog/hive.py: ########## @@ -111,6 +122,13 @@ HIVE2_COMPATIBLE = "hive.hive2-compatible" HIVE2_COMPATIBLE_DEFAULT = False +DEFAULT_LOCK_CHECK_MIN_WAIT_TIME = 2 +DEFAULT_LOCK_CHECK_MAX_WAIT_TIME = 30 +DEFAULT_LOCK_CHECK_RETRIES = 5 +DEFAULT_LOCK_CHECK_MULTIPLIER = 2 Review Comment: In java, they are in the [`HadoopConfiguration`](https://iceberg.apache.org/docs/nightly/configuration/#hadoop-configuration). The hadoop configuration for a hive catalog is set at the catalog-level: https://github.com/apache/iceberg/blob/817a5e1be1616af77329965ac3742c14ca3ae116/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java#L630 [Metastore parse these properties from the hadoop configuration](https://github.com/apache/iceberg/blob/e10453a10d3f9237032ba7e649808347c699f6a2/hive-metastore/src/main/java/org/apache/iceberg/hive/MetastoreLock.java#L110-L125) I call them "CatalogProperties" since pyiceberg does not have hadoop configuration. Sorry if that creates any confusion. The only hive-lock-related config in `TableProperties` is [`engine.hive.lock-enabled`](https://github.com/apache/iceberg/blob/e7999a194dc0f32fba8fb515c9108764ed6ba6b5/core/src/main/java/org/apache/iceberg/TableProperties.java#L319) which can enable/disable lock for a single table. But that is for https://issues.apache.org/jira/browse/HIVE-26882, see the "warn" section in the end of https://iceberg.apache.org/docs/nightly/configuration/#hadoop-configuration. I don't think we need that in pyiceberg now. WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org