HonahX commented on code in PR #490:
URL: https://github.com/apache/iceberg-python/pull/490#discussion_r1510041483
##########
pyiceberg/catalog/glue.py:
##########
@@ -252,6 +259,22 @@ def _construct_database_input(database_name: str,
properties: Properties) -> Dat
return database_input
+def _register_glue_catalog_id_with_glue_client(glue: GlueClient,
glue_catalog_id: str) -> None:
+ """
+ Register the Glue Catalog ID (AWS Account ID) as a parameter on all Glue
client methods.
+
+ It's more ergonomic to do this than to pass the CatalogId as a parameter
to every client call since it's an optional
+ parameter and boto3 does not support 'None' values for missing parameters.
+ """
+ event_system = glue.meta.events
+
+ def add_glue_catalog_id(params, **kwargs) -> None: # type:
ignore[no-untyped-def]
Review Comment:
```suggestion
def add_glue_catalog_id(params: Dict[str, str], **kwargs: Any) -> None:
```
I think it is better to write the type explicitly when possible. WDYT?
##########
pyiceberg/catalog/glue.py:
##########
@@ -266,6 +289,10 @@ def __init__(self, name: str, **properties: Any):
)
self.glue: GlueClient = session.client("glue")
+ self.glue_catalog_id: Optional[str] = properties.get("glue.id")
+ if self.glue_catalog_id is not None:
+ _register_glue_catalog_id_with_glue_client(self.glue,
self.glue_catalog_id)
Review Comment:
```suggestion
if glue_catalog_id := properties.get(GLUE_ID):
_register_glue_catalog_id_with_glue_client(self.glue,
glue_catalog_id)
```
Since a valid glue ID cannot be an empty string, I think we could use walrus
operator here. Also, seems we do not need to store glue_catalog_id in the
class. The information can be retrieved from `self.properties`. WDYT?
##########
pyiceberg/catalog/glue.py:
##########
@@ -252,6 +259,22 @@ def _construct_database_input(database_name: str,
properties: Properties) -> Dat
return database_input
+def _register_glue_catalog_id_with_glue_client(glue: GlueClient,
glue_catalog_id: str) -> None:
+ """
+ Register the Glue Catalog ID (AWS Account ID) as a parameter on all Glue
client methods.
+
+ It's more ergonomic to do this than to pass the CatalogId as a parameter
to every client call since it's an optional
+ parameter and boto3 does not support 'None' values for missing parameters.
Review Comment:
Thanks for providing all these possibilities. I like the current approach
since it make things simple and clear
##########
pyiceberg/catalog/glue.py:
##########
@@ -93,6 +93,13 @@
if TYPE_CHECKING:
import pyarrow as pa
+
+# There is a unique Glue metastore in each AWS account and each AWS region. By
default, GlueCatalog chooses the Glue
+# metastore to use based on the user's default AWS client credential and
region setup. You can specify the Glue catalog
+# ID through glue.id catalog property to point to a Glue catalog in a
different AWS account. The Glue catalog ID is your
+# numeric AWS account ID.
+GLUE_ID = "glue.id"
Review Comment:
Shall we call it `GLUE_CATALOG_ID` since we use `glue_catalog_id` below?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
