gh-yzou commented on code in PR #3826:
URL: https://github.com/apache/polaris/pull/3826#discussion_r2835213721
##########
spec/polaris-catalog-apis/generic-tables-api.yaml:
##########
@@ -256,6 +262,55 @@ components:
items:
$ref:
'../iceberg-rest-catalog-open-api.yaml#/components/schemas/TableIdentifier'
+
+ StorageAccessCredential:
+ type: object
+ required:
+ - prefix
+ - config
+ properties:
+ prefix:
+ type: string
+ description: Indicates a storage location prefix where the
credential is relevant. Clients should choose the most
+ specific prefix (by selecting the longest prefix) if several
credentials of the same type are available.
+ config:
+ type: object
+ description: |
+ Credential configurations for AWS S3, GCP GCS, and Azure ADLS are
supported. The following outlines
+ the currently supported configuration options:
+
+ ## AWS Configurations
+
+ The following configurations should be respected when working with
tables stored in AWS S3
Review Comment:
that is good point, I updated to S3 configuration to be more precise. Also
updated other ones to contain the storage type
##########
spec/polaris-catalog-apis/generic-tables-api.yaml:
##########
@@ -256,6 +262,55 @@ components:
items:
$ref:
'../iceberg-rest-catalog-open-api.yaml#/components/schemas/TableIdentifier'
+
+ StorageAccessCredential:
+ type: object
+ required:
+ - prefix
+ - config
+ properties:
+ prefix:
+ type: string
+ description: Indicates a storage location prefix where the
credential is relevant. Clients should choose the most
+ specific prefix (by selecting the longest prefix) if several
credentials of the same type are available.
+ config:
+ type: object
+ description: |
+ Credential configurations for AWS S3, GCP GCS, and Azure ADLS are
supported. The following outlines
+ the currently supported configuration options:
+
+ ## AWS Configurations
+
+ The following configurations should be respected when working with
tables stored in AWS S3
+ - `s3.access-key-id`: id for credentials that provide access to
the data in S3
+ - `s3.secret-access-key`: secret for credentials that provide
access to data in S3
Review Comment:
I don’t think the cloud-specific configuration fields are good candidates to
be extracted into dedicated fields. Different cloud providers rely on different
credential structures, and they regularly introduce new authentication methods
or storage access patterns. Keeping these settings in a general configuration
map makes it much easier to support new configurations without requiring schema
changes.
##########
spec/polaris-catalog-apis/generic-tables-api.yaml:
##########
@@ -256,6 +262,55 @@ components:
items:
$ref:
'../iceberg-rest-catalog-open-api.yaml#/components/schemas/TableIdentifier'
+
+ StorageAccessCredential:
+ type: object
+ required:
+ - prefix
+ - config
+ properties:
+ prefix:
+ type: string
+ description: Indicates a storage location prefix where the
credential is relevant. Clients should choose the most
+ specific prefix (by selecting the longest prefix) if several
credentials of the same type are available.
+ config:
+ type: object
+ description: |
+ Credential configurations for AWS S3, GCP GCS, and Azure ADLS are
supported. The following outlines
+ the currently supported configuration options:
+
+ ## AWS Configurations
+
+ The following configurations should be respected when working with
tables stored in AWS S3
+ - `s3.access-key-id`: id for credentials that provide access to
the data in S3
+ - `s3.secret-access-key`: secret for credentials that provide
access to data in S3
+ - `s3.session-token`: if present, this value should be used for
as the session token
+ - `s3.session-token-expires-at-ms`: the time the aws session
token expires, in milliseconds
+ Extra properties:
+ - `s3.endpoint`: the S3 endpoint to use for requests
Review Comment:
That is a good point, it seems like those fields are really needed by
iceberg FILE IO today. I think for generic table, we don't necessary need this.
I can take if off from generic for now, and if we really need those access
specific configuration, we can have a separate discussion to see if we need to
evolve a separate structure for those information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
