cyriltovena opened a new issue, #65432: URL: https://github.com/apache/doris/issues/65432
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description Today, an S3-type storage vault with `provider = GCP` can only authenticate with static HMAC keys (`s3.access_key` / `s3.secret_key`). The other credential mechanisms available to vaults (`s3.role_arn`, `s3.credentials_provider_type`) are AWS-specific, so on GCP there is currently no way to run a compute-storage decoupled cluster without minting and distributing long-lived HMAC key pairs. For deployments on GKE this is a real gap: the platform-native (and increasingly organization-mandated) way to authorize workloads is Workload Identity / Application Default Credentials, where pods obtain short-lived OAuth2 tokens from the metadata server and no static secret exists at all. AWS deployments already enjoy the equivalent via `WEB_IDENTITY`/instance-profile credentials (thanks to the great work in #62788 and #64766) — it would be lovely for GCP users to have parity. The GCS XML API (which Doris already uses for GCP vaults through the S3-compatible endpoint) accepts `Authorization: Bearer <oauth2-token>` as a first-class alternative to HMAC signatures, so this doesn't require a new SDK dependency — "only" a way to resolve ADC tokens and attach them to vault I/O requests across FE/BE/meta-service. ### Use case `CREATE STORAGE VAULT` on a GKE cluster with Workload Identity, with no static credential anywhere: ```sql CREATE STORAGE VAULT IF NOT EXISTS gcs_vault PROPERTIES ( "type" = "S3", "s3.endpoint" = "storage.googleapis.com", "s3.region" = "us-central1", "s3.bucket" = "my-bucket", "s3.root.path" = "doris", "provider" = "GCP", "s3.credentials_provider_type" = "gcp_adc" ); ``` Benefits: no key minting/distribution/rotation, tokens are short-lived, and access control lives entirely in IAM bindings on the bucket — consistent with the keyless direction the vault credential work has been taking on AWS. ### Related issues #62987 added Google authentication for Iceberg REST catalogs (FE-side); this proposes the analogous capability for storage vaults (data path). I have a working implementation following the pattern of #64766 (new `CredProviderTypePB` value, shared token provider built on the AWS SDK's HTTP/JSON utilities, bearer header on the S3 client with anonymous credentials) and will open a PR shortly — very happy to adjust the approach based on maintainer feedback. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
