Shubham Sharma created NIFI-15734:
-------------------------------------
Summary: GCP PubSub/BigQuery processors fail with
ACCESS_TOKEN_SCOPE_INSUFFICIENT when using Workload Identity Federation with
Service Account Impersonation
Key: NIFI-15734
URL: https://issues.apache.org/jira/browse/NIFI-15734
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 2.8.0, 2.3.0
Reporter: Shubham Sharma
h3. Problem
After NIFI-14277 was merged, ConsumeGCPubSub, PublishGCPubSub, and BigQuery
processors fail with {{ACCESS_TOKEN_SCOPE_INSUFFICIENT}} error when using GCP
Workload Identity Federation with Service Account Impersonation.
{code:java}
com.google.api.gax.rpc.UnauthenticatedException:
io.grpc.StatusRuntimeException: UNAUTHENTICATED: Failed computing credential
metadata
Caused by: com.google.api.client.http.HttpResponseException: 403 Forbidden
POST
https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/[SA_EMAIL]:generateAccessToken
{
"error": {
"code": 403,
"message": "Request had insufficient authentication scopes.",
"reason": "ACCESS_TOKEN_SCOPE_INSUFFICIENT",
"domain": "googleapis.com",
"metadata": {
"service": "iamcredentials.googleapis.com",
"method": "google.iam.credentials.v1.IAMCredentials.GenerateAccessToken"
}
}
}{code}
h3. Root Cause
NIFI-14277 introduced explicit OAuth scopes for GCP processors to support
private endpoints. The implementation uses narrow scopes:
* {{GOOGLE_CLOUD_PUBSUB_SCOPE}}
({{{}https://www.googleapis.com/auth/pubsub{}}}) for PubSub processors
* {{GOOGLE_CLOUD_BIGQUERY_SCOPE}}
({{{}https://www.googleapis.com/auth/bigquery{}}}) for BigQuery processors
When using Workload Identity Federation with Service Account Impersonation, the
authentication flow is:
# OIDC token → STS token exchange → Service Account Impersonation → Final
access token
The impersonation step calls the IAM Credentials API
({{{}iamcredentials.googleapis.com/generateAccessToken{}}}), which requires
{{cloud-platform}} scope. The narrow {{pubsub}} or {{bigquery}} scopes are
insufficient to call this API, causing the authentication failure.
h3. Reproduction Steps
# Configure GCP Workload Identity Federation with Service Account Impersonation
# Create an external account credential configuration file pointing to an OIDC
token file
# Configure GCPCredentialsControllerService with the external account
credential file
# Configure ConsumeGCPubSub or PublishGCPubSub processor to use the credential
service
# Start the processor
# Observe the {{ACCESS_TOKEN_SCOPE_INSUFFICIENT}} error when credentials
refresh/impersonation occurs
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
