Sbaia opened a new pull request, #59893:
URL: https://github.com/apache/doris/pull/59893
## Proposed changes
Currently, Doris requires explicit AWS credentials for both:
1. `iceberg.catalog.type = s3tables` (S3 Tables native catalog)
2. `iceberg.catalog.type = rest` with SigV4 authentication
This prevents users from leveraging AWS's default credentials chain, which
supports:
- Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`)
- EC2 instance profile
- EKS IAM Roles for Service Accounts (IRSA)
- ECS container credentials
### Root Cause
The Java Frontend was passing empty strings to the AWS SDK instead of
letting Iceberg use the default credentials chain:
- `CustomAwsCredentialsProvider` only supported explicit credentials
- `IcebergRestProperties` always set `rest.access-key-id` and
`rest.secret-access-key` even when empty
### Solution
**For S3 Tables catalog:**
Modified `CustomAwsCredentialsProvider` to fall back to AWS SDK's
`DefaultCredentialsProvider` when explicit credentials are not provided.
**For REST catalog:**
Modified `IcebergRestProperties` to only set `rest.access-key-id` and
`rest.secret-access-key` when explicitly provided, allowing Iceberg to use the
default credentials chain.
### Changes
- `fe/fe-core/.../iceberg/s3tables/CustomAwsCredentialsProvider.java`
- Added fallback to `DefaultCredentialsProvider`
- Added support for session tokens (temporary credentials)
- `fe/fe-core/.../property/metastore/IcebergRestProperties.java`
- Only set credentials properties when values are provided
- Updated validation to not require explicit credentials for Glue
### Usage
**S3 Tables (without explicit credentials):**
```sql
CREATE CATALOG lakehouse PROPERTIES (
'type'= 'iceberg',
'iceberg.catalog.type'= 's3tables',
'warehouse'= 'arn:aws:s3tables:eu-west-1:123456789:bucket/my-bucket',
's3.region'= 'eu-west-1'
);
```
**REST catalog with SigV4 (without explicit credentials):**
```sql
CREATE CATALOG lakehouse PROPERTIES (
'type'= 'iceberg',
'iceberg.catalog.type'= 'rest',
'iceberg.rest.uri'= 'https://s3tables.eu-west-1.amazonaws.com/iceberg',
'iceberg.rest.sigv4-enabled'= 'true',
'iceberg.rest.signing-name'= 's3tables',
'iceberg.rest.signing-region'= 'eu-west-1',
's3.region'= 'eu-west-1'
);
```
## Compatibility
- **No breaking changes**: Explicit credentials continue to work exactly as
before
- **Additive feature**: Users can now omit credentials to use IAM roles
## Further comments
The C++ Backend already supports IAM roles for S3 file access. This PR only
addresses the Java Frontend catalog initialization issue.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
