github-actions[bot] commented on code in PR #68540:
URL: https://github.com/apache/doris/pull/68540#discussion_r4356827954
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergCatalogFactory.java:
##########
@@ -450,6 +453,36 @@ public static Map<String, String>
buildCatalogProperties(IcebergCatalogPropertie
// s3tables: bespoke instantiation. Preserve the skeleton's
base+impl routing.
break;
}
+ chosenS3.ifPresent(storage ->
storage.toBackendProperties().ifPresent(backend -> {
+ Optional<GcsAuth> auth = GcsAuthResolver.resolve(backend.toMap());
+ if (auth.filter(GcsAuth::isAnonymous).isPresent()) {
+ opts.put(AwsClientProperties.CLIENT_CREDENTIALS_PROVIDER,
+
"software.amazon.awssdk.auth.credentials.AnonymousCredentialsProvider");
+ }
+ auth.flatMap(GcsAuth::getNativeCredential).ifPresent(credential ->
{
+ // HadoopCatalog resolves its namespace filesystem
independently of S3FileIO.
+ // Native GCS credentials configure fs.gs.*, so compatibility
warehouse schemes
+ // must select that same filesystem before HadoopCatalog
initializes.
+ String warehouse =
opts.get(CatalogProperties.WAREHOUSE_LOCATION);
+ if (IcebergCatalogProperties.TYPE_HADOOP.equals(flavor) &&
warehouse != null) {
+ if (warehouse.regionMatches(true, 0, "s3://", 0, 5)) {
+ opts.put(CatalogProperties.WAREHOUSE_LOCATION, "gs://"
+ warehouse.substring(5));
+ } else if (warehouse.regionMatches(true, 0, "s3a://", 0,
6)) {
+ opts.put(CatalogProperties.WAREHOUSE_LOCATION, "gs://"
+ warehouse.substring(6));
+ }
+ }
+ putS3FileIODialect(opts, storage);
Review Comment:
[P2] Preserve Glue's AWS region when enabling native GCP storage
For an Iceberg Glue catalog with `provider=GCP`,
`gs.credential_provider_type=COMPUTE_ENGINE` (or `DEFAULT`), valid
`glue.access_key`/`glue.secret_key`, and `glue.region=us-east-1`,
`appendGlueProperties` first sets `client.region` correctly. This call then
overwrites it with GCS's constant `us-east1`. Iceberg's [Glue client
factory](https://github.com/apache/iceberg/blob/apache-iceberg-1.11.0/aws/src/main/java/org/apache/iceberg/aws/AwsClientFactories.java#L134-L141)
uses that shared region, so the first namespace/table lookup targets an
invalid AWS region or signs an explicit Glue endpoint with the wrong region.
Preserve Glue's control-plane region when installing GCS FileIO settings, and
cover the Glue client's region or a signed request. This is separate from the
existing missing-Hadoop-configuration report: that configuration is now passed
correctly.
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergCatalogFactory.java:
##########
@@ -450,6 +453,36 @@ public static Map<String, String>
buildCatalogProperties(IcebergCatalogPropertie
// s3tables: bespoke instantiation. Preserve the skeleton's
base+impl routing.
break;
}
+ chosenS3.ifPresent(storage ->
storage.toBackendProperties().ifPresent(backend -> {
+ Optional<GcsAuth> auth = GcsAuthResolver.resolve(backend.toMap());
+ if (auth.filter(GcsAuth::isAnonymous).isPresent()) {
+ opts.put(AwsClientProperties.CLIENT_CREDENTIALS_PROVIDER,
Review Comment:
[P2] Keep anonymous GCS credentials off the Glue metadata client
With `iceberg.catalog.type=glue`, `provider=GCP`,
`gs.credential_provider_type=ANONYMOUS`, and valid
`glue.access_key`/`glue.secret_key`, `appendGlueProperties` has already
installed the authenticated AWS Glue provider. This assignment replaces its
shared `client.credentials-provider` with `AnonymousCredentialsProvider`.
Iceberg's [Glue factory applies that
provider](https://github.com/apache/iceberg/blob/apache-iceberg-1.11.0/aws/src/main/java/org/apache/iceberg/aws/AwsClientFactories.java#L134-L141),
so Glue metadata requests become unsigned and fail before public GCS objects
can be read. Scope anonymous credentials to the object FileIO client while
preserving Glue authentication, and cover this combination. This is independent
of the native-region finding: anonymous mode never enters that native branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
