laskoviymishka commented on code in PR #69031:
URL: https://github.com/apache/doris/pull/69031#discussion_r4263169983
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergCatalogFactory.java:
##########
@@ -154,9 +156,22 @@ public static Map<String, String>
buildBaseCatalogProperties(Map<String, String>
opts.put(CatalogProperties.WAREHOUSE_LOCATION, warehouse);
}
appendManifestCacheProperties(props, opts, metaCacheWeightLimited);
+ applyDefaultAwsHttpClient(opts);
return opts;
}
+ /**
+ * Defaults iceberg-aws's HTTP client to {@code urlconnection} unless
{@code opts} sets one. iceberg-aws's default
+ * "apache" client type is Apache5HttpClient since iceberg 1.12, which
needs an AWS SDK newer than the one Doris
+ * pins (2.29.x, see #58566), so every S3FileIO / Glue / STS client
iceberg builds from a map without this key fails
+ * with NoClassDefFoundError; url-connection-client ships with that SDK.
Every map that ends up building an
+ * iceberg AWS client must go through here. The key travels with
S3FileIO's properties, so the BE metadata
+ * scanner, which rebuilds the FileIO from them, uses the same client.
+ */
+ static void applyDefaultAwsHttpClient(Map<String, String> opts) {
+ opts.putIfAbsent(HttpClientProperties.CLIENT_TYPE,
HttpClientProperties.CLIENT_TYPE_URLCONNECTION);
Review Comment:
fixed in 8cf8ab6ec0
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergManifestCache.java:
##########
@@ -285,7 +285,7 @@ private static ManifestCacheValue
loadManifestCacheValue(ManifestFile manifest,
private static List<DataFile> loadDataFiles(ManifestFile manifest, Table
table) throws IOException {
List<DataFile> dataFiles = new ArrayList<>();
// .copy() is mandatory: the ManifestReader iterator reuses the same
object across iterations.
- try (ManifestReader<DataFile> reader = ManifestFiles.read(manifest,
table.io())) {
+ try (ManifestReader<DataFile> reader = ManifestFiles.read(manifest,
table.io(), table.specs())) {
Review Comment:
fixed in 98679abb2b
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
