jerryshao commented on code in PR #10789:
URL: https://github.com/apache/gravitino/pull/10789#discussion_r3145887151
##########
catalogs/catalog-hive/build.gradle.kts:
##########
@@ -49,12 +49,17 @@ dependencies {
implementation(libs.commons.io)
implementation(libs.commons.lang3)
implementation(libs.guava)
- implementation(libs.hadoop2.auth) {
+ implementation(libs.hadoop3.auth) {
exclude("*")
}
- implementation(libs.hadoop2.common) {
+ implementation(libs.hadoop3.common) {
exclude("*")
}
+ // Hadoop 3.x runtime requirements (stripped by exclude("*") above)
+ implementation(libs.hadoop3.shaded.guava)
+ implementation(libs.hadoop3.shaded.protobuf)
+ implementation(libs.commons.configuration2)
+ implementation(libs.re2j)
Review Comment:
Excluding everything and adding back dependencies one by one is very tricky.
Simply, because of bumping the major version will introduce lots of dependency
changes, the previous way that works with Hadoop 2 may not apply to Hadoop 3,
unless you check all the dependencies of Hadoop 3 and make sure that you pull
in the ones you wanted exactly.
##########
authorizations/authorization-chain/build.gradle.kts:
##########
@@ -104,6 +104,7 @@ dependencies {
exclude("javax.servlet", "servlet-api")
exclude("io.netty")
}
+ testImplementation(libs.hadoop3.hdfs.client)
Review Comment:
Why don't we need this before?
##########
clients/filesystem-hadoop3/src/test/java/org/apache/gravitino/filesystem/hadoop/integration/test/GravitinoVirtualFileSystemABSCredentialIT.java:
##########
@@ -111,7 +111,7 @@ public void startUp() throws Exception {
conf.set(AzureProperties.GRAVITINO_AZURE_STORAGE_ACCOUNT_KEY,
ABS_ACCOUNT_KEY);
conf.set("fs.abfss.impl",
"org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem");
- conf.set("fs.gravitino.client.useCloudStoreCredential", "true");
+ conf.set("fs.gravitino.enableCredentialVending", "true");
Review Comment:
@yuqi1129 @mchades can you confirm this change? Is it right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
