ByteYue commented on code in PR #22048:
URL: https://github.com/apache/doris/pull/22048#discussion_r1302848096
##########
be/src/agent/task_worker_pool.cpp:
##########
@@ -1176,6 +1179,22 @@ void
TaskWorkerPool::_push_storage_policy_worker_thread_callback() {
.tag("s3_conf", s3_conf.to_string());
put_storage_resource(resource.id, {std::move(fs),
resource.version});
}
+ } else if (resource.__isset.hdfs_storage_param) {
+ Status st;
+ std::shared_ptr<io::HdfsFileSystem> fs;
+ if (existed_resource.fs == nullptr) {
+ st =
io::HdfsFileSystem::create(resource.hdfs_storage_param, "", nullptr, &fs);
+ } else {
+ fs =
std::static_pointer_cast<io::HdfsFileSystem>(existed_resource.fs);
+ }
+ if (!st.ok()) {
+ LOG(WARNING) << "update hdfs resource failed: " << st;
+ } else {
+ LOG_INFO("successfully update hdfs resource")
+ .tag("resource_id", resource.id)
+ .tag("resource_name", resource.name);
Review Comment:
Are there any chance that we attach the hdfs resource message along with
prrvious log?
##########
fe/fe-core/pom.xml:
##########
Review Comment:
You should consider leaving this file unchanged using steps like the
following:
1. `git rm --cached fe/fe-core/pom.xml`
2. `git commit --amend`
##########
regression-test/conf/regression-conf.groovy:
##########
@@ -65,8 +65,8 @@ excludeDirectories = "segcompaction_p2"
customConf1 = "test_custom_conf_value"
// for test csv with header
-enableHdfs=false // set to true if hdfs is ready
-hdfsFs = "hdfs://127.0.0.1:9000"
+enableHdfs=true // set to true if hdfs is ready
Review Comment:
Leave this file unchanged
##########
be/src/agent/task_worker_pool.cpp:
##########
@@ -1142,6 +1144,7 @@ void
TaskWorkerPool::_push_storage_policy_worker_thread_callback() {
// refresh resource
for (auto& resource : push_storage_policy_req.resource) {
auto existed_resource = get_storage_resource(resource.id);
+ VLOG_DEBUG << "be successfully receive a storage policy task";
Review Comment:
I think there is no need for this log, since we would log no matter we
successfully update the resource or not.
##########
be/src/olap/storage_policy.cpp:
##########
@@ -48,7 +48,7 @@ Status get_remote_file_system(int64_t storage_policy_id,
return Status::InternalError("could not find resource, resouce_id={}",
storage_policy->resource_id);
}
- DCHECK(atol((*fs)->id().c_str()) == storage_policy->resource_id);
+ // DCHECK(atol((*fs)->id().c_str()) == storage_policy->resource_id);
Review Comment:
We should leave this DCHECK as it is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
