(doris) branch branch-2.0-var updated: [pick](branch-2.0) pick prs from branch-2.0 (#28176)

eldenmoon Fri, 08 Dec 2023 02:04:36 -0800

This is an automated email from the ASF dual-hosted git repository.

eldenmoon pushed a commit to branch branch-2.0-var
in repository https://gitbox.apache.org/repos/asf/doris.git



The following commit(s) were added to refs/heads/branch-2.0-var by this push:
     new 9d5f3141362 [pick](branch-2.0) pick prs from branch-2.0 (#28176)
9d5f3141362 is described below

commit 9d5f31413627e2f3ba9f629395e36fa537897dd2
Author: lihangyu <[email protected]>
AuthorDate: Fri Dec 8 18:04:21 2023 +0800

    [pick](branch-2.0) pick prs from branch-2.0 (#28176)
    
    * [fix](hudi-catalog) fix hudi catalog code (#27966)
    
    backport: #27963
    
    * [fix](Nereids): fix datetime fold-constant-be in Branch-2.0 (#27938)
    
    fix fold-constant-be datetimev2 will lose the scale of datetime.
    
    * [bug](function) fix compound expr coredump problem #27988 (#27989)
    
    * [branch-2.0](cache) Fix partition cache support DATEV2  #27978 (#27990)
    
    * [branch-2.0] delete auto partition docs (#28001)
    
    * [fix](stats) Drop stats or update updated rows after truncate table 
(#27956)
    
    1. Also clear follower's stats cache when doing drop stats.
    2. Drop stats when truncate a table.
    
    backport https://github.com/apache/doris/pull/27931
    
    * [minor](stats) truncate min/max if too long #27955 (#27997)
    
    * [agg](profile) fix incorrect profile (#28004) (#28035)
    
    * [fix](remote-scanner-pool) missing _remote_thread_pool_max_size value 
#28057 (#28053)
    
    bp #28057
    
    * [enhance](partitionid) check partition id to avoid unexpected behavior 
(#28045)
    
    * [opt](resource-tag) root and admin user can use any resource tag by 
default #28088 (#28080)
    
    bp #28088
    
    * [compile](compile option) add deflate to X86 only (#28089)
    
    * change version to 2.0.3-rc05-var05
    
    * [fix](Nereids): Preserve `""` in single quote strings and `''` in double 
quote strings. (#27959)
    
    ---------
    
    Co-authored-by: Tiewei Fang <[email protected]>
    Co-authored-by: jakevin <[email protected]>
    Co-authored-by: HappenLee <[email protected]>
    Co-authored-by: Xinyi Zou <[email protected]>
    Co-authored-by: zclllyybb <[email protected]>
    Co-authored-by: Jibing-Li <[email protected]>
    Co-authored-by: AKIRA <[email protected]>
    Co-authored-by: TengJianPing <[email protected]>
    Co-authored-by: Mingyu Chen <[email protected]>
    Co-authored-by: Yongqiang YANG 
<[email protected]>
    Co-authored-by: shuke <[email protected]>
    Co-authored-by: 谢健 <[email protected]>
---
 be/CMakeLists.txt                                  |   8 +-
 be/src/olap/data_dir.cpp                           |  17 +-
 be/src/olap/rowset/rowset_meta_manager.cpp         |   6 +
 be/src/olap/tablet.cpp                             |   5 +
 be/src/olap/tablet_meta_manager.cpp                |   6 +
 be/src/olap/txn_manager.cpp                        |   7 +-
 be/src/vec/exec/distinct_vaggregation_node.cpp     |   1 -
 be/src/vec/exec/scan/scanner_scheduler.cpp         |   7 +-
 be/src/vec/exec/vaggregation_node.cpp              |  13 +-
 be/src/vec/exec/vaggregation_node.h                |   2 -
 be/src/vec/exprs/vcompound_pred.h                  |   2 +-
 be/src/vec/exprs/vectorized_agg_fn.cpp             |   6 -
 be/src/vec/exprs/vectorized_agg_fn.h               |   5 +-
 be/test/olap/delete_handler_test.cpp               |   2 +
 be/test/olap/delta_writer_test.cpp                 |   2 +
 .../olap/engine_storage_migration_task_test.cpp    |   1 +
 be/test/olap/remote_rowset_gc_test.cpp             |   1 +
 be/test/olap/tablet_cooldown_test.cpp              |   1 +
 be/test/olap/tablet_mgr_test.cpp                   |   3 +
 docs/en/docs/admin-manual/multi-tenant.md          |   4 +-
 docs/en/docs/advanced/partition/auto-partition.md  | 190 ---------------------
 docs/sidebars.json                                 |   3 +-
 docs/zh-CN/docs/admin-manual/multi-tenant.md       |   4 +-
 .../docs/advanced/partition/auto-partition.md      | 190 ---------------------
 .../apache/doris/datasource/InternalCatalog.java   |  11 +-
 .../mysql/privilege/CommonUserProperties.java      |   4 -
 .../doris/mysql/privilege/UserPropertyMgr.java     |  11 +-
 .../doris/nereids/parser/LogicalPlanBuilder.java   |   8 +-
 .../doris/planner/external/hudi/HudiScanNode.java  |   3 +-
 .../org/apache/doris/qe/cache/PartitionRange.java  |   1 +
 .../apache/doris/service/FrontendServiceImpl.java  |   8 +
 .../apache/doris/statistics/AnalysisManager.java   |   6 +-
 .../doris/statistics/StatisticsAutoCollector.java  |  30 ----
 .../apache/doris/statistics/StatisticsCache.java   |  30 ++++
 .../doris/statistics/StatisticsRepository.java     |   3 +
 .../doris/statistics/util/StatisticsUtil.java      |   3 +-
 .../org/apache/doris/catalog/UserPropertyTest.java |   3 +-
 .../apache/doris/planner/ResourceTagQueryTest.java |   2 +-
 .../org/apache/doris/statistics/CacheTest.java     |   3 +-
 .../statistics/StatisticsAutoCollectorTest.java    |  74 --------
 .../doris/statistics/util/StatisticsUtilTest.java  |  14 +-
 gensrc/script/gen_build_version.sh                 |   2 +-
 gensrc/thrift/FrontendService.thrift               |   6 +
 .../nereids_syntax_p0/one_row_relation.groovy      |  10 +-
 .../suites/statistics/analyze_stats.groovy         |  33 +++-
 45 files changed, 199 insertions(+), 552 deletions(-)

diff --git a/be/CMakeLists.txt b/be/CMakeLists.txt
index 58e059ca6ec..116c4fd96e0 100644
--- a/be/CMakeLists.txt
+++ b/be/CMakeLists.txt
@@ -559,9 +559,15 @@ set(COMMON_THIRDPARTY
     xml2
     lzma
     simdjson
-    deflate
 )
 
+if (ARCH_AMD64)
+    set(COMMON_THIRDPARTY
+        ${COMMON_THIRDPARTY}
+        deflate
+    )
+endif()
+
 if ((ARCH_AMD64 OR ARCH_AARCH64) AND OS_LINUX)
     add_library(hadoop_hdfs STATIC IMPORTED)
     set_target_properties(hadoop_hdfs PROPERTIES IMPORTED_LOCATION 
${THIRDPARTY_DIR}/lib/hadoop_hdfs/native/libhdfs.a)
diff --git a/be/src/olap/data_dir.cpp b/be/src/olap/data_dir.cpp
index 7f80e01e82b..6b333fc040a 100644
--- a/be/src/olap/data_dir.cpp
+++ b/be/src/olap/data_dir.cpp
@@ -506,15 +506,22 @@ Status DataDir::load() {
                     _meta, rowset_meta->partition_id(), rowset_meta->txn_id(),
                     rowset_meta->tablet_id(), 
rowset_meta->tablet_schema_hash(),
                     rowset_meta->tablet_uid(), rowset_meta->load_id(), rowset, 
true);
-            if (!commit_txn_status && 
!commit_txn_status.is<PUSH_TRANSACTION_ALREADY_EXIST>()) {
-                LOG(WARNING) << "failed to add committed rowset: " << 
rowset_meta->rowset_id()
-                             << " to tablet: " << rowset_meta->tablet_id()
-                             << " for txn: " << rowset_meta->txn_id();
-            } else {
+            if (commit_txn_status || 
commit_txn_status.is<PUSH_TRANSACTION_ALREADY_EXIST>()) {
                 LOG(INFO) << "successfully to add committed rowset: " << 
rowset_meta->rowset_id()
                           << " to tablet: " << rowset_meta->tablet_id()
                           << " schema hash: " << 
rowset_meta->tablet_schema_hash()
                           << " for txn: " << rowset_meta->txn_id();
+            } else if (commit_txn_status.is<ErrorCode::INTERNAL_ERROR>()) {
+                LOG(WARNING) << "failed to add committed rowset: " << 
rowset_meta->rowset_id()
+                             << " to tablet: " << rowset_meta->tablet_id()
+                             << " for txn: " << rowset_meta->txn_id()
+                             << " error: " << commit_txn_status;
+                return commit_txn_status;
+            } else {
+                LOG(WARNING) << "failed to add committed rowset: " << 
rowset_meta->rowset_id()
+                             << " to tablet: " << rowset_meta->tablet_id()
+                             << " for txn: " << rowset_meta->txn_id()
+                             << " error: " << commit_txn_status;
             }
         } else if (rowset_meta->rowset_state() == RowsetStatePB::VISIBLE &&
                    rowset_meta->tablet_uid() == tablet->tablet_uid()) {
diff --git a/be/src/olap/rowset/rowset_meta_manager.cpp 
b/be/src/olap/rowset/rowset_meta_manager.cpp
index f879aaf8995..16bda887c35 100644
--- a/be/src/olap/rowset/rowset_meta_manager.cpp
+++ b/be/src/olap/rowset/rowset_meta_manager.cpp
@@ -91,6 +91,12 @@ Status RowsetMetaManager::get_json_rowset_meta(OlapMeta* 
meta, TabletUid tablet_
 }
 Status RowsetMetaManager::save(OlapMeta* meta, TabletUid tablet_uid, const 
RowsetId& rowset_id,
                                const RowsetMetaPB& rowset_meta_pb, bool 
enable_binlog) {
+    if (rowset_meta_pb.partition_id() <= 0) {
+        LOG(WARNING) << "invalid partition id " << 
rowset_meta_pb.partition_id() << " tablet "
+                     << rowset_meta_pb.tablet_id();
+        return Status::InternalError("invaid partition id {} tablet {}",
+                                     rowset_meta_pb.partition_id(), 
rowset_meta_pb.tablet_id());
+    }
     if (enable_binlog) {
         return _save_with_binlog(meta, tablet_uid, rowset_id, rowset_meta_pb);
     } else {
diff --git a/be/src/olap/tablet.cpp b/be/src/olap/tablet.cpp
index f02af78bb92..2cbd5463397 100644
--- a/be/src/olap/tablet.cpp
+++ b/be/src/olap/tablet.cpp
@@ -243,6 +243,11 @@ void 
WriteCooldownMetaExecutors::WriteCooldownMetaExecutors::submit(TabletShared
 
 TabletSharedPtr Tablet::create_tablet_from_meta(TabletMetaSharedPtr 
tablet_meta,
                                                 DataDir* data_dir) {
+    if (tablet_meta->partition_id() <= 0) {
+        LOG(WARNING) << "invalid partition id " << tablet_meta->partition_id() 
<< ", tablet "
+                     << tablet_meta->tablet_id();
+        return nullptr;
+    }
     return std::make_shared<Tablet>(tablet_meta, data_dir);
 }
 
diff --git a/be/src/olap/tablet_meta_manager.cpp 
b/be/src/olap/tablet_meta_manager.cpp
index 20e5747e2f0..20f3872a03e 100644
--- a/be/src/olap/tablet_meta_manager.cpp
+++ b/be/src/olap/tablet_meta_manager.cpp
@@ -92,6 +92,12 @@ Status TabletMetaManager::save(DataDir* store, TTabletId 
tablet_id, TSchemaHash
     std::string key = fmt::format("{}{}_{}", header_prefix, tablet_id, 
schema_hash);
     std::string value;
     tablet_meta->serialize(&value);
+    if (tablet_meta->partition_id() <= 0) {
+        LOG(WARNING) << "invalid partition id " << tablet_meta->partition_id() 
<< " tablet "
+                     << tablet_meta->tablet_id();
+        return Status::InternalError("invaid partition id {} tablet {}",
+                                     tablet_meta->partition_id(), 
tablet_meta->tablet_id());
+    }
     OlapMeta* meta = store->get_meta();
     VLOG_NOTICE << "save tablet meta"
                 << ", key:" << key << ", meta length:" << value.length();
diff --git a/be/src/olap/txn_manager.cpp b/be/src/olap/txn_manager.cpp
index 0b9b08418bb..7a06a08d838 100644
--- a/be/src/olap/txn_manager.cpp
+++ b/be/src/olap/txn_manager.cpp
@@ -257,9 +257,10 @@ Status TxnManager::commit_txn(OlapMeta* meta, TPartitionId 
partition_id,
                               const PUniqueId& load_id, const RowsetSharedPtr& 
rowset_ptr,
                               bool is_recovery) {
     if (partition_id < 1 || transaction_id < 1 || tablet_id < 1) {
-        LOG(FATAL) << "invalid commit req "
-                   << " partition_id=" << partition_id << " transaction_id=" 
<< transaction_id
-                   << " tablet_id=" << tablet_id;
+        LOG(WARNING) << "invalid commit req "
+                     << " partition_id=" << partition_id << " transaction_id=" 
<< transaction_id
+                     << " tablet_id=" << tablet_id;
+        return Status::InternalError("invalid partition id");
     }
 
     pair<int64_t, int64_t> key(partition_id, transaction_id);
diff --git a/be/src/vec/exec/distinct_vaggregation_node.cpp 
b/be/src/vec/exec/distinct_vaggregation_node.cpp
index bbbd1964119..0765a3f4910 100644
--- a/be/src/vec/exec/distinct_vaggregation_node.cpp
+++ b/be/src/vec/exec/distinct_vaggregation_node.cpp
@@ -35,7 +35,6 @@ DistinctAggregationNode::DistinctAggregationNode(ObjectPool* 
pool, const TPlanNo
 
 Status DistinctAggregationNode::_distinct_pre_agg_with_serialized_key(
         doris::vectorized::Block* in_block, doris::vectorized::Block* 
out_block) {
-    SCOPED_TIMER(_build_timer);
     DCHECK(!_probe_expr_ctxs.empty());
 
     size_t key_size = _probe_expr_ctxs.size();
diff --git a/be/src/vec/exec/scan/scanner_scheduler.cpp 
b/be/src/vec/exec/scan/scanner_scheduler.cpp
index ec1f7a818ca..7e5bdeedd67 100644
--- a/be/src/vec/exec/scan/scanner_scheduler.cpp
+++ b/be/src/vec/exec/scan/scanner_scheduler.cpp
@@ -116,9 +116,12 @@ Status ScannerScheduler::init(ExecEnv* env) {
             config::doris_scanner_thread_pool_queue_size, "local_scan");
 
     // 3. remote scan thread pool
+    _remote_thread_pool_max_size = 
config::doris_max_remote_scanner_thread_pool_thread_num != -1
+                                           ? 
config::doris_max_remote_scanner_thread_pool_thread_num
+                                           : std::max(512, 
CpuInfo::num_cores() * 10);
     _remote_scan_thread_pool = std::make_unique<PriorityThreadPool>(
-            config::doris_remote_scanner_thread_pool_thread_num,
-            config::doris_remote_scanner_thread_pool_queue_size, 
"RemoteScanThreadPool");
+            _remote_thread_pool_max_size, 
config::doris_remote_scanner_thread_pool_queue_size,
+            "RemoteScanThreadPool");
 
     // 4. limited scan thread pool
     ThreadPoolBuilder("LimitedScanThreadPool")
diff --git a/be/src/vec/exec/vaggregation_node.cpp 
b/be/src/vec/exec/vaggregation_node.cpp
index f2d14ea5957..19f5ce48173 100644
--- a/be/src/vec/exec/vaggregation_node.cpp
+++ b/be/src/vec/exec/vaggregation_node.cpp
@@ -106,7 +106,6 @@ AggregationNode::AggregationNode(ObjectPool* pool, const 
TPlanNode& tnode,
         : ExecNode(pool, tnode, descs),
           _hash_table_compute_timer(nullptr),
           _hash_table_input_counter(nullptr),
-          _build_timer(nullptr),
           _expr_timer(nullptr),
           _exec_timer(nullptr),
           _intermediate_tuple_id(tnode.agg_node.intermediate_tuple_id),
@@ -332,7 +331,6 @@ Status AggregationNode::prepare_profile(RuntimeState* 
state) {
     _serialize_key_arena_memory_usage = 
runtime_profile()->AddHighWaterMarkCounter(
             "SerializeKeyArena", TUnit::BYTES, "MemoryUsage");
 
-    _build_timer = ADD_TIMER(runtime_profile(), "BuildTime");
     _build_table_convert_timer = ADD_TIMER(runtime_profile(), 
"BuildConvertToPartitionedTime");
     _serialize_key_timer = ADD_TIMER(runtime_profile(), "SerializeKeyTime");
     _exec_timer = ADD_TIMER(runtime_profile(), "ExecTime");
@@ -375,7 +373,7 @@ Status AggregationNode::prepare_profile(RuntimeState* 
state) {
 
     // set profile timer to evaluators
     for (auto& evaluator : _aggregate_evaluators) {
-        evaluator->set_timer(_exec_timer, _merge_timer, _expr_timer);
+        evaluator->set_timer(_merge_timer, _expr_timer);
     }
 
     _offsets_of_aggregate_states.resize(_aggregate_evaluators.size());
@@ -486,7 +484,7 @@ Status AggregationNode::prepare_profile(RuntimeState* 
state) {
                    _is_merge ? "true" : "false", _needs_finalize ? "true" : 
"false",
                    _is_streaming_preagg ? "true" : "false",
                    std::to_string(_aggregate_evaluators.size()), 
std::to_string(_limit));
-    runtime_profile()->add_info_string("AggInfos:", fmt::to_string(msg));
+    runtime_profile()->add_info_string("AggInfos", fmt::to_string(msg));
     return Status::OK();
 }
 
@@ -494,11 +492,13 @@ Status AggregationNode::prepare(RuntimeState* state) {
     SCOPED_TIMER(_runtime_profile->total_time_counter());
 
     RETURN_IF_ERROR(ExecNode::prepare(state));
+    SCOPED_TIMER(_exec_timer);
     RETURN_IF_ERROR(prepare_profile(state));
     return Status::OK();
 }
 
 Status AggregationNode::alloc_resource(doris::RuntimeState* state) {
+    SCOPED_TIMER(_exec_timer);
     RETURN_IF_ERROR(ExecNode::alloc_resource(state));
 
     RETURN_IF_ERROR(VExpr::open(_probe_expr_ctxs, state));
@@ -547,6 +547,7 @@ Status AggregationNode::open(RuntimeState* state) {
 
 Status AggregationNode::do_pre_agg(vectorized::Block* input_block,
                                    vectorized::Block* output_block) {
+    SCOPED_TIMER(_exec_timer);
     RETURN_IF_ERROR(_executor.pre_agg(input_block, output_block));
 
     // pre stream agg need use _num_row_return to decide whether to do pre 
stream agg
@@ -585,6 +586,7 @@ Status AggregationNode::get_next(RuntimeState* state, 
Block* block, bool* eos) {
 }
 
 Status AggregationNode::pull(doris::RuntimeState* state, vectorized::Block* 
block, bool* eos) {
+    SCOPED_TIMER(_exec_timer);
     RETURN_IF_ERROR(_executor.get_result(state, block, eos));
     _make_nullable_output_key(block);
     // dispose the having clause, should not be execute in prestreaming agg
@@ -595,6 +597,7 @@ Status AggregationNode::pull(doris::RuntimeState* state, 
vectorized::Block* bloc
 }
 
 Status AggregationNode::sink(doris::RuntimeState* state, vectorized::Block* 
in_block, bool eos) {
+    SCOPED_TIMER(_exec_timer);
     if (in_block->rows() > 0) {
         RETURN_IF_ERROR(_executor.execute(in_block));
         RETURN_IF_ERROR(_try_spill_disk());
@@ -744,7 +747,6 @@ Status 
AggregationNode::_serialize_without_key(RuntimeState* state, Block* block
 
 Status AggregationNode::_execute_without_key(Block* block) {
     DCHECK(_agg_data->without_key != nullptr);
-    SCOPED_TIMER(_build_timer);
     for (int i = 0; i < _aggregate_evaluators.size(); ++i) {
         RETURN_IF_ERROR(_aggregate_evaluators[i]->execute_single_add(
                 block, _agg_data->without_key + 
_offsets_of_aggregate_states[i],
@@ -1059,7 +1061,6 @@ void 
AggregationNode::_find_in_hash_table(AggregateDataPtr* places, ColumnRawPtr
 
 Status AggregationNode::_pre_agg_with_serialized_key(doris::vectorized::Block* 
in_block,
                                                      doris::vectorized::Block* 
out_block) {
-    SCOPED_TIMER(_build_timer);
     DCHECK(!_probe_expr_ctxs.empty());
 
     size_t key_size = _probe_expr_ctxs.size();
diff --git a/be/src/vec/exec/vaggregation_node.h 
b/be/src/vec/exec/vaggregation_node.h
index 90b199869ea..6615166d7dd 100644
--- a/be/src/vec/exec/vaggregation_node.h
+++ b/be/src/vec/exec/vaggregation_node.h
@@ -855,7 +855,6 @@ protected:
     std::vector<size_t> _make_nullable_keys;
     RuntimeProfile::Counter* _hash_table_compute_timer;
     RuntimeProfile::Counter* _hash_table_input_counter;
-    RuntimeProfile::Counter* _build_timer;
     RuntimeProfile::Counter* _expr_timer;
     RuntimeProfile::Counter* _exec_timer;
 
@@ -973,7 +972,6 @@ protected:
 private:
     template <bool limit>
     Status _execute_with_serialized_key_helper(Block* block) {
-        SCOPED_TIMER(_build_timer);
         DCHECK(!_probe_expr_ctxs.empty());
 
         size_t key_size = _probe_expr_ctxs.size();
diff --git a/be/src/vec/exprs/vcompound_pred.h 
b/be/src/vec/exprs/vcompound_pred.h
index 20c7633e3c3..57bf07d4097 100644
--- a/be/src/vec/exprs/vcompound_pred.h
+++ b/be/src/vec/exprs/vcompound_pred.h
@@ -117,7 +117,7 @@ public:
             return res_id;
         };
 
-        auto create_null_map_column = [&](ColumnPtr null_map_column,
+        auto create_null_map_column = [&](ColumnPtr& null_map_column,
                                           uint8* __restrict null_map_data) {
             if (null_map_data == nullptr) {
                 null_map_column = ColumnUInt8::create(size, 0);
diff --git a/be/src/vec/exprs/vectorized_agg_fn.cpp 
b/be/src/vec/exprs/vectorized_agg_fn.cpp
index c27d95a7f74..885f53dc663 100644
--- a/be/src/vec/exprs/vectorized_agg_fn.cpp
+++ b/be/src/vec/exprs/vectorized_agg_fn.cpp
@@ -69,7 +69,6 @@ AggFnEvaluator::AggFnEvaluator(const TExprNode& desc)
           _return_type(TypeDescriptor::from_thrift(desc.fn.ret_type)),
           _intermediate_slot_desc(nullptr),
           _output_slot_desc(nullptr),
-          _exec_timer(nullptr),
           _merge_timer(nullptr),
           _expr_timer(nullptr) {
     bool nullable = true;
@@ -233,7 +232,6 @@ void AggFnEvaluator::destroy(AggregateDataPtr place) {
 
 Status AggFnEvaluator::execute_single_add(Block* block, AggregateDataPtr 
place, Arena* arena) {
     RETURN_IF_ERROR(_calc_argument_columns(block));
-    SCOPED_TIMER(_exec_timer);
     _function->add_batch_single_place(block->rows(), place, 
_agg_columns.data(), arena);
     return Status::OK();
 }
@@ -241,7 +239,6 @@ Status AggFnEvaluator::execute_single_add(Block* block, 
AggregateDataPtr place,
 Status AggFnEvaluator::execute_batch_add(Block* block, size_t offset, 
AggregateDataPtr* places,
                                          Arena* arena, bool agg_many) {
     RETURN_IF_ERROR(_calc_argument_columns(block));
-    SCOPED_TIMER(_exec_timer);
     _function->add_batch(block->rows(), places, offset, _agg_columns.data(), 
arena, agg_many);
     return Status::OK();
 }
@@ -249,7 +246,6 @@ Status AggFnEvaluator::execute_batch_add(Block* block, 
size_t offset, AggregateD
 Status AggFnEvaluator::execute_batch_add_selected(Block* block, size_t offset,
                                                   AggregateDataPtr* places, 
Arena* arena) {
     RETURN_IF_ERROR(_calc_argument_columns(block));
-    SCOPED_TIMER(_exec_timer);
     _function->add_batch_selected(block->rows(), places, offset, 
_agg_columns.data(), arena);
     return Status::OK();
 }
@@ -257,7 +253,6 @@ Status AggFnEvaluator::execute_batch_add_selected(Block* 
block, size_t offset,
 Status AggFnEvaluator::streaming_agg_serialize(Block* block, BufferWritable& 
buf,
                                                const size_t num_rows, Arena* 
arena) {
     RETURN_IF_ERROR(_calc_argument_columns(block));
-    SCOPED_TIMER(_exec_timer);
     _function->streaming_agg_serialize(_agg_columns.data(), buf, num_rows, 
arena);
     return Status::OK();
 }
@@ -265,7 +260,6 @@ Status AggFnEvaluator::streaming_agg_serialize(Block* 
block, BufferWritable& buf
 Status AggFnEvaluator::streaming_agg_serialize_to_column(Block* block, 
MutableColumnPtr& dst,
                                                          const size_t 
num_rows, Arena* arena) {
     RETURN_IF_ERROR(_calc_argument_columns(block));
-    SCOPED_TIMER(_exec_timer);
     _function->streaming_agg_serialize_to_column(_agg_columns.data(), dst, 
num_rows, arena);
     return Status::OK();
 }
diff --git a/be/src/vec/exprs/vectorized_agg_fn.h 
b/be/src/vec/exprs/vectorized_agg_fn.h
index 2688fae2608..f20ef2fd1c3 100644
--- a/be/src/vec/exprs/vectorized_agg_fn.h
+++ b/be/src/vec/exprs/vectorized_agg_fn.h
@@ -56,9 +56,7 @@ public:
                    const SlotDescriptor* intermediate_slot_desc,
                    const SlotDescriptor* output_slot_desc);
 
-    void set_timer(RuntimeProfile::Counter* exec_timer, 
RuntimeProfile::Counter* merge_timer,
-                   RuntimeProfile::Counter* expr_timer) {
-        _exec_timer = exec_timer;
+    void set_timer(RuntimeProfile::Counter* merge_timer, 
RuntimeProfile::Counter* expr_timer) {
         _merge_timer = merge_timer;
         _expr_timer = expr_timer;
     }
@@ -120,7 +118,6 @@ private:
     const SlotDescriptor* _intermediate_slot_desc;
     const SlotDescriptor* _output_slot_desc;
 
-    RuntimeProfile::Counter* _exec_timer;
     RuntimeProfile::Counter* _merge_timer;
     RuntimeProfile::Counter* _expr_timer;
 
diff --git a/be/test/olap/delete_handler_test.cpp 
b/be/test/olap/delete_handler_test.cpp
index 792fc18f78c..3029220903f 100644
--- a/be/test/olap/delete_handler_test.cpp
+++ b/be/test/olap/delete_handler_test.cpp
@@ -100,6 +100,7 @@ static void tear_down() {
 static void set_default_create_tablet_request(TCreateTabletReq* request) {
     request->tablet_id = 10003;
     request->__set_version(1);
+    request->partition_id = 10004;
     request->tablet_schema.schema_hash = 270068375;
     request->tablet_schema.short_key_column_count = 2;
     request->tablet_schema.keys_type = TKeysType::AGG_KEYS;
@@ -185,6 +186,7 @@ static void 
set_default_create_tablet_request(TCreateTabletReq* request) {
 
 static void set_create_duplicate_tablet_request(TCreateTabletReq* request) {
     request->tablet_id = 10009;
+    request->partition_id = 10010;
     request->__set_version(1);
     request->tablet_schema.schema_hash = 270068376;
     request->tablet_schema.short_key_column_count = 2;
diff --git a/be/test/olap/delta_writer_test.cpp 
b/be/test/olap/delta_writer_test.cpp
index f54570b88f1..b9bbd557976 100644
--- a/be/test/olap/delta_writer_test.cpp
+++ b/be/test/olap/delta_writer_test.cpp
@@ -101,6 +101,7 @@ static void tear_down() {
 static void create_tablet_request(int64_t tablet_id, int32_t schema_hash,
                                   TCreateTabletReq* request) {
     request->tablet_id = tablet_id;
+    request->partition_id = 1000;
     request->__set_version(1);
     request->tablet_schema.schema_hash = schema_hash;
     request->tablet_schema.short_key_column_count = 6;
@@ -264,6 +265,7 @@ static void create_tablet_request_with_sequence_col(int64_t 
tablet_id, int32_t s
                                                     TCreateTabletReq* request,
                                                     bool enable_mow = false) {
     request->tablet_id = tablet_id;
+    request->partition_id = 10000;
     request->__set_version(1);
     request->tablet_schema.schema_hash = schema_hash;
     request->tablet_schema.short_key_column_count = 2;
diff --git a/be/test/olap/engine_storage_migration_task_test.cpp 
b/be/test/olap/engine_storage_migration_task_test.cpp
index 9c99ef308ae..532cc38d05c 100644
--- a/be/test/olap/engine_storage_migration_task_test.cpp
+++ b/be/test/olap/engine_storage_migration_task_test.cpp
@@ -100,6 +100,7 @@ static void tear_down() {
 static void create_tablet_request_with_sequence_col(int64_t tablet_id, int32_t 
schema_hash,
                                                     TCreateTabletReq* request) 
{
     request->tablet_id = tablet_id;
+    request->partition_id = 20001;
     request->__set_version(1);
     request->tablet_schema.schema_hash = schema_hash;
     request->tablet_schema.short_key_column_count = 2;
diff --git a/be/test/olap/remote_rowset_gc_test.cpp 
b/be/test/olap/remote_rowset_gc_test.cpp
index a8fc13470f8..779722c9c48 100644
--- a/be/test/olap/remote_rowset_gc_test.cpp
+++ b/be/test/olap/remote_rowset_gc_test.cpp
@@ -117,6 +117,7 @@ public:
 static void create_tablet_request_with_sequence_col(int64_t tablet_id, int32_t 
schema_hash,
                                                     TCreateTabletReq* request) 
{
     request->tablet_id = tablet_id;
+    request->partition_id = 1000;
     request->__set_version(1);
     request->tablet_schema.schema_hash = schema_hash;
     request->tablet_schema.short_key_column_count = 2;
diff --git a/be/test/olap/tablet_cooldown_test.cpp 
b/be/test/olap/tablet_cooldown_test.cpp
index 2582746291a..d02586592c2 100644
--- a/be/test/olap/tablet_cooldown_test.cpp
+++ b/be/test/olap/tablet_cooldown_test.cpp
@@ -276,6 +276,7 @@ public:
 static void create_tablet_request_with_sequence_col(int64_t tablet_id, int32_t 
schema_hash,
                                                     TCreateTabletReq* request) 
{
     request->tablet_id = tablet_id;
+    request->partition_id = 1000;
     request->__set_version(1);
     request->tablet_schema.schema_hash = schema_hash;
     request->tablet_schema.short_key_column_count = 2;
diff --git a/be/test/olap/tablet_mgr_test.cpp b/be/test/olap/tablet_mgr_test.cpp
index 5954d0329fb..dd901df8137 100644
--- a/be/test/olap/tablet_mgr_test.cpp
+++ b/be/test/olap/tablet_mgr_test.cpp
@@ -107,6 +107,7 @@ TEST_F(TabletMgrTest, CreateTablet) {
     create_tablet_req.__set_tablet_schema(tablet_schema);
     create_tablet_req.__set_tablet_id(111);
     create_tablet_req.__set_version(2);
+    create_tablet_req.__set_partition_id(1000);
     std::vector<DataDir*> data_dirs;
     data_dirs.push_back(_data_dir);
     RuntimeProfile profile("CreateTablet");
@@ -167,6 +168,7 @@ TEST_F(TabletMgrTest, CreateTabletWithSequence) {
     TCreateTabletReq create_tablet_req;
     create_tablet_req.__set_tablet_schema(tablet_schema);
     create_tablet_req.__set_tablet_id(111);
+    create_tablet_req.__set_partition_id(1000);
     create_tablet_req.__set_version(2);
     std::vector<DataDir*> data_dirs;
     data_dirs.push_back(_data_dir);
@@ -210,6 +212,7 @@ TEST_F(TabletMgrTest, DropTablet) {
     TCreateTabletReq create_tablet_req;
     create_tablet_req.__set_tablet_schema(tablet_schema);
     create_tablet_req.__set_tablet_id(111);
+    create_tablet_req.__set_partition_id(1000);
     create_tablet_req.__set_version(2);
     std::vector<DataDir*> data_dirs;
     data_dirs.push_back(_data_dir);
diff --git a/docs/en/docs/admin-manual/multi-tenant.md 
b/docs/en/docs/admin-manual/multi-tenant.md
index b5659a7897e..d4f207ceae7 100644
--- a/docs/en/docs/admin-manual/multi-tenant.md
+++ b/docs/en/docs/admin-manual/multi-tenant.md
@@ -138,7 +138,7 @@ Node resource division refers to setting tags for BE nodes 
in a Doris cluster, a
     
     After the setting is complete, when user1 initiates a query on the 
UserTable table, it will only access the data copy on the nodes in the 
`group_a` resource group, and the query will only use the node computing 
resources in the `group_a` resource group. The query of user3 can use copies 
and computing resources in any resource group.
     
-    > Note: By default, the user's `resource_tags.location` attribute is 
empty. In versions prior to 2.0.2 (inclusive), by default, users are not 
restricted by tags and can use any resource group. After version 2.0.3, users 
can only use the `default` resource group by default.
+    > Note: By default, the user's `resource_tags.location` attribute is 
empty. In versions prior to 2.0.2 (inclusive), by default, users are not 
restricted by tags and can use any resource group. After version 2.0.3, normal 
users can only use the `default` resource group by default. Root and Admin user 
can use any resource group.
 
     In this way, we have achieved physical resource isolation for different 
user queries by dividing nodes and restricting user resource usage. 
Furthermore, we can create different users for different business departments 
and restrict each user from using different resource groups. In order to avoid 
the use of resource interference between different business parts. For example, 
there is a business table in the cluster that needs to be shared by all 9 
business departments, but it is hoped [...]
     
@@ -268,4 +268,4 @@ Through the above 4 steps, we can smoothly use the resource 
division function af
 
    The table creation statements for table2, table3, and table4 do not need to 
specify `replication_allocation` again.
 
-   Note: Changing the replica distribution policy of the database will not 
affect existing tables.
\ No newline at end of file
+   Note: Changing the replica distribution policy of the database will not 
affect existing tables.
diff --git a/docs/en/docs/advanced/partition/auto-partition.md 
b/docs/en/docs/advanced/partition/auto-partition.md
deleted file mode 100644
index c718f75454d..00000000000
--- a/docs/en/docs/advanced/partition/auto-partition.md
+++ /dev/null
@@ -1,190 +0,0 @@
----
-{
-    "title": "Auto Partition",
-    "language": "en"
-}
----
-
-<!-- 
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements.  See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership.  The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License.  You may obtain a copy of the License at
-
-  http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied.  See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# AUTO PARTITION
-
-<version since="2.0.3">
-
-</version>
-
-The Auto Partitioning feature supports automatic detection of whether the 
corresponding partition exists during the data import process. If it does not 
exist, the partition will be created automatically and imported normally.
-
-## Grammer
-
-When building a table, use the following syntax to populate 
[CREATE-TABLE](../../sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md)
 with the `partition_info` section:
-
-1. AUTO RANGE PARTITION:
-
-  ```SQL
-  AUTO PARTITION BY RANGE FUNC_CALL_EXPR
-  (
-  )
-  ```
-  where
-  ```SQL
-  FUNC_CALL_EXPR ::= date_trunc ( <partition_column>, '<interval>' )
-  ```
-
-2. AUTO LIST PARTITION:
-
-  ```SQL
-  AUTO PARTITION BY LIST(`partition_col`)
-  (
-  )
-  ```
-
-### Usage example
-
-1. AUTO RANGE PARTITION
-
-  ```SQL
-  CREATE TABLE `${tblDate}` (
-      `TIME_STAMP` datev2 NOT NULL COMMENT 'Date of collection'
-  ) ENGINE=OLAP
-  DUPLICATE KEY(`TIME_STAMP`)
-  AUTO PARTITION BY RANGE date_trunc(`TIME_STAMP`, 'month')
-  (
-  )
-  DISTRIBUTED BY HASH(`TIME_STAMP`) BUCKETS 10
-  PROPERTIES (
-  "replication_allocation" = "tag.location.default: 1"
-  );
-  ```
-
-2. AUTO LIST PARTITION
-
-  ```SQL
-  CREATE TABLE `${tblName1}` (
-      `str` varchar not null
-  ) ENGINE=OLAP
-  DUPLICATE KEY(`str`)
-  AUTO PARTITION BY LIST (`str`)
-  (
-  )
-  DISTRIBUTED BY HASH(`str`) BUCKETS 10
-  PROPERTIES (
-  "replication_allocation" = "tag.location.default: 1"
-  );
-  ```
-
-### Using constraints
-
-1. Currently the AUTO RANGE PARTITION function supports only one partition 
column;
-2. In AUTO RANGE PARTITION, the partition function supports only `date_trunc` 
and the partition column supports only `DATEV2` or `DATETIMEV2` format;
-3. In AUTO LIST PARTITION, function calls are not supported. Partitioned 
columns support BOOLEAN, TINYINT, SMALLINT, INT, BIGINT, LARGEINT, DATE, 
DATETIME, CHAR, VARCHAR datatypes, and partitioned values are enum values.
-4. In AUTO LIST PARTITION, a separate new PARTITION is created for each fetch 
of a partition column for which the corresponding partition does not currently 
exist.
-
-## Sample Scenarios
-
-In the [Dynamic Partitioning](./dynamic-partition.md) feature, we support the 
automatic creation of new partitions to accommodate real-time data at specific 
time periods. However, in more complex scenarios, such as processing 
non-real-time data, the partition columns are independent of the current system 
time. In this case, if you need to partition the data, you need to manually 
organise the partitions you belong to and create them before importing the 
data. This is cumbersome when the n [...]
-
-For example, we have a table as follows:
-
-```sql
-CREATE TABLE `DAILY_TRADE_VALUE`
-(
-    `TRADE_DATE`              datev2 NULL
-    `TRADE_ID`                varchar(40) NULL,
-    ......
-)
-UNIQUE KEY(`TRADE_DATE`, `TRADE_ID`)
-PARTITION BY RANGE(`TRADE_DATE`)
-(
-    PARTITION p_2000 VALUES [('2000-01-01'), ('2001-01-01')),
-    PARTITION p_2001 VALUES [('2001-01-01'), ('2002-01-01')),
-    PARTITION p_2002 VALUES [('2002-01-01'), ('2003-01-01')),
-    PARTITION p_2003 VALUES [('2003-01-01'), ('2004-01-01')),
-    PARTITION p_2004 VALUES [('2004-01-01'), ('2005-01-01')),
-    PARTITION p_2005 VALUES [('2005-01-01'), ('2006-01-01')),
-    PARTITION p_2006 VALUES [('2006-01-01'), ('2007-01-01')),
-    PARTITION p_2007 VALUES [('2007-01-01'), ('2008-01-01')),
-    PARTITION p_2008 VALUES [('2008-01-01'), ('2009-01-01')),
-    PARTITION p_2009 VALUES [('2009-01-01'), ('2010-01-01')),
-    PARTITION p_2010 VALUES [('2010-01-01'), ('2011-01-01')),
-    PARTITION p_2011 VALUES [('2011-01-01'), ('2012-01-01')),
-    PARTITION p_2012 VALUES [('2012-01-01'), ('2013-01-01')),
-    PARTITION p_2013 VALUES [('2013-01-01'), ('2014-01-01')),
-    PARTITION p_2014 VALUES [('2014-01-01'), ('2015-01-01')),
-    PARTITION p_2015 VALUES [('2015-01-01'), ('2016-01-01')),
-    PARTITION p_2016 VALUES [('2016-01-01'), ('2017-01-01')),
-    PARTITION p_2017 VALUES [('2017-01-01'), ('2018-01-01')),
-    PARTITION p_2018 VALUES [('2018-01-01'), ('2019-01-01')),
-    PARTITION p_2019 VALUES [('2019-01-01'), ('2020-01-01')),
-    PARTITION p_2020 VALUES [('2020-01-01'), ('2021-01-01')),
-    PARTITION p_2021 VALUES [('2021-01-01'), ('2022-01-01'))
-)
-DISTRIBUTED BY HASH(`TRADE_DATE`) BUCKETS 10
-PROPERTIES (
-  "replication_num" = "1"
-);
-```
-
-The table stores a large amount of business history data, partitioned based on 
the date the transaction occurred. As you can see when building the table, we 
need to manually create the partitions in advance. If the data range of the 
partitioned columns changes, for example, 2022 is added to the above table, we 
need to create a partition by 
[ALTER-TABLE-PARTITION](../../sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-PARTITION.md)
 to make changes to the table partiti [...]
-
-```SQL
-CREATE TABLE `DAILY_TRADE_VALUE`
-(
-    `TRADE_DATE`              datev2 NULL,
-    `TRADE_ID`                varchar(40) NULL,
-    ......
-)
-UNIQUE KEY(`TRADE_DATE`, `TRADE_ID`)
-AUTO PARTITION BY RANGE date_trunc(`TRADE_DATE`, 'year')
-(
-)
-DISTRIBUTED BY HASH(`TRADE_DATE`) BUCKETS 10
-PROPERTIES (
-  "replication_num" = "1"
-);
-```
-
-At this point the new table does not have a default partition:
-```SQL
-mysql> show partitions from `DAILY_TRADE_VALUE`;
-Empty set (0.12 sec)
-```
-
-After inserting the data and then viewing it again, we could found that the 
table has been created with corresponding partitions:
-```SQL
-mysql> insert into `DAILY_TRADE_VALUE` values ('2012-12-13', 1), 
('2008-02-03', 2), ('2014-11-11', 3);
-Query OK, 3 rows affected (0.88 sec)
-{'label':'insert_754e2a3926a345ea_854793fb2638f0ec', 'status':'VISIBLE', 
'txnId':'20014'}
-
-mysql> show partitions from `DAILY_TRADE_VALUE`;
-+-------------+-----------------+----------------+---------------------+--------+--------------+--------------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+-----------+
-| PartitionId | PartitionName   | VisibleVersion | VisibleVersionTime  | State 
 | PartitionKey | Range                                                         
                 | DistributionKey | Buckets | ReplicationNum | StorageMedium | 
CooldownTime        | RemoteStoragePolicy | LastConsistencyCheckTime | DataSize 
| IsInMemory | ReplicaAllocation       | IsMutable |
-+-------------+-----------------+----------------+---------------------+--------+--------------+--------------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+-----------+
-| 180060      | p20080101000000 | 2              | 2023-09-18 21:49:29 | 
NORMAL | TRADE_DATE   | [types: [DATEV2]; keys: [2008-01-01]; ..types: 
[DATEV2]; keys: [2009-01-01]; ) | TRADE_DATE      | 10      | 1              | 
HDD           | 9999-12-31 23:59:59 |                     | NULL                
     | 0.000    | false      | tag.location.default: 1 | true      |
-| 180039      | p20120101000000 | 2              | 2023-09-18 21:49:29 | 
NORMAL | TRADE_DATE   | [types: [DATEV2]; keys: [2012-01-01]; ..types: 
[DATEV2]; keys: [2013-01-01]; ) | TRADE_DATE      | 10      | 1              | 
HDD           | 9999-12-31 23:59:59 |                     | NULL                
     | 0.000    | false      | tag.location.default: 1 | true      |
-| 180018      | p20140101000000 | 2              | 2023-09-18 21:49:29 | 
NORMAL | TRADE_DATE   | [types: [DATEV2]; keys: [2014-01-01]; ..types: 
[DATEV2]; keys: [2015-01-01]; ) | TRADE_DATE      | 10      | 1              | 
HDD           | 9999-12-31 23:59:59 |                     | NULL                
     | 0.000    | false      | tag.location.default: 1 | true      |
-+-------------+-----------------+----------------+---------------------+--------+--------------+--------------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+-----------+
-3 rows in set (0.12 sec)
-```
-
-## caveat
-
-- If a partition is created during the insertion or importation of data and 
the process eventually fails, the created partition is not automatically 
deleted.
-- Tables that use AUTO PARTITION only have their partitions created 
automatically instead of manually. The original use of the table and the 
partitions it creates is the same as for non-AUTO PARTITION tables or 
partitions.
diff --git a/docs/sidebars.json b/docs/sidebars.json
index db0caa5e08e..601c843701f 100644
--- a/docs/sidebars.json
+++ b/docs/sidebars.json
@@ -142,8 +142,7 @@
                     "label": "Doris Partition",
                     "items": [
                         "advanced/partition/dynamic-partition",
-                        "advanced/partition/table-temp-partition",
-                        "advanced/partition/auto-partition"
+                        "advanced/partition/table-temp-partition"
                     ]
                 },
                 {
diff --git a/docs/zh-CN/docs/admin-manual/multi-tenant.md 
b/docs/zh-CN/docs/admin-manual/multi-tenant.md
index ec24a513825..7e346dbd8e5 100644
--- a/docs/zh-CN/docs/admin-manual/multi-tenant.md
+++ b/docs/zh-CN/docs/admin-manual/multi-tenant.md
@@ -138,7 +138,7 @@ FE 不参与用户数据的处理计算等工作，因此是一个资源消耗
 
    设置完成后，user1 在发起对 UserTable 表的查询时，只会访问 `group_a` 资源组内节点上的数据副本，并且查询仅会使用 
`group_a` 资源组内的节点计算资源。而 user3 的查询可以使用任意资源组内的副本和计算资源。
 
-   > 注：默认情况下，用户的 `resource_tags.location` 属性为空，在2.0.2（含）之前的版本中，默认情况下，用户不受 tag 
的限制，可以使用任意资源组。在 2.0.3 版本之后，默认情况下，用户只能使用 `default` 资源组。
+   > 注：默认情况下，用户的 `resource_tags.location` 属性为空，在2.0.2（含）之前的版本中，默认情况下，用户不受 tag 
的限制，可以使用任意资源组。在 2.0.3 版本之后，默认情况下，普通用户只能使用 `default` 资源组。root 和 admin 
用户可以使用任意资源组。
 
    
这样，我们通过对节点的划分，以及对用户的资源使用限制，实现了不同用户查询上的物理资源隔离。更进一步，我们可以给不同的业务部门创建不同的用户，并限制每个用户使用不同的资源组。以避免不同业务部分之间使用资源干扰。比如集群内有一张业务表需要共享给所有9个业务部门使用，但是希望能够尽量避免不同部门之间的资源抢占。则我们可以为这张表创建3个副本，分别存储在3个资源组中。接下来，我们为9个业务部门创建9个用户，每3个用户限制使用一个资源组。这样，资源的竞争程度就由9降低到了3。
 
@@ -266,4 +266,4 @@ FE 不参与用户数据的处理计算等工作，因此是一个资源消耗
 
    table2，table3,table4的建表语句无需再指定`replication_allocation`。
    
-   注意事项：更改database的副本分布策略不会对已有的table产生影响。
\ No newline at end of file
+   注意事项：更改database的副本分布策略不会对已有的table产生影响。
diff --git a/docs/zh-CN/docs/advanced/partition/auto-partition.md 
b/docs/zh-CN/docs/advanced/partition/auto-partition.md
deleted file mode 100644
index 193a0868b37..00000000000
--- a/docs/zh-CN/docs/advanced/partition/auto-partition.md
+++ /dev/null
@@ -1,190 +0,0 @@
----
-{
-    "title": "自动分区",
-    "language": "zh-CN"
-}
----
-
-<!-- 
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements.  See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership.  The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License.  You may obtain a copy of the License at
-
-  http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied.  See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# 自动分区
-
-<version since="2.0.3">
-
-</version>
-
-自动分区功能支持了在导入数据过程中自动检测是否存在对应所属分区。如果不存在，则会自动创建分区并正常进行导入。
-
-## 语法
-
-建表时，使用以下语法填充[CREATE-TABLE](../../sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md)时的`partition_info`部分：
-
-1. AUTO RANGE PARTITION:
-
-  ```SQL
-  AUTO PARTITION BY RANGE FUNC_CALL_EXPR
-  (
-  )
-  ```
-  其中
-  ```SQL
-  FUNC_CALL_EXPR ::= date_trunc ( <partition_column>, '<interval>' )
-  ```
-
-2. AUTO LIST PARTITION:
-
-  ```SQL
-  AUTO PARTITION BY LIST(`partition_col`)
-  (
-  )
-  ```
-
-### 用法示例
-
-1. AUTO RANGE PARTITION
-
-  ```SQL
-  CREATE TABLE `${tblDate}` (
-      `TIME_STAMP` datev2 NOT NULL COMMENT '采集日期'
-  ) ENGINE=OLAP
-  DUPLICATE KEY(`TIME_STAMP`)
-  AUTO PARTITION BY RANGE date_trunc(`TIME_STAMP`, 'month')
-  (
-  )
-  DISTRIBUTED BY HASH(`TIME_STAMP`) BUCKETS 10
-  PROPERTIES (
-  "replication_allocation" = "tag.location.default: 1"
-  );
-  ```
-
-2. AUTO LIST PARTITION
-
-  ```SQL
-  CREATE TABLE `${tblName1}` (
-      `str` varchar not null
-  ) ENGINE=OLAP
-  DUPLICATE KEY(`str`)
-  AUTO PARTITION BY LIST (`str`)
-  (
-  )
-  DISTRIBUTED BY HASH(`str`) BUCKETS 10
-  PROPERTIES (
-  "replication_allocation" = "tag.location.default: 1"
-  );
-  ```
-
-### 约束
-
-1. 当前自动分区功能仅支持一个分区列；
-2. 在AUTO RANGE PARTITION中，分区函数仅支持`date_trunc`，分区列仅支持`DATEV2`或者`DATETIMEV2`格式；
-3. 在AUTO LIST PARTITION中，不支持函数调用，分区列支持 BOOLEAN, TINYINT, SMALLINT, INT, 
BIGINT, LARGEINT, DATE, DATETIME, CHAR, VARCHAR 数据类型，分区值为枚举值。
-4. 在AUTO LIST PARTITION中，分区列的每个当前不存在对应分区的取值，都会创建一个独立的新PARTITION。
-
-## 场景示例
-
-在[动态分区](./dynamic-partition.md)功能中，我们支持了按特定时间周期自动创建新分区以容纳实时数据。但在更复杂的场景下，例如处理非实时数据时，分区列与当前系统时间无关。此时如果需要进行数据分区操作，则需要用户手动整理所属分区并在数据导入前进行创建。在分区列基数较大时比较繁琐。自动分区功能解决了这一问题。
-
-例如，我们有一张表如下：
-
-```sql
-CREATE TABLE `DAILY_TRADE_VALUE`
-(
-    `TRADE_DATE`              datev2 NULL COMMENT '交易日期',
-    `TRADE_ID`                varchar(40) NULL COMMENT '交易编号',
-    ......
-)
-UNIQUE KEY(`TRADE_DATE`, `TRADE_ID`)
-PARTITION BY RANGE(`TRADE_DATE`)
-(
-    PARTITION p_2000 VALUES [('2000-01-01'), ('2001-01-01')),
-    PARTITION p_2001 VALUES [('2001-01-01'), ('2002-01-01')),
-    PARTITION p_2002 VALUES [('2002-01-01'), ('2003-01-01')),
-    PARTITION p_2003 VALUES [('2003-01-01'), ('2004-01-01')),
-    PARTITION p_2004 VALUES [('2004-01-01'), ('2005-01-01')),
-    PARTITION p_2005 VALUES [('2005-01-01'), ('2006-01-01')),
-    PARTITION p_2006 VALUES [('2006-01-01'), ('2007-01-01')),
-    PARTITION p_2007 VALUES [('2007-01-01'), ('2008-01-01')),
-    PARTITION p_2008 VALUES [('2008-01-01'), ('2009-01-01')),
-    PARTITION p_2009 VALUES [('2009-01-01'), ('2010-01-01')),
-    PARTITION p_2010 VALUES [('2010-01-01'), ('2011-01-01')),
-    PARTITION p_2011 VALUES [('2011-01-01'), ('2012-01-01')),
-    PARTITION p_2012 VALUES [('2012-01-01'), ('2013-01-01')),
-    PARTITION p_2013 VALUES [('2013-01-01'), ('2014-01-01')),
-    PARTITION p_2014 VALUES [('2014-01-01'), ('2015-01-01')),
-    PARTITION p_2015 VALUES [('2015-01-01'), ('2016-01-01')),
-    PARTITION p_2016 VALUES [('2016-01-01'), ('2017-01-01')),
-    PARTITION p_2017 VALUES [('2017-01-01'), ('2018-01-01')),
-    PARTITION p_2018 VALUES [('2018-01-01'), ('2019-01-01')),
-    PARTITION p_2019 VALUES [('2019-01-01'), ('2020-01-01')),
-    PARTITION p_2020 VALUES [('2020-01-01'), ('2021-01-01')),
-    PARTITION p_2021 VALUES [('2021-01-01'), ('2022-01-01'))
-)
-DISTRIBUTED BY HASH(`TRADE_DATE`) BUCKETS 10
-PROPERTIES (
-  "replication_num" = "1"
-);
-```
-
-该表内存储了大量业务历史数据，依据交易发生的日期进行分区。可以看到在建表时，我们需要预先手动创建分区。如果分区列的数据范围发生变化，例如上表中增加了2022年的数据，则我们需要通过[ALTER-TABLE-PARTITION](../../sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-PARTITION.md)对表的分区进行更改。在使用AUTO
 PARTITION后，该表DDL可以改为：
-
-```SQL
-CREATE TABLE `DAILY_TRADE_VALUE`
-(
-    `TRADE_DATE`              datev2 NULL COMMENT '交易日期',
-    `TRADE_ID`                varchar(40) NULL COMMENT '交易编号',
-    ......
-)
-UNIQUE KEY(`TRADE_DATE`, `TRADE_ID`)
-AUTO PARTITION BY RANGE date_trunc(`TRADE_DATE`, 'year')
-(
-)
-DISTRIBUTED BY HASH(`TRADE_DATE`) BUCKETS 10
-PROPERTIES (
-  "replication_num" = "1"
-);
-```
-
-此时新表没有默认分区：
-```SQL
-mysql> show partitions from `DAILY_TRADE_VALUE`;
-Empty set (0.12 sec)
-```
-
-经过插入数据后再查看，发现该表已经创建了对应的分区：
-```SQL
-mysql> insert into `DAILY_TRADE_VALUE` values ('2012-12-13', 1), 
('2008-02-03', 2), ('2014-11-11', 3);
-Query OK, 3 rows affected (0.88 sec)
-{'label':'insert_754e2a3926a345ea_854793fb2638f0ec', 'status':'VISIBLE', 
'txnId':'20014'}
-
-mysql> show partitions from `DAILY_TRADE_VALUE`;
-+-------------+-----------------+----------------+---------------------+--------+--------------+--------------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+-----------+
-| PartitionId | PartitionName   | VisibleVersion | VisibleVersionTime  | State 
 | PartitionKey | Range                                                         
                 | DistributionKey | Buckets | ReplicationNum | StorageMedium | 
CooldownTime        | RemoteStoragePolicy | LastConsistencyCheckTime | DataSize 
| IsInMemory | ReplicaAllocation       | IsMutable |
-+-------------+-----------------+----------------+---------------------+--------+--------------+--------------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+-----------+
-| 180060      | p20080101000000 | 2              | 2023-09-18 21:49:29 | 
NORMAL | TRADE_DATE   | [types: [DATEV2]; keys: [2008-01-01]; ..types: 
[DATEV2]; keys: [2009-01-01]; ) | TRADE_DATE      | 10      | 1              | 
HDD           | 9999-12-31 23:59:59 |                     | NULL                
     | 0.000    | false      | tag.location.default: 1 | true      |
-| 180039      | p20120101000000 | 2              | 2023-09-18 21:49:29 | 
NORMAL | TRADE_DATE   | [types: [DATEV2]; keys: [2012-01-01]; ..types: 
[DATEV2]; keys: [2013-01-01]; ) | TRADE_DATE      | 10      | 1              | 
HDD           | 9999-12-31 23:59:59 |                     | NULL                
     | 0.000    | false      | tag.location.default: 1 | true      |
-| 180018      | p20140101000000 | 2              | 2023-09-18 21:49:29 | 
NORMAL | TRADE_DATE   | [types: [DATEV2]; keys: [2014-01-01]; ..types: 
[DATEV2]; keys: [2015-01-01]; ) | TRADE_DATE      | 10      | 1              | 
HDD           | 9999-12-31 23:59:59 |                     | NULL                
     | 0.000    | false      | tag.location.default: 1 | true      |
-+-------------+-----------------+----------------+---------------------+--------+--------------+--------------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+-----------+
-3 rows in set (0.12 sec)
-```
-
-## 注意事项
-
-- 在数据的插入或导入过程中如果创建了分区，而最终整个过程失败，被创建的分区不会被自动删除。
-- 使用AUTO PARTITION的表，只是分区创建方式上由手动转为了自动。表及其所创建分区的原本使用方法都与非AUTO PARTITION的表或分区相同。
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java 
b/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
index f255b1f303a..314ec990ebb 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
@@ -2870,6 +2870,8 @@ public class InternalCatalog implements 
CatalogIf<Database> {
         Database db = (Database) getDbOrDdlException(dbTbl.getDb());
         OlapTable olapTable = db.getOlapTableOrDdlException(dbTbl.getTbl());
 
+        long rowsToTruncate = 0;
+
         BinlogConfig binlogConfig;
         olapTable.readLock();
         try {
@@ -2882,6 +2884,7 @@ public class InternalCatalog implements 
CatalogIf<Database> {
                     }
                     origPartitions.put(partName, partition.getId());
                     partitionsDistributionInfo.put(partition.getId(), 
partition.getDistributionInfo());
+                    rowsToTruncate += partition.getBaseIndex().getRowCount();
                 }
             } else {
                 for (Partition partition : olapTable.getPartitions()) {
@@ -3016,7 +3019,13 @@ public class InternalCatalog implements 
CatalogIf<Database> {
         } finally {
             olapTable.writeUnlock();
         }
-
+        if (truncateEntireTable) {
+            // Drop the whole table stats after truncate the entire table
+            Env.getCurrentEnv().getAnalysisManager().dropStats(olapTable);
+        } else {
+            // Update the updated rows in table stats after truncate some 
partitions.
+            
Env.getCurrentEnv().getAnalysisManager().updateUpdatedRows(olapTable.getId(), 
rowsToTruncate);
+        }
         LOG.info("finished to truncate table {}, partitions: {}", 
tblRef.getName().toSql(), tblRef.getPartitionNames());
     }
 
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CommonUserProperties.java
 
b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CommonUserProperties.java
index 8fa28485ae5..cf36639c0a9 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CommonUserProperties.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CommonUserProperties.java
@@ -110,10 +110,6 @@ public class CommonUserProperties implements Writable {
     }
 
     public Set<Tag> getResourceTags() {
-        // If resource tags in user properties is empty, use default backend 
tag.
-        if (resourceTags.isEmpty()) {
-            return Sets.newHashSet(Tag.DEFAULT_BACKEND_TAG);
-        }
         return resourceTags;
     }
 
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/UserPropertyMgr.java
 
b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/UserPropertyMgr.java
index 7232417a2e8..46f900f655a 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/UserPropertyMgr.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/UserPropertyMgr.java
@@ -29,6 +29,7 @@ import org.apache.doris.load.DppConfig;
 import org.apache.doris.resource.Tag;
 
 import com.google.common.collect.Maps;
+import com.google.common.collect.Sets;
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 
@@ -123,7 +124,15 @@ public class UserPropertyMgr implements Writable {
         if (existProperty == null) {
             return UserProperty.INVALID_RESOURCE_TAGS;
         }
-        return existProperty.getCopiedResourceTags();
+        Set<Tag> tags = existProperty.getCopiedResourceTags();
+        // only root and admin can return empty tag.
+        // empty tag means user can access all backends.
+        // for normal user, if tag is empty, use default tag.
+        if (tags.isEmpty() && !(qualifiedUser.equalsIgnoreCase(Auth.ROOT_USER)
+                || qualifiedUser.equalsIgnoreCase(Auth.ADMIN_USER))) {
+            tags = Sets.newHashSet(Tag.DEFAULT_BACKEND_TAG);
+        }
+        return tags;
     }
 
     public Pair<String, DppConfig> getLoadClusterInfo(String qualifiedUser, 
String cluster) throws DdlException {
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
index 38ebb65c354..23cb6c572e2 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
@@ -1360,7 +1360,13 @@ public class LogicalPlanBuilder extends 
DorisParserBaseVisitor<Object> {
     public Literal visitStringLiteral(StringLiteralContext ctx) {
         String txt = ctx.STRING_LITERAL().getText();
         String s = txt.substring(1, txt.length() - 1);
-        s = s.replace("''", "'").replace("\"\"", "\"");
+        if (txt.charAt(0) == '\'') {
+            // for single quote string, '' should be converted to '
+            s = s.replace("''", "'");
+        } else if (txt.charAt(0) == '"') {
+            // for double quote string, "" should be converted to "
+            s = s.replace("\"\"", "\"");
+        }
         if (!SqlModeHelper.hasNoBackSlashEscapes()) {
             s = escapeBackSlash(s);
         }
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/hudi/HudiScanNode.java
 
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/hudi/HudiScanNode.java
index 4b8b5dcef0d..3921eb5cb89 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/hudi/HudiScanNode.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/hudi/HudiScanNode.java
@@ -223,7 +223,8 @@ public class HudiScanNode extends HiveScanNode {
 
         List<String> columnNames = new ArrayList<>();
         List<String> columnTypes = new ArrayList<>();
-        List<FieldSchema> allFields = 
hmsTable.getRemoteTable().getSd().getCols();
+        List<FieldSchema> allFields = Lists.newArrayList();
+        allFields.addAll(hmsTable.getRemoteTable().getSd().getCols());
         allFields.addAll(hmsTable.getRemoteTable().getPartitionKeys());
 
         for (Schema.Field hudiField : hudiSchema.getFields()) {
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/qe/cache/PartitionRange.java 
b/fe/fe-core/src/main/java/org/apache/doris/qe/cache/PartitionRange.java
index 0c36cb69e76..7aaf1977f64 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/cache/PartitionRange.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/cache/PartitionRange.java
@@ -153,6 +153,7 @@ public class PartitionRange {
         public boolean init(Type type, String str) {
             switch (type.getPrimitiveType()) {
                 case DATE:
+                case DATEV2:
                     try {
                         date = Date.from(
                                 LocalDate.parse(str, 
df10).atStartOfDay().atZone(ZoneId.systemDefault()).toInstant());
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java 
b/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
index 1b17d0829e2..770615e5ce4 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
@@ -128,6 +128,7 @@ import 
org.apache.doris.thrift.TGetTabletReplicaInfosRequest;
 import org.apache.doris.thrift.TGetTabletReplicaInfosResult;
 import org.apache.doris.thrift.TInitExternalCtlMetaRequest;
 import org.apache.doris.thrift.TInitExternalCtlMetaResult;
+import org.apache.doris.thrift.TInvalidateFollowerStatsCacheRequest;
 import org.apache.doris.thrift.TListPrivilegesResult;
 import org.apache.doris.thrift.TListTableMetadataNameIdsResult;
 import org.apache.doris.thrift.TListTableStatusResult;
@@ -2910,6 +2911,13 @@ public class FrontendServiceImpl implements 
FrontendService.Iface {
         return new TStatus(TStatusCode.OK);
     }
 
+    @Override
+    public TStatus invalidateStatsCache(TInvalidateFollowerStatsCacheRequest 
request) throws TException {
+        StatisticsCacheKey k = GsonUtils.GSON.fromJson(request.key, 
StatisticsCacheKey.class);
+        Env.getCurrentEnv().getStatisticsCache().invalidate(k.tableId, 
k.idxId, k.colName);
+        return new TStatus(TStatusCode.OK);
+    }
+
     public TGetMetaResult getMeta(TGetMetaRequest request) throws TException {
         String clientAddr = getClientAddrAsString();
         LOG.debug("receive get meta request: {}", request);
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisManager.java 
b/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisManager.java
index 24618fd38ca..1b9f71acbc5 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisManager.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisManager.java
@@ -711,8 +711,9 @@ public class AnalysisManager implements Writable {
             tableStats.reset();
         } else {
             dropStatsStmt.getColumnNames().forEach(tableStats::removeColumn);
+            StatisticsCache statisticsCache = 
Env.getCurrentEnv().getStatisticsCache();
             for (String col : cols) {
-                Env.getCurrentEnv().getStatisticsCache().invalidate(tblId, 
-1L, col);
+                statisticsCache.syncInvalidate(tblId, -1L, col);
             }
             tableStats.updatedTime = 0;
         }
@@ -726,9 +727,10 @@ public class AnalysisManager implements Writable {
             return;
         }
         Set<String> cols = 
table.getBaseSchema().stream().map(Column::getName).collect(Collectors.toSet());
+        StatisticsCache statisticsCache = 
Env.getCurrentEnv().getStatisticsCache();
         for (String col : cols) {
             tableStats.removeColumn(col);
-            Env.getCurrentEnv().getStatisticsCache().invalidate(table.getId(), 
-1L, col);
+            statisticsCache.syncInvalidate(table.getId(), -1L, col);
         }
         tableStats.updatedTime = 0;
         logCreateTableStats(tableStats);
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsAutoCollector.java
 
b/fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsAutoCollector.java
index bcc7f53f4d1..044eeffcccf 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsAutoCollector.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsAutoCollector.java
@@ -93,10 +93,6 @@ public class StatisticsAutoCollector extends 
StatisticsCollector {
         List<AnalysisInfo> analysisInfos = constructAnalysisInfo(databaseIf);
         for (AnalysisInfo analysisInfo : analysisInfos) {
             try {
-                if (needDropStaleStats(analysisInfo)) {
-                    
Env.getCurrentEnv().getAnalysisManager().dropStats(databaseIf.getTable(analysisInfo.tblId).get());
-                    continue;
-                }
                 analysisInfo = getReAnalyzeRequiredPart(analysisInfo);
                 if (analysisInfo == null) {
                     continue;
@@ -201,30 +197,4 @@ public class StatisticsAutoCollector extends 
StatisticsCollector {
 
         return new 
AnalysisInfoBuilder(jobInfo).setColToPartitions(needRunPartitions).build();
     }
-
-    /**
-     * Check if the given table should drop stale stats. User may truncate 
table,
-     * in this case, we need to drop the stale stats.
-     * @param jobInfo
-     * @return True if you need to drop, false otherwise.
-     */
-    protected boolean needDropStaleStats(AnalysisInfo jobInfo) {
-        TableIf table = StatisticsUtil
-                .findTable(jobInfo.catalogId, jobInfo.dbId, jobInfo.tblId);
-        if (!(table instanceof OlapTable)) {
-            return false;
-        }
-        AnalysisManager analysisManager = 
Env.getServingEnv().getAnalysisManager();
-        TableStatsMeta tblStats = 
analysisManager.findTableStatsStatus(table.getId());
-        if (tblStats == null) {
-            return false;
-        }
-        if (tblStats.analyzeColumns().isEmpty()) {
-            return false;
-        }
-        if (table.getRowCount() == 0) {
-            return true;
-        }
-        return false;
-    }
 }
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsCache.java 
b/fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsCache.java
index 84110d5bda1..d4b91b07364 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsCache.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsCache.java
@@ -27,6 +27,7 @@ import org.apache.doris.qe.ConnectContext;
 import org.apache.doris.statistics.util.StatisticsUtil;
 import org.apache.doris.system.Frontend;
 import org.apache.doris.thrift.FrontendService;
+import org.apache.doris.thrift.TInvalidateFollowerStatsCacheRequest;
 import org.apache.doris.thrift.TNetworkAddress;
 import org.apache.doris.thrift.TUpdateFollowerStatsCacheRequest;
 
@@ -138,6 +139,19 @@ public class StatisticsCache {
         columnStatisticsCache.synchronous().invalidate(new 
StatisticsCacheKey(tblId, idxId, colName));
     }
 
+    public void syncInvalidate(long tblId, long idxId, String colName) {
+        StatisticsCacheKey cacheKey = new StatisticsCacheKey(tblId, idxId, 
colName);
+        columnStatisticsCache.synchronous().invalidate(cacheKey);
+        TInvalidateFollowerStatsCacheRequest request = new 
TInvalidateFollowerStatsCacheRequest();
+        request.key = GsonUtils.GSON.toJson(cacheKey);
+        for (Frontend frontend : 
Env.getCurrentEnv().getFrontends(FrontendNodeType.FOLLOWER)) {
+            if (StatisticsUtil.isMaster(frontend)) {
+                continue;
+            }
+            invalidateStats(frontend, request);
+        }
+    }
+
     public void updateColStatsCache(long tblId, long idxId, String colName, 
ColumnStatistic statistic) {
         columnStatisticsCache.synchronous().put(new StatisticsCacheKey(tblId, 
idxId, colName), Optional.of(statistic));
     }
@@ -250,6 +264,22 @@ public class StatisticsCache {
         }
     }
 
+    @VisibleForTesting
+    public void invalidateStats(Frontend frontend, 
TInvalidateFollowerStatsCacheRequest request) {
+        TNetworkAddress address = new TNetworkAddress(frontend.getHost(), 
frontend.getRpcPort());
+        FrontendService.Client client = null;
+        try {
+            client = ClientPool.frontendPool.borrowObject(address);
+            client.invalidateStatsCache(request);
+        } catch (Throwable t) {
+            LOG.warn("Failed to sync invalidate to follower: {}", address, t);
+        } finally {
+            if (client != null) {
+                ClientPool.frontendPool.returnObject(address, client);
+            }
+        }
+    }
+
     public void putCache(StatisticsCacheKey k, ColumnStatistic c) {
         CompletableFuture<Optional<ColumnStatistic>> f = new 
CompletableFuture<Optional<ColumnStatistic>>();
         f.obtrudeValue(Optional.of(c));
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsRepository.java
 
b/fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsRepository.java
index 29e11ac75ad..4512bf1feff 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsRepository.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsRepository.java
@@ -184,6 +184,9 @@ public class StatisticsRepository {
     }
 
     public static void dropStatistics(long tblId, Set<String> colNames) throws 
DdlException {
+        if (colNames == null) {
+            return;
+        }
         dropStatisticsByColName(tblId, colNames, 
StatisticConstants.STATISTIC_TBL_NAME);
         dropStatisticsByColName(tblId, colNames, 
StatisticConstants.HISTOGRAM_TBL_NAME);
     }
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/statistics/util/StatisticsUtil.java 
b/fe/fe-core/src/main/java/org/apache/doris/statistics/util/StatisticsUtil.java
index 3f9abcad5f9..d6f61aa9ebc 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/statistics/util/StatisticsUtil.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/statistics/util/StatisticsUtil.java
@@ -787,8 +787,7 @@ public class StatisticsUtil {
             return null;
         }
         return str.replace("'", "''")
-                .replace("\\", "\\\\")
-                .replace("\"", "\"\"");
+                .replace("\\", "\\\\");
     }
 
     public static boolean isExternalTable(String catalogName, String dbName, 
String tblName) {
diff --git 
a/fe/fe-core/src/test/java/org/apache/doris/catalog/UserPropertyTest.java 
b/fe/fe-core/src/test/java/org/apache/doris/catalog/UserPropertyTest.java
index 3142d75691a..2a3252caadc 100644
--- a/fe/fe-core/src/test/java/org/apache/doris/catalog/UserPropertyTest.java
+++ b/fe/fe-core/src/test/java/org/apache/doris/catalog/UserPropertyTest.java
@@ -24,7 +24,6 @@ import org.apache.doris.common.Pair;
 import org.apache.doris.common.UserException;
 import org.apache.doris.load.DppConfig;
 import org.apache.doris.mysql.privilege.UserProperty;
-import org.apache.doris.resource.Tag;
 
 import com.google.common.collect.Lists;
 import com.google.common.collect.Sets;
@@ -118,7 +117,7 @@ public class UserPropertyTest {
         Assert.assertEquals(new String[]{"rule1", "rule2"}, 
userProperty.getSqlBlockRules());
         Assert.assertEquals(2, userProperty.getCpuResourceLimit());
         Assert.assertEquals(500, userProperty.getQueryTimeout());
-        Assert.assertEquals(Sets.newHashSet(Tag.DEFAULT_BACKEND_TAG), 
userProperty.getCopiedResourceTags());
+        Assert.assertEquals(Sets.newHashSet(), 
userProperty.getCopiedResourceTags());
 
         // fetch property
         List<List<String>> rows = userProperty.fetchProperty();
diff --git 
a/fe/fe-core/src/test/java/org/apache/doris/planner/ResourceTagQueryTest.java 
b/fe/fe-core/src/test/java/org/apache/doris/planner/ResourceTagQueryTest.java
index 9bbc7ab55eb..8bcd302a181 100644
--- 
a/fe/fe-core/src/test/java/org/apache/doris/planner/ResourceTagQueryTest.java
+++ 
b/fe/fe-core/src/test/java/org/apache/doris/planner/ResourceTagQueryTest.java
@@ -196,7 +196,7 @@ public class ResourceTagQueryTest {
         OlapTable tbl = (OlapTable) db.getTableNullable("tbl1");
 
         Set<Tag> userTags = 
Env.getCurrentEnv().getAuth().getResourceTags(Auth.ROOT_USER);
-        Assert.assertEquals(1, userTags.size());
+        Assert.assertEquals(0, userTags.size());
 
         // set default tag for root
         String setPropStr = "set property for 'root' 'resource_tags.location' 
= 'default';";
diff --git 
a/fe/fe-core/src/test/java/org/apache/doris/statistics/CacheTest.java 
b/fe/fe-core/src/test/java/org/apache/doris/statistics/CacheTest.java
index 587a5b859a1..b8e8e8df433 100644
--- a/fe/fe-core/src/test/java/org/apache/doris/statistics/CacheTest.java
+++ b/fe/fe-core/src/test/java/org/apache/doris/statistics/CacheTest.java
@@ -379,7 +379,7 @@ public class CacheTest extends TestWithFeService {
     }
 
     @Test
-    public void testEvict() {
+    public void testEvict() throws InterruptedException {
         ThreadPoolExecutor threadPool
                 = ThreadPoolManager.newDaemonFixedThreadPool(
                 1, Integer.MAX_VALUE, "STATS_FETCH", true);
@@ -400,6 +400,7 @@ public class CacheTest extends TestWithFeService {
         columnStatisticsCache.get(1);
         columnStatisticsCache.get(2);
         
Assertions.assertTrue(columnStatisticsCache.synchronous().asMap().containsKey(2));
+        Thread.sleep(100);
         Assertions.assertEquals(1, 
columnStatisticsCache.synchronous().asMap().size());
     }
 }
diff --git 
a/fe/fe-core/src/test/java/org/apache/doris/statistics/StatisticsAutoCollectorTest.java
 
b/fe/fe-core/src/test/java/org/apache/doris/statistics/StatisticsAutoCollectorTest.java
index 56475201765..fd7eaeaad90 100644
--- 
a/fe/fe-core/src/test/java/org/apache/doris/statistics/StatisticsAutoCollectorTest.java
+++ 
b/fe/fe-core/src/test/java/org/apache/doris/statistics/StatisticsAutoCollectorTest.java
@@ -27,7 +27,6 @@ import org.apache.doris.catalog.Table;
 import org.apache.doris.catalog.TableIf;
 import org.apache.doris.catalog.Type;
 import org.apache.doris.catalog.View;
-import org.apache.doris.catalog.external.ExternalTable;
 import org.apache.doris.cluster.ClusterNamespace;
 import org.apache.doris.common.Config;
 import org.apache.doris.common.DdlException;
@@ -450,77 +449,4 @@ public class StatisticsAutoCollectorTest {
             Assertions.assertNotNull(task.getTableSample());
         }
     }
-
-    @Test
-    public void testNeedDropStaleStats() {
-
-        TableIf olapTable = new OlapTable();
-        TableIf otherTable = new ExternalTable();
-
-        new MockUp<StatisticsUtil>() {
-            @Mock
-            public TableIf findTable(long catalogId, long dbId, long tblId) {
-                if (tblId == 0) {
-                    return olapTable;
-                } else {
-                    return otherTable;
-                }
-            }
-        };
-
-        new MockUp<OlapTable>() {
-            int count = 0;
-
-            int[] rowCounts = {100, 100, 100, 0, 0, 0, 0};
-            @Mock
-            public long getRowCount() {
-                return rowCounts[count++];
-            }
-
-            @Mock
-            public List<Column> getBaseSchema() {
-                return Lists.newArrayList(new Column("col1", Type.INT), new 
Column("col2", Type.INT));
-            }
-        };
-
-        AnalysisInfo analysisInfoOlap = new 
AnalysisInfoBuilder().setAnalysisMethod(AnalysisMethod.FULL)
-                .setColToPartitions(new HashMap<>())
-                .setAnalysisType(AnalysisType.FUNDAMENTALS)
-                .setColName("col1")
-                .setTblId(0)
-                .setJobType(JobType.SYSTEM).build();
-
-        new MockUp<AnalysisManager>() {
-            int count = 0;
-
-            TableStatsMeta[] tableStatsArr =
-                    new TableStatsMeta[] {null,
-                        new TableStatsMeta(0, analysisInfoOlap, olapTable),
-                        new TableStatsMeta(0, analysisInfoOlap, olapTable)};
-
-            {
-                tableStatsArr[1].updatedRows.addAndGet(100);
-                tableStatsArr[2].updatedRows.addAndGet(0);
-            }
-
-
-            @Mock
-            public TableStatsMeta findTableStatsStatus(long tblId) {
-                return tableStatsArr[count++];
-            }
-        };
-
-        AnalysisInfo analysisInfoOtherTable = new 
AnalysisInfoBuilder().setAnalysisMethod(AnalysisMethod.FULL)
-                .setColToPartitions(new HashMap<>())
-                .setAnalysisType(AnalysisType.FUNDAMENTALS)
-                .setColName("col1")
-                .setTblId(1)
-                .setJobType(JobType.SYSTEM).build();
-
-        StatisticsAutoCollector statisticsAutoCollector = new 
StatisticsAutoCollector();
-        
Assertions.assertFalse(statisticsAutoCollector.needDropStaleStats(analysisInfoOtherTable));
-        
Assertions.assertFalse(statisticsAutoCollector.needDropStaleStats(analysisInfoOlap));
-        
Assertions.assertFalse(statisticsAutoCollector.needDropStaleStats(analysisInfoOlap));
-        
Assertions.assertTrue(statisticsAutoCollector.needDropStaleStats(analysisInfoOlap));
-    }
 }
diff --git 
a/fe/fe-core/src/test/java/org/apache/doris/statistics/util/StatisticsUtilTest.java
 
b/fe/fe-core/src/test/java/org/apache/doris/statistics/util/StatisticsUtilTest.java
index c827a7d1690..724e0363833 100644
--- 
a/fe/fe-core/src/test/java/org/apache/doris/statistics/util/StatisticsUtilTest.java
+++ 
b/fe/fe-core/src/test/java/org/apache/doris/statistics/util/StatisticsUtilTest.java
@@ -34,9 +34,9 @@ import java.time.format.DateTimeFormatter;
 import java.util.ArrayList;
 import java.util.Base64;
 
-public class StatisticsUtilTest {
+class StatisticsUtilTest {
     @Test
-    public void testConvertToDouble() {
+    void testConvertToDouble() {
         try {
             //test DATE
             double date1 = StatisticsUtil.convertToDouble(Type.DATE, 
"1990-01-01");
@@ -80,7 +80,7 @@ public class StatisticsUtilTest {
     }
 
     @Test
-    public void testInAnalyzeTime1() {
+    void testInAnalyzeTime1() {
         new MockUp<StatisticsUtil>() {
 
             @Mock
@@ -99,7 +99,7 @@ public class StatisticsUtilTest {
     }
 
     @Test
-    public void testInAnalyzeTime2() {
+    void testInAnalyzeTime2() {
         new MockUp<StatisticsUtil>() {
 
             @Mock
@@ -119,7 +119,7 @@ public class StatisticsUtilTest {
 
 
     @Test
-    public void testEncodeValue() throws Exception {
+    void testEncodeValue() throws Exception {
         Assertions.assertEquals("NULL", StatisticsUtil.encodeValue(null, 0));
 
         ResultRow row = new ResultRow(null);
@@ -144,10 +144,10 @@ public class StatisticsUtilTest {
     }
 
     @Test
-    public void testEscape() {
+    void testEscape() {
         // \'"
         String origin = "\\'\"";
         // \\''""
-        Assertions.assertEquals("\\\\''\"\"", 
StatisticsUtil.escapeSQL(origin));
+        Assertions.assertEquals("\\\\''\"", StatisticsUtil.escapeSQL(origin));
     }
 }
diff --git a/gensrc/script/gen_build_version.sh 
b/gensrc/script/gen_build_version.sh
index 0b7af81e1ba..e850b33bca5 100755
--- a/gensrc/script/gen_build_version.sh
+++ b/gensrc/script/gen_build_version.sh
@@ -31,7 +31,7 @@ build_version_prefix="doris"
 build_version_major=2
 build_version_minor=0
 build_version_patch=3
-build_version_rc_version="rc04-var04"
+build_version_rc_version="rc05-var05"
 
 
build_version="${build_version_prefix}-${build_version_major}.${build_version_minor}.${build_version_patch}-${build_version_rc_version}"
 
diff --git a/gensrc/thrift/FrontendService.thrift 
b/gensrc/thrift/FrontendService.thrift
index 9c61f910678..1c19f0081cc 100644
--- a/gensrc/thrift/FrontendService.thrift
+++ b/gensrc/thrift/FrontendService.thrift
@@ -1085,6 +1085,10 @@ struct TUpdateFollowerStatsCacheRequest {
     2: list<string> statsRows;
 }
 
+struct TInvalidateFollowerStatsCacheRequest {
+    1: optional string key;
+}
+
 struct TGetMetaReplica {
     1: optional i64 id
 }
@@ -1263,4 +1267,6 @@ service FrontendService {
     TGetMetaResult getMeta(1: TGetMetaRequest request)
 
     TGetBackendMetaResult getBackendMeta(1: TGetBackendMetaRequest request)
+
+    Status.TStatus invalidateStatsCache(1: 
TInvalidateFollowerStatsCacheRequest request)
 }
diff --git a/regression-test/suites/nereids_syntax_p0/one_row_relation.groovy 
b/regression-test/suites/nereids_syntax_p0/one_row_relation.groovy
index 9b814cc4fe0..b9a43ac54de 100644
--- a/regression-test/suites/nereids_syntax_p0/one_row_relation.groovy
+++ b/regression-test/suites/nereids_syntax_p0/one_row_relation.groovy
@@ -32,10 +32,8 @@ suite("one_row_relation") {
         result([[100, "abc", "ab", "de", null]])
     }
 
-    test {
-        sql """
-            select sum(1);
-        """
-        exception "OneRowRelation can not contains any aggregate function"     
       
-    }
+    qt_string1 """ select 'A''B', 'A''''B', 'A\\'\\'B', ''; """
+    qt_string2 """ select "A""B", "A\\"\\"B", "";  """
+    qt_string3 """ select 'A""B', 'A\\"\\"B';  """
+    qt_string4 """ select "A''B", "A\\'\\'B";  """
 }
diff --git a/regression-test/suites/statistics/analyze_stats.groovy 
b/regression-test/suites/statistics/analyze_stats.groovy
index 3f0e5862ba0..e2ea11b8364 100644
--- a/regression-test/suites/statistics/analyze_stats.groovy
+++ b/regression-test/suites/statistics/analyze_stats.groovy
@@ -1248,7 +1248,6 @@ PARTITION `p599` VALUES IN (599)
 
     assert all_finished(show_result)
 
-
     // unique table update rows
     sql """
         CREATE TABLE unique_tbl_update_rows_test (col1 varchar(11451) not null,
@@ -1275,4 +1274,36 @@ PARTITION `p599` VALUES IN (599)
         return false
     }
     check_update_rows(unique_table_update_rows_result)
+
+    // Test truncate table will drop table stats too.
+    sql """ANALYZE TABLE ${tbl} WITH SYNC"""
+    def result_before_truncate = sql """show column stats ${tbl}"""
+    assertEquals(14, result_before_truncate.size())
+    sql """TRUNCATE TABLE ${tbl}"""
+    def result_after_truncate = sql """show column stats ${tbl}"""
+    assertEquals(0, result_after_truncate.size())
+    result_after_truncate = sql """show column cached stats ${tbl}"""
+    assertEquals(0, result_after_truncate.size())
+
+    sql """
+        delete from ${tbl} where analyzetestlimitedk3 >= -2147483648
+    """
+    sql """
+        INSERT INTO `${tbl}` VALUES 
(-2103297891,1,101,15248,4761818404925265645,939926.283,
+        'UTmCFKMbprf0zSVOIlBJRNOl3JcNBdOsnCDt','2022-09-28','2022-10-28 
01:56:56','tVvGDSrN6kyn',
+        
-954349107.187117,-40.46286,'11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
 [...]
+        '-1559301292834325905', NULL, NULL, NULL, NULL)
+    """
+
+    sql """
+        ANALYZE TABLE ${tbl} WITH SYNC
+    """
+
+    def truncate_test_result = sql """
+        SHOW COLUMN CACHED STATS ${tbl}(analyzetestlimitedk12)
+    """
+    assert 
"111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
 [...]
+    assert 
"111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
 [...]
+
+
 }


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

(doris) branch branch-2.0-var updated: [pick](branch-2.0) pick prs from branch-2.0 (#28176)

Reply via email to