Re: [PR] [feat](mtmv)Paimon queries the data in the cache instead of querying the latest data [doris]

[via GitHub]

github-actions[bot] commented on PR #44911:
URL: https://github.com/apache/doris/pull/44911#issuecomment-2533474352

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Job](Fix)Improve Event Publishing with Timeout [doris]

[via GitHub]

github-actions[bot] commented on PR #45103:
URL: https://github.com/apache/doris/pull/45103#issuecomment-2533488938

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix] add the window-function doc and multi-dimensional-analytics doc sql used table data [doris-website]

[via GitHub]

KassieZ commented on code in PR #1478:
URL: https://github.com/apache/doris-website/pull/1478#discussion_r1877267479


##
docs/query-data/multi-dimensional-analytics.md:
##
@@ -549,6 +549,6 @@ Using `CUBE` computes all possible aggregation levels 
(eight in this case), but
   );
   ```
 
-## Design Document
+## APPENDIX
 
-For detailed information, please refer to [SQL for Aggregation in Data 
Warehouses](https://www.oracle.com/path-to-reference-documentation) on the 
Oracle official website.
\ No newline at end of file
+For table creation statements and data files, see the [Window 
Function](window-function.md) appendix.

Review Comment:
   Please check whether the link is in the correct format.
   



##
i18n/zh-CN/docusaurus-plugin-content-docs/current/query-data/window-function.md:
##
@@ -698,39 +696,34 @@ PROPERTIES (
   "replication_num" = "1"
 );
 
-
 curl --location-trusted \
 -u "root:" \
 -H "column_separator:|" \
 -H "columns: i_item_sk, i_item_id, i_rec_start_date, i_rec_end_date, 
i_item_desc, i_current_price, i_wholesale_cost, i_brand_id, i_brand, 
i_class_id, i_class, i_category_id, i_category, i_manufact_id, i_manufact, 
i_size, i_formulation, i_color, i_units, i_container, i_manager_id, 
i_product_name" \
 -T "/path/to/data/item_1_10.dat" \
 http://127.0.0.1:8030/api/doc_tpcds/item/_stream_load
 
-
 curl --location-trusted \
 -u "root:" \
 -H "column_separator:|" \
 -H "columns: d_date_sk, d_date_id, d_date, d_month_seq, d_week_seq, 
d_quarter_seq, d_year, d_dow, d_moy, d_dom, d_qoy, d_fy_year, d_fy_quarter_seq, 
d_fy_week_seq, d_day_name, d_quarter_name, d_holiday, d_weekend, 
d_following_holiday, d_first_dom, d_last_dom, d_same_day_ly, d_same_day_lq, 
d_current_day, d_current_week, d_current_month, d_current_quarter, 
d_current_year" \
 -T "/path/to/data/date_dim_1_10.dat" \
 http://127.0.0.1:8030/api/doc_tpcds/date_dim/_stream_load
 
-
 curl --location-trusted \
 -u "root:" \
 -H "column_separator:|" \
 -H "columns: ss_sold_date_sk, ss_sold_time_sk, ss_item_sk, ss_customer_sk, 
ss_cdemo_sk, ss_hdemo_sk, ss_addr_sk, ss_store_sk, ss_promo_sk, 
ss_ticket_number, ss_quantity, ss_wholesale_cost, ss_list_price, 
ss_sales_price, ss_ext_discount_amt, ss_ext_sales_price, ss_ext_wholesale_cost, 
ss_ext_list_price, ss_ext_tax, ss_coupon_amt, ss_net_paid, ss_net_paid_inc_tax, 
ss_net_profit" \
 -T "/path/to/data/store_sales.csv" \
 http://127.0.0.1:8030/api/doc_tpcds/store_sales/_stream_load
 
-
-
 curl --location-trusted \
 -u "root:" \
 -H "column_separator:|" \
 -H "ca_address_sk, ca_address_id, ca_street_number, ca_street_name, 
ca_street_type, ca_suite_number, ca_city, ca_county, ca_state, ca_zip, 
ca_country, ca_gmt_offset, ca_location_type" \
 -T "/path/to/data/customer_address_1_10.dat" \
 http://127.0.0.1:8030/api/doc_tpcds/customer_address/_stream_load
+```
 
-数据路径：
-/Users/${username}/Documents/file/docs_write/doc_ddl_dir.tar  
-```
\ No newline at end of file
+数据文件item_1_10.dat，date_dim_1_10.dat，store_sales.csv，customer_address_1_10.dat可以从下面的链接下载：

Review Comment:
   数据文件 
`item_1_10.dat`，`date_dim_1_10.dat`，`store_sales.csv`，`customer_address_1_10.dat`
 可以[点击链接](https://cdn.selectdb.com/static/doc_ddl_dir_d27a752a7b.tar) 下载。
   
   
   1. 请用行内代码包裹
   2. 请讲链接设置成锚点，不暴露URL



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Job](Fix)Improve Event Publishing with Timeout [doris]

[via GitHub]

github-actions[bot] commented on PR #45103:
URL: https://github.com/apache/doris/pull/45103#issuecomment-2533488979

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](nereids) expression add min max scope for slot reference [doris]

[via GitHub]

yujun777 commented on code in PR #45081:
URL: https://github.com/apache/doris/pull/45081#discussion_r1879221458


##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/ExpressionOptimization.java:
##
@@ -58,6 +59,20 @@ public class ExpressionOptimization extends 
ExpressionRewrite {
 BetweenToEqual.INSTANCE
 )
 );
+
+/**
+ * don't use it with PushDownFilterThroughJoin, it may cause dead loop:
+ *   LogicalFilter(origin expr)
+ *  => LogicalFilter((origin expr) and (add min max range))
+ *  => LogicalFilter((origin expr)) // use PushDownFilterThroughJoin
+ *  => ...
+ */
+public static final List ADD_RANGE = 
ImmutableList.of(

Review Comment:
   the annotation said it cause dead loop with push down filter through join. 
so seperate it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Enhancement] (nereids)implement adminCleanTrashCommand in nereids [doris]

[via GitHub]

github-actions[bot] commented on PR #45051:
URL: https://github.com/apache/doris/pull/45051#issuecomment-2533472195

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feature] (Metric) add jni metrics for jdbc connection [doris]

[via GitHub]

dqz123 commented on PR #41752:
URL: https://github.com/apache/doris/pull/41752#issuecomment-2533480320

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Enhancement] (nereids)implement adminCleanTrashCommand in nereids [doris]

[via GitHub]

starocean999 merged PR #45051:
URL: https://github.com/apache/doris/pull/45051


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](nereids) expression add min max scope for slot reference [doris]

[via GitHub]

yujun777 commented on code in PR #45081:
URL: https://github.com/apache/doris/pull/45081#discussion_r1879223475


##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/ExpressionOptimization.java:
##
@@ -58,6 +59,20 @@ public class ExpressionOptimization extends 
ExpressionRewrite {
 BetweenToEqual.INSTANCE
 )
 );
+
+/**
+ * don't use it with PushDownFilterThroughJoin, it may cause dead loop:
+ *   LogicalFilter(origin expr)
+ *  => LogicalFilter((origin expr) and (add min max range))
+ *  => LogicalFilter((origin expr)) // use PushDownFilterThroughJoin
+ *  => ...
+ */
+public static final List ADD_RANGE = 
ImmutableList.of(

Review Comment:
   ming hong said it may add OrToIn to this rule set later



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch master updated: [Enhancement] (nereids)implement adminCleanTrashCommand in nereids (#45051)

[starocean999]

This is an automated email from the ASF dual-hosted git repository.

starocean999 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 94b44be3064 [Enhancement] (nereids)implement adminCleanTrashCommand in 
nereids (#45051)
94b44be3064 is described below

commit 94b44be3064dcb17d47190e6bb68ebadbc351ad5
Author: wudongliang <46414265+donglian...@users.noreply.github.com>
AuthorDate: Wed Dec 11 10:25:19 2024 +0800

[Enhancement] (nereids)implement adminCleanTrashCommand in nereids (#45051)

Issue Number: close #42846
---
 .../antlr4/org/apache/doris/nereids/DorisParser.g4 |   9 +-
 .../doris/nereids/parser/LogicalPlanBuilder.java   |  11 ++
 .../apache/doris/nereids/trees/plans/PlanType.java |   1 +
 .../plans/commands/AdminCleanTrashCommand.java | 120 +
 .../trees/plans/visitor/CommandVisitor.java|   5 +
 .../auth_call/test_database_management_auth.groovy |   2 +-
 .../ddl/show_trash/test_nereids_trash.groovy   |   4 +
 7 files changed, 147 insertions(+), 5 deletions(-)

diff --git a/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4 
b/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4
index fdad6baccdc..c833e7d4b62 100644
--- a/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4
+++ b/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4
@@ -183,9 +183,9 @@ supportedCreateStatement
 | CREATE (EXTERNAL)? TABLE (IF NOT EXISTS)? name=multipartIdentifier
 LIKE existedTable=multipartIdentifier
 (WITH ROLLUP (rollupNames=identifierList)?)?  
#createTableLike
-| CREATE ROLE (IF NOT EXISTS)? name=identifier (COMMENT STRING_LITERAL)?   
 #createRole  
+| CREATE ROLE (IF NOT EXISTS)? name=identifier (COMMENT STRING_LITERAL)?   
 #createRole
 | CREATE WORKLOAD GROUP (IF NOT EXISTS)?
-name=identifierOrText properties=propertyClause?   
 #createWorkloadGroup  
+name=identifierOrText properties=propertyClause?   
 #createWorkloadGroup
 | CREATE ROW POLICY (IF NOT EXISTS)? name=identifier
 ON table=multipartIdentifier
 AS type=(RESTRICTIVE | PERMISSIVE)
@@ -496,6 +496,9 @@ supportedAdminStatement
 | ADMIN COMPACT TABLE baseTableRef (WHERE TYPE EQ STRING_LITERAL)? 
 #adminCompactTable
 | ADMIN CHECK tabletList properties=propertyClause?
 #adminCheckTablets
 | ADMIN SHOW TABLET STORAGE FORMAT VERBOSE?
 #adminShowTabletStorageFormat
+| ADMIN CLEAN TRASH
+(ON LEFT_PAREN backends+=STRING_LITERAL
+  (COMMA backends+=STRING_LITERAL)* RIGHT_PAREN)?  
 #adminCleanTrash
 ;
 
 supportedRecoverStatement
@@ -517,8 +520,6 @@ unsupportedAdminStatement
 (COMMA backends+=STRING_LITERAL) RIGHT_PAREN)? 
 #adminRebalanceDisk
 | ADMIN CANCEL REBALANCE DISK (ON LEFT_PAREN backends+=STRING_LITERAL
 (COMMA backends+=STRING_LITERAL) RIGHT_PAREN)? 
 #adminCancelRebalanceDisk
-| ADMIN CLEAN TRASH (ON LEFT_PAREN backends+=STRING_LITERAL
-(COMMA backends+=STRING_LITERAL) RIGHT_PAREN)? 
 #adminCleanTrash
 | ADMIN SET TABLE name=multipartIdentifier
 PARTITION VERSION properties=propertyClause?   
 #adminSetPartitionVersion
 | ADMIN COPY TABLET tabletId=INTEGER_VALUE properties=propertyClause?  
 #adminCopyTablet
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
index 95dbbb50471..a825f80f4e0 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
@@ -477,6 +477,7 @@ import 
org.apache.doris.nereids.trees.plans.algebra.Aggregate;
 import org.apache.doris.nereids.trees.plans.algebra.SetOperation.Qualifier;
 import org.apache.doris.nereids.trees.plans.commands.AddConstraintCommand;
 import org.apache.doris.nereids.trees.plans.commands.AdminCheckTabletsCommand;
+import org.apache.doris.nereids.trees.plans.commands.AdminCleanTrashCommand;
 import org.apache.doris.nereids.trees.plans.commands.AdminCompactTableCommand;
 import 
org.apache.doris.nereids.trees.plans.commands.AdminShowReplicaStatusCommand;
 import org.apache.doris.nereids.trees.plans.commands.AlterMTMVCommand;
@@ -4579,6 +4580,16 @@ public class LogicalPlanBuilder extends 
DorisParserBaseVisitor {
 return new ShowTrashCommand();
 }
 
+@Override
+public LogicalPlan visitAdminCleanTrash(DorisParser.AdminCleanTrashContext 
ctx) {
+if (ctx.ON() != null) {
+  

Re: [I] [Enhancement] (nereids)implement adminCleanTrashCommand in nereids [doris]

[via GitHub]

starocean999 closed issue #42846: [Enhancement] (nereids)implement 
adminCleanTrashCommand in nereids
URL: https://github.com/apache/doris/issues/42846


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [I] [Enhancement] (nereids)implement adminCleanTrashCommand in nereids [doris]

[via GitHub]

starocean999 closed issue #42846: [Enhancement] (nereids)implement 
adminCleanTrashCommand in nereids
URL: https://github.com/apache/doris/issues/42846


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [I] [Bug] V2.1.6 query iceberg MOR v2 table crash [doris]

[via GitHub]

liuchunhua commented on issue #41460:
URL: https://github.com/apache/doris/issues/41460#issuecomment-2533482065

   读取iceberg v2表be经常在iceberg_read.cpp的_sort_delete_rows函数挂掉， 统计添加日志分析，发行有的data 
file会读取两次，但是结果不同。
   ``` c++
   void 
IcebergTableReader::_sort_delete_rows(std::vector*>& 
delete_rows_array,
  int64_t num_delete_rows) {
   LOG(INFO) << "num_delete_rows: " << num_delete_rows;
   LOG(INFO) << "delete_rows_array size: " << delete_rows_array.size();
   if (delete_rows_array.empty()) {
   return;
   }
   if (delete_rows_array.size() == 1) {
   LOG(INFO) << "delete_rows_array[0] size: " << 
delete_rows_array.front() ->size();
   _iceberg_delete_rows.resize(num_delete_rows);
   memcpy(&_iceberg_delete_rows[0], &((*delete_rows_array.front())[0]),
  sizeof(int64_t) * num_delete_rows);
   LOG(INFO) << "end";
   return;
   }
   if (delete_rows_array.size() == 2) {
   _iceberg_delete_rows.resize(num_delete_rows);
   std::merge(delete_rows_array.front()->begin(), 
delete_rows_array.front()->end(),
  delete_rows_array.back()->begin(), 
delete_rows_array.back()->end(),
  _iceberg_delete_rows.begin());
   LOG(INFO) << "end";
   return;
   }
   ```
   ``` log
   ~/doris$ grep -E "3833737|3833797" be/log/be.INFO
   I20241210 22:53:31.775945 3833737 iceberg_reader.cpp:302] date file path: 
oss://xx/warehouse/ods/x/data/gmt_create_month=2024-12/1-286-cc6330a4-2b6a-4e9d-b76d-b5dc5
   2a96007-1.parquet
   I20241210 22:53:31.775998 3833737 iceberg_reader.cpp:307] delete file: 
s3://xx/warehouse/ods/xx/data/gmt_create_month=2024-12/00113-5464-e7204633-b48c-4df7-83a8-f6ad27c5
   697c-2-deletes.parquet
   I20241210 22:53:31.776429 3833797 iceberg_reader.cpp:302] date file path: 
oss://xx/warehouse/ods/xx/data/gmt_create_month=2024-12/1-286-cc6330a4-2b6a-4e9d-b76d-b5dc5
   2a96007-1.parquet
   I20241210 22:53:31.776443 3833797 iceberg_reader.cpp:307] delete file: 
s3://xx/warehouse/ods/xxx/data/gmt_create_month=2024-12/00113-5464-e7204633-b48c-4df7-83a8-f6ad27c5
   697c-2-deletes.parquet
   I20241210 22:53:31.857023 3833737 iceberg_reader.cpp:307] delete file: 
s3://xxx/warehouse/ods/xxx/data/gmt_create_month=2024-12/00113-5445-33eab2a7-294d-4086-bfcd-63eac5ae
   0f5e-2-deletes.parquet
   I20241210 22:53:31.857031 3833797 iceberg_reader.cpp:307] delete file: 
s3://xxx/warehouse/ods/xx/data/gmt_create_month=2024-12/00113-5445-33eab2a7-294d-4086-bfcd-63eac5ae
   0f5e-2-deletes.parquet
   I20241210 22:53:31.915529 3833737 iceberg_reader.cpp:403] num_delete_rows: 8
   I20241210 22:53:31.915545 3833737 iceberg_reader.cpp:404] delete_rows_array 
size: 2
   I20241210 22:53:31.915551 3833737 iceberg_reader.cpp:421] end
   I20241210 22:53:31.915912 3833797 iceberg_reader.cpp:403] num_delete_rows: 3
   I20241210 22:53:31.915922 3833797 iceberg_reader.cpp:404] delete_rows_array 
size: 1
   I20241210 22:53:31.915927 3833797 iceberg_reader.cpp:409] 
delete_rows_array[0] size: 18446709446057983746
   ```
   怀疑是多线程并发问题, 通过注销掉以下代码后查询正常。
   ``` c++
   
   DeleteFile& delete_file_map = *((DeleteFile*)delete_file_cache);
   auto get_value = [&](const auto& v) {
   DeleteRows* row_ids = v.second.get();
   if (row_ids->size() > 0) {
   delete_rows_array.emplace_back(row_ids);
   num_delete_rows += row_ids->size();
   // 不释放内存
   //erase_data.emplace_back(delete_file_cache);
   }
   };
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](sql) Fix error result for column tosql [doris]

[via GitHub]

doris-robot commented on PR #44997:
URL: https://github.com/apache/doris/pull/44997#issuecomment-2533482486

   
   
   TPC-H: Total hot run time: 39836 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 7328f746960b5c65f392c203353560368f412473, 
data reload: false
   
   -- Round 1 --
   q1   17610   736273087308
   q2   2047170 165 165
   q3   10610   112612201126
   q4   10588   803 734 734
   q5   7606271027172710
   q6   235 147 145 145
   q7   999 612 610 610
   q8   9251186519451865
   q9   6644653664246424
   q10  7040228323212283
   q11  478 254 254 254
   q12  408 219 219 219
   q13  17778   308229852985
   q14  245 210 212 210
   q15  576 525 514 514
   q16  655 567 585 567
   q17  971 499 472 472
   q18  7272669867026698
   q19  13601013959 959
   q20  470 177 182 177
   q21  3990309831703098
   q22  383 327 313 313
   Total cold run time: 107216 ms
   Total hot run time: 39836 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7215717572347175
   q2   332 230 226 226
   q3   2892277329532773
   q4   2153187818581858
   q5   5683566556175617
   q6   231 142 149 142
   q7   2273182618331826
   q8   3401353134903490
   q9   8957894389418941
   q10  3600353035883530
   q11  600 503 504 503
   q12  849 616 630 616
   q13  11452   319931983198
   q14  304 267 278 267
   q15  568 528 527 527
   q16  663 651 653 651
   q17  1886165215971597
   q18  8342777575057505
   q19  1673146415051464
   q20  2090185918621859
   q21  5688548354425442
   q22  641 626 574 574
   Total cold run time: 71493 ms
   Total hot run time: 59781 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](nereids) expression add min max scope for slot reference [doris]

[via GitHub]

xzj7019 commented on code in PR #45081:
URL: https://github.com/apache/doris/pull/45081#discussion_r1879222906


##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/ExpressionOptimization.java:
##
@@ -58,6 +59,20 @@ public class ExpressionOptimization extends 
ExpressionRewrite {
 BetweenToEqual.INSTANCE
 )
 );
+
+/**
+ * don't use it with PushDownFilterThroughJoin, it may cause dead loop:
+ *   LogicalFilter(origin expr)
+ *  => LogicalFilter((origin expr) and (add min max range))
+ *  => LogicalFilter((origin expr)) // use PushDownFilterThroughJoin
+ *  => ...
+ */
+public static final List ADD_RANGE = 
ImmutableList.of(

Review Comment:
   is this possible to add other similar processing rule in the future?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Chore](brpc) add gc for abafreelist to avoid eagain and set brpc timeout_ms to 2s [doris]

[via GitHub]

eleven-f commented on PR #37888:
URL: https://github.com/apache/doris/pull/37888#issuecomment-2533465918

   version 2.1.7 error still exists
   
![image](https://github.com/user-attachments/assets/514b97de-7a4b-4757-9205-92cc3f94e019)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix] add the window-function doc and multi-dimensional-analytics doc sql used table data [doris-website]

[via GitHub]

feiniaofeiafei commented on PR #1478:
URL: https://github.com/apache/doris-website/pull/1478#issuecomment-2533481123

   run build


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [opt](ci) debug "Reach limit of connections" [doris]

[via GitHub]

hello-stephen commented on PR #45173:
URL: https://github.com/apache/doris/pull/45173#issuecomment-2533496935

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](nereids) expression add min max scope for slot reference [doris]

[via GitHub]

xzj7019 commented on code in PR #45081:
URL: https://github.com/apache/doris/pull/45081#discussion_r1879247382


##
regression-test/data/nereids_hint_tpcds_p0/shape/query13.out:
##
@@ -7,7 +7,7 @@ PhysicalResultSink
 PhysicalProject
 --hashJoin[INNER_JOIN broadcast] hashCondition=((store.s_store_sk = 
store_sales.ss_store_sk)) otherCondition=() build RFs:RF4 
s_store_sk->[ss_store_sk]
 PhysicalProject
---hashJoin[INNER_JOIN shuffle] 
hashCondition=((customer_demographics.cd_demo_sk = store_sales.ss_cdemo_sk)) 
otherCondition=(OR[AND[(household_demographics.hd_dep_count = 
1),cd_marital_status IN ('D', 'W'),cd_education_status IN ('2 yr Degree', 
'Primary'),OR[AND[(customer_demographics.cd_marital_status = 
'D'),(customer_demographics.cd_education_status = 
'Primary'),(store_sales.ss_sales_price >= 50.00),(store_sales.ss_sales_price <= 
100.00)],AND[(customer_demographics.cd_marital_status = 
'W'),(customer_demographics.cd_education_status = '2 yr 
Degree'),(store_sales.ss_sales_price >= 150.00),(store_sales.ss_sales_price <= 
200.00)]]],AND[(customer_demographics.cd_marital_status = 
'M'),(customer_demographics.cd_education_status = 
'College'),(store_sales.ss_sales_price >= 100.00),(store_sales.ss_sales_price 
<= 150.00),(household_demographics.hd_dep_count = 3)]]) build RFs:RF3 
ss_cdemo_sk->[cd_demo_sk]
+--hashJoin[INNER_JOIN shuffle] 
hashCondition=((customer_demographics.cd_demo_sk = store_sales.ss_cdemo_sk)) 
otherCondition=(OR[AND[(household_demographics.hd_dep_count = 
1),cd_marital_status IN ('D', 'W'),cd_education_status IN ('2 yr Degree', 
'Primary'),OR[AND[(customer_demographics.cd_marital_status = 
'D'),(customer_demographics.cd_education_status = 
'Primary'),(store_sales.ss_sales_price <= 
100.00)],AND[(customer_demographics.cd_marital_status = 
'W'),(customer_demographics.cd_education_status = '2 yr 
Degree'),(store_sales.ss_sales_price >= 
150.00)]]],AND[(customer_demographics.cd_marital_status = 
'M'),(customer_demographics.cd_education_status = 
'College'),(store_sales.ss_sales_price >= 100.00),(store_sales.ss_sales_price 
<= 150.00),(household_demographics.hd_dep_count = 3)]]) build RFs:RF3 
ss_cdemo_sk->[cd_demo_sk]

Review Comment:
   why >= 50.00 and <=200.00 are lost? pls check the result's correctness.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Job](Fix)Improve Event Publishing with Timeout [doris]

[via GitHub]

zddr commented on code in PR #45103:
URL: https://github.com/apache/doris/pull/45103#discussion_r1879239005


##
fe/fe-core/src/main/java/org/apache/doris/job/disruptor/TaskDisruptor.java:
##
@@ -73,20 +73,42 @@ public void start() {
  */
 public boolean publishEvent(Object... args) {
 try {
-RingBuffer ringBuffer = disruptor.getRingBuffer();
-// Check if the RingBuffer has enough capacity to reserve 10 slots 
for tasks
-// If there is insufficient capacity (less than 10 slots available)
-// log a warning and drop the current task
-if (!ringBuffer.hasAvailableCapacity(10)) {
-LOG.warn("ring buffer has no available capacity,task will be 
dropped,"
-+ "please check the task queue size.");
-return false;
+// Set the timeout to 1 second, converted to nanoseconds for 
precision
+long timeoutInNanos = TimeUnit.SECONDS.toNanos(1);  // Timeout set 
to 1 second
+long startTime = System.nanoTime();  // Record the start time
+
+// Loop until the timeout is reached
+while (System.nanoTime() - startTime < timeoutInNanos) {
+// Check if there is enough remaining capacity in the ring 
buffer
+// Adjusting to check if the required capacity is available 
(instead of hardcoding 1)
+if (disruptor.getRingBuffer().remainingCapacity() > 1) {
+// Publish the event if there is enough capacity
+disruptor.getRingBuffer().publishEvent(eventTranslator, 
args);
+if (LOG.isDebugEnabled()) {
+LOG.debug("publishEvent success,the remaining buffer 
size is {}",
+disruptor.getRingBuffer().remainingCapacity());

Review Comment:
   What is the purpose of 'remainingCapacity'？



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Job](Fix)Improve Event Publishing with Timeout [doris]

[via GitHub]

zddr commented on code in PR #45103:
URL: https://github.com/apache/doris/pull/45103#discussion_r1879236729


##
fe/fe-core/src/main/java/org/apache/doris/job/disruptor/TaskDisruptor.java:
##
@@ -73,20 +73,42 @@ public void start() {
  */
 public boolean publishEvent(Object... args) {
 try {
-RingBuffer ringBuffer = disruptor.getRingBuffer();
-// Check if the RingBuffer has enough capacity to reserve 10 slots 
for tasks
-// If there is insufficient capacity (less than 10 slots available)
-// log a warning and drop the current task
-if (!ringBuffer.hasAvailableCapacity(10)) {
-LOG.warn("ring buffer has no available capacity,task will be 
dropped,"
-+ "please check the task queue size.");
-return false;
+// Set the timeout to 1 second, converted to nanoseconds for 
precision
+long timeoutInNanos = TimeUnit.SECONDS.toNanos(1);  // Timeout set 
to 1 second
+long startTime = System.nanoTime();  // Record the start time
+
+// Loop until the timeout is reached
+while (System.nanoTime() - startTime < timeoutInNanos) {
+// Check if there is enough remaining capacity in the ring 
buffer
+// Adjusting to check if the required capacity is available 
(instead of hardcoding 1)
+if (disruptor.getRingBuffer().remainingCapacity() > 1) {
+// Publish the event if there is enough capacity

Review Comment:
   Will there be concurrency? When making a judgment, it is greater than 1, and 
when publishing, there is no capacity left



##
fe/fe-core/src/main/java/org/apache/doris/job/disruptor/TaskDisruptor.java:
##
@@ -73,20 +73,42 @@ public void start() {
  */
 public boolean publishEvent(Object... args) {
 try {
-RingBuffer ringBuffer = disruptor.getRingBuffer();
-// Check if the RingBuffer has enough capacity to reserve 10 slots 
for tasks
-// If there is insufficient capacity (less than 10 slots available)
-// log a warning and drop the current task
-if (!ringBuffer.hasAvailableCapacity(10)) {
-LOG.warn("ring buffer has no available capacity,task will be 
dropped,"
-+ "please check the task queue size.");
-return false;
+// Set the timeout to 1 second, converted to nanoseconds for 
precision
+long timeoutInNanos = TimeUnit.SECONDS.toNanos(1);  // Timeout set 
to 1 second
+long startTime = System.nanoTime();  // Record the start time
+
+// Loop until the timeout is reached
+while (System.nanoTime() - startTime < timeoutInNanos) {
+// Check if there is enough remaining capacity in the ring 
buffer
+// Adjusting to check if the required capacity is available 
(instead of hardcoding 1)
+if (disruptor.getRingBuffer().remainingCapacity() > 1) {
+// Publish the event if there is enough capacity
+disruptor.getRingBuffer().publishEvent(eventTranslator, 
args);
+if (LOG.isDebugEnabled()) {
+LOG.debug("publishEvent success,the remaining buffer 
size is {}",
+disruptor.getRingBuffer().remainingCapacity());

Review Comment:
   What is the purpose of this operation？



##
fe/fe-core/src/main/java/org/apache/doris/job/executor/TaskProcessor.java:
##
@@ -0,0 +1,87 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.job.executor;
+
+import org.apache.doris.job.task.AbstractTask;
+
+import lombok.extern.log4j.Log4j2;
+
+import java.util.concurrent.ArrayBlockingQueue;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.RejectedExecutionException;
+import java.util.concurrent.ThreadFactory;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+
+@Log4j2
+public class TaskProcessor {
+private ExecutorService executor;
+
+public  TaskProcessor(int nu

Re: [PR] [fix](sql) Fix error result for column tosql [doris]

[via GitHub]

doris-robot commented on PR #44997:
URL: https://github.com/apache/doris/pull/44997#issuecomment-2533504295

   
   
   ClickBench: Total hot run time: 32.84 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 7328f746960b5c65f392c203353560368f412473, 
data reload: false
   
   query1   0.040.030.02
   query2   0.070.030.03
   query3   0.230.070.08
   query4   1.620.100.10
   query5   0.420.400.40
   query6   1.150.650.65
   query7   0.020.010.02
   query8   0.040.030.03
   query9   0.580.520.50
   query10  0.540.570.54
   query11  0.140.100.11
   query12  0.150.110.11
   query13  0.610.620.60
   query14  2.702.732.72
   query15  0.910.830.82
   query16  0.390.380.39
   query17  1.071.051.04
   query18  0.220.210.22
   query19  1.991.902.02
   query20  0.020.010.01
   query21  15.36   0.600.60
   query22  2.372.341.89
   query23  16.99   1.040.84
   query24  3.191.001.19
   query25  0.210.120.11
   query26  0.540.140.13
   query27  0.060.050.05
   query28  10.51   1.101.08
   query29  12.57   3.283.26
   query30  0.250.070.06
   query31  2.850.380.36
   query32  3.300.460.46
   query33  3.003.033.14
   query34  17.07   4.484.46
   query35  4.524.524.50
   query36  0.670.480.51
   query37  0.090.060.06
   query38  0.050.030.03
   query39  0.030.020.02
   query40  0.160.120.13
   query41  0.070.020.02
   query42  0.030.020.02
   query43  0.040.030.02
   Total cold run time: 106.84 s
   Total hot run time: 32.84 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Job](Fix)Improve Event Publishing with Timeout [doris]

[via GitHub]

CalvinKirs commented on code in PR #45103:
URL: https://github.com/apache/doris/pull/45103#discussion_r1879248318


##
fe/fe-core/src/main/java/org/apache/doris/job/disruptor/TaskDisruptor.java:
##
@@ -73,20 +73,42 @@ public void start() {
  */
 public boolean publishEvent(Object... args) {
 try {
-RingBuffer ringBuffer = disruptor.getRingBuffer();
-// Check if the RingBuffer has enough capacity to reserve 10 slots 
for tasks
-// If there is insufficient capacity (less than 10 slots available)
-// log a warning and drop the current task
-if (!ringBuffer.hasAvailableCapacity(10)) {
-LOG.warn("ring buffer has no available capacity,task will be 
dropped,"
-+ "please check the task queue size.");
-return false;
+// Set the timeout to 1 second, converted to nanoseconds for 
precision
+long timeoutInNanos = TimeUnit.SECONDS.toNanos(1);  // Timeout set 
to 1 second
+long startTime = System.nanoTime();  // Record the start time
+
+// Loop until the timeout is reached
+while (System.nanoTime() - startTime < timeoutInNanos) {
+// Check if there is enough remaining capacity in the ring 
buffer
+// Adjusting to check if the required capacity is available 
(instead of hardcoding 1)
+if (disruptor.getRingBuffer().remainingCapacity() > 1) {
+// Publish the event if there is enough capacity

Review Comment:
   Currently, only the time wheel is in use, and since the time wheel itself is 
single-threaded, there are no issues.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [refactor](shuffle) (PART I) Unify all hash-partition based shuffling [doris]

[via GitHub]

github-actions[bot] commented on PR #45256:
URL: https://github.com/apache/doris/pull/45256#issuecomment-2533507205

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [refactor](shuffle) (PART I) Unify all hash-partition based shuffling [doris]

[via GitHub]

github-actions[bot] commented on PR #45256:
URL: https://github.com/apache/doris/pull/45256#issuecomment-2533507231

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Job](Fix)Improve Event Publishing with Timeout [doris]

[via GitHub]

CalvinKirs commented on code in PR #45103:
URL: https://github.com/apache/doris/pull/45103#discussion_r1879248885


##
fe/fe-core/src/main/java/org/apache/doris/job/disruptor/TaskDisruptor.java:
##
@@ -73,20 +73,42 @@ public void start() {
  */
 public boolean publishEvent(Object... args) {
 try {
-RingBuffer ringBuffer = disruptor.getRingBuffer();
-// Check if the RingBuffer has enough capacity to reserve 10 slots 
for tasks
-// If there is insufficient capacity (less than 10 slots available)
-// log a warning and drop the current task
-if (!ringBuffer.hasAvailableCapacity(10)) {
-LOG.warn("ring buffer has no available capacity,task will be 
dropped,"
-+ "please check the task queue size.");
-return false;
+// Set the timeout to 1 second, converted to nanoseconds for 
precision
+long timeoutInNanos = TimeUnit.SECONDS.toNanos(1);  // Timeout set 
to 1 second
+long startTime = System.nanoTime();  // Record the start time
+
+// Loop until the timeout is reached
+while (System.nanoTime() - startTime < timeoutInNanos) {
+// Check if there is enough remaining capacity in the ring 
buffer
+// Adjusting to check if the required capacity is available 
(instead of hardcoding 1)
+if (disruptor.getRingBuffer().remainingCapacity() > 1) {
+// Publish the event if there is enough capacity
+disruptor.getRingBuffer().publishEvent(eventTranslator, 
args);
+if (LOG.isDebugEnabled()) {
+LOG.debug("publishEvent success,the remaining buffer 
size is {}",
+disruptor.getRingBuffer().remainingCapacity());

Review Comment:
   Simply to add debug logs.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [regression-test](fix) fix case with invalid grammer [doris]

[via GitHub]

github-actions[bot] commented on PR #45211:
URL: https://github.com/apache/doris/pull/45211#issuecomment-2533508842

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](nereids) expression add min max scope for slot reference [doris]

[via GitHub]

xzj7019 commented on code in PR #45081:
URL: https://github.com/apache/doris/pull/45081#discussion_r1879247382


##
regression-test/data/nereids_hint_tpcds_p0/shape/query13.out:
##
@@ -7,7 +7,7 @@ PhysicalResultSink
 PhysicalProject
 --hashJoin[INNER_JOIN broadcast] hashCondition=((store.s_store_sk = 
store_sales.ss_store_sk)) otherCondition=() build RFs:RF4 
s_store_sk->[ss_store_sk]
 PhysicalProject
---hashJoin[INNER_JOIN shuffle] 
hashCondition=((customer_demographics.cd_demo_sk = store_sales.ss_cdemo_sk)) 
otherCondition=(OR[AND[(household_demographics.hd_dep_count = 
1),cd_marital_status IN ('D', 'W'),cd_education_status IN ('2 yr Degree', 
'Primary'),OR[AND[(customer_demographics.cd_marital_status = 
'D'),(customer_demographics.cd_education_status = 
'Primary'),(store_sales.ss_sales_price >= 50.00),(store_sales.ss_sales_price <= 
100.00)],AND[(customer_demographics.cd_marital_status = 
'W'),(customer_demographics.cd_education_status = '2 yr 
Degree'),(store_sales.ss_sales_price >= 150.00),(store_sales.ss_sales_price <= 
200.00)]]],AND[(customer_demographics.cd_marital_status = 
'M'),(customer_demographics.cd_education_status = 
'College'),(store_sales.ss_sales_price >= 100.00),(store_sales.ss_sales_price 
<= 150.00),(household_demographics.hd_dep_count = 3)]]) build RFs:RF3 
ss_cdemo_sk->[cd_demo_sk]
+--hashJoin[INNER_JOIN shuffle] 
hashCondition=((customer_demographics.cd_demo_sk = store_sales.ss_cdemo_sk)) 
otherCondition=(OR[AND[(household_demographics.hd_dep_count = 
1),cd_marital_status IN ('D', 'W'),cd_education_status IN ('2 yr Degree', 
'Primary'),OR[AND[(customer_demographics.cd_marital_status = 
'D'),(customer_demographics.cd_education_status = 
'Primary'),(store_sales.ss_sales_price <= 
100.00)],AND[(customer_demographics.cd_marital_status = 
'W'),(customer_demographics.cd_education_status = '2 yr 
Degree'),(store_sales.ss_sales_price >= 
150.00)]]],AND[(customer_demographics.cd_marital_status = 
'M'),(customer_demographics.cd_education_status = 
'College'),(store_sales.ss_sales_price >= 100.00),(store_sales.ss_sales_price 
<= 150.00),(household_demographics.hd_dep_count = 3)]]) build RFs:RF3 
ss_cdemo_sk->[cd_demo_sk]

Review Comment:
   why >= 50.00 and <=200.00 are lost? is it pushed down to the basic table?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](mtmv)fix concurrent problem when get partition names [doris]

[via GitHub]

zddr closed pull request #45260: [fix](mtmv)fix concurrent problem when get 
partition names
URL: https://github.com/apache/doris/pull/45260


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](mtmv)Paimon queries the data in the cache instead of querying the latest data [doris]

[via GitHub]

morrySnow merged PR #44911:
URL: https://github.com/apache/doris/pull/44911


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch master updated: [feat](mtmv)Paimon queries the data in the cache instead of querying the latest data (#44911)

[morrysnow]

This is an automated email from the ASF dual-hosted git repository.

morrysnow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 700719a93c5 [feat](mtmv)Paimon queries the data in the cache instead 
of querying the latest data (#44911)
700719a93c5 is described below

commit 700719a93c570b89761a7f0ecf78c04fd1c0a4ec
Author: zhangdong 
AuthorDate: Wed Dec 11 10:53:08 2024 +0800

[feat](mtmv)Paimon queries the data in the cache instead of querying the 
latest data (#44911)

### What problem does this PR solve?
Problem Summary:
- add `PaimonMetadataCacheMgr` in `ExternalMetaCacheMgr` to manage
snapshotCache of paimon table
- move paimonSchemaCache to PaimonMetadataCacheMgr, and add schemaId as
part of key
- PaimonExternalTable overrides the methods in ExternalTable and
supports partition pruning
- PaimonExternalTable implements the MvcTable interface, supporting the
retrieval of snapshot data from the cache during queries to avoid cache
refreshes that may result in different versions of metadata being used
in a single query
- MTMVTask retrieves snapshot data of mvccTable before the task starts
to avoid cache refresh that may result in different versions of metadata
being used in a single refresh task

### Release note
Paimon queries the data in the cache instead of querying the latest data

### behavior change
behavior changes of query  paimon table:
- FE has just started and is query the latest data
- Paimon data has changed, Doris is still query the previous data
- After the snapshot cache expires, Doris will query the latest data
- desc paimon; The schema corresponding to the snapshotId in the
snapshot cache is displayed
---
 .../apache/doris/datasource/ExternalCatalog.java   |   9 +-
 .../doris/datasource/ExternalMetaCacheMgr.java |  12 +
 .../doris/datasource/ExternalSchemaCache.java  |   6 +-
 .../org/apache/doris/datasource/ExternalTable.java |   7 +-
 .../doris/datasource/hive/HMSExternalTable.java|   5 +
 .../org/apache/doris/datasource/mvcc/MvccUtil.java |  44 +++
 .../datasource/paimon/PaimonExternalTable.java | 304 -
 .../datasource/paimon/PaimonMetadataCache.java | 144 ++
 .../datasource/paimon/PaimonMetadataCacheMgr.java  |  49 
 ...nPartitionInfo.java => PaimonMvccSnapshot.java} |  30 +-
 .../datasource/paimon/PaimonPartitionInfo.java |   4 +-
 ...{PaimonPartitionInfo.java => PaimonSchema.java} |  34 ++-
 .../datasource/paimon/PaimonSchemaCacheKey.java|  55 
 .../datasource/paimon/PaimonSchemaCacheValue.java  |  25 +-
 ...aimonPartitionInfo.java => PaimonSnapshot.java} |  32 +--
 .../datasource/paimon/PaimonSnapshotCacheKey.java  |  75 +
 ...cheValue.java => PaimonSnapshotCacheValue.java} |  37 +--
 .../apache/doris/datasource/paimon/PaimonUtil.java | 122 -
 .../datasource/paimon/source/PaimonSource.java |   3 +-
 .../apache/doris/job/extensions/mtmv/MTMVTask.java |  14 +
 .../org/apache/doris/nereids/StatementContext.java |  26 +-
 .../rules/rewrite/PruneFileScanPartition.java  |   9 +-
 .../plans/commands/UpdateMvByPartitionCommand.java |   7 +-
 .../trees/plans/logical/LogicalFileScan.java   |   7 +-
 .../data/mtmv_p0/test_paimon_rewrite_mtmv.out  |  16 ++
 .../suites/mtmv_p0/test_paimon_rewrite_mtmv.groovy |  95 +++
 26 files changed, 839 insertions(+), 332 deletions(-)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java 
b/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java
index cde08113373..d7cbee18c74 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java
@@ -41,6 +41,7 @@ import org.apache.doris.common.io.Text;
 import org.apache.doris.common.io.Writable;
 import 
org.apache.doris.common.security.authentication.PreExecutionAuthenticator;
 import org.apache.doris.common.util.Util;
+import org.apache.doris.datasource.ExternalSchemaCache.SchemaCacheKey;
 import org.apache.doris.datasource.es.EsExternalDatabase;
 import org.apache.doris.datasource.hive.HMSExternalCatalog;
 import org.apache.doris.datasource.hive.HMSExternalDatabase;
@@ -432,13 +433,13 @@ public abstract class ExternalCatalog
 }
 }
 
-public final Optional getSchema(String dbName, String 
tblName) {
+public final Optional getSchema(SchemaCacheKey key) {
 makeSureInitialized();
-Optional> db = getDb(dbName);
+Optional> db = 
getDb(key.getDbName());
 if (db.isPresent()) {
-Optional table = 
db.get().getTable(tblName);
+Optional table = 
db.get().getTable(key.getTblName());
 if (table.isPresent()) {
-return table.get().initSchemaAndU

(doris) branch auto-pick-44911-branch-3.0 created (now 635aa6c7d1d)

[github-bot]

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch auto-pick-44911-branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git


  at 635aa6c7d1d branch-3.0: [opt](test) Make the case stable #44910 
(#45007)

No new revisions were added by this update.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch auto-pick-44911-branch-2.1 created (now ca85238ccf0)

[github-bot]

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch auto-pick-44911-branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git


  at ca85238ccf0 [fix] (inverted index) Disallow variant columns from using 
inverted index format v1 (#43599) (#45179)

No new revisions were added by this update.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Enhancement] (nereids)implement adminRebalanceDiskCommand in nereids [doris]

[via GitHub]

DongLiang-0 commented on PR #45108:
URL: https://github.com/apache/doris/pull/45108#issuecomment-2533512797

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](docker)Add a BE ENV item 'SKIP_CHECK_ULIMIT' for Docker to start quickly [doris]

[via GitHub]

github-actions[bot] commented on PR #45267:
URL: https://github.com/apache/doris/pull/45267#issuecomment-2533513598

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](mtmv)use real snapshot instead of optional.empty() [doris]

[via GitHub]

zddr commented on PR #45273:
URL: https://github.com/apache/doris/pull/45273#issuecomment-2533538825

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](mtmv)use real snapshot instead of optional.empty() [doris]

[via GitHub]

doris-robot commented on PR #45273:
URL: https://github.com/apache/doris/pull/45273#issuecomment-2533538392

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting 
information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it 
now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and 
after the optimization?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](function) fixed some nested type func's param type which is not suitable and make result wrong [doris]

[via GitHub]

amorynan commented on PR #44923:
URL: https://github.com/apache/doris/pull/44923#issuecomment-2533534865

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](Nereids) fix create routine load forward strategy [doris]

[via GitHub]

LiBinfeng-01 commented on PR #45253:
URL: https://github.com/apache/doris/pull/45253#issuecomment-2533536929

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch auto-pick-45211-branch-3.0 created (now 635aa6c7d1d)

[github-bot]

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch auto-pick-45211-branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git


  at 635aa6c7d1d branch-3.0: [opt](test) Make the case stable #44910 
(#45007)

No new revisions were added by this update.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] branch-3.0: [fix](Outfile) fix the unclear error code #45119 [doris]

[via GitHub]

doris-robot commented on PR #45276:
URL: https://github.com/apache/doris/pull/45276#issuecomment-2533540619

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] branch-2.1: [fix](Outfile) add more regression-test cases for `Select into outfile` #45230 [doris]

[via GitHub]

dataroaring closed pull request #45275: branch-2.1: [fix](Outfile) add more 
regression-test cases for `Select into outfile` #45230
URL: https://github.com/apache/doris/pull/45275


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] branch-2.1: [fix](Outfile) add more regression-test cases for `Select into outfile` #45230 [doris]

[via GitHub]

doris-robot commented on PR #45275:
URL: https://github.com/apache/doris/pull/45275#issuecomment-2533540429

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] branch-2.1: [fix](Outfile) add more regression-test cases for `Select into outfile` #45230 [doris]

[via GitHub]

doris-robot commented on PR #45275:
URL: https://github.com/apache/doris/pull/45275#issuecomment-2533540375

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting 
information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it 
now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and 
after the optimization?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] branch-3.0: [fix](Outfile) fix the unclear error code #45119 [doris]

[via GitHub]

dataroaring closed pull request #45276: branch-3.0: [fix](Outfile) fix the 
unclear error code #45119
URL: https://github.com/apache/doris/pull/45276


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch auto-pick-45119-branch-3.0 updated (635aa6c7d1d -> 5d5fa4dc591)

[github-bot]

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch auto-pick-45119-branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git


from 635aa6c7d1d branch-3.0: [opt](test) Make the case stable #44910 
(#45007)
 add 5d5fa4dc591 [fix](Outfile) fix the unclear error code (#45119)

No new revisions were added by this update.

Summary of changes:
 be/src/util/arrow/utils.cpp | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[PR] branch-2.1: [fix](Outfile) add more regression-test cases for `Select into outfile` #45230 [doris]

[via GitHub]

github-actions[bot] opened a new pull request, #45275:
URL: https://github.com/apache/doris/pull/45275

   Cherry-picked from #45230


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[PR] branch-3.0: [fix](Outfile) fix the unclear error code #45119 [doris]

[via GitHub]

github-actions[bot] opened a new pull request, #45276:
URL: https://github.com/apache/doris/pull/45276

   Cherry-picked from #45119


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] branch-3.0: [fix](Nereids) move tables from connect context to statement context #44568 [doris]

[via GitHub]

LiBinfeng-01 commented on PR #44957:
URL: https://github.com/apache/doris/pull/44957#issuecomment-2533543709

   run feut


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch auto-pick-45119-branch-2.1 updated (ca85238ccf0 -> 865384edfda)

[github-bot]

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch auto-pick-45119-branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git


from ca85238ccf0 [fix] (inverted index) Disallow variant columns from using 
inverted index format v1 (#43599) (#45179)
 add 865384edfda [fix](Outfile) fix the unclear error code (#45119)

No new revisions were added by this update.

Summary of changes:
 be/src/util/arrow/utils.cpp | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](branch-2.1) Cacel the export job when the ring buffer have no enough slot [doris]

[via GitHub]

morningman commented on PR #45245:
URL: https://github.com/apache/doris/pull/45245#issuecomment-2533544452

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] branch-3.0: [fix](Outfile) add more regression-test cases for `Select into outfile` #45230 [doris]

[via GitHub]

doris-robot commented on PR #45274:
URL: https://github.com/apache/doris/pull/45274#issuecomment-2533539638

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting 
information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it 
now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and 
after the optimization?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[PR] [update] Add file_cache_statistics [doris-website]

[via GitHub]

KassieZ opened a new pull request, #1496:
URL: https://github.com/apache/doris-website/pull/1496

   ## Versions 
   
   - [ ] dev
   - [ ] 3.0
   - [ ] 2.1
   - [ ] 2.0
   
   ## Languages
   
   - [ ] Chinese
   - [ ] English
   
   ## Docs Checklist
   
   - [ ] Checked by AI
   - [ ] Test Cases Built
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](sql) Fix error result for column tosql [doris]

[via GitHub]

wyxxxcat commented on PR #44997:
URL: https://github.com/apache/doris/pull/44997#issuecomment-2533549118

   run build all


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Job](Fix)Improve Event Publishing with Timeout [doris]

[via GitHub]

CalvinKirs commented on PR #45103:
URL: https://github.com/apache/doris/pull/45103#issuecomment-2533547291

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](docker)Add a BE ENV item 'SKIP_CHECK_ULIMIT' for Docker to start quickly [doris]

[via GitHub]

github-actions[bot] commented on PR #45267:
URL: https://github.com/apache/doris/pull/45267#issuecomment-2533562932

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](Export) Cancel the export job when the ring buffer have no enough slot [doris]

[via GitHub]

morningman commented on PR #45244:
URL: https://github.com/apache/doris/pull/45244#issuecomment-2533543961

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feature](hive)support hive4 acid [doris]

[via GitHub]

hubgeter commented on code in PR #44001:
URL: https://github.com/apache/doris/pull/44001#discussion_r1879279169


##
fe/fe-core/src/main/java/org/apache/doris/datasource/hive/AcidUtil.java:
##
@@ -0,0 +1,429 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.datasource.hive;
+
+import org.apache.doris.backup.Status;
+import org.apache.doris.common.util.LocationPath;
+import org.apache.doris.datasource.hive.AcidInfo.DeleteDeltaInfo;
+import org.apache.doris.datasource.hive.HiveMetaStoreCache.FileCacheValue;
+import org.apache.doris.fs.remote.RemoteFile;
+import org.apache.doris.fs.remote.RemoteFileSystem;
+
+import lombok.EqualsAndHashCode;
+import lombok.Getter;
+import lombok.NonNull;
+import lombok.ToString;
+import org.apache.hadoop.hive.common.ValidReadTxnList;
+import org.apache.hadoop.hive.common.ValidReaderWriteIdList;
+import org.apache.hadoop.hive.common.ValidTxnList;
+import org.apache.hadoop.hive.common.ValidWriteIdList;
+import org.apache.hadoop.hive.common.ValidWriteIdList.RangeResponse;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class AcidUtil {
+private static final Logger LOG = LogManager.getLogger(AcidUtil.class);
+
+public static final String VALID_TXNS_KEY = "hive.txn.valid.txns";
+public static final String VALID_WRITEIDS_KEY = "hive.txn.valid.writeids";
+
+private static final String HIVE_TRANSACTIONAL_ORC_BUCKET_PREFIX = 
"bucket_";
+
+// An `_orc_acid_version` file is written to each base/delta/delete_delta 
dir written by a full acid write
+// or compaction.  This is the primary mechanism for versioning acid data.
+// Each individual ORC file written stores the current version as a user 
property in ORC footer. All data files
+// produced by Acid write should have this (starting with Hive 3.0), 
including those written by compactor.This is
+// more for sanity checking in case someone moved the files around or 
something like that.
+// In hive, methods for getting/reading the version from files were moved 
to test which is the only place they are
+// used (after HIVE-23506), in order to keep devs out of temptation, since 
they access the FileSystem which
+// is expensive.
+// After `HIVE-23825: Create a flag to turn off _orc_acid_version file 
creation`, introduce variables to
+// control whether to generate `_orc_acid_version` file. So don't need 
check this file exist.
+private static final String HIVE_ORC_ACID_VERSION_FILE = 
"_orc_acid_version";
+
+@Getter
+@ToString
+@EqualsAndHashCode
+private static class ParsedBase {
+private final long writeId;
+private final long visibilityId;
+
+public ParsedBase(long writeId, long visibilityId) {
+this.writeId = writeId;
+this.visibilityId = visibilityId;
+}
+}
+
+private static ParsedBase parseBase(String name) {
+//format1 : base_writeId
+//format2 : base_writeId_visibilityId  detail: 
https://issues.apache.org/jira/browse/HIVE-20823
+name = name.substring("base_".length());
+int index = name.indexOf("_v");
+if (index == -1) {
+return new ParsedBase(Long.parseLong(name), 0);
+}
+return new ParsedBase(
+Long.parseLong(name.substring(0, index)),
+Long.parseLong(name.substring(index + 2)));
+}
+
+@Getter
+@ToString
+@EqualsAndHashCode
+private static class ParsedDelta implements Comparable {
+private final long min;
+private final long max;
+private final String path;
+private final int statementId;
+private final boolean deleteDelta;
+private final long visibilityId;
+
+public ParsedDelta(long min, long max, @NonNull String path, int 
statementId,
+boolean deleteDelta, long visibilityId) {
+this.min = min;
+this.max = max;
+this.path = path;
+this.statementId = statementId;

Re: [PR] [feat](docker)Add a BE ENV item 'SKIP_CHECK_ULIMIT' for Docker to start quickly [doris]

[via GitHub]

github-actions[bot] commented on PR #45267:
URL: https://github.com/apache/doris/pull/45267#issuecomment-2533554980

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feature] (Metric) add jni metrics for jdbc connection [doris]

[via GitHub]

doris-robot commented on PR #41752:
URL: https://github.com/apache/doris/pull/41752#issuecomment-2533551854

   TeamCity be ut coverage result:
Function Coverage: 38.79% (10108/26056) 
Line Coverage: 29.70% (84766/285399)
Region Coverage: 28.76% (43508/151264)
Branch Coverage: 25.32% (22103/87292)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/99955f5c33ea3013f0954fa495ef5b54f2ff68e4_99955f5c33ea3013f0954fa495ef5b54f2ff68e4/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](docker)Add a BE ENV item 'SKIP_CHECK_ULIMIT' for Docker to start quickly [doris]

[via GitHub]

github-actions[bot] commented on PR #45267:
URL: https://github.com/apache/doris/pull/45267#issuecomment-2533558772

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch auto-pick-45230-branch-3.0 created (now 635aa6c7d1d)

[github-bot]

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch auto-pick-45230-branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git


  at 635aa6c7d1d branch-3.0: [opt](test) Make the case stable #44910 
(#45007)

No new revisions were added by this update.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](Outfile) add more regression-test cases for `Select into outfile` [doris]

[via GitHub]

morningman merged PR #45230:
URL: https://github.com/apache/doris/pull/45230


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch master updated (e8aab052bd6 -> 4ec8db51b79)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


from e8aab052bd6 [fix](Outfile) add more regression-test cases for `Select 
into outfile` (#45230)
 add 4ec8db51b79 [fix](Outfile) fix the unclear error code (#45119)

No new revisions were added by this update.

Summary of changes:
 be/src/util/arrow/utils.cpp | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[PR] branch-3.0: [fix](Outfile) add more regression-test cases for `Select into outfile` #45230 [doris]

[via GitHub]

github-actions[bot] opened a new pull request, #45274:
URL: https://github.com/apache/doris/pull/45274

   Cherry-picked from #45230


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch auto-pick-45230-branch-2.1 created (now ca85238ccf0)

[github-bot]

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch auto-pick-45230-branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git


  at ca85238ccf0 [fix] (inverted index) Disallow variant columns from using 
inverted index format v1 (#43599) (#45179)

No new revisions were added by this update.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [regression-test](fix) fix case with invalid grammer [doris]

[via GitHub]

dataroaring merged PR #45211:
URL: https://github.com/apache/doris/pull/45211


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch auto-pick-45119-branch-3.0 created (now 635aa6c7d1d)

[github-bot]

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch auto-pick-45119-branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git


  at 635aa6c7d1d branch-3.0: [opt](test) Make the case stable #44910 
(#45007)

No new revisions were added by this update.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] branch-3.0: [fix](Outfile) add more regression-test cases for `Select into outfile` #45230 [doris]

[via GitHub]

dataroaring closed pull request #45274: branch-3.0: [fix](Outfile) add more 
regression-test cases for `Select into outfile` #45230
URL: https://github.com/apache/doris/pull/45274


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch auto-pick-45230-branch-2.1 updated (ca85238ccf0 -> 347911ffc2f)

[github-bot]

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch auto-pick-45230-branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git


from ca85238ccf0 [fix] (inverted index) Disallow variant columns from using 
inverted index format v1 (#43599) (#45179)
 add 347911ffc2f [fix](Outfile) add more regression-test cases for `Select 
into outfile` (#45230)

No new revisions were added by this update.

Summary of changes:
 .../org/apache/doris/analysis/OutFileClause.java   |  6 ++
 .../export_p0/outfile/test_outfile_constant.out| 76 ++
 .../export_p0/outfile/test_outfile_null_type.out   | 12 +++
 ...ll_type.groovy => test_outfile_constant.groovy} | 92 --
 .../outfile/test_outfile_null_type.groovy  | 21 +++--
 5 files changed, 155 insertions(+), 52 deletions(-)
 create mode 100644 
regression-test/data/export_p0/outfile/test_outfile_constant.out
 copy regression-test/suites/export_p0/outfile/{test_outfile_null_type.groovy 
=> test_outfile_constant.groovy} (56%)


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch auto-pick-45230-branch-3.0 updated (635aa6c7d1d -> 54741e3b5a4)

[github-bot]

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch auto-pick-45230-branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git


from 635aa6c7d1d branch-3.0: [opt](test) Make the case stable #44910 
(#45007)
 add 54741e3b5a4 [fix](Outfile) add more regression-test cases for `Select 
into outfile` (#45230)

No new revisions were added by this update.

Summary of changes:
 .../org/apache/doris/analysis/OutFileClause.java   |  6 ++
 .../export_p0/outfile/test_outfile_constant.out| 76 ++
 .../export_p0/outfile/test_outfile_null_type.out   | 12 +++
 ...ll_type.groovy => test_outfile_constant.groovy} | 92 --
 .../outfile/test_outfile_null_type.groovy  | 21 +++--
 5 files changed, 155 insertions(+), 52 deletions(-)
 create mode 100644 
regression-test/data/export_p0/outfile/test_outfile_constant.out
 copy regression-test/suites/export_p0/outfile/{test_outfile_null_type.groovy 
=> test_outfile_constant.groovy} (56%)


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](Outfile) fix the unclear error code [doris]

[via GitHub]

morningman merged PR #45119:
URL: https://github.com/apache/doris/pull/45119


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [opt](ci) debug "Reach limit of connections" [doris]

[via GitHub]

doris-robot commented on PR #45173:
URL: https://github.com/apache/doris/pull/45173#issuecomment-2533541627

   
   
   TPC-DS: Total hot run time: 198506 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 135024cb55b716d7c17373bba1091368790a8362, 
data reload: false
   
   query1   1255970 921 921
   query2   6244208519991999
   query3   10926   447945504479
   query4   67154   27521   23599   23599
   query5   4950446 455 446
   query6   387 182 168 168
   query7   5458314 305 305
   query8   340 245 248 245
   query9   8466273227252725
   query10  432 263 250 250
   query11  16971   15443   15813   15443
   query12  166 104 99  99
   query13  1458451 427 427
   query14  10626   739776517397
   query15  208 189 202 189
   query16  7126490 453 453
   query17  1089598 590 590
   query18  1874314 342 314
   query19  204 154 159 154
   query20  118 109 116 109
   query21  208 100 104 100
   query22  4818455047044550
   query23  35089   34616   34314   34314
   query24  5443259524952495
   query25  510 385 379 379
   query26  664 153 156 153
   query27  1889276 281 276
   query28  4785250924592459
   query29  655 424 431 424
   query30  221 153 161 153
   query31  1002884 881 881
   query32  63  57  54  54
   query33  478 299 292 292
   query34  965 520 520 520
   query35  904 772 766 766
   query36  1092973 970 970
   query37  122 76  70  70
   query38  4478442944194419
   query39  1528148314771477
   query40  212 104 101 101
   query41  46  43  47  43
   query42  111 101 101 101
   query43  550 518 506 506
   query44  1248832 841 832
   query45  192 177 175 175
   query46  1202720 752 720
   query47  2080192418911891
   query48  457 342 331 331
   query49  738 415 399 399
   query50  850 394 394 394
   query51  7317712972267129
   query52  99  85  92  85
   query53  267 178 187 178
   query54  522 393 395 393
   query55  80  82  82  82
   query56  256 251 248 248
   query57  1279113511311131
   query58  229 221 216 216
   query59  3356303931493039
   query60  278 265 246 246
   query61  107 109 103 103
   query62  818 682 666 666
   query63  219 211 187 187
   query64  1481671 702 671
   query65  3258324432373237
   query66  757 304 301 301
   query67  16010   15657   15720   15657
   query68  3753584 573 573
   query69  438 258 254 254
   query70  1225115011541150
   query71  397 250 254 250
   query72  6462416339843984
   query73  789 361 357 357
   query74  10296   917391879173
   query75  3537273727372737
   query76  1910111011851110
   query77  492 299 281 281
   query78  10465   949494869486
   query79  1744584 595 584
   query80  1157420 441 420
   query81  513 236 227 227
   query82  1287120 122 120
   query83  174 149 143 143
   query84  286 69  73  69
   query85  958 300 305 300
   query86  394 292 301 292
   query87  4689456946764569
   query88  3680221721762176
   query89  429 313 291 291
   query90  1945191 190 190
   query91  150 102 106 102
   query92  64  56  53  53
   query93  1971538 529 529
   query94  806 292 294 292
   query95  359 250 256 250
   query96  633 290 274 274
   query97  2857264726772647
   query98  233 192 193 192
   query99  1586133713051305
   Total cold run time: 320064 ms
   Total hot run time: 198506 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t

Re: [PR] branch-3.0: [fix](Outfile) fix the unclear error code #45119 [doris]

[via GitHub]

doris-robot commented on PR #45276:
URL: https://github.com/apache/doris/pull/45276#issuecomment-2533540569

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting 
information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it 
now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and 
after the optimization?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[PR] branch-2.1: [fix](Outfile) fix the unclear error code #45119 [doris]

[via GitHub]

github-actions[bot] opened a new pull request, #45277:
URL: https://github.com/apache/doris/pull/45277

   Cherry-picked from #45119


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch auto-pick-45119-branch-2.1 created (now ca85238ccf0)

[github-bot]

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch auto-pick-45119-branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git


  at ca85238ccf0 [fix] (inverted index) Disallow variant columns from using 
inverted index format v1 (#43599) (#45179)

No new revisions were added by this update.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch master updated: [fix](Outfile) add more regression-test cases for `Select into outfile` (#45230)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new e8aab052bd6 [fix](Outfile) add more regression-test cases for `Select 
into outfile` (#45230)
e8aab052bd6 is described below

commit e8aab052bd6ba299d8f585e8b5d23096a924d7b8
Author: Tiewei Fang 
AuthorDate: Wed Dec 11 11:19:47 2024 +0800

[fix](Outfile) add more regression-test cases for `Select into outfile` 
(#45230)

Problem Summary:

1. add more regression-test cases for `Select into outfile`
2. Support to export the NULL type data in Doris to Orc file format.
---
 .../org/apache/doris/analysis/OutFileClause.java   |  6 ++
 .../export_p0/outfile/test_outfile_constant.out| 76 ++
 .../export_p0/outfile/test_outfile_null_type.out   | 12 +++
 ...ll_type.groovy => test_outfile_constant.groovy} | 92 --
 .../outfile/test_outfile_null_type.groovy  | 21 +++--
 5 files changed, 155 insertions(+), 52 deletions(-)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/analysis/OutFileClause.java 
b/fe/fe-core/src/main/java/org/apache/doris/analysis/OutFileClause.java
index 026e4da29b5..d05a52ec905 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/OutFileClause.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/OutFileClause.java
@@ -289,6 +289,9 @@ public class OutFileClause {
 private String dorisTypeToOrcTypeMap(Type dorisType) throws 
AnalysisException {
 String orcType = "";
 switch (dorisType.getPrimitiveType()) {
+case NULL_TYPE:
+orcType = "tinyint";
+break;
 case BOOLEAN:
 case TINYINT:
 case SMALLINT:
@@ -413,6 +416,9 @@ public class OutFileClause {
 Pair schema = this.orcSchemas.get(i);
 Type resultType = resultExprs.get(i).getType();
 switch (resultType.getPrimitiveType()) {
+case NULL_TYPE:
+checkOrcType(schema.second, "tinyint", true, 
resultType.getPrimitiveType().toString());
+break;
 case BOOLEAN:
 case TINYINT:
 case SMALLINT:
diff --git a/regression-test/data/export_p0/outfile/test_outfile_constant.out 
b/regression-test/data/export_p0/outfile/test_outfile_constant.out
new file mode 100644
index 000..855a66a129f
--- /dev/null
+++ b/regression-test/data/export_p0/outfile/test_outfile_constant.out
@@ -0,0 +1,76 @@
+-- This file is automatically generated. You should know what you did if you 
want to edit this
+-- !select_export --
+1  ftw-1   19
+2  ftw-2   20
+3  ftw-3   21
+4  ftw-4   22
+5  ftw-5   23
+6  ftw-6   24
+7  ftw-7   25
+8  ftw-8   26
+9  ftw-9   27
+10 \N  \N
+
+-- !s3_select --
+1
+
+-- !s3_select --
+3.1415926
+
+-- !s3_select --
+2.718281828459045
+
+-- !s3_select --
+3.141592653589793
+
+-- !s3_select --
+2003
+
+-- !s3_select --
+2024-12-09
+
+-- !s3_select --
+\N
+
+-- !s3_select --
+1
+
+-- !s3_select --
+3.1415926
+
+-- !s3_select --
+2.718281828459045
+
+-- !s3_select --
+3.141592653589793
+
+-- !s3_select --
+2003
+
+-- !s3_select --
+2024-12-09
+
+-- !s3_select --
+\N
+
+-- !s3_select --
+1
+
+-- !s3_select --
+3.1415926
+
+-- !s3_select --
+2.718281828459045
+
+-- !s3_select --
+3.141592653589793
+
+-- !s3_select --
+2003
+
+-- !s3_select --
+2024-12-09
+
+-- !s3_select --
+\N
+
diff --git a/regression-test/data/export_p0/outfile/test_outfile_null_type.out 
b/regression-test/data/export_p0/outfile/test_outfile_null_type.out
index 1777bba72c1..a39131b61e2 100644
--- a/regression-test/data/export_p0/outfile/test_outfile_null_type.out
+++ b/regression-test/data/export_p0/outfile/test_outfile_null_type.out
@@ -23,6 +23,18 @@
 8  ftw-8   26  \N
 9  ftw-9   27  \N
 
+-- !select_load_orc --
+1  ftw-1   19  \N
+10 \N  \N  \N
+2  ftw-2   20  \N
+3  ftw-3   21  \N
+4  ftw-4   22  \N
+5  ftw-5   23  \N
+6  ftw-6   24  \N
+7  ftw-7   25  \N
+8  ftw-8   26  \N
+9  ftw-9   27  \N
+
 -- !select_load_csv --
 1  ftw-1   19  \N
 10 \N  \N  \N
diff --git 
a/regression-test/suites/export_p0/outfile/test_outfile_null_type.groovy 
b/regression-test/suites/export_p0/outfile/test_outfile_constant.groovy
similarity index 56%
copy from regression-test/suites/export_p0/outfile/test_outfile_null_type.groovy
copy to regression-test/suites/export_p0/outfile/test_outfile_constant.groovy
index 492b6770195..3df29499237 100644
--- a/regression-test/suites/export_p0/outfile/test_outfile_null_type.groovy
+++ b/regression-test/suites/export_p0/outfile/test_outfile_constant.groovy
@@ -15,20 +15,20 @@
 // specific language governing permissions and limitations

[PR] [feat](mtmv)use real snapshot instead of optional.empty() [doris]

[via GitHub]

zddr opened a new pull request, #45273:
URL: https://github.com/apache/doris/pull/45273

   ### What problem does this PR solve?
   
   Issue Number: close #xxx
   
   Related PR: #xxx
   
   Problem Summary:
   use real snapshot instead of optional.empty()
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test 
   - [ ] Regression test
   - [ ] Unit Test
   - [ ] Manual test (add detailed scripts or steps below)
   - [ ] No need to test or manual test. Explain why:
   - [ ] This is a refactor/code format and no logic has been changed.
   - [ ] Previous test can cover this change.
   - [ ] No code files have been changed.
   - [ ] Other reason 
   
   - Behavior changed:
   - [ ] No.
   - [ ] Yes. 
   
   - Does this need documentation?
   - [ ] No.
   - [ ] Yes. 
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [opt](ci) debug "Reach limit of connections" [doris]

[via GitHub]

doris-robot commented on PR #45173:
URL: https://github.com/apache/doris/pull/45173#issuecomment-2533546700

   
   
   ClickBench: Total hot run time: 32.61 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 135024cb55b716d7c17373bba1091368790a8362, 
data reload: false
   
   query1   0.030.030.03
   query2   0.080.040.04
   query3   0.240.080.07
   query4   1.610.110.11
   query5   0.440.400.40
   query6   1.180.660.66
   query7   0.020.020.02
   query8   0.040.030.03
   query9   0.580.490.52
   query10  0.580.560.58
   query11  0.140.100.10
   query12  0.140.110.12
   query13  0.610.610.60
   query14  2.722.822.73
   query15  0.920.830.83
   query16  0.390.390.38
   query17  1.061.051.06
   query18  0.220.200.20
   query19  1.931.802.02
   query20  0.020.010.02
   query21  15.36   0.600.59
   query22  2.362.041.51
   query23  17.07   0.930.74
   query24  2.731.461.32
   query25  0.110.210.09
   query26  0.610.120.13
   query27  0.050.050.04
   query28  10.50   1.101.06
   query29  12.55   3.353.32
   query30  0.250.060.06
   query31  2.860.390.39
   query32  3.260.460.47
   query33  2.993.003.07
   query34  16.99   4.474.43
   query35  4.464.484.46
   query36  0.710.480.49
   query37  0.090.060.06
   query38  0.050.040.04
   query39  0.030.030.03
   query40  0.160.130.12
   query41  0.080.020.02
   query42  0.030.030.02
   query43  0.030.030.03
   Total cold run time: 106.28 s
   Total hot run time: 32.61 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] branch-3.0: [fix](Outfile) fix the unclear error code #45119 [doris]

[via GitHub]

github-actions[bot] commented on PR #45276:
URL: https://github.com/apache/doris/pull/45276#issuecomment-2533546708

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Job](Fix)Improve Event Publishing with Timeout [doris]

[via GitHub]

doris-robot commented on PR #45103:
URL: https://github.com/apache/doris/pull/45103#issuecomment-2533569358

   
   
   TPC-H: Total hot run time: 40061 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 68a38eeebdf392d91f874034aebcf6b5108d1999, 
data reload: false
   
   -- Round 1 --
   q1   17606   732973027302
   q2   2042179 172 172
   q3   10542   111811451118
   q4   10222   702 709 702
   q5   7593270527022702
   q6   239 148 147 147
   q7   1010620 619 619
   q8   9235187419461874
   q9   6727645464296429
   q10  6954229322732273
   q11  469 260 251 251
   q12  428 232 223 223
   q13  17786   302930123012
   q14  250 205 208 205
   q15  569 539 522 522
   q16  628 583 586 583
   q17  968 595 609 595
   q18  7507678667156715
   q19  13471010983 983
   q20  469 180 181 180
   q21  4011314031433140
   q22  394 320 314 314
   Total cold run time: 106996 ms
   Total hot run time: 40061 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7193721472257214
   q2   324 231 243 231
   q3   2871277229942772
   q4   2087185817801780
   q5   5537569556205620
   q6   217 141 142 141
   q7   2201182717821782
   q8   3329360134783478
   q9   8932897389598959
   q10  3588356735403540
   q11  591 522 506 506
   q12  819 637 624 624
   q13  12611   322431633163
   q14  298 265 272 265
   q15  564 526 525 525
   q16  690 650 616 616
   q17  1779153815531538
   q18  7820760874647464
   q19  1628147614901476
   q20  2048181718431817
   q21  5443527451175117
   q22  627 570 572 570
   Total cold run time: 71197 ms
   Total hot run time: 59198 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](cluster key) fix cluster key schema change [doris]

[via GitHub]

github-actions[bot] commented on PR #45203:
URL: https://github.com/apache/doris/pull/45203#issuecomment-2533531211

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](nereids) expression add min max scope for slot reference [doris]

[via GitHub]

starocean999 merged PR #45081:
URL: https://github.com/apache/doris/pull/45081


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [draft](mtmv) support rewrite when mv has date_trunc but query doesn't have [doris]

[via GitHub]

seawinde commented on PR #44948:
URL: https://github.com/apache/doris/pull/44948#issuecomment-2533570660

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](function) fixed some nested type func's param type which is not suitable and make result wrong [doris]

[via GitHub]

doris-robot commented on PR #44923:
URL: https://github.com/apache/doris/pull/44923#issuecomment-2533573068

   
   
   TPC-H: Total hot run time: 39785 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 8a56be8d9144a5a482634cd4771e7b7b104b8663, 
data reload: false
   
   -- Round 1 --
   q1   17597   751272787278
   q2   2040179 176 176
   q3   10683   110311511103
   q4   10552   727 685 685
   q5   7604272626292629
   q6   239 151 150 150
   q7   997 632 594 594
   q8   9239188419001884
   q9   6633649264196419
   q10  6994227323302273
   q11  468 262 270 262
   q12  432 224 234 224
   q13  17783   300730123007
   q14  247 210 238 210
   q15  558 540 526 526
   q16  663 580 610 580
   q17  988 624 534 534
   q18  7283670666496649
   q19  13551062962 962
   q20  472 177 182 177
   q21  4175326831463146
   q22  384 317 321 317
   Total cold run time: 107386 ms
   Total hot run time: 39785 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7248728672127212
   q2   349 226 221 221
   q3   2904277329932773
   q4   2050178917941789
   q5   5725563256815632
   q6   227 141 145 141
   q7   2246178218361782
   q8   3343352134523452
   q9   9095902790299027
   q10  3590357635373537
   q11  614 510 494 494
   q12  828 607 625 607
   q13  11015   335533023302
   q14  302 263 279 263
   q15  577 527 534 527
   q16  707 673 654 654
   q17  1853162516181618
   q18  8399778074707470
   q19  1697162415281528
   q20  2135188418471847
   q21  5637551754695469
   q22  673 611 563 563
   Total cold run time: 71214 ms
   Total hot run time: 59908 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](mtmv)use real snapshot instead of optional.empty() [doris]

[via GitHub]

github-actions[bot] commented on PR #45273:
URL: https://github.com/apache/doris/pull/45273#issuecomment-2533577127

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](mtmv)use real snapshot instead of optional.empty() [doris]

[via GitHub]

github-actions[bot] commented on PR #45273:
URL: https://github.com/apache/doris/pull/45273#issuecomment-2533577082

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](mtmv)use real snapshot instead of optional.empty() [doris]

[via GitHub]

doris-robot commented on PR #45273:
URL: https://github.com/apache/doris/pull/45273#issuecomment-2533579449

   
   
   TPC-H: Total hot run time: 39780 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 5477fd10f2b6525eb9fa38c30480879f50120af3, 
data reload: false
   
   -- Round 1 --
   q1   17606   746172327232
   q2   2044181 179 179
   q3   10749   107611631076
   q4   10560   696 761 696
   q5   7614276926892689
   q6   238 147 143 143
   q7   993 619 590 590
   q8   9244186519221865
   q9   6637646564156415
   q10  7001225723302257
   q11  477 265 270 265
   q12  431 220 217 217
   q13  17760   299830442998
   q14  251 204 211 204
   q15  583 527 514 514
   q16  669 592 582 582
   q17  994 527 491 491
   q18  7249675767056705
   q19  13341026999 999
   q20  470 192 185 185
   q21  3995325631523152
   q22  377 329 326 326
   Total cold run time: 107276 ms
   Total hot run time: 39780 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7231728572377237
   q2   324 228 229 228
   q3   2893280329502803
   q4   2055181618131813
   q5   5712563656505636
   q6   232 137 140 137
   q7   2233178217851782
   q8   3351354334603460
   q9   8928912090589058
   q10  3582355135553551
   q11  596 507 504 504
   q12  833 600 604 600
   q13  12018   332732473247
   q14  297 274 271 271
   q15  564 522 525 522
   q16  693 648 628 628
   q17  1858159916061599
   q18  8307764875627562
   q19  1652154716271547
   q20  2107185419091854
   q21  5692544554085408
   q22  676 578 588 578
   Total cold run time: 71834 ms
   Total hot run time: 60025 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](function) fixed some nested type func's param type which is not suitable and make result wrong [doris]

[via GitHub]

doris-robot commented on PR #44923:
URL: https://github.com/apache/doris/pull/44923#issuecomment-2533585208

   
   
   TPC-DS: Total hot run time: 196774 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 8a56be8d9144a5a482634cd4771e7b7b104b8663, 
data reload: false
   
   query1   1323972 966 966
   query2   6244203720102010
   query3   10983   435643394339
   query4   66442   28779   23483   23483
   query5   4958473 448 448
   query6   409 185 175 175
   query7   5442297 297 297
   query8   333 240 239 239
   query9   8617270526942694
   query10  421 242 244 242
   query11  17057   15279   15874   15279
   query12  160 98  97  97
   query13  1404405 408 405
   query14  10472   679668506796
   query15  222 202 198 198
   query16  7163473 460 460
   query17  1108617 638 617
   query18  2009318 307 307
   query19  219 159 160 159
   query20  122 117 122 117
   query21  203 112 109 109
   query22  4726464844374437
   query23  35021   34616   34868   34616
   query24  5482249125232491
   query25  492 383 391 383
   query26  640 156 161 156
   query27  1943280 285 280
   query28  4732247624472447
   query29  712 411 422 411
   query30  208 152 152 152
   query31  1020853 855 853
   query32  66  55  54  54
   query33  405 288 298 288
   query34  942 517 536 517
   query35  891 764 757 757
   query36  1109961 978 961
   query37  123 86  82  82
   query38  4595442345514423
   query39  1529150114721472
   query40  204 110 100 100
   query41  46  43  45  43
   query42  106 93  110 93
   query43  544 502 494 494
   query44  1232850 823 823
   query45  194 171 169 169
   query46  1202720 732 720
   query47  2016197019231923
   query48  433 326 327 326
   query49  759 386 375 375
   query50  838 407 394 394
   query51  7392722370927092
   query52  97  84  88  84
   query53  252 181 174 174
   query54  510 396 383 383
   query55  79  78  77  77
   query56  254 248 235 235
   query57  1234112611391126
   query58  213 206 224 206
   query59  3319326528672867
   query60  266 243 240 240
   query61  111 102 108 102
   query62  766 692 677 677
   query63  221 189 186 186
   query64  1401644 672 644
   query65  3273322631723172
   query66  694 299 304 299
   query67  16002   15731   15558   15558
   query68  3848552 550 550
   query69  427 270 247 247
   query70  1165115011381138
   query71  362 251 243 243
   query72  6459413239993999
   query73  776 351 347 347
   query74  10242   902691139026
   query75  3411268026802680
   query76  21261195971 971
   query77  509 276 270 270
   query78  10519   951394609460
   query79  1506597 594 594
   query80  889 438 438 438
   query81  500 230 234 230
   query82  1293127 119 119
   query83  248 150 144 144
   query84  277 73  69  69
   query85  959 311 292 292
   query86  345 301 313 301
   query87  4807456447064564
   query88  3660226421702170
   query89  423 301 289 289
   query90  2026188 193 188
   query91  136 102 109 102
   query92  70  48  51  48
   query93  1854530 531 530
   query94  791 279 288 279
   query95  347 243 243 243
   query96  623 275 273 273
   query97  2862267526742674
   query98  222 192 196 192
   query99  1738131813051305
   Total cold run time: 319025 ms
   Total hot run time: 196774 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to 

Re: [PR] [Job](Fix)Improve Event Publishing with Timeout [doris]

[via GitHub]

doris-robot commented on PR #45103:
URL: https://github.com/apache/doris/pull/45103#issuecomment-2533580182

   
   
   TPC-DS: Total hot run time: 191201 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 68a38eeebdf392d91f874034aebcf6b5108d1999, 
data reload: false
   
   query1   971 381 364 364
   query2   6508205320312031
   query3   6703234 223 223
   query4   34200   23700   23446   23446
   query5   4314497 462 462
   query6   271 187 184 184
   query7   4619305 319 305
   query8   309 244 236 236
   query9   9485270927152709
   query10  465 260 263 260
   query11  18288   15219   15317   15219
   query12  149 106 103 103
   query13  1657423 409 409
   query14  10308   684672496846
   query15  303 175 183 175
   query16  8205453 469 453
   query17  1652566 537 537
   query18  2120295 292 292
   query19  358 150 150 150
   query20  117 105 110 105
   query21  205 100 102 100
   query22  4575432342884288
   query23  35374   34194   34228   34194
   query24  11518   249925452499
   query25  682 384 383 383
   query26  1855152 155 152
   query27  2841276 277 276
   query28  8205243924672439
   query29  1038408 413 408
   query30  305 149 150 149
   query31  1136850 825 825
   query32  108 60  58  58
   query33  756 311 283 283
   query34  977 505 543 505
   query35  862 735 744 735
   query36  1094966 933 933
   query37  283 79  74  74
   query38  4420419043144190
   query39  1492143414381434
   query40  277 106 101 101
   query41  48  47  46  46
   query42  107 108 103 103
   query43  538 501 478 478
   query44  1271829 813 813
   query45  185 205 164 164
   query46  1159703 711 703
   query47  1910185318281828
   query48  411 307 315 307
   query49  1243381 382 381
   query50  797 399 392 392
   query51  7165711271997112
   query52  99  92  88  88
   query53  258 182 181 181
   query54  1088416 404 404
   query55  79  83  77  77
   query56  298 246 256 246
   query57  1268111311021102
   query58  252 229 214 214
   query59  3423311329472947
   query60  272 249 254 249
   query61  118 107 107 107
   query62  863 706 678 678
   query63  226 190 190 190
   query64  5037673 636 636
   query65  3293320932643209
   query66  1258313 307 307
   query67  15821   15595   15540   15540
   query68  4890546 553 546
   query69  433 256 297 256
   query70  11771164
   query71  338 262 249 249
   query72  6356410839903990
   query73  764 355 357 355
   query74  10209   902090739020
   query75  3452270227132702
   query76  3081112011811120
   query77  513 283 330 283
   query78  10464   943594039403
   query79  1612613 605 605
   query80  1141439 456 439
   query81  525 251 226 226
   query82  933 118 116 116
   query83  236 146 157 146
   query84  236 68  73  68
   query85  1288308 307 307
   query86  449 304 306 304
   query87  4736460146644601
   query88  3539222921812181
   query89  413 299 310 299
   query90  2014190 190 190
   query91  137 105 108 105
   query92  68  51  52  51
   query93  1592539 576 539
   query94  845 300 292 292
   query95  364 253 252 252
   query96  615 279 285 279
   query97  2867265027002650
   query98  223 191 206 191
   query99  1572130312881288
   Total cold run time: 304574 ms
   Total hot run time: 191201 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t

Re: [PR] [feat](docker)Modify the init_be and start_be scripts to meet the requirements for rapid Docker startup. [doris]

[via GitHub]

FreeOnePlus commented on PR #45269:
URL: https://github.com/apache/doris/pull/45269#issuecomment-2533589379

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](docker)Add a BE ENV item 'SKIP_CHECK_ULIMIT' for Docker to start quickly [doris]

[via GitHub]

github-actions[bot] commented on PR #45267:
URL: https://github.com/apache/doris/pull/45267#issuecomment-2533578746

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Job](Fix)Improve Event Publishing with Timeout [doris]

[via GitHub]

doris-robot commented on PR #45103:
URL: https://github.com/apache/doris/pull/45103#issuecomment-2533585889

   
   
   ClickBench: Total hot run time: 33.22 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 68a38eeebdf392d91f874034aebcf6b5108d1999, 
data reload: false
   
   query1   0.030.030.03
   query2   0.060.030.04
   query3   0.240.080.06
   query4   1.620.100.10
   query5   0.430.400.42
   query6   1.160.650.66
   query7   0.010.020.01
   query8   0.040.040.03
   query9   0.570.530.50
   query10  0.560.580.57
   query11  0.140.110.10
   query12  0.140.100.12
   query13  0.610.600.60
   query14  2.722.722.74
   query15  0.910.830.82
   query16  0.370.380.39
   query17  1.021.021.04
   query18  0.230.220.23
   query19  1.931.811.94
   query20  0.010.010.01
   query21  15.36   0.580.58
   query22  2.712.342.56
   query23  16.97   1.020.87
   query24  2.641.111.16
   query25  0.400.070.05
   query26  0.440.150.13
   query27  0.050.040.04
   query28  10.87   1.101.07
   query29  12.60   3.233.23
   query30  0.250.070.08
   query31  2.850.390.38
   query32  3.300.460.46
   query33  2.983.053.03
   query34  17.22   4.474.47
   query35  4.504.504.46
   query36  0.640.500.48
   query37  0.090.060.06
   query38  0.040.030.03
   query39  0.030.020.02
   query40  0.170.130.12
   query41  0.080.030.02
   query42  0.030.020.02
   query43  0.040.020.03
   Total cold run time: 107.06 s
   Total hot run time: 33.22 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](mtmv)use real snapshot instead of optional.empty() [doris]

[via GitHub]

doris-robot commented on PR #45273:
URL: https://github.com/apache/doris/pull/45273#issuecomment-2533591654

   
   
   TPC-DS: Total hot run time: 197950 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 5477fd10f2b6525eb9fa38c30480879f50120af3, 
data reload: false
   
   query1   1272992 963 963
   query2   6248214420052005
   query3   11024   460446424604
   query4   33317   23813   23616   23616
   query5   3510485 456 456
   query6   291 195 192 192
   query7   3988296 303 296
   query8   293 228 242 228
   query9   9618271026982698
   query10  461 248 262 248
   query11  18193   15313   15655   15313
   query12  151 98  96  96
   query13  1570400 394 394
   query14  9604754478507544
   query15  293 218 205 205
   query16  7961456 463 456
   query17  1739640 593 593
   query18  1986332 318 318
   query19  393 164 161 161
   query20  119 119 117 117
   query21  214 107 111 107
   query22  4860442946694429
   query23  35384   34452   34678   34452
   query24  10610   248424292429
   query25  598 379 391 379
   query26  1163152 149 149
   query27  2552286 288 286
   query28  7754245224902452
   query29  807 412 463 412
   query30  243 151 153 151
   query31  1051842 863 842
   query32  99  55  51  51
   query33  782 295 303 295
   query34  998 508 529 508
   query35  930 766 797 766
   query36  1121966 947 947
   query37  132 76  76  76
   query38  4501450243934393
   query39  1501148414691469
   query40  211 114 104 104
   query41  45  43  44  43
   query42  111 100 96  96
   query43  547 514 483 483
   query44  1329840 816 816
   query45  198 169 172 169
   query46  1229738 719 719
   query47  2003194418931893
   query48  425 322 323 322
   query49  1021390 375 375
   query50  813 405 385 385
   query51  7371731073817310
   query52  110 92  94  92
   query53  268 196 183 183
   query54  1205410 403 403
   query55  83  79  76  76
   query56  253 229 245 229
   query57  1267111211231112
   query58  231 209 205 205
   query59  3357301328212821
   query60  265 241 246 241
   query61  106 108 138 108
   query62  887 686 668 668
   query63  211 193 190 190
   query64  3991658 645 645
   query65  3294317931943179
   query66  837 298 297 297
   query67  16169   15662   15617   15617
   query68  4963543 544 543
   query69  429 250 255 250
   query70  1241113011421130
   query71  337 240 250 240
   query72  6402415440644064
   query73  773 354 355 354
   query74  10452   911090449044
   query75  3417265526692655
   query76  2769104710661047
   query77  383 284 283 283
   query78  10647   947994749474
   query79  1505594 592 592
   query80  1193445 425 425
   query81  546 233 238 233
   query82  863 117 164 117
   query83  240 143 148 143
   query84  225 64  73  64
   query85  1511303 307 303
   query86  480 258 304 258
   query87  4712454645754546
   query88  3630220721622162
   query89  417 287 293 287
   query90  2076189 182 182
   query91  136 101 108 101
   query92  68  49  51  49
   query93  2301539 537 537
   query94  1050362 295 295
   query95  356 251 250 250
   query96  616 282 280 280
   query97  2854268027242680
   query98  238 193 206 193
   query99  1624135912921292
   Total cold run time: 303622 ms
   Total hot run time: 197950 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to

Re: [PR] [feat](docker)Add a BE ENV item 'SKIP_CHECK_ULIMIT' for Docker to start quickly [doris]

[via GitHub]

FreeOnePlus commented on PR #45267:
URL: https://github.com/apache/doris/pull/45267#issuecomment-2533594503

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](mtmv)use real snapshot instead of optional.empty() [doris]

[via GitHub]

doris-robot commented on PR #45273:
URL: https://github.com/apache/doris/pull/45273#issuecomment-2533597049

   
   
   ClickBench: Total hot run time: 32.53 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 5477fd10f2b6525eb9fa38c30480879f50120af3, 
data reload: false
   
   query1   0.030.030.03
   query2   0.070.040.03
   query3   0.230.080.06
   query4   1.620.100.10
   query5   0.430.380.40
   query6   1.140.660.65
   query7   0.020.020.02
   query8   0.040.030.03
   query9   0.570.520.50
   query10  0.550.570.55
   query11  0.140.110.11
   query12  0.150.130.12
   query13  0.610.600.60
   query14  2.702.742.71
   query15  0.900.840.83
   query16  0.400.370.38
   query17  1.061.081.07
   query18  0.220.210.21
   query19  1.901.911.94
   query20  0.010.010.01
   query21  15.38   0.580.57
   query22  2.802.861.37
   query23  16.89   1.100.86
   query24  2.461.471.19
   query25  0.140.190.17
   query26  0.370.140.13
   query27  0.050.040.03
   query28  10.57   1.111.07
   query29  12.59   3.263.25
   query30  0.250.070.06
   query31  2.880.390.38
   query32  3.260.480.47
   query33  3.093.033.06
   query34  16.92   4.424.42
   query35  4.534.504.47
   query36  0.660.480.47
   query37  0.100.060.06
   query38  0.050.030.03
   query39  0.040.030.02
   query40  0.170.120.13
   query41  0.080.020.03
   query42  0.030.020.02
   query43  0.030.030.03
   Total cold run time: 106.13 s
   Total hot run time: 32.53 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Enhancement] (nereids)implement adminSetTableStatusCommand in nereids [doris]

[via GitHub]

Vallishp commented on PR #45272:
URL: https://github.com/apache/doris/pull/45272#issuecomment-2533596973

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feat](nereids) expression add min max scope for slot reference [doris]

[via GitHub]

yujun777 commented on code in PR #45081:
URL: https://github.com/apache/doris/pull/45081#discussion_r1879312615


##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/AddMinMax.java:
##
@@ -0,0 +1,378 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rules;
+
+import org.apache.doris.nereids.rules.expression.ExpressionPatternMatcher;
+import org.apache.doris.nereids.rules.expression.ExpressionPatternRuleFactory;
+import org.apache.doris.nereids.rules.expression.ExpressionRewriteContext;
+import 
org.apache.doris.nereids.rules.expression.rules.RangeInference.DiscreteValue;
+import 
org.apache.doris.nereids.rules.expression.rules.RangeInference.EmptyValue;
+import 
org.apache.doris.nereids.rules.expression.rules.RangeInference.RangeValue;
+import 
org.apache.doris.nereids.rules.expression.rules.RangeInference.UnknownValue;
+import 
org.apache.doris.nereids.rules.expression.rules.RangeInference.ValueDesc;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.SlotReference;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.util.ExpressionUtils;
+
+import com.google.common.collect.BoundType;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+import com.google.common.collect.Range;
+import com.google.common.collect.Sets;
+import org.apache.commons.lang3.NotImplementedException;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * This class implements the function to add min max to or expression.
+ * for example:

Review Comment:
   add test later



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix] (inverted index) fix index file in gc binlogs [doris]

[via GitHub]

github-actions[bot] commented on code in PR #44994:
URL: https://github.com/apache/doris/pull/44994#discussion_r1879312363


##
be/src/http/action/download_binlog_action.cpp:
##
@@ -144,8 +144,19 @@ void handle_get_segment_index_file(StorageEngine& engine, 
HttpRequest* req,
 const auto& rowset_id = get_http_param(req, kRowsetIdParameter);
 const auto& segment_index = get_http_param(req, 
kSegmentIndexParameter);
 const auto& segment_index_id = req->param(kSegmentIndexIdParameter);
-segment_index_file_path =
-tablet->get_segment_index_filepath(rowset_id, segment_index, 
segment_index_id);
+auto segment_file_path = tablet->get_segment_filepath(rowset_id, 
segment_index);
+if (tablet->tablet_schema()->get_inverted_index_storage_format() ==
+InvertedIndexStorageFormatPB::V1) {
+// now CCR not support for variant + index v1
+constexpr std::string_view index_suffix = "";

Review Comment:
   warning: redundant string initialization [readability-redundant-string-init]
   
   ```suggestion
   constexpr std::string_view index_suffix;
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [regression-test](framework) try to disable defining global variable in test groovy files [doris]

[via GitHub]

shuke987 commented on PR #44576:
URL: https://github.com/apache/doris/pull/44576#issuecomment-2533601963

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org