Re: [PR] [Improment]Add schema table workload_group_privileges [doris]
github-actions[bot] commented on code in PR #38436:
URL: https://github.com/apache/doris/pull/38436#discussion_r1694125324
##
be/src/exec/schema_scanner/schema_workload_group_privileges.cpp:
##
@@ -0,0 +1,134 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "exec/schema_scanner/schema_workload_group_privileges.h"
+
+#include "runtime/client_cache.h"
+#include "runtime/exec_env.h"
+#include "runtime/runtime_state.h"
+#include "util/thrift_rpc_helper.h"
+#include "vec/common/string_ref.h"
+#include "vec/core/block.h"
+#include "vec/data_types/data_type_factory.hpp"
+
+namespace doris {
+std::vector
SchemaWorkloadGroupPrivilegesScanner::_s_tbls_columns = {
+{"GRANTEE", TYPE_VARCHAR, sizeof(StringRef), true},
+{"WORKLOAD_GROUP_NAME", TYPE_VARCHAR, sizeof(StringRef), true},
+{"PRIVILEGE_TYPE", TYPE_VARCHAR, sizeof(StringRef), true},
+{"IS_GRANTABLE", TYPE_VARCHAR, sizeof(StringRef), true},
+};
+
+SchemaWorkloadGroupPrivilegesScanner::SchemaWorkloadGroupPrivilegesScanner()
+: SchemaScanner(_s_tbls_columns,
TSchemaTableType::SCH_WORKLOAD_GROUPS) {}
+
+SchemaWorkloadGroupPrivilegesScanner::~SchemaWorkloadGroupPrivilegesScanner()
{}
Review Comment:
warning: use '= default' to define a trivial destructor
[modernize-use-equals-default]
```suggestion
SchemaWorkloadGroupPrivilegesScanner::~SchemaWorkloadGroupPrivilegesScanner() =
default;
```
##
be/src/exec/schema_scanner/schema_workload_group_privileges.cpp:
##
@@ -0,0 +1,134 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "exec/schema_scanner/schema_workload_group_privileges.h"
+
+#include "runtime/client_cache.h"
+#include "runtime/exec_env.h"
+#include "runtime/runtime_state.h"
+#include "util/thrift_rpc_helper.h"
+#include "vec/common/string_ref.h"
+#include "vec/core/block.h"
+#include "vec/data_types/data_type_factory.hpp"
+
+namespace doris {
+std::vector
SchemaWorkloadGroupPrivilegesScanner::_s_tbls_columns = {
+{"GRANTEE", TYPE_VARCHAR, sizeof(StringRef), true},
+{"WORKLOAD_GROUP_NAME", TYPE_VARCHAR, sizeof(StringRef), true},
+{"PRIVILEGE_TYPE", TYPE_VARCHAR, sizeof(StringRef), true},
+{"IS_GRANTABLE", TYPE_VARCHAR, sizeof(StringRef), true},
+};
+
+SchemaWorkloadGroupPrivilegesScanner::SchemaWorkloadGroupPrivilegesScanner()
Review Comment:
warning: use '= default' to define a trivial default constructor
[modernize-use-equals-default]
be/src/exec/schema_scanner/schema_workload_group_privileges.cpp:36:
```diff
- : SchemaScanner(_s_tbls_columns,
TSchemaTableType::SCH_WORKLOAD_GROUPS) {}
+ : SchemaScanner(_s_tbls_columns,
TSchemaTableType::SCH_WORKLOAD_GROUPS) = default;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](profile) Fix read load profile failed [doris]
zhiqiang- opened a new pull request, #38437: URL: https://github.com/apache/doris/pull/38437 Profile ID of load task is a long string, `DebugUtil.parseTUniqueIdFromString` will throw exception instead of null when we pass long string to it. So we should handle the exception inside of parseProfileFileName. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](profile) Fix read load profile failed [doris]
doris-robot commented on PR #38437: URL: https://github.com/apache/doris/pull/38437#issuecomment-2254390071 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](profile) Fix read load profile failed [doris]
zhiqiang- commented on PR #38437: URL: https://github.com/apache/doris/pull/38437#issuecomment-2254390082 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [DEBUG] debug hist [doris]
zhiqiang- commented on PR #38242: URL: https://github.com/apache/doris/pull/38242#issuecomment-2254391492 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improment]Add schema table workload_group_privileges [doris]
doris-robot commented on PR #38436: URL: https://github.com/apache/doris/pull/38436#issuecomment-2254395140 TPC-H: Total hot run time: 39494 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 6c95b9a2c9f23e20e35eedf63d670fce6458db54, data reload: false -- Round 1 -- q1 17962 451643154315 q2 2011205 201 201 q3 10494 122211561156 q4 10159 791 709 709 q5 7530272926822682 q6 224 139 140 139 q7 957 596 612 596 q8 9275195719281928 q9 8747656965656565 q10 8832380538083805 q11 448 253 247 247 q12 393 230 215 215 q13 17763 300329972997 q14 275 235 242 235 q15 526 487 488 487 q16 518 378 379 378 q17 983 723 736 723 q18 8234738673697369 q19 6082985 941 941 q20 690 340 351 340 q21 5034317532333175 q22 354 291 299 291 Total cold run time: 117491 ms Total hot run time: 39494 ms - Round 2, with runtime_filter_mode=off - q1 4429425243094252 q2 381 273 270 270 q3 3031282129082821 q4 1995173716591659 q5 5732554655415541 q6 222 132 132 132 q7 2210184518901845 q8 3268342234203420 q9 8843878688838786 q10 4131395737883788 q11 599 511 497 497 q12 811 635 631 631 q13 16263 318631763176 q14 332 292 290 290 q15 539 490 496 490 q16 477 447 439 439 q17 1804152315091509 q18 8225813278047804 q19 1709171415721572 q20 2180192818781878 q21 5047498546214621 q22 639 529 505 505 Total cold run time: 72867 ms Total hot run time: 55926 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improment]Add schema table workload_group_privileges [doris]
doris-robot commented on PR #38436: URL: https://github.com/apache/doris/pull/38436#issuecomment-2254398235 TPC-DS: Total hot run time: 172902 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 6c95b9a2c9f23e20e35eedf63d670fce6458db54, data reload: false query1 916 381 369 369 query2 6437195318901890 query3 6651209 222 209 query4 28177 17475 17442 17442 query5 3663498 503 498 query6 260 184 160 160 query7 4582292 289 289 query8 239 195 197 195 query9 8455248024502450 query10 438 294 293 293 query11 12206 10056 10048 10048 query12 122 98 87 87 query13 1641383 383 383 query14 10057 777269786978 query15 213 170 168 168 query16 7553498 472 472 query17 1511539 524 524 query18 1936273 287 273 query19 190 141 140 140 query20 91 87 83 83 query21 207 112 98 98 query22 4287402838403840 query23 34097 33846 33635 33635 query24 10776 290429202904 query25 604 391 411 391 query26 708 151 152 151 query27 2225282 286 282 query28 6280211021262110 query29 806 461 418 418 query30 254 157 157 157 query31 970 765 745 745 query32 97 53 59 53 query33 662 337 338 337 query34 910 493 497 493 query35 892 748 779 748 query36 1147978 963 963 query37 134 82 83 82 query38 2986289829182898 query39 907 850 852 850 query40 197 111 114 111 query41 45 49 42 42 query42 112 101 95 95 query43 505 476 473 473 query44 1095729 729 729 query45 213 177 177 177 query46 1085750 719 719 query47 1851177417681768 query48 382 291 297 291 query49 838 414 425 414 query50 792 396 401 396 query51 6877665266626652 query52 95 93 93 93 query53 259 187 181 181 query54 851 455 444 444 query55 77 74 75 74 query56 314 283 284 283 query57 1133105110201020 query58 262 260 273 260 query59 2852262926642629 query60 310 286 285 285 query61 96 93 95 93 query62 798 647 615 615 query63 212 181 184 181 query64 9208224717911791 query65 3165307531103075 query66 758 341 353 341 query67 15322 14828 14990 14828 query68 4504563 567 563 query69 600 435 339 339 query70 1143113911401139 query71 439 284 279 279 query72 7771538758335387 query73 771 342 337 337 query74 6085565957015659 query75 3383267127402671 query76 2648933 845 845 query77 636 317 305 305 query78 9732895289158915 query79 2582537 515 515 query80 2747490 496 490 query81 606 222 221 221 query82 1388149 141 141 query83 311 176 176 176 query84 263 87 79 79 query85 1601327 305 305 query86 475 308 320 308 query87 3250312030943094 query88 4187249024902490 query89 390 285 286 285 query90 1974193 203 193 query91 128 154 108 108 query92 67 49 49 49 query93 2399549 548 548 query94 1040300 304 300 query95 357 262 260 260 query96 606 271 272 271 query97 3215302030293020 query98 218 209 194 194 query99 1539125012651250 Total cold run time: 281046 ms Total hot run time: 172902 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [fix](profile) Fix read load profile failed [doris]
zhiqiang- commented on PR #38437: URL: https://github.com/apache/doris/pull/38437#issuecomment-2254398652 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improment]Add schema table workload_group_privileges [doris]
doris-robot commented on PR #38436: URL: https://github.com/apache/doris/pull/38436#issuecomment-2254400812 ClickBench: Total hot run time: 31.11 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 6c95b9a2c9f23e20e35eedf63d670fce6458db54, data reload: false query1 0.040.040.03 query2 0.070.040.04 query3 0.220.050.05 query4 1.670.080.09 query5 0.500.500.48 query6 1.130.720.72 query7 0.020.010.02 query8 0.060.040.04 query9 0.540.490.48 query10 0.550.550.55 query11 0.160.110.11 query12 0.140.120.13 query13 0.600.580.60 query14 0.770.800.77 query15 0.850.820.83 query16 0.380.370.37 query17 0.960.980.97 query18 0.220.220.22 query19 1.901.761.85 query20 0.010.000.01 query21 15.42 0.740.66 query22 4.656.252.36 query23 18.31 1.441.22 query24 2.080.250.24 query25 0.140.090.08 query26 0.290.210.21 query27 0.450.220.23 query28 13.19 1.000.99 query29 12.61 3.323.31 query30 0.250.060.06 query31 2.860.410.39 query32 3.280.490.47 query33 2.942.932.88 query34 16.92 4.374.35 query35 4.444.434.44 query36 0.660.470.46 query37 0.190.150.16 query38 0.150.150.14 query39 0.040.030.03 query40 0.150.120.12 query41 0.100.050.05 query42 0.050.050.05 query43 0.050.040.04 Total cold run time: 110.01 s Total hot run time: 31.11 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [opt](memory) Optimize `ClearCacheAction`implementation [doris]
xinyiZzz opened a new pull request, #38438: URL: https://github.com/apache/doris/pull/38438 1. `ClearCacheAction` support for specifying cache. 2. Optimize result returned by `ClearCacheAction`. 3. Optimize Allocator large memory printing stacktrace 4. `cache_prune_stale_thread` no longer check conf to prune cache, which has delays and brings a bad experience. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](memory) Optimize `ClearCacheAction`implementation [doris]
xinyiZzz commented on PR #38438: URL: https://github.com/apache/doris/pull/38438#issuecomment-2254402385 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](memory) Optimize `ClearCacheAction`implementation [doris]
doris-robot commented on PR #38438: URL: https://github.com/apache/doris/pull/38438#issuecomment-2254402380 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](profile) Fix read load profile failed [doris]
doris-robot commented on PR #38437: URL: https://github.com/apache/doris/pull/38437#issuecomment-2254404635 TPC-H: Total hot run time: 38979 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit e3c905dcb1444f4653866fc6a2ef9690048a5be6, data reload: false -- Round 1 -- q1 17627 430242454245 q2 2009198 195 195 q3 10431 122210841084 q4 10142 663 731 663 q5 7538271226392639 q6 218 139 140 139 q7 948 624 586 586 q8 9319191718701870 q9 8692659765096509 q10 8829380037663766 q11 475 243 239 239 q12 537 217 216 216 q13 18798 296429702964 q14 283 235 243 235 q15 529 483 488 483 q16 504 380 378 378 q17 962 695 699 695 q18 8037741173447344 q19 5527978 1060978 q20 654 334 328 328 q21 5030313032703130 q22 357 293 294 293 Total cold run time: 117446 ms Total hot run time: 38979 ms - Round 2, with runtime_filter_mode=off - q1 4422428542184218 q2 388 276 268 268 q3 3004272928882729 q4 1958171016851685 q5 5622552955175517 q6 220 133 135 133 q7 2142182618981826 q8 3271343733933393 q9 8767877687698769 q10 4086387538243824 q11 578 505 477 477 q12 798 615 670 615 q13 16707 315331443144 q14 318 304 293 293 q15 531 498 496 496 q16 468 434 432 432 q17 1811149814991498 q18 8034784677757775 q19 1735163516261626 q20 2047192618681868 q21 9633486747754775 q22 565 499 543 499 Total cold run time: 77105 ms Total hot run time: 55860 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](memory) Optimize `ClearCacheAction`implementation [doris]
github-actions[bot] commented on PR #38438: URL: https://github.com/apache/doris/pull/38438#issuecomment-2254404759 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [DEBUG] debug hist [doris]
doris-robot commented on PR #38242: URL: https://github.com/apache/doris/pull/38242#issuecomment-2254404974 TPC-H: Total hot run time: 39727 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit acd50f8a804ccf91c96958bf7f89f8ddb3a4d50c, data reload: false -- Round 1 -- q1 17625 429242924292 q2 2008196 195 195 q3 10624 1216 q4 10145 746 706 706 q5 7522270526892689 q6 225 138 140 138 q7 961 604 597 597 q8 9291193019041904 q9 9006660965536553 q10 8962386638093809 q11 480 256 255 255 q12 423 230 225 225 q13 18975 304530313031 q14 286 241 259 241 q15 524 489 514 489 q16 505 398 385 385 q17 961 731 721 721 q18 8216749974537453 q19 70761067987 987 q20 738 359 371 359 q21 5133329541223295 q22 360 300 292 292 Total cold run time: 120046 ms Total hot run time: 39727 ms - Round 2, with runtime_filter_mode=off - q1 4635438943454345 q2 380 276 283 276 q3 3366316030823082 q4 2095185417301730 q5 5613558455995584 q6 238 142 142 142 q7 2287189718581858 q8 3290342334703423 q9 8754894587938793 q10 4049383938903839 q11 591 509 499 499 q12 794 661 633 633 q13 15906 317132243171 q14 303 301 293 293 q15 549 479 483 479 q16 512 456 460 456 q17 1815152815181518 q18 8231792578877887 q19 1753159815311531 q20 3100187118621862 q21 7303479548274795 q22 559 491 538 491 Total cold run time: 76123 ms Total hot run time: 56687 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](profile) Fix read load profile failed [doris]
doris-robot commented on PR #38437: URL: https://github.com/apache/doris/pull/38437#issuecomment-2254416607 TPC-DS: Total hot run time: 171936 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit e3c905dcb1444f4653866fc6a2ef9690048a5be6, data reload: false query1 914 371 363 363 query2 6440187318291829 query3 6632204 217 204 query4 28580 17437 17315 17315 query5 3623460 504 460 query6 257 173 169 169 query7 4584290 290 290 query8 248 201 197 197 query9 8430247324412441 query10 457 289 295 289 query11 12127 10002 10016 10002 query12 121 90 84 84 query13 1645380 369 369 query14 10262 611476636114 query15 207 162 159 159 query16 7580462 500 462 query17 1372556 535 535 query18 1894295 275 275 query19 182 139 142 139 query20 89 86 83 83 query21 206 99 97 97 query22 4516407840264026 query23 34121 33863 33815 33815 query24 10735 292728942894 query25 593 398 397 397 query26 718 156 155 155 query27 2268288 275 275 query28 6113211320922092 query29 763 443 424 424 query30 250 155 153 153 query31 1030796 779 779 query32 97 52 56 52 query33 662 343 320 320 query34 886 495 504 495 query35 889 729 741 729 query36 1085956 956 956 query37 147 84 88 84 query38 3009283028492830 query39 904 841 831 831 query40 197 131 117 117 query41 46 43 44 43 query42 101 98 94 94 query43 520 489 451 451 query44 1081719 724 719 query45 210 176 176 176 query46 1095739 725 725 query47 1843178917791779 query48 387 295 291 291 query49 835 405 414 405 query50 793 396 402 396 query51 6875667566696669 query52 94 88 84 84 query53 252 181 180 180 query54 924 438 438 438 query55 74 70 72 70 query56 283 271 265 265 query57 1136103910411039 query58 252 276 290 276 query59 2923254525832545 query60 296 281 301 281 query61 96 90 92 90 query62 807 635 630 630 query63 212 185 182 182 query64 9214223316671667 query65 3172310430983098 query66 737 346 335 335 query67 15432 14911 14952 14911 query68 6099551 562 551 query69 653 405 327 327 query70 1176109910971097 query71 438 269 269 269 query72 7376563758235637 query73 773 327 328 327 query74 6092564056205620 query75 3554267927092679 query76 3284935 901 901 query77 635 313 308 308 query78 9622954888778877 query79 4850517 513 513 query80 1145484 557 484 query81 577 217 221 217 query82 789 141 132 132 query83 204 179 175 175 query84 267 78 80 78 query85 1278323 305 305 query86 365 314 312 312 query87 3244308930873087 query88 4973246024742460 query89 420 282 290 282 query90 1692194 193 193 query91 121 103 99 99 query92 55 48 50 48 query93 5231554 542 542 query94 684 294 281 281 query95 356 258 261 258 query96 610 277 272 272 query97 3202309030143014 query98 213 201 196 196 query99 1606128912561256 Total cold run time: 286107 ms Total hot run time: 171936 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s
Re: [PR] [DEBUG] debug hist [doris]
doris-robot commented on PR #38242: URL: https://github.com/apache/doris/pull/38242#issuecomment-2254419118 TPC-DS: Total hot run time: 173291 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit acd50f8a804ccf91c96958bf7f89f8ddb3a4d50c, data reload: false query1 895 372 363 363 query2 6366197618511851 query3 6632206 214 206 query4 28453 17604 17259 17259 query5 3756473 489 473 query6 268 168 178 168 query7 4591289 287 287 query8 250 202 203 202 query9 8501245824362436 query10 437 269 273 269 query11 12259 10172 99889988 query12 122 87 89 87 query13 1624377 369 369 query14 10381 776177147714 query15 247 171 172 171 query16 7622454 461 454 query17 1399572 549 549 query18 1784289 286 286 query19 195 148 152 148 query20 95 83 87 83 query21 208 105 104 104 query22 4241391038683868 query23 34262 33785 33814 33785 query24 11134 297229462946 query25 656 417 413 413 query26 1174157 158 157 query27 2889282 298 282 query28 7745210220822082 query29 883 460 461 460 query30 293 156 160 156 query31 976 783 749 749 query32 99 53 58 53 query33 778 356 343 343 query34 921 473 504 473 query35 868 760 735 735 query36 1103945 951 945 query37 153 83 82 82 query38 3013284428422842 query39 904 836 851 836 query40 205 119 118 118 query41 48 44 43 43 query42 109 98 103 98 query43 509 459 476 459 query44 1234710 716 710 query45 207 177 176 176 query46 1090759 710 710 query47 1844179417361736 query48 366 293 300 293 query49 910 409 411 409 query50 788 398 398 398 query51 6797674466266626 query52 103 85 91 85 query53 252 175 182 175 query54 857 450 454 450 query55 75 73 73 73 query56 298 274 283 274 query57 1141105210581052 query58 254 255 273 255 query59 2914276926562656 query60 314 284 278 278 query61 93 92 95 92 query62 814 633 646 633 query63 209 182 183 182 query64 9538225816751675 query65 3149309630913091 query66 749 337 338 337 query67 15344 14840 15037 14840 query68 6613561 568 561 query69 681 394 344 344 query70 1145102310821023 query71 554 272 261 261 query72 8966567456305630 query73 817 327 329 327 query74 6099561556615615 query75 5047268126292629 query76 4902976 892 892 query77 809 303 293 293 query78 9730893789298929 query79 9379517 520 517 query80 1138478 484 478 query81 585 219 221 219 query82 792 128 133 128 query83 339 169 172 169 query84 263 76 79 76 query85 1432322 307 307 query86 436 303 314 303 query87 3258303730543037 query88 5423246724602460 query89 515 290 282 282 query90 2021190 190 190 query91 123 100 146 100 query92 60 47 51 47 query93 7197541 544 541 query94 926 298 293 293 query95 353 255 260 255 query96 632 287 282 282 query97 3218304930333033 query98 217 198 194 194 query99 1562129612781278 Total cold run time: 303620 ms Total hot run time: 173291 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [fix](profile) Fix read load profile failed [doris]
doris-robot commented on PR #38437: URL: https://github.com/apache/doris/pull/38437#issuecomment-2254426397 ClickBench: Total hot run time: 30.86 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit e3c905dcb1444f4653866fc6a2ef9690048a5be6, data reload: false query1 0.040.040.03 query2 0.080.030.04 query3 0.230.050.05 query4 1.680.070.07 query5 0.510.500.47 query6 1.120.720.74 query7 0.020.010.02 query8 0.050.040.04 query9 0.540.500.48 query10 0.520.550.53 query11 0.150.120.11 query12 0.140.120.12 query13 0.600.590.58 query14 0.770.770.77 query15 0.850.800.80 query16 0.380.370.37 query17 0.950.961.06 query18 0.220.210.22 query19 1.901.801.75 query20 0.010.000.01 query21 15.40 0.810.66 query22 5.556.152.34 query23 18.28 1.301.27 query24 2.080.230.22 query25 0.160.090.08 query26 0.290.200.20 query27 0.460.230.24 query28 13.39 1.030.99 query29 12.63 3.283.23 query30 0.250.070.05 query31 2.880.390.39 query32 3.260.490.48 query33 2.892.872.90 query34 17.19 4.304.30 query35 4.384.384.39 query36 0.660.460.48 query37 0.190.160.16 query38 0.150.150.15 query39 0.040.040.04 query40 0.140.120.13 query41 0.090.050.05 query42 0.060.050.05 query43 0.050.040.04 Total cold run time: 111.23 s Total hot run time: 30.86 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [DEBUG] debug hist [doris]
doris-robot commented on PR #38242: URL: https://github.com/apache/doris/pull/38242#issuecomment-2254428814 ClickBench: Total hot run time: 30.47 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit acd50f8a804ccf91c96958bf7f89f8ddb3a4d50c, data reload: false query1 0.050.030.03 query2 0.080.030.03 query3 0.220.050.06 query4 1.670.080.08 query5 0.500.480.49 query6 1.130.730.72 query7 0.020.020.01 query8 0.040.040.04 query9 0.560.480.49 query10 0.550.540.56 query11 0.150.110.11 query12 0.150.130.12 query13 0.590.580.58 query14 0.750.780.78 query15 0.860.800.82 query16 0.360.360.35 query17 1.021.021.01 query18 0.220.210.22 query19 1.771.691.72 query20 0.010.010.01 query21 15.40 0.820.67 query22 4.617.061.94 query23 18.27 1.401.23 query24 2.150.230.23 query25 0.170.070.09 query26 0.300.210.21 query27 0.460.230.23 query28 13.19 1.011.00 query29 12.65 3.293.24 query30 0.250.060.06 query31 2.870.390.38 query32 3.290.470.46 query33 2.892.952.95 query34 17.07 4.384.32 query35 4.384.344.38 query36 0.650.470.45 query37 0.190.150.16 query38 0.150.160.14 query39 0.040.030.04 query40 0.150.120.13 query41 0.090.050.05 query42 0.050.050.04 query43 0.050.040.04 Total cold run time: 110.02 s Total hot run time: 30.47 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](memory) Optimize `ClearCacheAction`implementation [doris]
doris-robot commented on PR #38438: URL: https://github.com/apache/doris/pull/38438#issuecomment-2254460923 TPC-H: Total hot run time: 39816 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 7b9b20b63343b247e6cd88e87bbdb7b7d1799518, data reload: false -- Round 1 -- q1 17619 456443344334 q2 2018201 204 201 q3 10585 136311521152 q4 10614 766 706 706 q5 7989281327282728 q6 226 147 143 143 q7 1012634 621 621 q8 9516198619711971 q9 9170669667336696 q10 9338385137693769 q11 457 251 244 244 q12 401 227 221 221 q13 17773 296329742963 q14 288 250 241 241 q15 509 471 500 471 q16 495 381 382 381 q17 967 685 652 652 q18 8118753474957495 q19 6251962 976 962 q20 695 338 342 338 q21 5015324332793243 q22 354 284 289 284 Total cold run time: 119410 ms Total hot run time: 39816 ms - Round 2, with runtime_filter_mode=off - q1 4544429842724272 q2 379 275 272 272 q3 3093292229322922 q4 2015175216871687 q5 5541561055075507 q6 238 142 139 139 q7 2281188219411882 q8 3292347334313431 q9 8737871188218711 q10 4108388738763876 q11 596 507 534 507 q12 804 633 638 633 q13 16824 325631723172 q14 341 288 283 283 q15 540 485 492 485 q16 512 447 435 435 q17 1831152015481520 q18 8342798577057705 q19 1773148417201484 q20 2178187518641864 q21 5198500350215003 q22 620 538 536 536 Total cold run time: 73787 ms Total hot run time: 56326 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [DEBUG] debug hist [doris]
zhiqiang- commented on PR #38242: URL: https://github.com/apache/doris/pull/38242#issuecomment-2254462988 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](memory) Optimize `ClearCacheAction`implementation [doris]
doris-robot commented on PR #38438: URL: https://github.com/apache/doris/pull/38438#issuecomment-2254463915 TPC-DS: Total hot run time: 173290 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 7b9b20b63343b247e6cd88e87bbdb7b7d1799518, data reload: false query1 914 376 375 375 query2 6304197718671867 query3 6657206 218 206 query4 28287 17441 17335 17335 query5 3630483 491 483 query6 263 170 173 170 query7 4580293 291 291 query8 247 203 205 203 query9 8555249424702470 query10 442 283 275 275 query11 11565 10014 10054 10014 query12 138 87 86 86 query13 1633374 370 370 query14 10315 775378917753 query15 225 173 167 167 query16 7678489 469 469 query17 1547582 548 548 query18 1979290 316 290 query19 197 151 152 151 query20 94 89 86 86 query21 208 108 107 107 query22 4349410539483948 query23 34197 33805 33713 33713 query24 12038 297529812975 query25 704 421 417 417 query26 1761162 160 160 query27 3022282 290 282 query28 7409209320802080 query29 1182471 457 457 query30 256 156 165 156 query31 1016774 758 758 query32 104 58 58 58 query33 790 367 359 359 query34 877 488 515 488 query35 896 763 774 763 query36 1119961 960 960 query37 288 88 90 88 query38 3010296528272827 query39 927 830 829 829 query40 244 114 115 114 query41 47 45 42 42 query42 110 92 98 92 query43 496 479 476 476 query44 1202731 725 725 query45 210 172 173 172 query46 1099749 712 712 query47 1830175217821752 query48 367 291 291 291 query49 931 409 410 409 query50 793 400 409 400 query51 6801663666806636 query52 99 91 103 91 query53 255 181 187 181 query54 958 454 440 440 query55 76 76 73 73 query56 287 272 275 272 query57 1149108510431043 query58 264 301 292 292 query59 2968279126712671 query60 302 285 300 285 query61 97 93 93 93 query62 805 645 653 645 query63 218 186 183 183 query64 10315 229417071707 query65 3182309831133098 query66 1246355 342 342 query67 15597 14694 14573 14573 query68 8718543 541 541 query69 721 387 345 345 query70 1361111610481048 query71 542 274 269 269 query72 8543590153345334 query73 2277334 326 326 query74 6146566055765576 query75 5144268226502650 query76 5523924 919 919 query77 769 323 306 306 query78 9725901589518951 query79 8072521 527 521 query80 879 483 494 483 query81 578 215 218 215 query82 284 132 130 130 query83 333 173 175 173 query84 265 77 78 77 query85 971 376 297 297 query86 351 310 296 296 query87 3325307730603060 query88 4494249124802480 query89 497 287 281 281 query90 2038199 190 190 query91 129 101 100 100 query92 65 47 55 47 query93 5856531 520 520 query94 1013305 304 304 query95 360 256 258 256 query96 625 281 282 281 query97 3214301430383014 query98 218 200 198 198 query99 1605125812721258 Total cold run time: 305992 ms Total hot run time: 173290 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [feature](hive)Support reading renamed Parquet Hive and Orc Hive tables. [doris]
hubgeter commented on PR #38432: URL: https://github.com/apache/doris/pull/38432#issuecomment-2254464262 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](hive)Support reading renamed Parquet Hive and Orc Hive tables. [doris]
github-actions[bot] commented on PR #38432: URL: https://github.com/apache/doris/pull/38432#issuecomment-2254465158 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](memory) Optimize `ClearCacheAction`implementation [doris]
doris-robot commented on PR #38438: URL: https://github.com/apache/doris/pull/38438#issuecomment-2254465328 ClickBench: Total hot run time: 31.02 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 7b9b20b63343b247e6cd88e87bbdb7b7d1799518, data reload: false query1 0.050.040.03 query2 0.080.030.04 query3 0.230.050.05 query4 1.680.100.10 query5 0.500.480.48 query6 1.140.730.73 query7 0.020.010.02 query8 0.050.040.04 query9 0.540.480.49 query10 0.540.550.54 query11 0.160.120.11 query12 0.150.120.12 query13 0.610.580.58 query14 0.780.770.79 query15 0.850.810.81 query16 0.370.370.38 query17 1.051.041.03 query18 0.220.220.22 query19 1.921.801.76 query20 0.010.010.01 query21 15.40 0.760.65 query22 3.766.682.17 query23 18.30 1.451.31 query24 2.080.230.22 query25 0.150.080.08 query26 0.280.210.21 query27 0.450.230.24 query28 13.28 1.021.00 query29 12.68 3.363.33 query30 0.250.060.06 query31 2.880.390.39 query32 3.290.480.46 query33 2.872.962.90 query34 17.03 4.324.34 query35 4.384.384.44 query36 0.650.470.47 query37 0.180.160.15 query38 0.170.150.14 query39 0.050.030.03 query40 0.160.120.12 query41 0.100.040.05 query42 0.060.050.06 query43 0.040.040.04 Total cold run time: 109.44 s Total hot run time: 31.02 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](hive)Support reading renamed Parquet Hive and Orc Hive tables. [doris]
github-actions[bot] commented on PR #38432: URL: https://github.com/apache/doris/pull/38432#issuecomment-2254465769 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](hive)Support reading renamed Parquet Hive and Orc Hive tables. [doris]
github-actions[bot] commented on PR #38432: URL: https://github.com/apache/doris/pull/38432#issuecomment-2254465968 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]
github-actions[bot] commented on PR #38215: URL: https://github.com/apache/doris/pull/38215#issuecomment-2254467721 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [DEBUG] debug hist [doris]
doris-robot commented on PR #38242: URL: https://github.com/apache/doris/pull/38242#issuecomment-2254473751 TPC-H: Total hot run time: 39135 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit acd50f8a804ccf91c96958bf7f89f8ddb3a4d50c, data reload: false -- Round 1 -- q1 17624 437642394239 q2 2014195 194 194 q3 10447 116211311131 q4 10145 742 659 659 q5 7575270526852685 q6 218 145 141 141 q7 948 593 582 582 q8 9222188919321889 q9 8892650465376504 q10 8800380337513751 q11 456 245 242 242 q12 478 217 226 217 q13 18597 296229692962 q14 282 234 247 234 q15 517 479 484 479 q16 524 385 373 373 q17 962 655 664 655 q18 7952747073987398 q19 32481060971 971 q20 682 338 329 329 q21 5417321240093212 q22 352 292 288 288 Total cold run time: 115352 ms Total hot run time: 39135 ms - Round 2, with runtime_filter_mode=off - q1 4379424742854247 q2 387 271 274 271 q3 2996278628252786 q4 1998165817301658 q5 5614556055435543 q6 226 135 136 135 q7 2173182018601820 q8 3252343433663366 q9 8741871588378715 q10 4143394638713871 q11 569 478 472 472 q12 820 651 638 638 q13 17280 316332743163 q14 341 305 299 299 q15 509 473 496 473 q16 489 447 439 439 q17 1823157315261526 q18 8406798877097709 q19 1886160015381538 q20 2107187718631863 q21 5010492848704870 q22 585 515 497 497 Total cold run time: 73734 ms Total hot run time: 55899 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Opt](load) don't print stack when some errors occur for stream load #38332 [doris]
liaoxin01 merged PR #38418: URL: https://github.com/apache/doris/pull/38418 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [Opt](load) don't print stack when some errors occur for stream load #38332 (#38418)
This is an automated email from the ASF dual-hosted git repository.
liaoxin pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new d8744cd3d08 [Opt](load) don't print stack when some errors occur for
stream load #38332 (#38418)
d8744cd3d08 is described below
commit d8744cd3d087dd918a2c59765973f9379d09096e
Author: Xin Liao
AuthorDate: Sun Jul 28 19:04:24 2024 +0800
[Opt](load) don't print stack when some errors occur for stream load #38332
(#38418)
cherry pick from #38332
---
be/src/http/action/stream_load.cpp | 29 -
1 file changed, 16 insertions(+), 13 deletions(-)
diff --git a/be/src/http/action/stream_load.cpp
b/be/src/http/action/stream_load.cpp
index a8768b83860..feea93446c8 100644
--- a/be/src/http/action/stream_load.cpp
+++ b/be/src/http/action/stream_load.cpp
@@ -144,7 +144,7 @@ Status
StreamLoadAction::_handle(std::shared_ptr ctx) {
if (ctx->body_bytes > 0 && ctx->receive_bytes != ctx->body_bytes) {
LOG(WARNING) << "recevie body don't equal with body bytes,
body_bytes=" << ctx->body_bytes
<< ", receive_bytes=" << ctx->receive_bytes << ", id=" <<
ctx->id;
-return Status::InternalError("receive body don't equal with body
bytes");
+return Status::InternalError("receive body don't equal with
body bytes");
}
// if we use non-streaming, MessageBodyFileSink.finish will close the file
@@ -230,13 +230,13 @@ Status StreamLoadAction::_on_header(HttpRequest*
http_req, std::shared_ptrauth)) {
LOG(WARNING) << "parse basic authorization failed." << ctx->brief();
-return Status::InternalError("no valid Basic authorization");
+return Status::InternalError("no valid Basic authorization");
}
// get format of this put
if (!http_req->header(HTTP_COMPRESS_TYPE).empty() &&
iequal(http_req->header(HTTP_FORMAT_KEY), "JSON")) {
-return Status::InternalError("compress data of JSON format is not
supported.");
+return Status::InternalError("compress data of JSON format is
not supported.");
}
std::string format_str = http_req->header(HTTP_FORMAT_KEY);
if (iequal(format_str, BeConsts::CSV_WITH_NAMES) ||
@@ -252,8 +252,8 @@ Status StreamLoadAction::_on_header(HttpRequest* http_req,
std::shared_ptrheader(HTTP_COMPRESS_TYPE),
&ctx->format,
&ctx->compress_type);
if (ctx->format == TFileFormatType::FORMAT_UNKNOWN) {
-return Status::InternalError("unknown data format, format={}",
- http_req->header(HTTP_FORMAT_KEY));
+return Status::InternalError("unknown data format, format={}",
+http_req->header(HTTP_FORMAT_KEY));
}
// check content length
@@ -271,7 +271,7 @@ Status StreamLoadAction::_on_header(HttpRequest* http_req,
std::shared_ptrformat == TFileFormatType::FORMAT_JSON) &&
(ctx->body_bytes > json_max_body_bytes) && !read_json_by_line) {
-return Status::InternalError(
+return Status::InternalError(
"The size of this batch exceed the max size [{}] of json
type data "
" data [ {} ]. Split the file, or use 'read_json_by_line'",
json_max_body_bytes, ctx->body_bytes);
@@ -279,8 +279,8 @@ Status StreamLoadAction::_on_header(HttpRequest* http_req,
std::shared_ptrbody_bytes > csv_max_body_bytes) {
LOG(WARNING) << "body exceed max size." << ctx->brief();
-return Status::InternalError("body exceed max size: {}, data: {}",
csv_max_body_bytes,
- ctx->body_bytes);
+return Status::InternalError("body exceed max size: {},
data: {}",
+csv_max_body_bytes,
ctx->body_bytes);
}
} else {
#ifndef BE_TEST
@@ -298,13 +298,14 @@ Status StreamLoadAction::_on_header(HttpRequest*
http_req, std::shared_ptris_chunked_transfer))) {
LOG(WARNING) << "content_length is empty and
transfer-encoding!=chunked, please set "
"content_length or transfer-encoding=chunked";
-return Status::InternalError(
+return Status::InternalError(
"content_length is empty and transfer-encoding!=chunked,
please set content_length "
"or transfer-encoding=chunked");
} else if (UNLIKELY(!http_req->header(HttpHeaders::CONTENT_LENGTH).empty()
&&
ctx->is_chunked_transfer)) {
LOG(WARNING) << "please do not set both content_length and
transfer-encoding";
-return Status::InternalError("please do not set both content_length
and transfer-encoding");
+return Status::InternalError(
+"please do not set both content_length and transfer-e
Re: [PR] [fix](load) fix no error url for stream load #38325 [doris]
liaoxin01 merged PR #38417: URL: https://github.com/apache/doris/pull/38417 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [fix](load) fix no error url for stream load #38325 (#38417)
This is an automated email from the ASF dual-hosted git repository.
liaoxin pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new e9f12fac47e [fix](load) fix no error url for stream load #38325
(#38417)
e9f12fac47e is described below
commit e9f12fac47e5703b5aadf9b0d34ef56c94c5706e
Author: Xin Liao
AuthorDate: Sun Jul 28 19:06:57 2024 +0800
[fix](load) fix no error url for stream load #38325 (#38417)
cherry pick from #38325
---
.../runtime/stream_load/stream_load_executor.cpp | 4 +--
.../load_p0/stream_load/test_stream_load.groovy| 42 +-
2 files changed, 42 insertions(+), 4 deletions(-)
diff --git a/be/src/runtime/stream_load/stream_load_executor.cpp
b/be/src/runtime/stream_load/stream_load_executor.cpp
index 051aca3e130..0fcd4af1ce5 100644
--- a/be/src/runtime/stream_load/stream_load_executor.cpp
+++ b/be/src/runtime/stream_load/stream_load_executor.cpp
@@ -90,9 +90,7 @@ Status
StreamLoadExecutor::execute_plan_fragment(std::shared_ptrnumber_filtered_rows > 0 &&
!state->get_error_log_file_path().empty()) {
-ctx->error_url =
to_load_error_http_path(state->get_error_log_file_path());
-}
+ctx->error_url =
to_load_error_http_path(state->get_error_log_file_path());
if (status->ok()) {
DorisMetrics::instance()->stream_receive_bytes_total->increment(ctx->receive_bytes);
diff --git a/regression-test/suites/load_p0/stream_load/test_stream_load.groovy
b/regression-test/suites/load_p0/stream_load/test_stream_load.groovy
index 6c002b2d29b..574e2e1466e 100644
--- a/regression-test/suites/load_p0/stream_load/test_stream_load.groovy
+++ b/regression-test/suites/load_p0/stream_load/test_stream_load.groovy
@@ -66,6 +66,15 @@ suite("test_stream_load", "p0") {
file 'test_strict_mode.csv'
time 1 // limit inflight 10s
+check { result, exception, startTime, endTime ->
+if (exception != null) {
+throw exception
+}
+log.info("Stream load result: ${result}".toString())
+def json = parseJson(result)
+assertEquals("success", json.Status.toLowerCase())
+assertTrue(!result.contains("ErrorURL"))
+}
}
sql "sync"
@@ -90,6 +99,7 @@ suite("test_stream_load", "p0") {
log.info("Stream load result: ${result}".toString())
def json = parseJson(result)
assertEquals("fail", json.Status.toLowerCase())
+assertTrue(result.contains("ErrorURL"))
assertEquals(2, json.NumberTotalRows)
assertEquals(1, json.NumberFilteredRows)
}
@@ -117,6 +127,7 @@ suite("test_stream_load", "p0") {
log.info("Stream load result: ${result}".toString())
def json = parseJson(result)
assertEquals("success", json.Status.toLowerCase())
+assertTrue(result.contains("ErrorURL"))
assertEquals(2, json.NumberTotalRows)
assertEquals(1, json.NumberFilteredRows)
}
@@ -160,6 +171,7 @@ suite("test_stream_load", "p0") {
log.info("Stream load result: ${result}".toString())
def json = parseJson(result)
assertEquals("success", json.Status.toLowerCase())
+assertTrue(!result.contains("ErrorURL"))
assertEquals(3, json.NumberTotalRows)
assertEquals(0, json.NumberFilteredRows)
}
@@ -210,6 +222,7 @@ suite("test_stream_load", "p0") {
log.info("Stream load result: ${result}".toString())
def json = parseJson(result)
assertEquals("success", json.Status.toLowerCase())
+assertTrue(!result.contains("ErrorURL"))
assertEquals(1, json.NumberTotalRows)
assertEquals(0, json.NumberFilteredRows)
}
@@ -428,6 +441,7 @@ suite("test_stream_load", "p0") {
log.info("Stream load result: ${result}".toString())
def json = parseJson(result)
assertEquals("success", json.Status.toLowerCase())
+assertTrue(!result.contains("ErrorURL"))
}
}
sql "sync"
@@ -453,6 +467,7 @@ suite("test_stream_load", "p0") {
log.info("Stream load result: ${result}".toString())
def json = parseJson(result)
assertEquals("success", json.Status.toLowerCase())
+assertTrue(!result.contains("ErrorURL"))
}
}
sql "sync"
@@ -474,6 +489,7 @@ suite("test_stream_load", "p0") {
log.info("Stream load result: ${result}".toString())
def json = parseJson(result)
assertEquals("success", json.Status.toLowerCase())
+assertTrue(!result.contains("ErrorURL"))
assertEquals(4, json.NumberTotalRows)
assertEquals(0, json.NumberFilteredRows)
}
@@ -49
Re: [PR] [DEBUG] debug hist [doris]
doris-robot commented on PR #38242: URL: https://github.com/apache/doris/pull/38242#issuecomment-2254477214 TPC-DS: Total hot run time: 172870 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit acd50f8a804ccf91c96958bf7f89f8ddb3a4d50c, data reload: false query1 910 363 362 362 query2 6434197419051905 query3 6655211 218 211 query4 28185 17439 17421 17421 query5 3601486 497 486 query6 261 172 171 171 query7 4568296 285 285 query8 239 191 196 191 query9 8447246924602460 query10 457 277 277 277 query11 11447 10187 10175 10175 query12 122 93 86 86 query13 1641387 379 379 query14 10288 698876526988 query15 237 163 165 163 query16 7595483 466 466 query17 1588574 521 521 query18 1948281 285 281 query19 190 137 136 136 query20 91 83 83 83 query21 203 98 96 96 query22 4300411140304030 query23 33995 33880 33574 33574 query24 11093 294728892889 query25 597 404 394 394 query26 705 148 147 147 query27 2253278 282 278 query28 5971210621112106 query29 788 425 433 425 query30 257 153 153 153 query31 1003766 756 756 query32 100 53 54 53 query33 743 335 326 326 query34 888 479 514 479 query35 878 761 758 758 query36 1097948 952 948 query37 156 85 84 84 query38 3012287628312831 query39 893 857 856 856 query40 221 115 119 115 query41 48 46 44 44 query42 111 92 94 92 query43 523 467 464 464 query44 1157726 725 725 query45 214 180 172 172 query46 1088731 752 731 query47 1821177017631763 query48 368 301 289 289 query49 860 408 416 408 query50 798 402 408 402 query51 6720682966496649 query52 98 93 89 89 query53 260 178 184 178 query54 878 434 434 434 query55 73 74 74 74 query56 283 256 263 256 query57 1126103410251025 query58 261 255 263 255 query59 3110276325392539 query60 302 281 279 279 query61 99 95 92 92 query62 805 655 632 632 query63 202 183 182 182 query64 9323222817091709 query65 3158309430993094 query66 739 332 331 331 query67 15221 14785 14909 14785 query68 5840561 553 553 query69 621 402 332 332 query70 1137112211211121 query71 476 272 270 270 query72 7895550360495503 query73 784 339 337 337 query74 6243567856805678 query75 3784262526802625 query76 3684962 987 962 query77 709 307 292 292 query78 10914 899491168994 query79 4168532 520 520 query80 1988487 488 487 query81 598 215 221 215 query82 1415137 135 135 query83 320 186 179 179 query84 273 81 77 77 query85 1471325 308 308 query86 474 321 315 315 query87 3288307030503050 query88 4531250124842484 query89 434 291 279 279 query90 1880201 197 197 query91 127 100 101 100 query92 62 51 51 51 query93 5181540 541 540 query94 932 262 295 262 query95 356 276 329 276 query96 621 273 282 273 query97 3281300230693002 query98 231 193 188 188 query99 1551122712621227 Total cold run time: 288972 ms Total hot run time: 172870 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [feature](hive)Support reading renamed Parquet Hive and Orc Hive tables. [doris]
doris-robot commented on PR #38432: URL: https://github.com/apache/doris/pull/38432#issuecomment-2254478029 TPC-H: Total hot run time: 39823 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit bcf4eaa6d9442783a8ff689f5c6ee38449481871, data reload: false -- Round 1 -- q1 18030 518844754475 q2 2551205 204 204 q3 11764 125511561156 q4 10409 796 734 734 q5 7578273028632730 q6 224 143 141 141 q7 989 606 611 606 q8 9278193719591937 q9 8957662766616627 q10 8723383437903790 q11 464 242 254 242 q12 405 219 215 215 q13 17737 298529962985 q14 287 235 242 235 q15 529 483 498 483 q16 492 398 384 384 q17 978 662 726 662 q18 8209734074367340 q19 13931057989 989 q20 694 323 345 323 q21 5012328133393281 q22 346 298 284 284 Total cold run time: 115049 ms Total hot run time: 39823 ms - Round 2, with runtime_filter_mode=off - q1 4315429042784278 q2 368 270 268 268 q3 2992277027422742 q4 1938164816171617 q5 5310534653215321 q6 224 128 131 128 q7 2119173717641737 q8 3202337433373337 q9 8452842384598423 q10 3924373637133713 q11 610 524 488 488 q12 747 588 589 588 q13 17434 296029662960 q14 292 268 286 268 q15 523 469 482 469 q16 479 419 423 419 q17 1806151714611461 q18 7606758276177582 q19 1683158014441444 q20 1964177517581758 q21 4964472246744674 q22 577 478 492 478 Total cold run time: 71529 ms Total hot run time: 54153 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [DEBUG] debug hist [doris]
doris-robot commented on PR #38242: URL: https://github.com/apache/doris/pull/38242#issuecomment-2254478845 ClickBench: Total hot run time: 31.45 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit acd50f8a804ccf91c96958bf7f89f8ddb3a4d50c, data reload: false query1 0.040.040.04 query2 0.080.040.03 query3 0.220.050.05 query4 1.680.070.07 query5 0.480.470.50 query6 1.140.720.73 query7 0.020.020.01 query8 0.060.040.04 query9 0.550.490.49 query10 0.540.550.54 query11 0.160.110.12 query12 0.150.120.13 query13 0.590.590.58 query14 0.770.780.80 query15 0.840.800.81 query16 0.370.380.36 query17 0.990.970.96 query18 0.220.220.21 query19 1.831.731.77 query20 0.010.010.01 query21 15.39 0.770.66 query22 3.547.422.79 query23 18.36 1.391.33 query24 2.130.230.23 query25 0.160.100.08 query26 0.300.210.21 query27 0.460.230.23 query28 13.27 1.030.99 query29 12.60 3.243.27 query30 0.250.060.06 query31 2.860.400.38 query32 3.270.480.47 query33 2.882.932.92 query34 16.98 4.364.33 query35 4.384.364.42 query36 0.660.480.46 query37 0.190.160.16 query38 0.160.150.14 query39 0.050.030.03 query40 0.150.130.13 query41 0.100.050.04 query42 0.050.050.06 query43 0.060.050.04 Total cold run time: 108.99 s Total hot run time: 31.45 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](func)fix array_with_const with larger than max_array_size [doris]
xiaokang merged PR #38150: URL: https://github.com/apache/doris/pull/38150 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [fix](func)fix array_with_const with larger than max_array_size #37495 (#38150)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 1079af3e8a9 [fix](func)fix array_with_const with larger than
max_array_size #37495 (#38150)
1079af3e8a9 is described below
commit 1079af3e8a99ea3dadb5889a26609567b4b85fa9
Author: amory
AuthorDate: Sun Jul 28 19:20:34 2024 +0800
[fix](func)fix array_with_const with larger than max_array_size #37495
(#38150)
---
be/src/vec/functions/array/function_array_with_constant.cpp | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/be/src/vec/functions/array/function_array_with_constant.cpp
b/be/src/vec/functions/array/function_array_with_constant.cpp
index 9e3885b1c3c..724f3cc1316 100644
--- a/be/src/vec/functions/array/function_array_with_constant.cpp
+++ b/be/src/vec/functions/array/function_array_with_constant.cpp
@@ -92,9 +92,9 @@ public:
array_sizes.reserve(input_rows_count);
for (size_t i = 0; i < input_rows_count; ++i) {
auto array_size = num->get_int(i);
-if (UNLIKELY(array_size < 0)) {
-return Status::RuntimeError("Array size can not be negative in
function:" +
-get_name());
+if (UNLIKELY(array_size < 0) || UNLIKELY(array_size >
max_array_size_as_field)) {
+return Status::RuntimeError("Array size should in range(0, {})
in function: {}",
+max_array_size_as_field,
get_name());
}
offset += array_size;
offsets.push_back(offset);
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]
xiaokang merged PR #38149: URL: https://github.com/apache/doris/pull/38149 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated (1079af3e8a9 -> aedec6ef212)
This is an automated email from the ASF dual-hosted git repository. kxiao pushed a change to branch branch-2.0 in repository https://gitbox.apache.org/repos/asf/doris.git from 1079af3e8a9 [fix](func)fix array_with_const with larger than max_array_size #37495 (#38150) add aedec6ef212 [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf #37936 (#38149) No new revisions were added by this update. Summary of changes: be/src/util/mysql_row_buffer.cpp | 3 +- .../test_large_string_and_nested_type.out | 4 +++ .../test_large_string_and_nested_type.groovy | 36 ++ 3 files changed, 42 insertions(+), 1 deletion(-) create mode 100644 regression-test/data/datatype_p0/nested_types/corner_cases/test_large_string_and_nested_type.out create mode 100644 regression-test/suites/datatype_p0/nested_types/corner_cases/test_large_string_and_nested_type.groovy - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](hive)Support reading renamed Parquet Hive and Orc Hive tables. [doris]
doris-robot commented on PR #38432: URL: https://github.com/apache/doris/pull/38432#issuecomment-2254481688 TPC-DS: Total hot run time: 172930 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit bcf4eaa6d9442783a8ff689f5c6ee38449481871, data reload: false query1 923 374 355 355 query2 6440194318611861 query3 6663206 215 206 query4 28266 17713 17576 17576 query5 4203497 476 476 query6 277 167 150 150 query7 4579294 288 288 query8 248 221 194 194 query9 8750246824382438 query10 454 281 264 264 query11 11861 10045 10042 10042 query12 133 85 89 85 query13 1627380 378 378 query14 10284 774777007700 query15 227 172 167 167 query16 7836491 486 486 query17 1597564 549 549 query18 2010288 283 283 query19 199 148 148 148 query20 94 88 94 88 query21 213 101 103 101 query22 4150425238973897 query23 34015 33171 32972 32972 query24 12225 291628872887 query25 698 393 399 393 query26 1803149 155 149 query27 2982274 278 274 query28 7331205320332033 query29 1122438 429 429 query30 290 153 151 151 query31 952 751 764 751 query32 96 55 57 55 query33 792 349 349 349 query34 906 470 475 470 query35 873 730 755 730 query36 1097901 929 901 query37 297 83 78 78 query38 2848273627402736 query39 901 792 831 792 query40 272 119 116 116 query41 49 47 46 46 query42 119 102 110 102 query43 516 471 455 455 query44 1208726 728 726 query45 213 180 178 178 query46 1094785 735 735 query47 1838175117441744 query48 378 304 299 299 query49 1209435 430 430 query50 828 408 405 405 query51 6706669566686668 query52 101 94 95 94 query53 260 183 186 183 query54 978 463 451 451 query55 75 75 75 75 query56 328 286 293 286 query57 1137103610231023 query58 388 256 276 256 query59 2914282826842684 query60 310 284 292 284 query61 94 114 103 103 query62 843 651 658 651 query63 207 191 183 183 query64 10476 228117581758 query65 3262310230903090 query66 1370335 340 335 query67 15624 14667 14648 14648 query68 9384567 580 567 query69 766 413 324 324 query70 1402108310801080 query71 549 271 265 265 query72 9166560859125608 query73 2248328 327 327 query74 6162572157245721 query75 6137273626742674 query76 5629963 942 942 query77 792 325 304 304 query78 9602912389638963 query79 9445534 536 534 query80 921 502 510 502 query81 584 225 214 214 query82 283 135 133 133 query83 344 177 179 177 query84 267 77 80 77 query85 989 316 300 300 query86 359 329 300 300 query87 3314316630523052 query88 5082249124902490 query89 496 285 285 285 query90 2056197 196 196 query91 126 101 101 101 query92 59 48 51 48 query93 5945556 558 556 query94 1047285 268 268 query95 364 265 263 263 query96 625 279 273 273 query97 3157305630293029 query98 217 206 203 203 query99 1519127912641264 Total cold run time: 312095 ms Total hot run time: 172930 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t
Re: [PR] [feature](hive)Support reading renamed Parquet Hive and Orc Hive tables. [doris]
doris-robot commented on PR #38432: URL: https://github.com/apache/doris/pull/38432#issuecomment-2254483212 ClickBench: Total hot run time: 30.46 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit bcf4eaa6d9442783a8ff689f5c6ee38449481871, data reload: false query1 0.050.040.04 query2 0.080.030.04 query3 0.220.040.05 query4 1.680.090.08 query5 0.510.490.48 query6 1.130.720.73 query7 0.020.020.01 query8 0.050.040.04 query9 0.560.500.50 query10 0.530.550.54 query11 0.160.110.12 query12 0.150.120.12 query13 0.610.580.59 query14 0.760.780.78 query15 0.860.810.81 query16 0.360.370.36 query17 0.971.030.95 query18 0.230.220.21 query19 1.881.701.73 query20 0.020.010.03 query21 15.40 0.780.66 query22 3.887.671.89 query23 18.28 1.361.18 query24 2.190.240.23 query25 0.160.080.10 query26 0.300.220.21 query27 0.460.240.23 query28 13.20 1.021.02 query29 12.64 3.303.25 query30 0.250.060.06 query31 2.890.400.38 query32 3.280.490.47 query33 2.922.882.92 query34 17.00 4.334.38 query35 4.394.454.37 query36 0.650.490.49 query37 0.200.160.17 query38 0.170.160.16 query39 0.040.030.04 query40 0.170.130.13 query41 0.100.050.05 query42 0.060.050.05 query43 0.050.040.05 Total cold run time: 109.51 s Total hot run time: 30.46 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](bloom filter)Fix rename column with bloom filter [doris]
github-actions[bot] commented on PR #38431: URL: https://github.com/apache/doris/pull/38431#issuecomment-2254483459 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](bloom filter)Fix rename column with bloom filter [doris]
github-actions[bot] commented on PR #38431: URL: https://github.com/apache/doris/pull/38431#issuecomment-2254483473 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](bloom filter)Fix rename column with bloom filter [doris]
xiaokang commented on code in PR #38431:
URL: https://github.com/apache/doris/pull/38431#discussion_r1694226231
##
fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java:
##
@@ -5051,6 +5052,18 @@ private void renameColumn(Database db, OlapTable table,
String colName,
table.setSequenceMapCol(newColName);
}
+// 6. modify bloom filter col
+Set bfCols = table.getCopiedBfColumns();
+Set newBfCols = new HashSet<>();
+for (String bfCol : bfCols) {
Review Comment:
You should check bfCols != null first
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](http) fix http url with incorrect character notation [doris]
xiaokang commented on code in PR #38420:
URL: https://github.com/apache/doris/pull/38420#discussion_r1694227605
##
be/src/olap/single_replica_compaction.cpp:
##
@@ -404,20 +404,7 @@ Status SingleReplicaCompaction::_download_files(DataDir*
data_dir,
return Status::InternalError("single compaction init curl failed");
}
for (auto& file_name : file_name_list) {
-// The file name of the variant column with the inverted index
contains %
-// such as:
023f624c4c322c568271060f9b5b274a4a95_0_10133@properties%2Emessage.idx
-//
{rowset_id}_{seg_num}_{index_id}_{variant_column_name}{%2E}{extracted_column_name}.idx
-// We need to handle %, otherwise it will cause an HTTP 404 error.
-// Because the percent ("%") character serves as the indicator for
percent-encoded octets,
-// it must be percent-encoded as "%25" for that octet to be used as
data within a URI.
-// https://datatracker.ietf.org/doc/html/rfc3986
-auto output = std::unique_ptr(
-curl_easy_escape(curl.get(), file_name.c_str(),
file_name.length()), &curl_free);
-if (!output) {
-return Status::InternalError("escape file name failed, file
name={}", file_name);
-}
-std::string encoded_filename(output.get());
-auto remote_file_url = remote_url_prefix + encoded_filename;
+auto remote_file_url = remote_url_prefix + file_name;
Review Comment:
Do you mean do not escape here and do escape in HttpClient uniformly?
##
be/src/http/http_client.cpp:
##
@@ -290,4 +293,59 @@ Status HttpClient::execute_with_retry(int retry_times, int
sleep_time,
return status;
}
+// http://example.com/page?param1=value1¶m2=value+with+spaces#section
+Status HttpClient::_escape_url(const std::string& url, std::string*
escaped_url) {
Review Comment:
can you find a library function to do escape instead of manual code which is
error prune.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Variant) check storage type before apply `get_row_ranges_by_dict` [doris]
xiaokang commented on code in PR #38413:
URL: https://github.com/apache/doris/pull/38413#discussion_r1694230848
##
be/src/olap/rowset/segment_v2/segment_iterator.cpp:
##
@@ -699,6 +699,11 @@ Status
SegmentIterator::_get_row_ranges_from_conditions(RowRanges* condition_row
if (_opts.io_ctx.reader_type == ReaderType::READER_QUERY) {
RowRanges dict_row_ranges = RowRanges::create_single(num_rows());
for (auto cid : cids) {
+if (!_segment->can_apply_predicate_safely(cid,
Review Comment:
There is check for can_apply_predicate_safely() in bf and zonemap branch.
Why it's not check in this branch before? Does this modification affect other
logic in this branch?
##
be/src/vec/data_types/data_type_object.cpp:
##
@@ -121,6 +126,11 @@ char* DataTypeObject::serialize(const IColumn& column,
char* buf, int be_exec_ve
}
// serialize num of subcolumns
*reinterpret_cast(size_pos) = num_of_columns;
+if (num_of_columns == 0) {
+// Make sure the num_of_rows in ColumnObject not lost
+*reinterpret_cast(buf) = column_object.rows();
Review Comment:
1. Why only serialize num_rows when num_of_columns == 0?
2. It may cause error when rolling upgrade be if serialization is changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](array-func)fix func array_enumerate_uniq with param is nullable return [doris]
xiaokang commented on code in PR #38384:
URL: https://github.com/apache/doris/pull/38384#discussion_r1694231347
##
be/src/vec/functions/array/function_array_enumerate_uniq.cpp:
##
@@ -95,15 +95,13 @@ class FunctionArrayEnumerateUniq : public IFunction {
"The {} -th argument for function: {} .must be an
array but it type is {}",
i, get_name(), arguments[i]->get_name());
}
-if (i == 0) {
-is_nested_nullable =
array_type->get_nested_type()->is_nullable();
-}
+is_nested_nullable = is_nested_nullable |
array_type->get_nested_type()->is_nullable();
Review Comment:
Do you mean `||` instead of `|` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cases)fixed one_nested_types groovy cases [doris]
github-actions[bot] commented on PR #38410: URL: https://github.com/apache/doris/pull/38410#issuecomment-2254492367 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cases)fixed one_nested_types groovy cases [doris]
github-actions[bot] commented on PR #38410: URL: https://github.com/apache/doris/pull/38410#issuecomment-2254492376 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]
liugddx commented on PR #38215: URL: https://github.com/apache/doris/pull/38215#issuecomment-2254497683 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix] (inverted index ) fix inverted index queries in AGG and MOR table [doris]
xiaokang commented on code in PR #38371:
URL: https://github.com/apache/doris/pull/38371#discussion_r1694234157
##
be/src/olap/rowset/segment_v2/segment_writer.cpp:
##
@@ -137,10 +137,15 @@ SegmentWriter::SegmentWriter(io::FileWriter* file_writer,
uint32_t segment_id,
}
}
if (_tablet_schema->has_inverted_index()) {
+std::string segment_file_path = file_writer->path().c_str();
+// if file_writer is a StreamSinkFileWriter, path is empty
+if (segment_file_path.empty()) {
+segment_file_path = _opts.rowset_ctx->segment_path(segment_id);
+}
_inverted_index_file_writer =
std::make_unique(
_opts.rowset_ctx->fs(),
-std::string
{InvertedIndexDescriptor::get_index_file_path_prefix(
-file_writer->path().c_str())},
+std::string {
+
InvertedIndexDescriptor::get_index_file_path_prefix(segment_file_path)},
Review Comment:
why change it?
##
be/src/vec/olap/block_reader.h:
##
@@ -97,6 +97,10 @@ class BlockReader final : public TabletReader {
std::vector _normal_columns_idx; // key column on agg mode, all
column on uniq mode
std::vector _agg_columns_idx;
std::vector _return_columns_loc;
+// before reading the segment, some expressions will compute results
through the index to generate result columns
+// retain this result column when calculating the agg value
+int _index_result_columns = 0;
Review Comment:
It seems to be not safe.
##
be/src/vec/olap/block_reader.cpp:
##
@@ -233,7 +233,9 @@ Status BlockReader::init(const ReaderParams& read_params) {
}
return status;
}
-
+if (size_t cols = _next_row.block->columns() - _return_columns_loc.size()
> 0; cols > 0) {
Review Comment:
It not a common style to write multiple statements in if.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](inverted index) add test for need read data opt [doris]
github-actions[bot] commented on PR #38261: URL: https://github.com/apache/doris/pull/38261#issuecomment-2254498983 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](inverted index) add test for need read data opt [doris]
xiaokang commented on PR #38261: URL: https://github.com/apache/doris/pull/38261#issuecomment-2254498874 check license header in new files. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](inverted index) add test for need read data opt [doris]
github-actions[bot] commented on PR #38261: URL: https://github.com/apache/doris/pull/38261#issuecomment-2254498972 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix] (compaction) fix time series compaction policy [doris]
xiaokang commented on code in PR #38220:
URL: https://github.com/apache/doris/pull/38220#discussion_r1694235068
##
be/src/cloud/cloud_cumulative_compaction_policy.h:
##
@@ -34,6 +34,8 @@ namespace doris {
class Tablet;
struct Version;
+static constexpr int MAX_COMPACT_ROWSET_COUNT = 1;
Review Comment:
It's better to add a be config.
##
be/src/olap/compaction.cpp:
##
@@ -1223,8 +1223,10 @@ Status
CloudCompactionMixin::construct_output_rowset_writer(RowsetWriterContext&
ctx.write_type = DataWriteType::TYPE_COMPACTION;
auto compaction_policy = _tablet->tablet_meta()->compaction_policy();
-ctx.compaction_level =
-
_engine.cumu_compaction_policy(compaction_policy)->new_compaction_level(_input_rowsets);
+if (_tablet->tablet_meta()->time_series_compaction_level_threshold() >= 2)
{
+ctx.compaction_level =
_engine.cumu_compaction_policy(compaction_policy)
Review Comment:
What problem this code try to fix?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](label) add a doing label for every pr [doris]
xiaokang merged PR #37886: URL: https://github.com/apache/doris/pull/37886 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [chore](label) add a doing label for every pr (#37886)
This is an automated email from the ASF dual-hosted git repository. kxiao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new fe7b674c575 [chore](label) add a doing label for every pr (#37886) fe7b674c575 is described below commit fe7b674c575f4638d6c85084e8424a5ad73d96be Author: Kang AuthorDate: Sun Jul 28 20:20:45 2024 +0800 [chore](label) add a doing label for every pr (#37886) Add a `doing` label for every pr to indicate that it is going on and more work need to be done, eg. pick it to other branches. It can be used to filter the prs that need to be picked to other branches. After all done, the `doing` label can be removed manually. --- .github/workflows/labeler/scope-label-conf.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/labeler/scope-label-conf.yml b/.github/workflows/labeler/scope-label-conf.yml index 5a88d046a63..47f6e1c4ab4 100644 --- a/.github/workflows/labeler/scope-label-conf.yml +++ b/.github/workflows/labeler/scope-label-conf.yml @@ -23,3 +23,6 @@ meta-change: - fe/fe-core/src/main/java/org/apache/doris/persist/EditLog.java - gensrc/thrift/* - gensrc/proto/* + +doing: + - * - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]
liugddx commented on PR #38215: URL: https://github.com/apache/doris/pull/38215#issuecomment-2254503321 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]
github-actions[bot] commented on PR #38215: URL: https://github.com/apache/doris/pull/38215#issuecomment-2254504650 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]
github-actions[bot] commented on PR #38215: URL: https://github.com/apache/doris/pull/38215#issuecomment-2254504827 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](nereids) support partitionTopn for multi window exprs [doris]
xzj7019 commented on PR #38393: URL: https://github.com/apache/doris/pull/38393#issuecomment-2254512185 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]
doris-robot commented on PR #38215: URL: https://github.com/apache/doris/pull/38215#issuecomment-2254517987 TPC-H: Total hot run time: 39691 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 0d988836fde4b274f87c6cbf6a5d5347e4c5b467, data reload: false -- Round 1 -- q1 17927 474242884288 q2 2036197 197 197 q3 10482 126711471147 q4 10161 758 726 726 q5 7546279927142714 q6 220 137 136 136 q7 964 610 607 607 q8 9217195419391939 q9 8690657765526552 q10 8829378438013784 q11 446 248 247 247 q12 405 224 213 213 q13 18553 295629922956 q14 275 236 247 236 q15 530 485 494 485 q16 522 382 382 382 q17 997 651 758 651 q18 8216751975237519 q19 5301107711231077 q20 678 342 349 342 q21 5074321833003218 q22 342 289 275 275 Total cold run time: 117411 ms Total hot run time: 39691 ms - Round 2, with runtime_filter_mode=off - q1 4539428842594259 q2 375 259 267 259 q3 3025287129182871 q4 1987170517261705 q5 5671556555465546 q6 226 130 130 130 q7 2220189418451845 q8 3274349434743474 q9 8870869289698692 q10 4101387437633763 q11 577 523 494 494 q12 819 689 638 638 q13 17196 314431653144 q14 311 305 303 303 q15 515 481 480 480 q16 510 436 427 427 q17 1839152915391529 q18 8116825378507850 q19 1768162716501627 q20 2100190018681868 q21 7205482847674767 q22 593 486 512 486 Total cold run time: 75837 ms Total hot run time: 56157 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]
doris-robot commented on PR #38215: URL: https://github.com/apache/doris/pull/38215#issuecomment-2254521290 TPC-DS: Total hot run time: 173373 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 0d988836fde4b274f87c6cbf6a5d5347e4c5b467, data reload: false query1 910 379 375 375 query2 6453200019091909 query3 6639211 215 211 query4 27461 17509 17336 17336 query5 3675481 488 481 query6 259 162 177 162 query7 4574304 302 302 query8 239 185 195 185 query9 8511247724222422 query10 437 290 263 263 query11 11780 10163 10037 10037 query12 113 87 84 84 query13 1621388 371 371 query14 9627722276447222 query15 212 162 163 162 query16 7519490 466 466 query17 1144559 531 531 query18 1799291 281 281 query19 192 140 138 138 query20 92 81 83 81 query21 224 107 102 102 query22 4242397738453845 query23 34105 33773 33768 33768 query24 11044 289429112894 query25 603 423 437 423 query26 718 158 162 158 query27 2317276 281 276 query28 6255215021172117 query29 784 427 429 427 query30 259 156 153 153 query31 995 754 782 754 query32 93 57 53 53 query33 650 349 336 336 query34 901 477 487 477 query35 859 753 787 753 query36 1119953 961 953 query37 141 81 84 81 query38 2924290228272827 query39 864 802 835 802 query40 186 110 109 109 query41 45 47 43 43 query42 112 100 92 92 query43 529 486 477 477 query44 1087719 717 717 query45 209 177 181 177 query46 1096742 719 719 query47 1822174817541748 query48 382 295 286 286 query49 841 408 409 408 query50 796 406 408 406 query51 6869677666336633 query52 104 90 92 90 query53 255 187 174 174 query54 834 444 443 443 query55 76 74 77 74 query56 292 255 274 255 query57 1143106110491049 query58 268 306 270 270 query59 3132265227562652 query60 315 288 281 281 query61 97 96 99 96 query62 785 660 642 642 query63 214 176 175 175 query64 9185225916981698 query65 3175311230973097 query66 733 355 346 346 query67 15589 14951 14893 14893 query68 6622567 567 567 query69 728 376 316 316 query70 1199113911391139 query71 469 268 283 268 query72 8361570056675667 query73 848 330 326 326 query74 6090567056595659 query75 3766266626782666 query76 39811018956 956 query77 690 301 312 301 query78 9747904589708970 query79 3452528 527 527 query80 1150488 490 488 query81 564 215 215 215 query82 453 139 136 136 query83 204 172 176 172 query84 269 80 82 80 query85 864 312 301 301 query86 337 315 302 302 query87 3237307730843077 query88 4326248924862486 query89 414 296 287 287 query90 1887197 195 195 query91 126 101 98 98 query92 57 48 48 48 query93 4795546 548 546 query94 864 291 290 290 query95 369 340 263 263 query96 613 279 275 275 query97 3154302430533024 query98 216 197 193 193 query99 1495126712881267 Total cold run time: 283872 ms Total hot run time: 173373 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]
doris-robot commented on PR #38215: URL: https://github.com/apache/doris/pull/38215#issuecomment-2254522879 ClickBench: Total hot run time: 30.47 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 0d988836fde4b274f87c6cbf6a5d5347e4c5b467, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.220.060.05 query4 1.670.070.08 query5 0.500.490.51 query6 1.160.730.71 query7 0.020.010.01 query8 0.050.040.05 query9 0.540.510.49 query10 0.550.540.55 query11 0.160.120.12 query12 0.140.120.12 query13 0.590.580.59 query14 0.770.760.80 query15 0.850.810.81 query16 0.360.350.37 query17 0.960.961.01 query18 0.220.210.21 query19 1.761.671.71 query20 0.020.010.01 query21 15.42 0.750.66 query22 4.177.391.90 query23 18.28 1.351.24 query24 1.990.240.23 query25 0.150.090.08 query26 0.300.210.20 query27 0.450.230.24 query28 13.27 1.031.00 query29 12.65 3.283.28 query30 0.260.060.05 query31 2.880.390.38 query32 3.310.480.46 query33 2.882.922.92 query34 17.25 4.334.32 query35 4.474.424.42 query36 0.650.460.48 query37 0.180.150.16 query38 0.160.150.14 query39 0.040.040.04 query40 0.150.130.13 query41 0.090.040.04 query42 0.050.040.05 query43 0.050.040.05 Total cold run time: 109.76 s Total hot run time: 30.47 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](nereids) support partitionTopn for multi window exprs [doris]
doris-robot commented on PR #38393: URL: https://github.com/apache/doris/pull/38393#issuecomment-2254522929 TPC-H: Total hot run time: 39385 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 303158a26f2d04939dca91a69f787f08d773b84a, data reload: false -- Round 1 -- q1 17675 433542794279 q2 2022200 193 193 q3 10448 118911191119 q4 10148 703 713 703 q5 7544272925982598 q6 215 135 136 135 q7 959 599 603 599 q8 9233190919301909 q9 8808656865406540 q10 8804378937803780 q11 449 248 254 248 q12 438 226 221 221 q13 17772 299829702970 q14 280 237 241 237 q15 514 489 496 489 q16 499 400 384 384 q17 979 659 683 659 q18 8042750574607460 q19 4256979 972 972 q20 679 336 331 331 q21 5023327132733271 q22 350 297 288 288 Total cold run time: 115137 ms Total hot run time: 39385 ms - Round 2, with runtime_filter_mode=off - q1 4376420842294208 q2 354 265 276 265 q3 3168274727942747 q4 1995176217601760 q5 5577559155155515 q6 228 143 135 135 q7 2178179817811781 q8 3299345933843384 q9 8834875289128752 q10 4171387238413841 q11 571 477 489 477 q12 825 643 627 627 q13 15889 319731113111 q14 326 284 284 284 q15 535 493 496 493 q16 486 454 432 432 q17 1813153315101510 q18 8114796079587958 q19 1749160715161516 q20 2057192618561856 q21 5206491548114811 q22 589 516 497 497 Total cold run time: 72340 ms Total hot run time: 55960 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](nereids) support partitionTopn for multi window exprs [doris]
doris-robot commented on PR #38393: URL: https://github.com/apache/doris/pull/38393#issuecomment-2254526406 TPC-DS: Total hot run time: 173456 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 303158a26f2d04939dca91a69f787f08d773b84a, data reload: false query1 907 368 368 368 query2 6433195618941894 query3 6627206 214 206 query4 28340 17562 17432 17432 query5 3652471 475 471 query6 267 184 172 172 query7 4578298 290 290 query8 261 203 189 189 query9 8546246924482448 query10 442 297 271 271 query11 11952 10065 10138 10065 query12 115 88 86 86 query13 1618386 373 373 query14 10085 766776457645 query15 237 174 170 170 query16 7685520 496 496 query17 1619564 552 552 query18 1996293 288 288 query19 194 150 147 147 query20 93 83 83 83 query21 207 106 101 101 query22 4338408738333833 query23 34051 33724 33635 33635 query24 11108 292129492921 query25 626 413 424 413 query26 717 161 161 161 query27 2317292 293 292 query28 6408211721152115 query29 837 448 468 448 query30 274 160 156 156 query31 991 778 773 773 query32 97 55 62 55 query33 784 354 425 354 query34 860 486 488 486 query35 864 737 748 737 query36 1112987 939 939 query37 140 88 83 83 query38 3065287327972797 query39 916 873 859 859 query40 199 118 113 113 query41 46 47 42 42 query42 120 98 106 98 query43 510 457 485 457 query44 1238726 724 724 query45 211 174 184 174 query46 1098735 748 735 query47 1839174817891748 query48 363 292 295 292 query49 852 414 418 414 query50 793 402 423 402 query51 6762671167176711 query52 102 87 91 87 query53 248 183 183 183 query54 879 455 452 452 query55 77 76 76 76 query56 295 267 274 267 query57 1128105510511051 query58 258 258 269 258 query59 2993292225982598 query60 309 309 287 287 query61 99 99 98 98 query62 804 645 650 645 query63 205 177 182 177 query64 9270227817411741 query65 3183311630843084 query66 730 332 325 325 query67 15519 14731 14670 14670 query68 8458581 570 570 query69 701 422 335 335 query70 1140110310171017 query71 523 281 284 281 query72 9142550058375500 query73 1236337 333 333 query74 6179562356735623 query75 5062265427082654 query76 51781008897 897 query77 813 320 301 301 query78 9638901889558955 query79 6916550 535 535 query80 1312511 553 511 query81 576 216 222 216 query82 796 141 138 138 query83 299 181 177 177 query84 268 82 83 82 query85 1347326 308 308 query86 402 306 318 306 query87 3234306830503050 query88 4307248024772477 query89 428 292 291 291 query90 1934204 198 198 query91 129 104 104 104 query92 64 48 52 48 query93 5202555 549 549 query94 837 286 299 286 query95 362 270 260 260 query96 612 282 275 275 query97 3257302530113011 query98 223 200 203 200 query99 1506126412551255 Total cold run time: 297570 ms Total hot run time: 173456 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [opt](nereids) support partitionTopn for multi window exprs [doris]
doris-robot commented on PR #38393: URL: https://github.com/apache/doris/pull/38393#issuecomment-2254528077 ClickBench: Total hot run time: 30.41 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 303158a26f2d04939dca91a69f787f08d773b84a, data reload: false query1 0.040.030.04 query2 0.080.030.04 query3 0.230.050.05 query4 1.680.080.08 query5 0.500.490.49 query6 1.130.730.73 query7 0.020.020.02 query8 0.050.040.04 query9 0.540.480.49 query10 0.530.530.55 query11 0.150.110.12 query12 0.150.120.13 query13 0.610.600.58 query14 0.750.790.76 query15 0.860.810.82 query16 0.380.370.37 query17 1.011.041.05 query18 0.230.210.22 query19 1.921.801.80 query20 0.010.000.00 query21 15.38 0.780.65 query22 3.958.311.58 query23 18.29 1.501.31 query24 2.110.220.22 query25 0.160.080.08 query26 0.310.210.22 query27 0.450.240.23 query28 13.37 1.030.98 query29 12.60 3.313.27 query30 0.250.060.06 query31 2.870.400.39 query32 3.260.480.47 query33 2.922.932.98 query34 16.98 4.344.35 query35 4.454.414.38 query36 0.650.460.48 query37 0.190.150.16 query38 0.160.150.15 query39 0.040.040.03 query40 0.170.130.12 query41 0.100.040.05 query42 0.050.050.04 query43 0.040.040.04 Total cold run time: 109.62 s Total hot run time: 30.41 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](jdbc catalog) improvement some jdbc catalog properties check order [doris]
doris-robot commented on PR #38439: URL: https://github.com/apache/doris/pull/38439#issuecomment-2254533323 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [improvement](jdbc catalog) improvement some jdbc catalog properties check order [doris]
zy-kkk opened a new pull request, #38439: URL: https://github.com/apache/doris/pull/38439 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](jdbc catalog) improvement some jdbc catalog properties check order [doris]
zy-kkk commented on PR #38439: URL: https://github.com/apache/doris/pull/38439#issuecomment-2254533436 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](jdbc catalog) improvement some jdbc catalog properties check order [doris]
doris-robot commented on PR #38439: URL: https://github.com/apache/doris/pull/38439#issuecomment-2254544621 TPC-H: Total hot run time: 39386 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 99b8e6718209d6c5d81d56d3a56b9e67f5b86fb7, data reload: false -- Round 1 -- q1 17640 432842994299 q2 2021199 195 195 q3 10437 117611131113 q4 10147 759 702 702 q5 7564269626722672 q6 216 138 137 137 q7 951 601 603 601 q8 9301190319141903 q9 8824655265306530 q10 8796380737983798 q11 482 246 255 246 q12 493 221 220 220 q13 18722 297930022979 q14 277 244 240 240 q15 524 478 479 478 q16 509 389 377 377 q17 985 684 704 684 q18 7949744174007400 q19 6958109510811081 q20 697 332 337 332 q21 4911311533043115 q22 356 319 284 284 Total cold run time: 118760 ms Total hot run time: 39386 ms - Round 2, with runtime_filter_mode=off - q1 4566425442534253 q2 378 262 275 262 q3 2993291828922892 q4 1992167717581677 q5 5656559354965496 q6 215 134 133 133 q7 2186182518341825 q8 3265338334073383 q9 8743875989328759 q10 4092390637683768 q11 616 516 496 496 q12 820 662 627 627 q13 16983 318531763176 q14 306 286 290 286 q15 533 518 501 501 q16 499 442 427 427 q17 1834159015241524 q18 8094819379227922 q19 1734156315751563 q20 2134186218491849 q21 8305483647554755 q22 627 516 539 516 Total cold run time: 76571 ms Total hot run time: 56090 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](partial-update) insert only without auto_inc column should not use partial update [doris]
cambyzju commented on PR #38229: URL: https://github.com/apache/doris/pull/38229#issuecomment-2254545803 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](partial-update) insert only without auto_inc column should not use partial update [doris]
cambyzju commented on code in PR #38229:
URL: https://github.com/apache/doris/pull/38229#discussion_r1694263143
##
regression-test/suites/unique_with_mow_p0/partial_update/test_partial_update_auto_inc.groovy:
##
@@ -0,0 +1,65 @@
+
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_partial_update_auto_inc") {
+String db = context.config.getDbNameByFile(context.file)
+sql "select 1;" // to create database
+
+for (def use_mow : [false, true]) {
+for (def use_nereids_planner : [false, true]) {
+logger.info("current params: use_mow: ${use_mow},
use_nereids_planner: ${use_nereids_planner}")
+connect(user = context.config.jdbcUser, password =
context.config.jdbcPassword, url = context.config.jdbcUrl) {
+sql "use ${db};"
+
+if (use_nereids_planner) {
+sql """ set enable_nereids_dml = true; """
+sql """ set enable_nereids_planner=true; """
+sql """ set enable_fallback_to_original_planner=false; """
+} else {
+sql """ set enable_nereids_dml = false; """
+sql """ set enable_nereids_planner = false; """
+}
+
+// create table
+sql """ DROP TABLE IF EXISTS
test_primary_key_partial_update_auto_inc """
+sql """ CREATE TABLE test_primary_key_partial_update_auto_inc (
+`id` BIGINT NOT NULL AUTO_INCREMENT,
+`name` varchar(65533) NOT NULL COMMENT "用户姓名" )
+UNIQUE KEY(`id`) DISTRIBUTED BY HASH(`id`) BUCKETS
1
+PROPERTIES("replication_num" = "1",
"enable_unique_key_merge_on_write" = "${use_mow}"); """
+
+sql """ set enable_unique_key_partial_update=true; """
+sql """ insert into
test_primary_key_partial_update_auto_inc(name) values("doris1"); """
+sql """ set enable_unique_key_partial_update=false; """
+sql """ insert into
test_primary_key_partial_update_auto_inc(name) values("doris2"); """
+sql "sync"
+
+qt_select_1 """ select name from
test_primary_key_partial_update_auto_inc order by name; """
+
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](jdbc catalog) improvement some jdbc catalog properties check order [doris]
doris-robot commented on PR #38439: URL: https://github.com/apache/doris/pull/38439#issuecomment-2254547654 TPC-DS: Total hot run time: 173424 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 99b8e6718209d6c5d81d56d3a56b9e67f5b86fb7, data reload: false query1 911 367 363 363 query2 6426194318811881 query3 6628209 213 209 query4 28400 17422 17505 17422 query5 3660513 511 511 query6 264 160 163 160 query7 4578307 307 307 query8 236 204 201 201 query9 8690249724522452 query10 449 291 277 277 query11 10521 10061 10120 10061 query12 121 91 85 85 query13 1643383 362 362 query14 10150 771777517717 query15 228 168 164 164 query16 7653471 441 441 query17 1306557 542 542 query18 1927301 287 287 query19 184 142 141 141 query20 92 86 82 82 query21 205 103 101 101 query22 4176401139133913 query23 34070 33897 33362 33362 query24 10885 294729022902 query25 599 395 393 393 query26 712 158 158 158 query27 2277281 287 281 query28 6343213321002100 query29 792 437 435 435 query30 264 159 157 157 query31 1009764 757 757 query32 96 84 56 56 query33 675 357 337 337 query34 888 482 519 482 query35 885 766 792 766 query36 1139955 947 947 query37 143 94 86 86 query38 2960290128072807 query39 946 828 800 800 query40 199 115 116 115 query41 47 44 43 43 query42 118 103 103 103 query43 514 481 463 463 query44 1128727 731 727 query45 207 177 175 175 query46 1088740 776 740 query47 1847175917481748 query48 363 306 294 294 query49 843 438 454 438 query50 812 414 410 410 query51 6853664666966646 query52 101 88 93 88 query53 265 190 184 184 query54 880 459 456 456 query55 76 78 77 77 query56 324 287 293 287 query57 1148103410341034 query58 275 275 284 275 query59 2885273826222622 query60 338 302 315 302 query61 121 115 117 115 query62 804 636 660 636 query63 223 186 188 186 query64 9267235118071807 query65 3237313931533139 query66 754 349 347 347 query67 15329 14835 14719 14719 query68 6261569 579 569 query69 706 416 363 363 query70 1186111511091109 query71 518 278 283 278 query72 9290537359955373 query73 812 331 330 330 query74 6024565056245624 query75 4731268526152615 query76 4353996 931 931 query77 791 315 314 314 query78 9689897591198975 query79 6482540 544 540 query80 2728500 512 500 query81 594 227 228 227 query82 973 142 137 137 query83 319 181 179 179 query84 268 83 81 81 query85 1471333 307 307 query86 472 287 285 285 query87 3261309330643064 query88 5397250624842484 query89 432 278 286 278 query90 2031199 200 199 query91 129 145 104 104 query92 61 51 49 49 query93 6014557 549 549 query94 1069283 295 283 query95 370 262 262 262 query96 617 271 279 271 query97 3208300430133004 query98 232 216 204 204 query99 1512125912551255 Total cold run time: 295178 ms Total hot run time: 173424 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t
Re: [PR] [improvement](jdbc catalog) improvement some jdbc catalog properties check order [doris]
doris-robot commented on PR #38439: URL: https://github.com/apache/doris/pull/38439#issuecomment-2254549521 ClickBench: Total hot run time: 30.79 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 99b8e6718209d6c5d81d56d3a56b9e67f5b86fb7, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.220.040.04 query4 1.680.070.08 query5 0.500.500.48 query6 1.120.720.73 query7 0.020.010.01 query8 0.050.040.04 query9 0.550.480.50 query10 0.540.540.54 query11 0.150.120.12 query12 0.150.120.12 query13 0.610.600.58 query14 0.790.780.78 query15 0.870.820.82 query16 0.360.360.35 query17 1.041.031.00 query18 0.230.220.22 query19 1.771.701.73 query20 0.010.010.01 query21 15.40 0.770.67 query22 4.856.561.92 query23 18.30 1.461.36 query24 2.120.240.24 query25 0.160.090.09 query26 0.290.220.21 query27 0.450.240.23 query28 13.35 1.011.02 query29 12.65 3.303.28 query30 0.250.070.05 query31 2.880.390.40 query32 3.260.480.47 query33 2.922.942.91 query34 17.22 4.384.33 query35 4.414.414.39 query36 0.650.460.49 query37 0.200.170.17 query38 0.160.160.16 query39 0.050.030.03 query40 0.150.130.13 query41 0.100.050.05 query42 0.060.050.06 query43 0.050.050.04 Total cold run time: 110.71 s Total hot run time: 30.79 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [chore](nereids) Added compatibility with mysql alias conflict (#38104) [doris]
Toms1999 opened a new pull request, #38440: URL: https://github.com/apache/doris/pull/38440 throw table name/alias conflict exception to keep same behavior with mysql for example: ```sql select * from test.a b, test.b ``` error: ``` Not unique table/alias: 'b' ``` ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](nereids) Added compatibility with mysql alias conflict (#38104) [doris]
doris-robot commented on PR #38440: URL: https://github.com/apache/doris/pull/38440#issuecomment-2254554649 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](partial-update) insert only without auto_inc column should not use partial update [doris]
doris-robot commented on PR #38229: URL: https://github.com/apache/doris/pull/38229#issuecomment-2254556904 TPC-H: Total hot run time: 39191 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 2e1557a5fdccb1d70df81e27cfee5b8954b47c3e, data reload: false -- Round 1 -- q1 17637 433442614261 q2 2015196 204 196 q3 10424 123210921092 q4 10138 725 671 671 q5 7519268426372637 q6 220 144 139 139 q7 943 597 604 597 q8 9225188318851883 q9 8707654764886488 q10 8695372737453727 q11 480 243 242 242 q12 453 226 220 220 q13 17768 301129592959 q14 263 227 231 227 q15 509 468 468 468 q16 501 374 369 369 q17 962 693 694 693 q18 8230759275257525 q19 5746105910891059 q20 674 326 343 326 q21 4958321631283128 q22 347 286 284 284 Total cold run time: 116414 ms Total hot run time: 39191 ms - Round 2, with runtime_filter_mode=off - q1 4467430442464246 q2 383 262 254 254 q3 2979282428622824 q4 1969173316441644 q5 5642555655025502 q6 224 133 132 132 q7 2180179018621790 q8 3256341833943394 q9 8756866987688669 q10 4156389237573757 q11 578 491 499 491 q12 785 627 655 627 q13 17015 320332053203 q14 308 309 285 285 q15 529 494 510 494 q16 484 430 447 430 q17 1801151615411516 q18 8087806377947794 q19 1785156617241566 q20 2168191518661866 q21 5140485549064855 q22 552 488 506 488 Total cold run time: 73244 ms Total hot run time: 55827 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](partial-update) insert only without auto_inc column should not use partial update [doris]
doris-robot commented on PR #38229: URL: https://github.com/apache/doris/pull/38229#issuecomment-2254560373 TPC-DS: Total hot run time: 173625 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 2e1557a5fdccb1d70df81e27cfee5b8954b47c3e, data reload: false query1 916 377 362 362 query2 6428187418801874 query3 6646206 212 206 query4 28103 17645 17110 17110 query5 3696474 473 473 query6 253 171 156 156 query7 4578289 289 289 query8 239 189 183 183 query9 8444245424302430 query10 447 288 275 275 query11 12023 10249 10183 10183 query12 120 91 84 84 query13 1623367 366 366 query14 10256 753377187533 query15 224 172 172 172 query16 7751494 440 440 query17 1608581 545 545 query18 1998294 287 287 query19 197 150 148 148 query20 94 88 84 84 query21 208 101 100 100 query22 4581415641394139 query23 34324 33819 33690 33690 query24 10537 288929772889 query25 609 390 385 385 query26 715 151 152 151 query27 2272282 286 282 query28 5960210321022102 query29 775 429 436 429 query30 249 152 155 152 query31 981 757 800 757 query32 94 55 70 55 query33 659 343 331 331 query34 890 491 506 491 query35 869 763 785 763 query36 1122972 939 939 query37 137 85 81 81 query38 2944288928602860 query39 884 800 810 800 query40 192 112 109 109 query41 43 45 42 42 query42 110 95 101 95 query43 501 479 452 452 query44 1060732 726 726 query45 216 177 171 171 query46 1072752 738 738 query47 1837176218021762 query48 358 298 287 287 query49 819 416 413 413 query50 778 401 404 401 query51 6777674966226622 query52 103 91 89 89 query53 259 184 174 174 query54 877 440 444 440 query55 74 73 72 72 query56 304 267 262 262 query57 1105105710521052 query58 264 261 285 261 query59 2910275827072707 query60 303 281 299 281 query61 92 92 91 91 query62 793 651 662 651 query63 210 182 181 181 query64 9149223417331733 query65 3200310931913109 query66 705 334 328 328 query67 15391 15166 14965 14965 query68 4557541 546 541 query69 511 351 309 309 query70 1126108111371081 query71 370 268 277 268 query72 7218600555045504 query73 759 326 327 326 query74 6230577457095709 query75 3331264926712649 query76 2192909 915 909 query77 421 316 307 307 query78 10378 895189288928 query79 2823518 532 518 query80 2565502 474 474 query81 592 273 221 221 query82 864 131 142 131 query83 307 179 179 179 query84 275 77 81 77 query85 1970304 300 300 query86 458 303 289 289 query87 3279309330933093 query88 4398249624952495 query89 395 293 283 283 query90 1772193 199 193 query91 124 100 101 100 query92 58 49 50 49 query93 4276548 539 539 query94 719 299 301 299 query95 344 261 268 261 query96 604 277 277 277 query97 3188304330333033 query98 233 201 202 201 query99 1691127213011272 Total cold run time: 281984 ms Total hot run time: 173625 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [fix](partial-update) insert only without auto_inc column should not use partial update [doris]
doris-robot commented on PR #38229: URL: https://github.com/apache/doris/pull/38229#issuecomment-2254561904 ClickBench: Total hot run time: 30.89 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 2e1557a5fdccb1d70df81e27cfee5b8954b47c3e, data reload: false query1 0.040.040.04 query2 0.080.040.04 query3 0.220.050.05 query4 1.690.080.07 query5 0.480.490.47 query6 1.120.730.73 query7 0.020.020.01 query8 0.050.050.05 query9 0.540.490.49 query10 0.550.540.54 query11 0.160.110.11 query12 0.150.130.12 query13 0.620.580.58 query14 0.750.780.78 query15 0.850.820.81 query16 0.370.370.38 query17 0.941.021.05 query18 0.220.210.21 query19 1.811.751.75 query20 0.020.010.01 query21 15.41 0.780.67 query22 4.476.762.06 query23 18.27 1.371.31 query24 2.140.260.23 query25 0.160.090.08 query26 0.300.210.21 query27 0.460.230.23 query28 13.17 1.020.99 query29 12.61 3.283.25 query30 0.250.060.05 query31 2.870.380.38 query32 3.300.470.47 query33 2.872.922.86 query34 17.01 4.364.35 query35 4.424.444.43 query36 0.640.490.47 query37 0.200.160.17 query38 0.170.160.16 query39 0.050.030.03 query40 0.150.120.13 query41 0.090.060.05 query42 0.070.060.04 query43 0.040.040.04 Total cold run time: 109.8 s Total hot run time: 30.89 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](doc) Remove useless doc files [doris]
zclllyybb commented on PR #38441: URL: https://github.com/apache/doris/pull/38441#issuecomment-2254581154 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [chore](doc) Remove useless doc files [doris]
zclllyybb opened a new pull request, #38441: URL: https://github.com/apache/doris/pull/38441 ## Proposed changes Issue Number: close #xxx Now they are maintained in doris-website. leave a copy here is useless and may leading to misunderstanding. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](doc) Remove useless doc files [doris]
doris-robot commented on PR #38441: URL: https://github.com/apache/doris/pull/38441#issuecomment-2254581134 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe (#38103) [doris]
zclllyybb opened a new pull request, #38442: URL: https://github.com/apache/doris/pull/38442 pick https://github.com/apache/doris/pull/38103 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe (#38103) [doris]
doris-robot commented on PR #38442: URL: https://github.com/apache/doris/pull/38442#issuecomment-2254581403 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe (#38103) [doris]
zclllyybb commented on PR #38442: URL: https://github.com/apache/doris/pull/38442#issuecomment-2254581413 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Feature] use cmake fetch instead of download with shell script [doris]
zclllyybb commented on issue #38333: URL: https://github.com/apache/doris/issues/38333#issuecomment-2254582881 cmake script is obviously harder to understand and modify than shells. could you explain what benifits will the refactor take? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix] add missing quote function docs [doris-website]
zclllyybb opened a new pull request, #923: URL: https://github.com/apache/doris-website/pull/923 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [DEBUG] debug hist [doris]
zhiqiang- commented on PR #38242: URL: https://github.com/apache/doris/pull/38242#issuecomment-2254590864 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [ci] print debug log in fragment_mgr.cpp [doris]
zhiqiang- commented on PR #38247: URL: https://github.com/apache/doris/pull/38247#issuecomment-2254591888 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [DEBUG] debug hist [doris]
zhiqiang- commented on PR #38242: URL: https://github.com/apache/doris/pull/38242#issuecomment-2254592133 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [DEBUG] debug hist [doris]
doris-robot commented on PR #38242: URL: https://github.com/apache/doris/pull/38242#issuecomment-2254601774 TPC-H: Total hot run time: 39329 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit acd50f8a804ccf91c96958bf7f89f8ddb3a4d50c, data reload: false -- Round 1 -- q1 17663 441142774277 q2 2004197 195 195 q3 10444 118711121112 q4 10151 701 697 697 q5 7566272326992699 q6 218 143 142 142 q7 945 612 589 589 q8 9230191619031903 q9 8718657365426542 q10 8747371537853715 q11 455 240 243 240 q12 435 229 225 225 q13 17771 303530103010 q14 284 236 240 236 q15 522 490 485 485 q16 516 398 387 387 q17 950 650 684 650 q18 8060742672997299 q19 5561998 1093998 q20 668 342 356 342 q21 5024330533663305 q22 337 281 291 281 Total cold run time: 116269 ms Total hot run time: 39329 ms - Round 2, with runtime_filter_mode=off - q1 4377424942254225 q2 373 261 277 261 q3 3024280828932808 q4 2034174316481648 q5 5638558255135513 q6 221 132 131 131 q7 2186187718971877 q8 3267336233983362 q9 8743880488918804 q10 4069392337663766 q11 616 530 506 506 q12 817 644 649 644 q13 15961 319832093198 q14 308 290 292 290 q15 530 510 485 485 q16 512 461 452 452 q17 1823153415191519 q18 8076802577587758 q19 1832166516971665 q20 2198187118661866 q21 5062487648204820 q22 574 517 495 495 Total cold run time: 72241 ms Total hot run time: 56093 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [DEBUG] debug hist [doris]
doris-robot commented on PR #38242: URL: https://github.com/apache/doris/pull/38242#issuecomment-2254604411 TPC-DS: Total hot run time: 173039 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit acd50f8a804ccf91c96958bf7f89f8ddb3a4d50c, data reload: false query1 919 379 374 374 query2 6445196919001900 query3 6640202 219 202 query4 26435 17552 17499 17499 query5 3700497 495 495 query6 275 182 188 182 query7 4578286 319 286 query8 254 206 201 201 query9 8639247924562456 query10 439 293 292 292 query11 10635 10005 10095 10005 query12 120 85 87 85 query13 1615372 356 356 query14 10217 766071547154 query15 217 171 168 168 query16 7566442 498 442 query17 1301563 532 532 query18 1718289 290 289 query19 242 143 145 143 query20 96 84 94 84 query21 209 103 108 103 query22 4212403638753875 query23 34052 33796 33518 33518 query24 11003 292029392920 query25 603 407 401 401 query26 733 158 158 158 query27 2239281 281 281 query28 6131210520962096 query29 783 423 453 423 query30 257 158 154 154 query31 956 756 764 756 query32 94 52 56 52 query33 669 339 329 329 query34 901 489 494 489 query35 882 747 779 747 query36 1109966 958 958 query37 154 88 85 85 query38 2956285928612859 query39 920 857 866 857 query40 216 125 130 125 query41 50 49 44 44 query42 115 101 103 101 query43 498 488 484 484 query44 1134735 762 735 query45 210 177 180 177 query46 1104721 733 721 query47 1821175217521752 query48 360 287 299 287 query49 834 429 429 429 query50 795 405 405 405 query51 6809673166916691 query52 102 86 92 86 query53 257 187 190 187 query54 928 445 441 441 query55 73 76 72 72 query56 298 282 277 277 query57 1132104610671046 query58 251 274 264 264 query59 2850262926502629 query60 313 293 298 293 query61 125 92 95 92 query62 799 642 656 642 query63 211 188 190 188 query64 9468226616991699 query65 3152313530993099 query66 748 348 350 348 query67 15280 14855 14744 14744 query68 5276564 567 564 query69 587 410 319 319 query70 1183113710701070 query71 507 279 279 279 query72 6949580954415441 query73 783 338 331 331 query74 6220575555935593 query75 3555269127072691 query76 3152918 948 918 query77 638 325 312 312 query78 9631905089808980 query79 3259527 531 527 query80 2383495 503 495 query81 598 221 229 221 query82 1437146 141 141 query83 312 178 182 178 query84 270 84 80 80 query85 1395327 297 297 query86 478 308 319 308 query87 3225305530383038 query88 4428249024782478 query89 401 286 294 286 query90 1731198 196 196 query91 127 105 102 102 query92 62 51 55 51 query93 3705559 547 547 query94 814 285 283 283 query95 347 261 266 261 query96 605 276 277 276 query97 3271304130503041 query98 228 206 204 204 query99 1598126712991267 Total cold run time: 280002 ms Total hot run time: 173039 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th
Re: [PR] [DEBUG] debug hist [doris]
doris-robot commented on PR #38242: URL: https://github.com/apache/doris/pull/38242#issuecomment-2254605971 ClickBench: Total hot run time: 30.08 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit acd50f8a804ccf91c96958bf7f89f8ddb3a4d50c, data reload: false query1 0.040.040.04 query2 0.080.040.04 query3 0.230.050.05 query4 1.680.070.08 query5 0.490.490.49 query6 1.130.720.72 query7 0.020.010.02 query8 0.050.040.04 query9 0.560.470.49 query10 0.550.540.53 query11 0.160.110.12 query12 0.160.120.12 query13 0.600.590.59 query14 0.770.780.79 query15 0.860.810.82 query16 0.360.350.37 query17 1.010.951.03 query18 0.230.220.21 query19 1.881.701.74 query20 0.010.010.01 query21 15.41 0.780.66 query22 4.048.051.44 query23 18.27 1.381.24 query24 2.160.240.22 query25 0.170.080.10 query26 0.290.210.21 query27 0.450.240.23 query28 13.19 1.020.99 query29 12.57 3.313.27 query30 0.250.060.06 query31 2.870.400.39 query32 3.260.490.47 query33 2.902.962.94 query34 17.08 4.344.36 query35 4.474.414.40 query36 0.640.480.47 query37 0.200.160.15 query38 0.170.150.16 query39 0.040.030.04 query40 0.160.130.12 query41 0.100.050.05 query42 0.060.040.05 query43 0.050.040.04 Total cold run time: 109.67 s Total hot run time: 30.08 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [draft](cloud-merge) Support to abort txn when coordinate be restart and do schema change [doris]
gavinchou commented on code in PR #37669:
URL: https://github.com/apache/doris/pull/37669#discussion_r1694295326
##
cloud/src/meta-service/meta_service_txn.cpp:
##
@@ -2553,6 +2568,100 @@ void
MetaServiceImpl::abort_sub_txn(::google::protobuf::RpcController* controlle
response->mutable_txn_info()->CopyFrom(txn_info);
}
+void
MetaServiceImpl::abort_txn_with_coordinator(::google::protobuf::RpcController*
controller,
+ const
AbortTxnWithCoordinatorRequest* request,
+
AbortTxnWithCoordinatorResponse* response,
+ ::google::protobuf::Closure*
done) {
+RPC_PREPROCESS(abort_txn_with_coordinator);
+if (!request->has_id() || !request->has_ip() ||
!request->has_start_time()) {
+code = MetaServiceCode::INVALID_ARGUMENT;
+msg = "invalid coordinate id, coordinate ip or coordinate start time.";
+return;
+}
+// TODO: For auth
+std::string cloud_unique_id = request->has_cloud_unique_id() ?
request->cloud_unique_id() : "";
+instance_id = get_instance_id(resource_mgr_, cloud_unique_id);
+if (instance_id.empty()) {
+code = MetaServiceCode::INVALID_ARGUMENT;
+ss << "cannot find instance_id with cloud_unique_id="
+ << (cloud_unique_id.empty() ? "(empty)" : cloud_unique_id);
+msg = ss.str();
+return;
+}
+RPC_RATE_LIMIT(abort_txn_with_coordinator);
+std::string begin_info_key = txn_info_key({instance_id, 0, 0});
+std::string end_info_key = txn_info_key({instance_id, INT64_MAX,
INT64_MAX});
+LOG(INFO) << "begin_info_key:" << hex(begin_info_key) << " end_info_key:"
<< hex(end_info_key);
+
+std::unique_ptr txn;
+TxnErrorCode err = txn_kv_->create_txn(&txn);
+if (err != TxnErrorCode::TXN_OK) {
+msg = "failed to create txn";
+code = cast_as(err);
+return;
+}
+std::unique_ptr it;
+int64_t abort_txn_cnt = 0;
+int64_t total_iteration_cnt = 0;
+bool need_commit = false;
+do {
+err = txn->get(begin_info_key, end_info_key, &it, true);
+if (err != TxnErrorCode::TXN_OK) {
+code = cast_as(err);
+ss << "failed to get txn info. err=" << err;
+msg = ss.str();
+LOG(WARNING) << msg;
+return;
+}
+
+while (it->has_next()) {
+total_iteration_cnt++;
+auto [k, v] = it->next();
+LOG(INFO) << "check txn info txn_info_key=" << hex(k);
+TxnInfoPB info_pb;
+if (!info_pb.ParseFromArray(v.data(), v.size())) {
+code = MetaServiceCode::PROTOBUF_PARSE_ERR;
+ss << "malformed txn running info";
+msg = ss.str();
+ss << " key=" << hex(k);
+LOG(WARNING) << ss.str();
+return;
+}
+const auto& coordinate = info_pb.coordinator();
+if (info_pb.status() == TxnStatusPB::TXN_STATUS_PREPARED &&
+coordinate.sourcetype() == TXN_SOURCE_TYPE_BE &&
coordinate.id() == request->id() &&
+coordinate.ip() == request->ip() &&
Review Comment:
what if there are multiple BEs on the same host?
##
cloud/src/meta-service/meta_service_txn.cpp:
##
@@ -2640,10 +2750,43 @@ void
MetaServiceImpl::check_txn_conflict(::google::protobuf::RpcController* cont
running_table_ids.end(), result.begin());
result.resize(iter - result.begin());
if (result.size() > 0) {
-response->set_finished(false);
-LOG(INFO) << "skip timeout txn count: " <<
skip_timeout_txn_cnt
- << " total iteration count: " <<
total_iteration_cnt;
-return;
+finished = false;
+std::vector,
int, int>> out;
+std::string_view key_view = k;
+key_view.remove_prefix(1);
+int ret = decode_key(&key_view, &out);
+if (ret != 0) [[unlikely]] {
+// decode version key error means this is something
wrong,
+// we can not continue this txn
+LOG(WARNING) << "failed to decode key, ret=" << ret <<
" key=" << hex(k);
+} else {
+DCHECK(out.size() == 5) << " key=" << hex(k) << " " <<
out.size();
+const std::string& decode_instance_id =
std::get<1>(std::get<0>(out[1]));
+int64_t db_id = std::get<0>(std::get<0>(out[3]));
+int64_t txn_id = std::get<0>(std::get<0>(out[4]));
+std::string conflict_txn_info_key =
+txn_info_key({decode_instance_id, db_id,
txn_id});
Review Com
Re: [PR] [draft](cloud-merge) Support shadow tablet to do cumulative compaction in cloud mode [doris]
gavinchou commented on code in PR #37293:
URL: https://github.com/apache/doris/pull/37293#discussion_r1694298446
##
gensrc/proto/cloud.proto:
##
@@ -536,6 +536,7 @@ message TabletCompactionJobPB {
optional int64 lease = 23; // prepare
optional int64 delete_bitmap_lock_initiator = 24;
optional int64 full_compaction_cnt = 25; // prepare
+optional bool judge_input_versions_range = 26;
Review Comment:
judge_input_versions_range -> check_input_versions_range
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [draft](cloud-merge) Support shadow tablet to do cumulative compaction in cloud mode [doris]
gavinchou commented on code in PR #37293:
URL: https://github.com/apache/doris/pull/37293#discussion_r1694299082
##
fe/fe-core/src/main/java/org/apache/doris/cloud/datasource/CloudInternalCatalog.java:
##
@@ -810,6 +810,48 @@ private void dropCloudPartition(long dbId, long tableId,
List partitionIds
}
}
+public void removeSchemaChangeJob(long dbId, long tableId, long indexId,
long partitionId, long tabletId)
+throws DdlException {
+Cloud.FinishTabletJobRequest.Builder finishTabletJobRequestBuilder =
Cloud.FinishTabletJobRequest.newBuilder();
+finishTabletJobRequestBuilder.setCloudUniqueId(Config.cloud_unique_id);
+
finishTabletJobRequestBuilder.setAction(Cloud.FinishTabletJobRequest.Action.ABORT);
+Cloud.TabletJobInfoPB.Builder tabletJobInfoPBBuilder =
Cloud.TabletJobInfoPB.newBuilder();
+
+Cloud.TabletIndexPB.Builder tabletIndexPBBuilder =
Cloud.TabletIndexPB.newBuilder();
+tabletIndexPBBuilder.setDbId(dbId);
+tabletIndexPBBuilder.setTableId(tableId);
+tabletIndexPBBuilder.setIndexId(indexId);
+tabletIndexPBBuilder.setPartitionId(partitionId);
+tabletIndexPBBuilder.setTabletId(tabletId);
+final Cloud.TabletIndexPB tabletIndex = tabletIndexPBBuilder.build();
+tabletJobInfoPBBuilder.setIdx(tabletIndex);
+final Cloud.TabletJobInfoPB tabletJobInfoPB =
tabletJobInfoPBBuilder.build();
+finishTabletJobRequestBuilder.setJob(tabletJobInfoPB);
+
+final Cloud.FinishTabletJobRequest request =
finishTabletJobRequestBuilder.build();
+
+Cloud.FinishTabletJobResponse response = null;
+int tryTimes = 0;
+while (tryTimes++ < Config.metaServiceRpcRetryTimes()) {
+try {
+response =
MetaServiceProxy.getInstance().finishTabletJob(request);
+if (response.getStatus().getCode() !=
Cloud.MetaServiceCode.KV_TXN_CONFLICT) {
+break;
+}
+} catch (RpcException e) {
+LOG.warn("tryTimes:{}, dropIndex RpcException", tryTimes, e);
+if (tryTimes + 1 >= Config.metaServiceRpcRetryTimes()) {
+throw new DdlException(e.getMessage());
+}
+}
+sleepSeveralMs();
+}
+
+if (response.getStatus().getCode() != Cloud.MetaServiceCode.OK) {
+LOG.warn("dropIndex response: {} ", response);
Review Comment:
typo? wrong log content
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [draft](cloud-merge) Support shadow tablet to do cumulative compaction in cloud mode [doris]
gavinchou commented on code in PR #37293:
URL: https://github.com/apache/doris/pull/37293#discussion_r1694299096
##
fe/fe-core/src/main/java/org/apache/doris/cloud/datasource/CloudInternalCatalog.java:
##
@@ -810,6 +810,48 @@ private void dropCloudPartition(long dbId, long tableId,
List partitionIds
}
}
+public void removeSchemaChangeJob(long dbId, long tableId, long indexId,
long partitionId, long tabletId)
+throws DdlException {
+Cloud.FinishTabletJobRequest.Builder finishTabletJobRequestBuilder =
Cloud.FinishTabletJobRequest.newBuilder();
+finishTabletJobRequestBuilder.setCloudUniqueId(Config.cloud_unique_id);
+
finishTabletJobRequestBuilder.setAction(Cloud.FinishTabletJobRequest.Action.ABORT);
+Cloud.TabletJobInfoPB.Builder tabletJobInfoPBBuilder =
Cloud.TabletJobInfoPB.newBuilder();
+
+Cloud.TabletIndexPB.Builder tabletIndexPBBuilder =
Cloud.TabletIndexPB.newBuilder();
+tabletIndexPBBuilder.setDbId(dbId);
+tabletIndexPBBuilder.setTableId(tableId);
+tabletIndexPBBuilder.setIndexId(indexId);
+tabletIndexPBBuilder.setPartitionId(partitionId);
+tabletIndexPBBuilder.setTabletId(tabletId);
+final Cloud.TabletIndexPB tabletIndex = tabletIndexPBBuilder.build();
+tabletJobInfoPBBuilder.setIdx(tabletIndex);
+final Cloud.TabletJobInfoPB tabletJobInfoPB =
tabletJobInfoPBBuilder.build();
+finishTabletJobRequestBuilder.setJob(tabletJobInfoPB);
+
+final Cloud.FinishTabletJobRequest request =
finishTabletJobRequestBuilder.build();
+
+Cloud.FinishTabletJobResponse response = null;
+int tryTimes = 0;
+while (tryTimes++ < Config.metaServiceRpcRetryTimes()) {
+try {
+response =
MetaServiceProxy.getInstance().finishTabletJob(request);
+if (response.getStatus().getCode() !=
Cloud.MetaServiceCode.KV_TXN_CONFLICT) {
+break;
+}
+} catch (RpcException e) {
+LOG.warn("tryTimes:{}, dropIndex RpcException", tryTimes, e);
Review Comment:
typo?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [draft](cloud-merge) Support shadow tablet to do cumulative compaction in cloud mode [doris]
gavinchou commented on code in PR #37293:
URL: https://github.com/apache/doris/pull/37293#discussion_r1694299814
##
fe/fe-core/src/main/java/org/apache/doris/alter/CloudSchemaChangeJobV2.java:
##
@@ -106,14 +110,39 @@ protected void commitShadowIndex() throws
AlterCancelException {
}
@Override
-protected void postProcessShadowIndex() {
+protected void onCancel() {
if (Config.enable_check_compatibility_mode) {
LOG.info("skip drop shadown indexes in checking compatibility
mode");
return;
}
List shadowIdxList =
indexIdMap.keySet().stream().collect(Collectors.toList());
dropIndex(shadowIdxList);
+
+long tryTimes = 1;
+while (true) {
+try {
+Set>> tableSet =
partitionIndexTabletMap.cellSet();
+Iterator>> it =
tableSet.iterator();
+while (it.hasNext()) {
+Table.Cell> data = it.next();
+Long partitionId = data.getRowKey();
+Long shadowIndexId = data.getColumnKey();
+Long originIndexId = indexIdMap.get(shadowIndexId);
+Map shadowTabletIdToOriginTabletId =
data.getValue();
+for (Map.Entry entry :
shadowTabletIdToOriginTabletId.entrySet()) {
+Long originTabletId = entry.getValue();
+((CloudInternalCatalog)
Env.getCurrentInternalCatalog())
+.removeSchemaChangeJob(dbId, tableId,
originIndexId, partitionId, originTabletId);
+}
+}
Review Comment:
log here:
* which table/index has been processed (id name etc.)
* how many talbets have been processed here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
