Re: [PR] [Enhancement] (nereids)implement showDataTypesCommand in nereids [doris]
doris-robot commented on PR #44299: URL: https://github.com/apache/doris/pull/44299#issuecomment-2552072612 TPC-H: Total hot run time: 39813 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 09be3f21cdc8b684d0fb6ea1e185e9d1b40e16a6, data reload: false -- Round 1 -- q1 17589 748973067306 q2 2050186 170 170 q3 10554 111411781114 q4 10225 746 665 665 q5 7615273426362636 q6 240 150 151 150 q7 998 629 613 613 q8 9257185619101856 q9 6634644963836383 q10 6942227322922273 q11 460 263 265 263 q12 441 227 226 226 q13 17778 299329792979 q14 249 226 217 217 q15 552 510 512 510 q16 645 602 586 586 q17 973 575 543 543 q18 7264680667046704 q19 1334957 1002957 q20 456 190 186 186 q21 4325329731653165 q22 387 311 312 311 Total cold run time: 106968 ms Total hot run time: 39813 ms - Round 2, with runtime_filter_mode=off - q1 7328728273307282 q2 335 233 233 233 q3 2898277230312772 q4 2117187717891789 q5 5610564755775577 q6 221 143 142 142 q7 2210185817481748 q8 3309355234863486 q9 8842889588848884 q10 3585358635373537 q11 619 520 500 500 q12 788 625 562 562 q13 13029 315331213121 q14 314 280 278 278 q15 561 523 513 513 q16 688 627 642 627 q17 1785160115841584 q18 7854747773397339 q19 1645157214711471 q20 2033183917991799 q21 5329521252585212 q22 634 572 626 572 Total cold run time: 71734 ms Total hot run time: 59028 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Opt](parquet-reader) parquet reader merge io. [doris]
kaka11chen commented on PR #45562: URL: https://github.com/apache/doris/pull/45562#issuecomment-2551916885 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement helpCommand in nereids [doris]
msridhar78 commented on PR #44819: URL: https://github.com/apache/doris/pull/44819#issuecomment-2551862417 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [feat](docker)Add a BE ENV item 'SKIP_CHECK_ULIMIT' for Docker to start quickly #45267 [doris]
github-actions[bot] commented on PR #45468: URL: https://github.com/apache/doris/pull/45468#issuecomment-2551885100 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showSyncJobCommand in nereids [doris]
msridhar78 commented on PR #44817: URL: https://github.com/apache/doris/pull/44817#issuecomment-2551859346 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](cloud) reduce cache hotspot table write amplification [doris]
github-actions[bot] commented on PR #45557: URL: https://github.com/apache/doris/pull/45557#issuecomment-2551950348 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](cloud) reduce cache hotspot table write amplification [doris]
github-actions[bot] commented on PR #45557: URL: https://github.com/apache/doris/pull/45557#issuecomment-2551950415 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showSyncJobCommand in nereids [doris]
doris-robot commented on PR #44817: URL: https://github.com/apache/doris/pull/44817#issuecomment-2551942092 TPC-DS: Total hot run time: 196143 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 9f2b09176f6a63df73a725defac6d68be4327946, data reload: false query1 1298947 963 947 query2 6243234223252325 query3 11125 478848504788 query4 33035 23448 23572 23448 query5 3718456 456 456 query6 263 208 171 171 query7 3975309 302 302 query8 301 244 228 228 query9 9465270026982698 query10 464 246 254 246 query11 17657 15134 15169 15134 query12 158 104 104 104 query13 1589428 424 424 query14 8835676468656764 query15 268 201 201 201 query16 8262442 437 437 query17 1586617 600 600 query18 2179299 314 299 query19 213 176 161 161 query20 123 112 115 112 query21 211 109 108 108 query22 4990443045954430 query23 34788 33561 33796 33561 query24 11725 253224742474 query25 670 402 409 402 query26 1296154 153 153 query27 3307344 338 338 query28 7527245424462446 query29 900 427 436 427 query30 234 151 155 151 query31 1045857 847 847 query32 101 61 59 59 query33 772 334 309 309 query34 1130543 532 532 query35 877 777 796 777 query36 1119944 964 944 query37 146 75 105 75 query38 4323419941514151 query39 1501146314711463 query40 215 110 104 104 query41 50 46 44 44 query42 118 109 101 101 query43 550 496 495 495 query44 1333853 833 833 query45 192 170 176 170 query46 1210737 749 737 query47 2039192619171917 query48 425 326 336 326 query49 947 387 388 387 query50 817 421 408 408 query51 7431730073557300 query52 100 99 92 92 query53 265 190 188 188 query54 1097417 433 417 query55 82 79 84 79 query56 270 256 242 242 query57 1282115511591155 query58 230 215 235 215 query59 3283317629982998 query60 297 244 246 244 query61 112 108 107 107 query62 871 695 700 695 query63 235 200 195 195 query64 4046700 700 700 query65 3289326732493249 query66 833 301 328 301 query67 16291 15512 15468 15468 query68 6166556 534 534 query69 505 256 256 256 query70 1168116011301130 query71 514 248 244 244 query72 6606406739633963 query73 787 355 364 355 query74 10353 885987148714 query75 4044263926392639 query76 4427113711671137 query77 692 281 287 281 query78 10271 939293059305 query79 1519604 590 590 query80 944 445 510 445 query81 515 235 240 235 query82 733 117 118 117 query83 196 144 140 140 query84 281 68 71 68 query85 1373326 310 310 query86 460 290 287 287 query87 4561441344434413 query88 3855221721882188 query89 423 286 293 286 query90 1884187 187 187 query91 150 105 103 103 query92 68 51 49 49 query93 2602557 533 533 query94 748 290 284 284 query95 355 247 251 247 query96 629 279 278 278 query97 2875271026672667 query98 214 200 199 199 query99 1622132313001300 Total cold run time: 306869 ms Total hot run time: 196143 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [opt](Nereids) lock table in ascending order of table IDs [doris]
doris-robot commented on PR #45045: URL: https://github.com/apache/doris/pull/45045#issuecomment-2551941427 ClickBench: Total hot run time: 32.03 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 7bfbff09eede59e7a7bb94ec982dce76457441c6, data reload: false query1 0.040.030.05 query2 0.070.040.03 query3 0.230.070.07 query4 1.620.110.10 query5 0.430.410.42 query6 1.170.660.65 query7 0.020.020.02 query8 0.040.040.03 query9 0.590.500.49 query10 0.540.590.56 query11 0.150.110.10 query12 0.140.110.11 query13 0.620.600.59 query14 2.872.752.74 query15 0.900.820.83 query16 0.360.390.39 query17 1.001.071.06 query18 0.230.210.21 query19 1.921.902.05 query20 0.010.010.01 query21 15.36 0.590.60 query22 2.731.682.04 query23 17.10 1.020.82 query24 2.840.390.74 query25 0.350.100.18 query26 0.250.130.14 query27 0.050.050.04 query28 11.68 1.101.07 query29 12.66 3.283.29 query30 0.240.060.07 query31 2.850.370.40 query32 3.240.460.46 query33 3.073.083.15 query34 16.97 4.424.44 query35 4.474.484.44 query36 0.680.470.47 query37 0.090.070.07 query38 0.050.030.04 query39 0.040.020.03 query40 0.160.120.12 query41 0.080.020.02 query42 0.040.020.02 query43 0.040.030.03 Total cold run time: 107.99 s Total hot run time: 32.03 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement alterCatalogRenameCommand in nereids [doris]
msridhar78 commented on PR #45163: URL: https://github.com/apache/doris/pull/45163#issuecomment-2551941104 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](script) fix `start_fe.sh --version` not work and MetaService scripts occur error in Debian GNU/Linux 11 (bullseye) [doris]
github-actions[bot] commented on PR #45610: URL: https://github.com/apache/doris/pull/45610#issuecomment-2551943915 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression-test](fix) Fix the incorrect invocation of Awaitility.await.untilAsserted [doris]
github-actions[bot] commented on PR #45605: URL: https://github.com/apache/doris/pull/45605#issuecomment-2551944919 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression-test](fix) Fix the incorrect invocation of Awaitility.await.untilAsserted [doris]
github-actions[bot] commented on PR #45605: URL: https://github.com/apache/doris/pull/45605#issuecomment-2551944977 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](script) fix `start_fe.sh --version` not work and MetaService scripts occur error in Debian GNU/Linux 11 (bullseye) [doris]
github-actions[bot] commented on PR #45610: URL: https://github.com/apache/doris/pull/45610#issuecomment-2551943864 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement alterCatalogRenameCommand in nereids [doris]
msridhar78 commented on PR #45163: URL: https://github.com/apache/doris/pull/45163#issuecomment-2551945082 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression](metrics) add a regression for version metrics [doris]
gavinchou commented on code in PR #45593:
URL: https://github.com/apache/doris/pull/45593#discussion_r1890632378
##
regression-test/suites/metrics_p0/test_version_metrics.groovy:
##
@@ -0,0 +1,60 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_version_metrics") {
+httpTest {
+endpoint context.config.feHttpAddress
+uri "/metrics"
+op "get"
+check { code, body ->
+logger.debug("code:${code} body:${body}");
+assertEquals(200, code)
+assertTrue(body.contains("doris_fe_version"))
Review Comment:
also check the version is numeric
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression](metrics) add a regression for version metrics [doris]
gavinchou commented on code in PR #45593:
URL: https://github.com/apache/doris/pull/45593#discussion_r1890632583
##
regression-test/suites/metrics_p0/test_version_metrics.groovy:
##
@@ -0,0 +1,60 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_version_metrics") {
+httpTest {
+endpoint context.config.feHttpAddress
+uri "/metrics"
+op "get"
+check { code, body ->
+logger.debug("code:${code} body:${body}");
+assertEquals(200, code)
+assertTrue(body.contains("doris_fe_version"))
+}
+}
+
+def res = sql_return_maparray("show backends")
+def beBrpcEndpoint = res[0].Host + ":" + res[0].BrpcPort
+
+httpTest {
+endpoint beBrpcEndpoint
+uri "/brpc_metrics"
+op "get"
+check { code, body ->
+logger.debug("code:${code} body:${body}");
+assertEquals(200, code)
+assertTrue(body.contains("doris_be_version"))
Review Comment:
ditto
##
regression-test/suites/metrics_p0/test_version_metrics.groovy:
##
@@ -0,0 +1,60 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_version_metrics") {
+httpTest {
+endpoint context.config.feHttpAddress
+uri "/metrics"
+op "get"
+check { code, body ->
+logger.debug("code:${code} body:${body}");
+assertEquals(200, code)
+assertTrue(body.contains("doris_fe_version"))
+}
+}
+
+def res = sql_return_maparray("show backends")
+def beBrpcEndpoint = res[0].Host + ":" + res[0].BrpcPort
+
+httpTest {
+endpoint beBrpcEndpoint
+uri "/brpc_metrics"
+op "get"
+check { code, body ->
+logger.debug("code:${code} body:${body}");
+assertEquals(200, code)
+assertTrue(body.contains("doris_be_version"))
+}
+}
+
+if (cluster.isRunning() && cluster.isCloudMode()) {
+
+def ms = cluster.getAllMetaservices().get(0)
+def msEndpoint = ms.host + ":" + ms.httpPort
+
+httpTest {
+endpoint msEndpoint
+uri "/brpc_metrics"
+op "get"
+check { code, body ->
+logger.debug("code:${code} body:${body}");
+assertEquals(200, code)
+assertTrue(body.contains("doris_cloud_version"))
Review Comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showStoragePolicyCommand in nereids [doris]
msridhar78 commented on PR #44393: URL: https://github.com/apache/doris/pull/44393#issuecomment-2551889323 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showDataTypesCommand in nereids [doris]
msridhar78 commented on PR #44299: URL: https://github.com/apache/doris/pull/44299#issuecomment-2551890882 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showUserPropertiesCommand and showAllPropertiesCommand in nereids [doris]
msridhar78 commented on PR #45145: URL: https://github.com/apache/doris/pull/45145#issuecomment-2551893963 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement DropCatalogCommand in nereids [doris]
msridhar78 commented on PR #45372: URL: https://github.com/apache/doris/pull/45372#issuecomment-2551951776 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [test](regression-test) add a regression test case for s3 vault #45261 [doris]
gavinchou merged PR #45548: URL: https://github.com/apache/doris/pull/45548 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-3.0 updated: branch-3.0: [test](regression-test) add a regression test case for s3 vault #45261 (#45548)
This is an automated email from the ASF dual-hosted git repository.
gavinchou pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 73a313ce2d6 branch-3.0: [test](regression-test) add a regression test
case for s3 vault #45261 (#45548)
73a313ce2d6 is described below
commit 73a313ce2d6f35b24d065757dc8d1fdbcde58b48
Author: github-actions[bot]
<41898282+github-actions[bot]@users.noreply.github.com>
AuthorDate: Thu Dec 19 01:59:42 2024 +0800
branch-3.0: [test](regression-test) add a regression test case for s3 vault
#45261 (#45548)
Cherry-picked from #45261
Co-authored-by: Xujian Duan <50550370+darvend...@users.noreply.github.com>
---
.../test_s3_vault_path_start_with_slash.groovy | 70 ++
1 file changed, 70 insertions(+)
diff --git
a/regression-test/suites/vault_p0/default/test_s3_vault_path_start_with_slash.groovy
b/regression-test/suites/vault_p0/default/test_s3_vault_path_start_with_slash.groovy
new file mode 100644
index 000..704a4b67836
--- /dev/null
+++
b/regression-test/suites/vault_p0/default/test_s3_vault_path_start_with_slash.groovy
@@ -0,0 +1,70 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_s3_vault_path_start_with_slash", "nonConcurrent") {
+if (!isCloudMode()) {
+logger.info("skip ${name} case, because not cloud mode")
+return
+}
+
+if (!enableStoragevault()) {
+logger.info("skip ${name} case")
+return
+}
+
+def tableName = "table_test_s3_vault_path_start_with_slash"
+try {
+def vault_name = "test_s3_vault_path_start_with_slash_vault"
+sql """
+CREATE STORAGE VAULT IF NOT EXISTS ${vault_name}
+PROPERTIES (
+"type"="S3",
+"s3.endpoint"="${getS3Endpoint()}",
+"s3.region" = "${getS3Region()}",
+"s3.access_key" = "${getS3AK()}",
+"s3.secret_key" = "${getS3SK()}",
+"s3.root.path" = "/test_s3_vault_path_start_with_slash_vault",
+"s3.bucket" = "${getS3BucketName()}",
+"s3.external_endpoint" = "",
+"provider" = "${getS3Provider()}",
+"set_as_default" = "true"
+);
+"""
+
+sql "DROP TABLE IF EXISTS ${tableName}"
+sql """
+CREATE TABLE ${tableName} (
+`key` INT,
+value INT
+) DUPLICATE KEY (`key`)
+DISTRIBUTED BY HASH (`key`) BUCKETS 1
+PROPERTIES (
+"replication_num" = "1",
+"storage_vault_name" = "${vault_name}"
+)
+"""
+
+sql """ insert into ${tableName} values(1, 1); """
+sql """ sync;"""
+def result = sql """ select * from ${tableName}; """
+logger.info("result:${result}");
+assertTrue(result.size() == 1)
+assertTrue(result[0][0].toInteger() == 1)
+} finally {
+sql "DROP TABLE IF EXISTS ${tableName}"
+}
+}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-3.0 updated: branch-3.0: [fix](export) Make enable_outfile_to_local available for export command #45241 (#45514)
This is an automated email from the ASF dual-hosted git repository.
gavinchou pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new d8170c5aa2e branch-3.0: [fix](export) Make enable_outfile_to_local
available for export command #45241 (#45514)
d8170c5aa2e is described below
commit d8170c5aa2e704db44a80db1f4c3465a1cf8eeb3
Author: github-actions[bot]
<41898282+github-actions[bot]@users.noreply.github.com>
AuthorDate: Thu Dec 19 02:00:50 2024 +0800
branch-3.0: [fix](export) Make enable_outfile_to_local available for export
command #45241 (#45514)
Cherry-picked from #45241
Co-authored-by: Siyang Tang
---
fe/fe-core/src/main/java/org/apache/doris/analysis/ExportStmt.java | 6 ++
.../apache/doris/nereids/trees/plans/commands/ExportCommand.java| 4
2 files changed, 10 insertions(+)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/ExportStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ExportStmt.java
index 1ca6c4e3f5b..cba01d22fca 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/ExportStmt.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/ExportStmt.java
@@ -47,6 +47,7 @@ import lombok.Getter;
import java.util.List;
import java.util.Map;
+import java.util.Objects;
import java.util.Optional;
import java.util.UUID;
@@ -146,6 +147,11 @@ public class ExportStmt extends StatementBase implements
NotFallbackInParser {
public void analyze(Analyzer analyzer) throws UserException {
super.analyze(analyzer);
+if (!Config.enable_outfile_to_local && Objects.requireNonNull(path)
+.startsWith(OutFileClause.LOCAL_FILE_PREFIX)) {
+throw new AnalysisException("`enable_outfile_to_local` = false,
exporting file to local fs is disabled.");
+}
+
tableRef = analyzer.resolveTableRef(tableRef);
Preconditions.checkNotNull(tableRef);
tableRef.analyze(analyzer);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExportCommand.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExportCommand.java
index c5d61f58586..b08c6a6b54a 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExportCommand.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExportCommand.java
@@ -140,6 +140,10 @@ public class ExportCommand extends Command implements
ForwardWithSync {
tblName.getDb() + ": " + tblName.getTbl());
}
+if (!Config.enable_outfile_to_local &&
path.startsWith(OutFileClause.LOCAL_FILE_PREFIX)) {
+throw new AnalysisException("`enable_outfile_to_local` = false,
exporting file to local fs is disabled.");
+}
+
// check phases
checkAllParameters(ctx, tblName, fileProperties);
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45241-branch-3.0 deleted (was a409e5ea527)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45241-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git was a409e5ea527 [fix](export) Make enable_outfile_to_local available for export command (#45241) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showSyncJobCommand in nereids [doris]
doris-robot commented on PR #44817: URL: https://github.com/apache/doris/pull/44817#issuecomment-2551951555 ClickBench: Total hot run time: 32.65 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 9f2b09176f6a63df73a725defac6d68be4327946, data reload: false query1 0.040.040.03 query2 0.100.030.04 query3 0.230.070.07 query4 1.610.100.11 query5 0.430.390.42 query6 1.150.650.65 query7 0.020.020.02 query8 0.040.030.03 query9 0.580.510.47 query10 0.540.550.55 query11 0.150.110.11 query12 0.140.110.11 query13 0.600.600.60 query14 2.712.732.74 query15 0.900.820.83 query16 0.390.370.38 query17 1.071.091.04 query18 0.230.210.21 query19 1.841.851.91 query20 0.020.010.01 query21 15.36 0.590.59 query22 2.372.061.68 query23 17.05 0.940.88 query24 3.290.950.99 query25 0.240.150.10 query26 0.470.140.13 query27 0.050.050.04 query28 10.75 1.101.08 query29 12.57 3.273.23 query30 0.240.070.07 query31 2.860.380.38 query32 3.250.480.46 query33 3.093.113.17 query34 16.96 4.494.46 query35 4.504.564.53 query36 0.680.470.48 query37 0.090.060.06 query38 0.040.030.04 query39 0.030.020.03 query40 0.160.130.12 query41 0.080.020.02 query42 0.030.020.02 query43 0.030.030.03 Total cold run time: 106.98 s Total hot run time: 32.65 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45261-branch-3.0 deleted (was af0f3990abe)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45261-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git was af0f3990abe [test](regression-test) add a regression test case for s3 vault (#45261) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [fix](export) Make enable_outfile_to_local available for export command #45241 [doris]
gavinchou merged PR #45514: URL: https://github.com/apache/doris/pull/45514 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showStoragePolicyCommand in nereids [doris]
doris-robot commented on PR #44393: URL: https://github.com/apache/doris/pull/44393#issuecomment-2551959037 TPC-H: Total hot run time: 39550 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit bd828574107e20c1877100458daf7b0e7b4f6e09, data reload: false -- Round 1 -- q1 17551 745872467246 q2 2047172 167 167 q3 10737 106911991069 q4 10570 755 653 653 q5 7608267227182672 q6 238 146 144 144 q7 998 624 599 599 q8 9246184619251846 q9 6632640063526352 q10 6978229723292297 q11 464 266 260 260 q12 429 224 216 216 q13 17760 294429772944 q14 255 204 210 204 q15 549 522 509 509 q16 659 598 591 591 q17 982 506 544 506 q18 7114669867096698 q19 13381058915 915 q20 462 179 180 179 q21 4052325931783178 q22 380 309 305 305 Total cold run time: 107049 ms Total hot run time: 39550 ms - Round 2, with runtime_filter_mode=off - q1 7278725472927254 q2 338 232 236 232 q3 2954292730252927 q4 2109184819171848 q5 5826577956615661 q6 227 139 140 139 q7 2263184618351835 q8 3370356835133513 q9 8956901289378937 q10 3609353535653535 q11 608 516 499 499 q12 844 634 592 592 q13 13812 320732863207 q14 339 297 300 297 q15 577 509 510 509 q16 729 661 647 647 q17 1871165716521652 q18 8648847279777977 q19 2916159216321592 q20 2117185718061806 q21 552054795479 q22 689 583 596 583 Total cold run time: 75635 ms Total hot run time: 60721 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement helpCommand in nereids [doris]
doris-robot commented on PR #44819: URL: https://github.com/apache/doris/pull/44819#issuecomment-2551959231 TPC-DS: Total hot run time: 196581 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 461df0d6418747e10e5a53fb0f4b5920654d5154, data reload: false query1 1285975 903 903 query2 6252234223562342 query3 10964 451946424519 query4 33489 23695 23334 23334 query5 4616452 435 435 query6 264 179 176 176 query7 3995312 299 299 query8 320 266 237 237 query9 9464272327062706 query10 478 245 243 243 query11 18049 15090 15109 15090 query12 153 104 100 100 query13 1579420 409 409 query14 9844692267666766 query15 288 211 192 192 query16 8319451 482 451 query17 1560608 639 608 query18 2160305 311 305 query19 216 167 164 164 query20 125 118 117 117 query21 205 109 107 107 query22 4901458745874587 query23 34622 34034 34009 34009 query24 11286 251724592459 query25 677 404 424 404 query26 1746161 160 160 query27 2848326 336 326 query28 7727248324792479 query29 1033431 432 431 query30 237 150 155 150 query31 1059819 838 819 query32 98 55 57 55 query33 790 318 303 303 query34 1022514 525 514 query35 900 775 791 775 query36 1091945 953 945 query37 193 72 75 72 query38 4458410741794107 query39 1511143514701435 query40 270 100 103 100 query41 45 46 43 43 query42 112 110 103 103 query43 547 499 502 499 query44 1338867 837 837 query45 197 169 176 169 query46 1165723 735 723 query47 2011194519501945 query48 421 342 338 338 query49 1143440 413 413 query50 824 407 402 402 query51 7360726771367136 query52 108 92 93 92 query53 264 192 186 186 query54 1186431 419 419 query55 86 79 79 79 query56 266 234 247 234 query57 1325119011751175 query58 245 224 234 224 query59 3406312931883129 query60 289 242 252 242 query61 106 112 108 108 query62 886 684 691 684 query63 224 204 204 204 query64 5133773 722 722 query65 3298323932633239 query66 1352318 323 318 query67 16086 15513 15581 15513 query68 6122552 543 543 query69 518 263 248 248 query70 1141114111391139 query71 510 257 255 255 query72 6860407740934077 query73 798 362 356 356 query74 10009 882888638828 query75 3723264426292629 query76 3845104910621049 query77 567 260 290 260 query78 10295 941096269410 query79 1335616 595 595 query80 949 422 433 422 query81 509 232 234 232 query82 656 121 123 121 query83 202 139 152 139 query84 283 69 71 69 query85 1361315 295 295 query86 415 319 297 297 query87 4431456244824482 query88 3335223821762176 query89 423 294 289 289 query90 2164185 186 185 query91 144 102 110 102 query92 65 53 52 52 query93 1821531 538 531 query94 941 274 287 274 query95 337 245 244 244 query96 607 280 287 280 query97 2872275126982698 query98 218 203 193 193 query99 1633130213291302 Total cold run time: 308606 ms Total hot run time: 196581 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [Enhancement] (nereids)implement showStorageVaultCommand in nereids [doris]
doris-robot commented on PR #44805: URL: https://github.com/apache/doris/pull/44805#issuecomment-2551958002 TPC-H: Total hot run time: 39461 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 6aff26af6f5bb38e64c59b649e6f1088223309ca, data reload: false -- Round 1 -- q1 17603 742973167316 q2 2061179 168 168 q3 10777 106911781069 q4 10547 758 735 735 q5 7598267826312631 q6 234 144 141 141 q7 982 622 595 595 q8 9247188019101880 q9 6587636363356335 q10 7030228423022284 q11 468 266 258 258 q12 425 230 221 221 q13 17820 284929272849 q14 240 209 217 209 q15 564 499 496 496 q16 691 573 595 573 q17 968 495 522 495 q18 7201681266326632 q19 13441008942 942 q20 455 190 181 181 q21 3983313633973136 q22 376 316 315 315 Total cold run time: 107201 ms Total hot run time: 39461 ms - Round 2, with runtime_filter_mode=off - q1 7218722872037203 q2 328 236 232 232 q3 2897277928412779 q4 2085188918381838 q5 5713562856825628 q6 223 141 138 138 q7 2214186018061806 q8 3377353334993499 q9 9021889189338891 q10 3621353535543535 q11 621 499 535 499 q12 812 617 582 582 q13 10688 306031323060 q14 304 289 266 266 q15 554 503 501 501 q16 708 628 641 628 q17 1834162016031603 q18 8268795577187718 q19 1735154716241547 q20 2062183418721834 q21 5624542955365429 q22 675 575 610 575 Total cold run time: 70582 ms Total hot run time: 59791 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [fix](s3) improve error msg #45360 [doris]
gavinchou commented on PR #45431: URL: https://github.com/apache/doris/pull/45431#issuecomment-2551958288 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement alterCatalogRenameCommand in nereids [doris]
doris-robot commented on PR #45163: URL: https://github.com/apache/doris/pull/45163#issuecomment-2552001050 TPC-DS: Total hot run time: 196999 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit a1dc00846602167e650db7707f9d0e9794f9a647, data reload: false query1 1289947 917 917 query2 6246225722792257 query3 10996 448544924485 query4 33391 23565 23554 23554 query5 4046471 458 458 query6 284 182 196 182 query7 3991304 310 304 query8 312 249 240 240 query9 9534271527222715 query10 477 258 250 250 query11 17655 15281 15430 15281 query12 150 101 102 101 query13 1558419 402 402 query14 9810722578627225 query15 257 184 193 184 query16 8300441 414 414 query17 1593594 585 585 query18 2157304 315 304 query19 372 154 163 154 query20 125 116 115 115 query21 209 115 110 110 query22 4697442344684423 query23 34688 33907 33886 33886 query24 10338 262125392539 query25 630 410 384 384 query26 1186159 156 156 query27 2297337 352 337 query28 7409244024372437 query29 849 436 427 427 query30 235 164 155 155 query31 1043837 846 837 query32 104 60 59 59 query33 767 297 318 297 query34 1058511 519 511 query35 936 756 768 756 query36 1134988 964 964 query37 146 84 83 83 query38 4319425242074207 query39 1541146014541454 query40 215 110 142 110 query41 45 42 44 42 query42 115 107 107 107 query43 559 500 481 481 query44 1319856 835 835 query45 193 179 175 175 query46 1189715 734 715 query47 2052189419371894 query48 416 325 325 325 query49 907 404 396 396 query50 876 407 401 401 query51 7372726672557255 query52 100 93 90 90 query53 273 183 184 183 query54 1150415 421 415 query55 95 79 79 79 query56 257 244 254 244 query57 1298117311541154 query58 231 229 222 222 query59 3290310830923092 query60 297 254 258 254 query61 113 110 132 110 query62 875 707 692 692 query63 237 238 186 186 query64 4506710 679 679 query65 3351324133073241 query66 740 306 311 306 query67 16620 15579 15489 15489 query68 5918549 532 532 query69 516 254 256 254 query70 1204113211281128 query71 469 260 261 260 query72 7122408740444044 query73 786 356 355 355 query74 10060 877688028776 query75 3656269426792679 query76 3884110411081104 query77 590 283 264 264 query78 10237 944793689368 query79 1904597 582 582 query80 686 432 446 432 query81 509 239 235 235 query82 684 119 155 119 query83 181 148 157 148 query84 237 73 77 73 query85 1348306 295 295 query86 436 305 293 293 query87 4511441343194319 query88 3695222021912191 query89 439 286 289 286 query90 1925191 190 190 query91 139 114 105 105 query92 67 51 49 49 query93 2231540 538 538 query94 744 283 289 283 query95 352 254 247 247 query96 641 280 283 280 query97 2859271327212713 query98 218 200 196 196 query99 1572134013491340 Total cold run time: 304640 ms Total hot run time: 196999 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [Enhancement] (nereids)implement alterCatalogRenameCommand in nereids [doris]
doris-robot commented on PR #45163: URL: https://github.com/apache/doris/pull/45163#issuecomment-2552010925 ClickBench: Total hot run time: 32.68 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit a1dc00846602167e650db7707f9d0e9794f9a647, data reload: false query1 0.030.030.03 query2 0.060.040.03 query3 0.240.070.08 query4 1.610.100.11 query5 0.430.390.39 query6 1.150.660.65 query7 0.020.020.01 query8 0.040.040.02 query9 0.580.500.50 query10 0.550.580.55 query11 0.140.100.10 query12 0.150.110.11 query13 0.610.600.59 query14 2.742.792.86 query15 0.900.820.82 query16 0.390.380.38 query17 1.041.061.04 query18 0.220.220.21 query19 1.961.882.00 query20 0.020.010.01 query21 15.36 0.590.59 query22 2.262.022.25 query23 16.91 1.060.85 query24 3.260.652.00 query25 0.220.130.08 query26 0.510.140.14 query27 0.040.040.05 query28 10.08 1.111.08 query29 12.60 3.293.23 query30 0.250.060.06 query31 2.850.390.40 query32 3.250.460.46 query33 3.153.113.15 query34 16.83 4.444.46 query35 4.484.464.47 query36 0.690.490.48 query37 0.090.060.06 query38 0.040.030.03 query39 0.030.020.02 query40 0.160.130.13 query41 0.080.030.03 query42 0.040.030.02 query43 0.040.030.03 Total cold run time: 106.1 s Total hot run time: 32.68 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showDatabasesCommand in nereids [doris]
doris-robot commented on PR #45421: URL: https://github.com/apache/doris/pull/45421#issuecomment-2552015866 TPC-H: Total hot run time: 39506 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 33c8d8c02066379b9d11ca22627b981149e93c96, data reload: false -- Round 1 -- q1 17593 740272887288 q2 2048181 167 167 q3 10552 113211571132 q4 10220 750 665 665 q5 7592272726912691 q6 239 149 146 146 q7 978 616 611 611 q8 9258181019021810 q9 6577636464006364 q10 6983232323192319 q11 476 269 273 269 q12 431 225 227 225 q13 17766 293429492934 q14 252 207 207 207 q15 563 511 487 487 q16 644 591 582 582 q17 988 528 579 528 q18 7196654467006544 q19 1362899 931 899 q20 458 185 185 185 q21 4049314231743142 q22 369 311 328 311 Total cold run time: 106594 ms Total hot run time: 39506 ms - Round 2, with runtime_filter_mode=off - q1 7214719371937193 q2 334 229 232 229 q3 2896276928882769 q4 2140185118741851 q5 5559566956265626 q6 230 144 139 139 q7 2200179417911791 q8 3336360635413541 q9 8865885488048804 q10 3621353135183518 q11 594 509 500 500 q12 788 605 617 605 q13 12122 306631513066 q14 303 266 280 266 q15 548 504 504 504 q16 673 667 612 612 q17 1830158415801580 q18 7914733973387338 q19 1671148115581481 q20 2073183617961796 q21 5308521150965096 q22 646 570 617 570 Total cold run time: 70865 ms Total hot run time: 58875 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](nereids)implement showStatusCommand in nereids [doris]
doris-robot commented on PR #45427: URL: https://github.com/apache/doris/pull/45427#issuecomment-2552000233 TPC-H: Total hot run time: 40146 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit e6c420e4d40a593be0a5d1719af947665ef12926, data reload: false -- Round 1 -- q1 17624 753173367336 q2 2045180 173 173 q3 10539 112611521126 q4 10208 737 676 676 q5 7588269027212690 q6 238 150 148 148 q7 994 637 631 631 q8 9270182119711821 q9 6651643264226422 q10 6976229723392297 q11 457 269 266 266 q12 456 239 230 230 q13 17787 293529792935 q14 242 214 224 214 q15 564 504 508 504 q16 658 594 594 594 q17 990 537 621 537 q18 7458677467026702 q19 1345102710521027 q20 477 194 188 188 q21 4209342033193319 q22 388 310 318 310 Total cold run time: 107164 ms Total hot run time: 40146 ms - Round 2, with runtime_filter_mode=off - q1 7284726673037266 q2 327 231 225 225 q3 2932280030962800 q4 2208189918351835 q5 5577566356865663 q6 225 142 143 142 q7 2246178317771777 q8 3410356734903490 q9 8918891689038903 q10 3576351335673513 q11 597 507 514 507 q12 811 642 626 626 q13 12937 316731513151 q14 297 286 266 266 q15 572 527 516 516 q16 709 637 655 637 q17 1813159415821582 q18 7961727073927270 q19 1682164014711471 q20 2019181017951795 q21 5456518553225185 q22 644 573 593 573 Total cold run time: 72201 ms Total hot run time: 59193 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showUserPropertiesCommand and showAllPropertiesCommand in nereids [doris]
doris-robot commented on PR #45145: URL: https://github.com/apache/doris/pull/45145#issuecomment-2552005673 ClickBench: Total hot run time: 33.56 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit ba531eb1d3febd85f50ad4270fec0caab703fba1, data reload: false query1 0.040.040.03 query2 0.090.040.03 query3 0.240.080.07 query4 1.610.110.10 query5 0.430.420.43 query6 1.150.650.64 query7 0.020.020.02 query8 0.040.040.04 query9 0.600.500.49 query10 0.550.580.57 query11 0.150.110.11 query12 0.140.110.11 query13 0.610.610.60 query14 2.722.842.76 query15 0.890.820.82 query16 0.380.400.38 query17 1.071.001.06 query18 0.230.210.20 query19 1.941.862.02 query20 0.010.010.01 query21 15.38 0.590.58 query22 2.832.182.69 query23 16.90 1.080.78 query24 3.251.651.45 query25 0.190.170.06 query26 0.530.150.13 query27 0.050.040.05 query28 9.691.121.07 query29 12.62 3.253.23 query30 0.250.060.06 query31 2.860.390.38 query32 3.230.470.45 query33 3.113.183.16 query34 16.99 4.464.47 query35 4.514.504.49 query36 0.680.480.49 query37 0.100.060.06 query38 0.040.040.03 query39 0.030.020.02 query40 0.170.130.12 query41 0.080.020.02 query42 0.030.020.02 query43 0.040.030.03 Total cold run time: 106.47 s Total hot run time: 33.56 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showStagesCommand in nereids [doris]
msridhar78 commented on PR #45373: URL: https://github.com/apache/doris/pull/45373#issuecomment-2552005387 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](nereids)implement showStatusCommand in nereids [doris]
doris-robot commented on PR #45427: URL: https://github.com/apache/doris/pull/45427#issuecomment-2552021839 TPC-DS: Total hot run time: 190545 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit e6c420e4d40a593be0a5d1719af947665ef12926, data reload: false query1 981 415 376 376 query2 6528242722972297 query3 6704224 216 216 query4 33844 23343 23547 23343 query5 4350470 448 448 query6 293 188 192 188 query7 4622302 318 302 query8 303 236 232 232 query9 9598272727162716 query10 479 265 265 265 query11 17913 15268 15185 15185 query12 160 108 104 104 query13 1669430 431 430 query14 10794 769571177117 query15 309 183 185 183 query16 8475465 466 465 query17 1812603 573 573 query18 2169305 332 305 query19 368 163 152 152 query20 124 106 117 106 query21 206 104 112 104 query22 4567425343554253 query23 34480 33566 33721 33566 query24 11540 242625282426 query25 674 382 380 380 query26 1827151 155 151 query27 2990325 320 320 query28 7950241924372419 query29 1042423 410 410 query30 301 148 149 148 query31 1010798 814 798 query32 105 82 58 58 query33 776 312 292 292 query34 1013519 534 519 query35 890 738 750 738 query36 1090962 980 962 query37 270 76 76 76 query38 4275425743224257 query39 1489143314181418 query40 295 102 99 99 query41 48 43 47 43 query42 114 101 102 101 query43 545 488 484 484 query44 1331820 822 820 query45 182 162 163 162 query46 1169723 717 717 query47 1954185918581858 query48 406 320 331 320 query49 1287388 392 388 query50 807 380 374 374 query51 7218719270277027 query52 107 95 94 94 query53 256 188 184 184 query54 1175416 417 416 query55 84 82 79 79 query56 262 239 238 238 query57 1290112911151115 query58 234 210 227 210 query59 3348308530123012 query60 270 249 245 245 query61 110 105 116 105 query62 889 701 671 671 query63 218 193 189 189 query64 5065659 696 659 query65 3250321331873187 query66 1428328 306 306 query67 15742 15478 15482 15478 query68 5890535 549 535 query69 456 266 258 258 query70 1197115610731073 query71 445 258 261 258 query72 6474417940594059 query73 783 361 356 356 query74 9675880988528809 query75 3445265426612654 query76 3554109311681093 query77 753 277 268 268 query78 10343 941293659365 query79 1820594 603 594 query80 838 451 415 415 query81 549 241 228 228 query82 826 120 122 120 query83 263 154 143 143 query84 234 70 74 70 query85 1310312 362 312 query86 432 305 298 298 query87 4593445844414441 query88 3766221421822182 query89 400 301 302 301 query90 2137192 190 190 query91 137 103 105 103 query92 70 52 53 52 query93 1652543 530 530 query94 1134281 283 281 query95 358 319 246 246 query96 619 282 285 282 query97 2826272126562656 query98 220 196 197 196 query99 1539129813051298 Total cold run time: 305812 ms Total hot run time: 190545 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Enhancement](nereids)implement showStatusCommand in nereids [doris]
doris-robot commented on PR #45427: URL: https://github.com/apache/doris/pull/45427#issuecomment-2552032210 ClickBench: Total hot run time: 32.53 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit e6c420e4d40a593be0a5d1719af947665ef12926, data reload: false query1 0.030.030.03 query2 0.070.030.03 query3 0.230.080.07 query4 1.600.100.10 query5 0.430.410.41 query6 1.170.660.65 query7 0.020.010.01 query8 0.040.030.03 query9 0.570.500.51 query10 0.550.580.55 query11 0.140.100.10 query12 0.130.110.11 query13 0.600.600.60 query14 2.752.882.70 query15 0.890.840.81 query16 0.370.390.38 query17 1.051.021.05 query18 0.240.210.20 query19 1.921.871.93 query20 0.010.010.01 query21 15.35 0.590.59 query22 3.032.041.64 query23 17.13 0.910.72 query24 3.271.042.40 query25 0.250.190.17 query26 0.570.140.14 query27 0.050.040.04 query28 9.291.121.07 query29 12.57 3.253.21 query30 0.250.070.06 query31 2.870.390.38 query32 3.240.460.46 query33 3.143.103.06 query34 17.06 4.514.50 query35 4.584.504.50 query36 0.660.480.47 query37 0.100.060.06 query38 0.050.030.04 query39 0.040.020.02 query40 0.180.130.13 query41 0.080.020.02 query42 0.030.020.02 query43 0.040.030.02 Total cold run time: 106.64 s Total hot run time: 32.53 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showDatabasesCommand in nereids [doris]
doris-robot commented on PR #45421: URL: https://github.com/apache/doris/pull/45421#issuecomment-2552037881 TPC-DS: Total hot run time: 190055 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 33c8d8c02066379b9d11ca22627b981149e93c96, data reload: false query1 992 397 397 397 query2 6518232523182318 query3 6699219 216 216 query4 33974 23479 23417 23417 query5 4387470 471 470 query6 297 207 210 207 query7 4621298 322 298 query8 304 240 240 240 query9 9582272727242724 query10 483 262 268 262 query11 18234 15235 15159 15159 query12 169 103 103 103 query13 1679445 408 408 query14 10975 726474347264 query15 304 181 183 181 query16 8099457 441 441 query17 1803617 540 540 query18 2101294 290 290 query19 362 158 157 157 query20 119 106 111 106 query21 212 105 110 105 query22 4561428341214121 query23 34703 33652 33121 33121 query24 11510 250724912491 query25 667 380 371 371 query26 1774155 150 150 query27 3016323 312 312 query28 7873241124422411 query29 1022425 413 413 query30 302 147 153 147 query31 1042787 821 787 query32 102 60 64 60 query33 765 297 292 292 query34 1054517 498 498 query35 936 769 733 733 query36 1107941 962 941 query37 274 79 79 79 query38 4184413941534139 query39 1463143116041431 query40 283 102 97 97 query41 48 44 48 44 query42 117 103 102 102 query43 532 486 482 482 query44 1244819 823 819 query45 195 161 162 161 query46 1130710 713 710 query47 1935182218551822 query48 421 314 345 314 query49 1266391 396 391 query50 782 388 400 388 query51 7313716870427042 query52 101 91 91 91 query53 267 191 185 185 query54 1197403 414 403 query55 79 77 84 77 query56 259 239 245 239 query57 1254112311101110 query58 235 221 236 221 query59 3215304530123012 query60 284 248 258 248 query61 113 107 109 107 query62 897 670 689 670 query63 219 188 200 188 query64 5180666 718 666 query65 3265321232393212 query66 1415331 315 315 query67 15844 15483 15474 15474 query68 5196546 548 546 query69 441 253 256 253 query70 1231115611461146 query71 435 256 254 254 query72 6920409241464092 query73 778 365 359 359 query74 10591 882488428824 query75 3521266626602660 query76 3123104811171048 query77 570 279 271 271 query78 10411 944693789378 query79 1729607 596 596 query80 1037449 427 427 query81 529 248 237 237 query82 959 129 120 120 query83 251 145 146 145 query84 241 73 72 72 query85 1253314 305 305 query86 358 306 310 306 query87 4566428043544280 query88 3413225521982198 query89 418 287 296 287 query90 2078198 189 189 query91 142 111 109 109 query92 60 54 53 53 query93 1091529 529 529 query94 1070287 283 283 query95 352 254 257 254 query96 604 285 287 285 query97 2834264926752649 query98 212 204 197 197 query99 1544133613131313 Total cold run time: 305317 ms Total hot run time: 190055 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] branch-3.0: [Chore](load) print error url to the log when encountering data quality issues during load #45212 [doris]
gavinchou merged PR #45320: URL: https://github.com/apache/doris/pull/45320 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-3.0 updated: branch-3.0: [Chore](load) print error url to the log when encountering data quality issues during load #45212 (#45320)
This is an automated email from the ASF dual-hosted git repository.
gavinchou pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new aaaf59fe4bb branch-3.0: [Chore](load) print error url to the log when
encountering data quality issues during load #45212 (#45320)
aaaf59fe4bb is described below
commit aaaf59fe4bbae624082323c91ef3d98258449090
Author: github-actions[bot]
<41898282+github-actions[bot]@users.noreply.github.com>
AuthorDate: Thu Dec 19 02:04:24 2024 +0800
branch-3.0: [Chore](load) print error url to the log when encountering data
quality issues during load #45212 (#45320)
Cherry-picked from #45212
Co-authored-by: Xin Liao
---
be/src/http/action/stream_load.cpp | 3 ++-
be/src/runtime/fragment_mgr.cpp| 14 +++---
be/src/runtime/runtime_state.cpp | 4 +++-
.../src/main/java/org/apache/doris/qe/Coordinator.java | 1 +
4 files changed, 17 insertions(+), 5 deletions(-)
diff --git a/be/src/http/action/stream_load.cpp
b/be/src/http/action/stream_load.cpp
index e3ad7f44866..117c6eaa981 100644
--- a/be/src/http/action/stream_load.cpp
+++ b/be/src/http/action/stream_load.cpp
@@ -145,7 +145,8 @@ void StreamLoadAction::handle(HttpRequest* req) {
<< ctx->commit_and_publish_txn_cost_nanos / 100
<< ", number_total_rows=" << ctx->number_total_rows
<< ", number_loaded_rows=" << ctx->number_loaded_rows
- << ", receive_bytes=" << ctx->receive_bytes << ", loaded_bytes="
<< ctx->loaded_bytes;
+ << ", receive_bytes=" << ctx->receive_bytes << ", loaded_bytes="
<< ctx->loaded_bytes
+ << ", error_url=" << ctx->error_url;
// update statistics
streaming_load_requests_total->increment(1);
diff --git a/be/src/runtime/fragment_mgr.cpp b/be/src/runtime/fragment_mgr.cpp
index a5392276282..c3d4341e5ea 100644
--- a/be/src/runtime/fragment_mgr.cpp
+++ b/be/src/runtime/fragment_mgr.cpp
@@ -427,12 +427,20 @@ void FragmentMgr::coordinator_callback(const
ReportStatusRequest& req) {
params.load_counters.emplace(s_unselected_rows,
std::to_string(num_rows_load_unselected));
if (!req.runtime_state->get_error_log_file_path().empty()) {
-params.__set_tracking_url(
-
to_load_error_http_path(req.runtime_state->get_error_log_file_path()));
+std::string error_log_url =
+
to_load_error_http_path(req.runtime_state->get_error_log_file_path());
+LOG(INFO) << "error log file path: " << error_log_url
+ << ", query id: " << print_id(req.query_id)
+ << ", fragment instance id: " <<
print_id(req.fragment_instance_id);
+params.__set_tracking_url(error_log_url);
} else if (!req.runtime_states.empty()) {
for (auto* rs : req.runtime_states) {
if (!rs->get_error_log_file_path().empty()) {
-
params.__set_tracking_url(to_load_error_http_path(rs->get_error_log_file_path()));
+std::string error_log_url =
to_load_error_http_path(rs->get_error_log_file_path());
+LOG(INFO) << "error log file path: " << error_log_url
+ << ", query id: " << print_id(req.query_id)
+ << ", fragment instance id: " <<
print_id(rs->fragment_instance_id());
+params.__set_tracking_url(error_log_url);
}
if (rs->wal_id() > 0) {
params.__set_txn_id(rs->wal_id());
diff --git a/be/src/runtime/runtime_state.cpp b/be/src/runtime/runtime_state.cpp
index f6f70f5c679..a22ad18ce04 100644
--- a/be/src/runtime/runtime_state.cpp
+++ b/be/src/runtime/runtime_state.cpp
@@ -376,7 +376,9 @@ Status RuntimeState::create_error_log_file() {
LOG(WARNING) << error_msg.str();
return Status::InternalError(error_msg.str());
}
-VLOG_FILE << "create error log file: " << _error_log_file_path;
+LOG(INFO) << "create error log file: " << _error_log_file_path
+ << ", query id: " << print_id(_query_id)
+ << ", fragment instance id: " << print_id(_fragment_instance_id);
return Status::OK();
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
index 2889e943e93..ae68f64d75f 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
@@ -2413,6 +2413,7 @@ public class Coordinator implements CoordInterface {
updateLoadCounters(params.getLoadCounters());
}
if (params.isSetTrackingUrl()) {
+LOG.info("query_id={} tracking_url: {}",
DebugUtil.printId(queryId), params.getTrackingUrl());
trackingUrl = para
(doris) branch auto-pick-45212-branch-3.0 deleted (was e3420b24316)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45212-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git was e3420b24316 [Chore](load) print error url to the log when encountering data quality issues during load (#45212) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](nereids)implement showStatusCommand in nereids [doris]
msridhar78 commented on PR #45427: URL: https://github.com/apache/doris/pull/45427#issuecomment-2551967249 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](nereids)implement showStatusCommand in nereids [doris]
msridhar78 commented on code in PR #45427: URL: https://github.com/apache/doris/pull/45427#discussion_r1890645513 ## regression-test/suites/nereids_p0/show/test_show_status_command.groovy: ## @@ -0,0 +1,32 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// distributed under the License is distributed on an +// "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY Review Comment: fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement helpCommand in nereids [doris]
doris-robot commented on PR #44819: URL: https://github.com/apache/doris/pull/44819#issuecomment-2551969500 ClickBench: Total hot run time: 31.55 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 461df0d6418747e10e5a53fb0f4b5920654d5154, data reload: false query1 0.030.040.03 query2 0.070.030.03 query3 0.220.070.07 query4 1.610.100.10 query5 0.450.400.42 query6 1.150.650.64 query7 0.020.020.02 query8 0.040.040.03 query9 0.560.500.50 query10 0.580.580.57 query11 0.140.100.11 query12 0.140.110.11 query13 0.600.600.60 query14 2.862.742.72 query15 0.890.830.84 query16 0.370.390.38 query17 1.021.080.98 query18 0.220.210.22 query19 1.861.731.99 query20 0.020.020.01 query21 15.36 0.620.60 query22 2.673.011.51 query23 17.03 0.930.75 query24 3.610.481.47 query25 0.110.130.06 query26 0.650.140.13 query27 0.050.040.04 query28 10.36 1.131.08 query29 12.55 3.263.24 query30 0.250.070.06 query31 2.870.380.38 query32 3.250.450.46 query33 3.083.053.15 query34 16.92 4.454.48 query35 4.444.454.43 query36 0.680.500.48 query37 0.100.070.06 query38 0.040.040.03 query39 0.030.020.03 query40 0.170.120.12 query41 0.080.030.02 query42 0.030.020.02 query43 0.030.030.03 Total cold run time: 107.21 s Total hot run time: 31.55 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showUserPropertiesCommand and showAllPropertiesCommand in nereids [doris]
doris-robot commented on PR #45145: URL: https://github.com/apache/doris/pull/45145#issuecomment-2551974311 TPC-H: Total hot run time: 40147 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit ba531eb1d3febd85f50ad4270fec0caab703fba1, data reload: false -- Round 1 -- q1 17581 742373237323 q2 2052180 168 168 q3 10758 107412371074 q4 10575 701 704 701 q5 7601268627302686 q6 241 154 148 148 q7 999 623 611 611 q8 9253185218761852 q9 6669652264856485 q10 7025227122912271 q11 465 266 255 255 q12 424 228 230 228 q13 17796 298130202981 q14 250 212 215 212 q15 572 536 513 513 q16 658 592 593 592 q17 997 620 696 620 q18 7300682666896689 q19 13451005970 970 q20 495 183 186 183 q21 4204336832783278 q22 379 316 307 307 Total cold run time: 107639 ms Total hot run time: 40147 ms - Round 2, with runtime_filter_mode=off - q1 7247724072887240 q2 330 243 239 239 q3 2881278829462788 q4 2052180619191806 q5 5641566856015601 q6 230 145 152 145 q7 2224181717811781 q8 3379351334903490 q9 8798893888728872 q10 3559358535003500 q11 612 504 516 504 q12 850 616 588 588 q13 12541 318830653065 q14 316 265 280 265 q15 567 506 509 506 q16 688 653 638 638 q17 1885163316041604 q18 8086778978057789 q19 1718155315581553 q20 2055188618841884 q21 550053735373 q22 676 604 614 604 Total cold run time: 71890 ms Total hot run time: 59835 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showStoragePolicyCommand in nereids [doris]
doris-robot commented on PR #44393: URL: https://github.com/apache/doris/pull/44393#issuecomment-2551980767 TPC-DS: Total hot run time: 196085 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit bd828574107e20c1877100458daf7b0e7b4f6e09, data reload: false query1 1320926 912 912 query2 6356227222762272 query3 11003 479246404640 query4 33396 23276 23394 23276 query5 5052451 467 451 query6 271 185 185 185 query7 3995305 315 305 query8 304 258 233 233 query9 9301269626822682 query10 504 252 247 247 query11 18040 15263 15151 15151 query12 151 103 104 103 query13 1770426 413 413 query14 11818 730270897089 query15 279 186 180 180 query16 8152443 480 443 query17 1623602 568 568 query18 2157311 312 311 query19 373 161 155 155 query20 115 108 112 108 query21 205 109 120 109 query22 4772463744384438 query23 34274 33576 33667 33576 query24 9593256224762476 query25 575 391 390 390 query26 718 156 159 156 query27 2137331 341 331 query28 6479245824402440 query29 772 411 408 408 query30 229 152 153 152 query31 1015847 838 838 query32 95 54 56 54 query33 643 292 285 285 query34 943 531 558 531 query35 902 763 754 754 query36 1126966 977 966 query37 125 72 72 72 query38 4399418841524152 query39 1498147614361436 query40 215 111 100 100 query41 47 44 47 44 query42 118 106 98 98 query43 536 499 489 489 query44 1180835 840 835 query45 190 173 173 173 query46 1190706 748 706 query47 2066196219771962 query48 421 333 331 331 query49 855 387 384 384 query50 836 394 388 388 query51 7314724772717247 query52 99 94 89 89 query53 253 184 180 180 query54 1185415 392 392 query55 84 78 80 78 query56 273 242 271 242 query57 1274117011841170 query58 262 227 237 227 query59 3279309730483048 query60 300 250 245 245 query61 108 99 105 99 query62 861 689 688 688 query63 224 197 193 193 query64 3642667 632 632 query65 3295331632643264 query66 755 311 306 306 query67 16807 15628 15650 15628 query68 5052559 551 551 query69 481 252 250 250 query70 1203113011081108 query71 464 243 251 243 query72 7080416040794079 query73 786 352 367 352 query74 9944880288158802 query75 3656264226112611 query76 3567113110651065 query77 570 271 280 271 query78 10306 940493989398 query79 1113593 586 586 query80 758 419 444 419 query81 535 235 222 222 query82 582 117 124 117 query83 169 154 141 141 query84 244 68 66 66 query85 1181296 299 296 query86 393 304 296 296 query87 4652438043014301 query88 3345225122562251 query89 436 289 297 289 query90 1995186 192 186 query91 150 102 103 102 query92 65 53 50 50 query93 2404532 524 524 query94 753 283 273 273 query95 347 250 248 248 query96 614 278 275 275 query97 2840272326872687 query98 210 200 194 194 query99 1570133613061306 Total cold run time: 301614 ms Total hot run time: 196085 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Enhancement] (nereids)implement alterCatalogRenameCommand in nereids [doris]
doris-robot commented on PR #45163: URL: https://github.com/apache/doris/pull/45163#issuecomment-2551979454 TPC-H: Total hot run time: 39591 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit a1dc00846602167e650db7707f9d0e9794f9a647, data reload: false -- Round 1 -- q1 17639 793172867286 q2 2042187 176 176 q3 10630 108711681087 q4 10560 683 752 683 q5 7597268926272627 q6 232 149 145 145 q7 967 610 603 603 q8 9231183319241833 q9 6502635463986354 q10 7049227223032272 q11 464 269 265 265 q12 423 227 227 227 q13 17783 295529192919 q14 253 216 204 204 q15 552 504 510 504 q16 668 592 579 579 q17 970 493 510 493 q18 7213669266486648 q19 1354998 1088998 q20 460 186 183 183 q21 4042328631973197 q22 383 313 308 308 Total cold run time: 107014 ms Total hot run time: 39591 ms - Round 2, with runtime_filter_mode=off - q1 7219720971997199 q2 330 226 226 226 q3 2884280928822809 q4 2060187518321832 q5 5681567156265626 q6 225 148 143 143 q7 2250181118071807 q8 3392355834753475 q9 8766902988828882 q10 3576351935393519 q11 600 525 503 503 q12 825 591 596 591 q13 13776 306131253061 q14 318 274 300 274 q15 562 531 515 515 q16 682 628 662 628 q17 1832160716491607 q18 8385796177007700 q19 1727154015071507 q20 2072187718801877 q21 5691556655405540 q22 660 579 608 579 Total cold run time: 73513 ms Total hot run time: 59900 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showStorageVaultCommand in nereids [doris]
doris-robot commented on PR #44805: URL: https://github.com/apache/doris/pull/44805#issuecomment-2551979887 TPC-DS: Total hot run time: 196362 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 6aff26af6f5bb38e64c59b649e6f1088223309ca, data reload: false query1 1302962 905 905 query2 6263235823572357 query3 3 461747494617 query4 32875 23306 23410 23306 query5 4102450 470 450 query6 284 187 186 186 query7 4001306 298 298 query8 284 230 222 222 query9 9542269726842684 query10 477 239 238 238 query11 17978 15081 15076 15076 query12 155 108 106 106 query13 1570445 424 424 query14 9540705269276927 query15 282 197 201 197 query16 8233486 470 470 query17 1667607 602 602 query18 2189322 318 318 query19 361 173 168 168 query20 122 115 114 114 query21 205 107 115 107 query22 4761456944304430 query23 34522 33887 34215 33887 query24 10570 243925002439 query25 624 382 397 382 query26 1231154 151 151 query27 2345334 345 334 query28 7951245024392439 query29 825 435 423 423 query30 227 147 147 147 query31 1052828 808 808 query32 103 57 57 57 query33 762 299 289 289 query34 1151527 567 527 query35 900 764 781 764 query36 1113963 956 956 query37 139 76 74 74 query38 4436430244034302 query39 1491146414461446 query40 209 97 98 97 query41 47 42 45 42 query42 112 107 101 101 query43 540 492 486 486 query44 1292843 825 825 query45 187 174 179 174 query46 1206768 715 715 query47 2079195319291929 query48 414 331 321 321 query49 906 382 399 382 query50 880 400 389 389 query51 7398703271417032 query52 112 92 92 92 query53 282 190 185 185 query54 1159418 405 405 query55 84 82 84 82 query56 268 255 274 255 query57 1288119311561156 query58 236 221 227 221 query59 3456319431013101 query60 277 252 249 249 query61 108 105 123 105 query62 870 690 726 690 query63 220 183 189 183 query64 4038693 650 650 query65 3281324232903242 query66 776 303 312 303 query67 16675 15584 15529 15529 query68 5661533 558 533 query69 501 262 247 247 query70 1192109711371097 query71 490 262 245 245 query72 7101425341484148 query73 800 368 365 365 query74 10008 890189908901 query75 3733262726242624 query76 3757116811591159 query77 592 282 259 259 query78 10329 947493189318 query79 1827578 595 578 query80 784 421 421 421 query81 509 228 226 226 query82 689 116 123 116 query83 164 139 149 139 query84 244 70 79 70 query85 1316315 328 315 query86 405 283 308 283 query87 4484442245084422 query88 3602222521882188 query89 470 287 287 287 query90 1967185 192 185 query91 135 113 104 104 query92 59 52 56 52 query93 1988536 529 529 query94 752 282 290 282 query95 348 248 257 248 query96 639 285 284 284 query97 2877269526812681 query98 218 195 199 195 query99 1590129912931293 Total cold run time: 304379 ms Total hot run time: 196362 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [Enhancement] (nereids)implement showDatabasesCommand in nereids [doris]
msridhar78 commented on PR #45421: URL: https://github.com/apache/doris/pull/45421#issuecomment-2551982058 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showStorageVaultCommand in nereids [doris]
doris-robot commented on PR #44805: URL: https://github.com/apache/doris/pull/44805#issuecomment-2551989140 ClickBench: Total hot run time: 32.74 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 6aff26af6f5bb38e64c59b649e6f1088223309ca, data reload: false query1 0.030.030.03 query2 0.070.040.04 query3 0.240.070.06 query4 1.630.110.11 query5 0.420.390.41 query6 1.140.650.65 query7 0.020.020.02 query8 0.040.030.03 query9 0.570.520.48 query10 0.540.570.56 query11 0.150.100.10 query12 0.140.110.10 query13 0.610.590.60 query14 2.872.762.89 query15 0.890.820.81 query16 0.380.380.38 query17 1.011.011.04 query18 0.220.210.21 query19 1.871.761.96 query20 0.010.010.01 query21 15.36 0.580.60 query22 2.492.412.37 query23 16.88 1.320.82 query24 2.930.491.22 query25 0.240.190.19 query26 0.270.140.13 query27 0.050.030.04 query28 10.83 1.131.08 query29 12.54 3.283.22 query30 0.240.050.05 query31 2.880.380.38 query32 3.250.470.47 query33 3.063.143.12 query34 17.28 4.494.46 query35 4.544.514.47 query36 0.670.480.47 query37 0.100.060.06 query38 0.040.030.04 query39 0.040.020.03 query40 0.170.130.13 query41 0.080.020.02 query42 0.040.020.03 query43 0.030.030.03 Total cold run time: 106.86 s Total hot run time: 32.74 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showStagesCommand in nereids [doris]
msridhar78 commented on PR #45373: URL: https://github.com/apache/doris/pull/45373#issuecomment-2551991415 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showStagesCommand in nereids [doris]
msridhar78 commented on code in PR #45373:
URL: https://github.com/apache/doris/pull/45373#discussion_r1890660289
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ShowStagesCommand.java:
##
@@ -0,0 +1,136 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.plans.commands;
+
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.cloud.datasource.CloudInternalCatalog;
+import org.apache.doris.cloud.proto.Cloud;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.DdlException;
+import org.apache.doris.nereids.trees.plans.PlanType;
+import org.apache.doris.nereids.trees.plans.visitor.PlanVisitor;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.ShowResultSet;
+import org.apache.doris.qe.ShowResultSetMetaData;
+import org.apache.doris.qe.StmtExecutor;
+
+import com.google.gson.GsonBuilder;
+import org.apache.commons.lang3.StringUtils;
+
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * Command for SHOW STAGES.
+ */
+public class ShowStagesCommand extends ShowCommand {
+
+private static final String NAME_COL = "StageName";
+private static final String ID_COL = "StageId";
+private static final String ENDPOINT_COL = "Endpoint";
+private static final String REGION_COL = "Region";
+private static final String BUCKET_COL = "Bucket";
+private static final String PREFIX_COL = "Prefix";
+private static final String AK_COL = "AK";
+private static final String SK_COL = "SK";
+private static final String PROVIDER_COL = "Provider";
+private static final String DEFAULT_PROP_COL = "DefaultProperties";
+private static final String COMMENT = "Comment";
+private static final String CREATE_TIME = "CreateTime";
+private static final String ACCESS_TYPE = "AccessType";
+private static final String ROLE_NAME = "RoleName";
+private static final String ARN = "Arn";
+
+private static final ShowResultSetMetaData META_DATA =
+ShowResultSetMetaData.builder()
+.addColumn(new Column(NAME_COL,
ScalarType.createVarchar(64)))
+.addColumn(new Column(ID_COL,
ScalarType.createVarchar(64)))
+.addColumn(new Column(ENDPOINT_COL,
ScalarType.createVarchar(128)))
+.addColumn(new Column(REGION_COL,
ScalarType.createVarchar(64)))
+.addColumn(new Column(BUCKET_COL,
ScalarType.createVarchar(64)))
+.addColumn(new Column(PREFIX_COL,
ScalarType.createVarchar(128)))
+.addColumn(new Column(AK_COL,
ScalarType.createVarchar(128)))
+.addColumn(new Column(SK_COL,
ScalarType.createVarchar(128)))
+.addColumn(new Column(PROVIDER_COL,
ScalarType.createVarchar(10)))
+.addColumn(new Column(DEFAULT_PROP_COL,
ScalarType.createVarchar(512)))
+.addColumn(new Column(COMMENT,
ScalarType.createVarchar(512)))
+.addColumn(new Column(CREATE_TIME,
ScalarType.createVarchar(64)))
+.addColumn(new Column(ACCESS_TYPE,
ScalarType.createVarchar(64)))
+.addColumn(new Column(ROLE_NAME,
ScalarType.createVarchar(64)))
+.addColumn(new Column(ARN, ScalarType.createVarchar(64)))
+.build();
+
+public ShowStagesCommand() {
+super(PlanType.SHOW_STAGES_COMMAND);
+}
+
+@Override
+public ShowResultSet doRun(ConnectContext ctx, StmtExecutor executor)
throws Exception {
+try {
+List stages = ((CloudInternalCatalog)
Env.getCurrentInternalCatalog())
+
.getStage(Cloud.StagePB.StageType.EXTERNAL, null, null, null);
+if (stages == null) {
+throw new AnalysisException("get stage err");
+}
+List> results = new ArrayList<>();
+for (Cloud.StagePB stage :
Re: [PR] [Enhancement] (nereids)implement showStoragePolicyCommand in nereids [doris]
doris-robot commented on PR #44393: URL: https://github.com/apache/doris/pull/44393#issuecomment-2551990103 ClickBench: Total hot run time: 32.87 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit bd828574107e20c1877100458daf7b0e7b4f6e09, data reload: false query1 0.040.030.02 query2 0.070.040.03 query3 0.240.070.07 query4 1.610.100.11 query5 0.430.400.41 query6 1.200.650.66 query7 0.020.020.02 query8 0.030.030.03 query9 0.570.520.51 query10 0.550.570.55 query11 0.150.100.10 query12 0.130.110.11 query13 0.610.600.59 query14 2.742.842.74 query15 0.900.820.82 query16 0.390.380.39 query17 0.991.081.06 query18 0.220.210.21 query19 1.851.822.00 query20 0.020.010.01 query21 15.36 0.600.59 query22 2.522.171.23 query23 17.15 0.940.79 query24 3.711.762.41 query25 0.300.130.14 query26 0.650.130.12 query27 0.050.040.04 query28 8.551.121.08 query29 12.61 3.263.26 query30 0.250.060.06 query31 2.840.380.38 query32 3.260.460.46 query33 3.243.113.12 query34 16.86 4.444.49 query35 4.464.454.40 query36 0.690.490.50 query37 0.100.060.06 query38 0.050.040.04 query39 0.040.030.02 query40 0.160.140.13 query41 0.080.030.02 query42 0.030.020.02 query43 0.040.020.02 Total cold run time: 105.76 s Total hot run time: 32.87 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showDataTypesCommand in nereids [doris]
msridhar78 commented on PR #44299: URL: https://github.com/apache/doris/pull/44299#issuecomment-2552039684 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement adminSetTableStatusCommand in nereids [doris]
Vallishp commented on PR #45272: URL: https://github.com/apache/doris/pull/45272#issuecomment-2552043599 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showDatabasesCommand in nereids [doris]
doris-robot commented on PR #45421: URL: https://github.com/apache/doris/pull/45421#issuecomment-2552047162 ClickBench: Total hot run time: 32.36 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 33c8d8c02066379b9d11ca22627b981149e93c96, data reload: false query1 0.030.030.04 query2 0.060.040.02 query3 0.240.070.07 query4 1.610.100.10 query5 0.440.420.40 query6 1.180.650.64 query7 0.020.020.02 query8 0.040.040.03 query9 0.580.490.50 query10 0.540.590.55 query11 0.140.100.10 query12 0.130.110.12 query13 0.600.600.60 query14 2.712.702.82 query15 0.880.810.81 query16 0.370.360.37 query17 1.021.051.06 query18 0.220.220.23 query19 1.971.812.05 query20 0.010.010.01 query21 15.36 0.600.59 query22 2.902.992.00 query23 16.78 0.960.92 query24 3.260.481.81 query25 0.290.120.16 query26 0.370.140.14 query27 0.050.060.05 query28 10.53 1.101.07 query29 12.56 3.253.24 query30 0.250.060.06 query31 2.870.380.39 query32 3.220.460.47 query33 3.103.053.06 query34 16.82 4.414.43 query35 4.514.544.48 query36 0.680.480.50 query37 0.100.060.06 query38 0.050.040.04 query39 0.030.020.03 query40 0.160.130.12 query41 0.070.020.02 query42 0.030.020.02 query43 0.040.040.03 Total cold run time: 106.82 s Total hot run time: 32.36 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [only test] [doris]
github-actions[bot] commented on PR #45558:
URL: https://github.com/apache/doris/pull/45558#issuecomment-2551235259
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/12393230622";) output.
shellcheck errors
```
'shellcheck ' returned error 1 finding the following syntactical issues:
--
In gensrc/script/gen_build_version.sh line 38:
if [[ ${build_version_hotfix} > 0 ]]; then
^-- SC2071 (error): > is for string
comparisons. Use -gt instead.
In gensrc/script/gen_build_version.sh line 228:
if [[ ${build_version_hotfix} > 0 ]]; then
^-- SC2071 (error): > is for string
comparisons. Use -gt instead.
For more information:
https://www.shellcheck.net/wiki/SC2071 -- > is for string comparisons. Use
...
--
You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
# shellcheck disable=
above the line that contains the issue, where is the error code;
3. Add '-e ' to the SHELLCHECK_OPTS setting in your .yml action file.
```
shfmt errors
```
'shfmt ' found no issues.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](ORC) Not push down fixed char type in orc reader [doris]
suxiaogang223 commented on PR #45484: URL: https://github.com/apache/doris/pull/45484#issuecomment-2551236455 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](tools) Support automatic profile fetching under the tools directory. [doris]
github-actions[bot] commented on PR #45612:
URL: https://github.com/apache/doris/pull/45612#issuecomment-2551289655
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/12393672915";) output.
shellcheck errors
```
'shellcheck ' returned error 1 finding the following syntactical issues:
--
In gensrc/script/gen_build_version.sh line 38:
if [[ ${build_version_hotfix} > 0 ]]; then
^-- SC2071 (error): > is for string
comparisons. Use -gt instead.
In gensrc/script/gen_build_version.sh line 228:
if [[ ${build_version_hotfix} > 0 ]]; then
^-- SC2071 (error): > is for string
comparisons. Use -gt instead.
In tools/ssb-tools/bin/run-ssb-queries.sh line 89:
if [[ -z "$profile" ]]; then
^--^ SC2250 (style): Prefer putting braces around variable
references even when not strictly required.
Did you mean:
if [[ -z "${profile}" ]]; then
For more information:
https://www.shellcheck.net/wiki/SC2071 -- > is for string comparisons. Use
...
https://www.shellcheck.net/wiki/SC2250 -- Prefer putting braces around
vari...
--
You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
# shellcheck disable=
above the line that contains the issue, where is the error code;
3. Add '-e ' to the SHELLCHECK_OPTS setting in your .yml action file.
```
shfmt errors
```
'shfmt ' returned error 1 finding the following formatting issues:
--
--- tools/ssb-tools/bin/run-ssb-queries.sh.orig
+++ tools/ssb-tools/bin/run-ssb-queries.sh
@@ -91,7 +91,7 @@
return 1
fi
sleep 1s
-curl -s -X GET -u "${USER}:${PASSWORD}"
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/text? >>
"${PROFILE_DIR}/${profile}"
+curl -s -X GET -u "${USER}:${PASSWORD}"
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/text?
>>"${PROFILE_DIR}/${profile}"
}
check_prerequest "mysql --version" "mysql"
--
You can reformat the above files to meet shfmt's requirements by typing:
shfmt -w filename
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [feature](tools) Support automatic profile fetching under the tools directory. [doris]
Mryange opened a new pull request, #45612: URL: https://github.com/apache/doris/pull/45612 ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [ ] No. - [ ] Yes. - Does this need documentation? - [ ] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](tools) Support automatic profile fetching under the tools directory. [doris]
Mryange commented on PR #45612: URL: https://github.com/apache/doris/pull/45612#issuecomment-2551287999 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](tools) Support automatic profile fetching under the tools directory. [doris]
hello-stephen commented on PR #45612: URL: https://github.com/apache/doris/pull/45612#issuecomment-2551287953 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refine](Column)Disallow implicit conversion of ColumnPtr to IColumn* [doris]
Mryange commented on PR #45588: URL: https://github.com/apache/doris/pull/45588#issuecomment-2551239139 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Stream load supports writing data from one topic to multipl… [doris-kafka-connector]
chuang-wang-pre closed pull request #54: [improve] Stream load supports writing data from one topic to multipl… URL: https://github.com/apache/doris-kafka-connector/pull/54 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [only test] [doris]
Mryange commented on PR #45558: URL: https://github.com/apache/doris/pull/45558#issuecomment-2551234420 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](build) Building Failed on x86 Mac [doris]
github-actions[bot] commented on code in PR #45613: URL: https://github.com/apache/doris/pull/45613#discussion_r1890232558 ## be/src/http/http_channel.cpp: ## @@ -25,6 +25,7 @@ #include #include #include +#include Review Comment: warning: inclusion of deprecated C++ header 'inttypes.h'; consider using 'cinttypes' instead [modernize-deprecated-headers] ```suggestion #include ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [only test] [doris]
doris-robot commented on PR #45558: URL: https://github.com/apache/doris/pull/45558#issuecomment-2551314147 TeamCity be ut coverage result: Function Coverage: 38.94% (10152/26068) Line Coverage: 29.85% (85356/285973) Region Coverage: 28.91% (43799/151507) Branch Coverage: 25.45% (22259/87474) Coverage Report: http://coverage.selectdb-in.cc/coverage/a1b4a8c4bdeb85ff51ccf43ba1b024fe55ab68fa_a1b4a8c4bdeb85ff51ccf43ba1b024fe55ab68fa/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [only test] [doris]
Mryange commented on PR #45594: URL: https://github.com/apache/doris/pull/45594#issuecomment-2551315138 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [opt](join) Check the property of nullable from intermediate row #45017 [doris]
github-actions[bot] commented on PR #45476: URL: https://github.com/apache/doris/pull/45476#issuecomment-2551483803 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: branch-2.1: [opt](join) Check the property of nullable from intermediate row #45017 (#45476)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new 59b3760fdd8 branch-2.1: [opt](join) Check the property of nullable
from intermediate row #45017 (#45476)
59b3760fdd8 is described below
commit 59b3760fdd8ce87ce25e90f9466a8ee561fbcda1
Author: github-actions[bot]
<41898282+github-actions[bot]@users.noreply.github.com>
AuthorDate: Wed Dec 18 22:40:17 2024 +0800
branch-2.1: [opt](join) Check the property of nullable from intermediate
row #45017 (#45476)
Cherry-picked from #45017
Co-authored-by: Jerry Hu
---
be/src/pipeline/exec/hashjoin_probe_operator.cpp | 29 ++--
1 file changed, 22 insertions(+), 7 deletions(-)
diff --git a/be/src/pipeline/exec/hashjoin_probe_operator.cpp
b/be/src/pipeline/exec/hashjoin_probe_operator.cpp
index b7dd0622fe3..0b88a174f9e 100644
--- a/be/src/pipeline/exec/hashjoin_probe_operator.cpp
+++ b/be/src/pipeline/exec/hashjoin_probe_operator.cpp
@@ -17,6 +17,8 @@
#include "hashjoin_probe_operator.h"
+#include
+
#include
#include "common/logging.h"
@@ -671,21 +673,34 @@ Status HashJoinProbeOperatorX::prepare(RuntimeState*
state) {
size_t idx = 0;
for (const auto* slot : slots_to_check) {
auto data_type = slot->get_data_type_ptr();
-auto target_data_type = idx < right_col_idx ?
_left_table_data_types[idx]
-:
_right_table_data_types[idx - right_col_idx];
+const auto slot_on_left = idx < right_col_idx;
+auto target_data_type = slot_on_left ? _left_table_data_types[idx]
+ : _right_table_data_types[idx -
right_col_idx];
++idx;
if (data_type->equals(*target_data_type)) {
continue;
}
-auto data_type_non_nullable = vectorized::remove_nullable(data_type);
-if (data_type_non_nullable->equals(*target_data_type)) {
+/// For outer join(left/right/full), the non-nullable columns may be
converted to nullable.
+const auto accept_nullable_not_match =
+_join_op == TJoinOp::FULL_OUTER_JOIN ||
+(slot_on_left ? _join_op == TJoinOp::RIGHT_OUTER_JOIN
+ : _join_op == TJoinOp::LEFT_OUTER_JOIN);
+
+if (accept_nullable_not_match) {
+auto data_type_non_nullable =
vectorized::remove_nullable(data_type);
+if (data_type_non_nullable->equals(*target_data_type)) {
+continue;
+}
+} else if (data_type->equals(*target_data_type)) {
continue;
}
-return Status::InternalError("intermediate slot({}) data type not
match: '{}' vs '{}'",
- slot->id(), data_type->get_name(),
- _left_table_data_types[idx]->get_name());
+return Status::InternalError(
+"Join node(id={}, OP={}) intermediate slot({}, #{})'s on {}
table data type not "
+"match: '{}' vs '{}'",
+_node_id, _join_op, slot->col_name(), slot->id(),
(slot_on_left ? "left" : "right"),
+data_type->get_name(), target_data_type->get_name());
}
_build_side_child.reset();
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [opt](join) Check the property of nullable from intermediate row #45017 [doris]
yiguolei merged PR #45476: URL: https://github.com/apache/doris/pull/45476 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45017-branch-2.1 deleted (was f6958fa5d27)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45017-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git was f6958fa5d27 [opt](join) Check the property of nullable from intermediate row (#45017) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: branch-2.1: [Test](binlog) Check correctly for show create database #45472 (#45551)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-2.1 by this push: new 1980e098bcf branch-2.1: [Test](binlog) Check correctly for show create database #45472 (#45551) 1980e098bcf is described below commit 1980e098bcfda2fe8720c97e55117e13cd65dff3 Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> AuthorDate: Wed Dec 18 22:43:15 2024 +0800 branch-2.1: [Test](binlog) Check correctly for show create database #45472 (#45551) Cherry-picked from #45472 Co-authored-by: Uniqueyou --- regression-test/data/ddl_p0/test_recover.out | 216 -- regression-test/suites/ddl_p0/test_recover.groovy | 27 ++- 2 files changed, 18 insertions(+), 225 deletions(-) diff --git a/regression-test/data/ddl_p0/test_recover.out b/regression-test/data/ddl_p0/test_recover.out index 149c4c5459b..646ec068a0e 100644 --- a/regression-test/data/ddl_p0/test_recover.out +++ b/regression-test/data/ddl_p0/test_recover.out @@ -27,33 +27,6 @@ test_recover_tb_new test_recover_tb test_recover_tb_new --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_db_newCREATE DATABASE `test_recover_db_new` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_db_newCREATE DATABASE `test_recover_db_new` - -- !select -- test_recover_tb test_recover_tb_2 @@ -89,33 +62,6 @@ test_recover_tb_new test_recover_tb test_recover_tb_new --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_db_newCREATE DATABASE `test_recover_db_new` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_db_newCREATE DATABASE `test_recover_db_new` - -- !select -- test_recover_tb test_recover_tb_2 @@ -151,33 +97,6 @@ test_recover_tb_new test_recover_tb test_recover_tb_new --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_db_newCREATE DATABASE `test_recover_db_new` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_db_newCREATE DATABASE `test_recover_db_new` - -- !select -- test_recover_tb test_recover_tb_2 @@ -213,33 +132,6 @@ test_recover_tb_new test_recover_tb test_recover_tb_new --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_db_newCREATE DATABASE `test_recover_db_new` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_db_newCREATE DATABASE `test_recover_db_new` - -- !select -- test_recover_tb test_recover_tb_2 @@ -275,33 +167,6 @@ test_recover_tb_new test_recover_tb test_recover_tb_new --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_dbCREATE DATABASE `test_recover_db` - --- !select -- -test_recover_db_newCRE
(doris) branch auto-pick-45472-branch-2.1 deleted (was 0f5bb84f510)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45472-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git was 0f5bb84f510 [Test](binlog) Check correctly for show create database (#45472) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [Test](binlog) Check correctly for show create database #45472 [doris]
yiguolei merged PR #45551: URL: https://github.com/apache/doris/pull/45551 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [fix](export) Make enable_outfile_to_local available for export command #45241 [doris]
yiguolei commented on PR #45515: URL: https://github.com/apache/doris/pull/45515#issuecomment-2551501718 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45241-branch-2.1 updated (40f1bfa4cde -> 0c86a955064)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch auto-pick-45241-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git omit 40f1bfa4cde [fix](export) Make enable_outfile_to_local available for export command (#45241) add 191ef9b8b01 branch-2.1: [fix](schema-change) fix array/map/struct in schema-change not-null to null will make core #45305 (#45482) add 7856662ecf4 [Bug](pipeline) make sink operator process eos signals after wake_up_early #45207 (#45400) add 900086667ff [Chore](pipeline) catch exception on task::prepare to avoid exception make backend coredump #45479 (#45516) add 01684ce3b13 branch-2.1: [fix](mysql-buffer) fix special buffer size with nested type #45126 (#45458) add a8e172ea5d3 branch-2.1: [Fix](show-stmt) Raise an error when show partitions with complex predicate #45257 (#45519) add 748648ef1eb branch-2.1: [fix](nereids) fix bug when sum0 distinct appear in cte #45447 (#45511) add 55170455982 branch-2.1: [chore](checkpoint) add enable_checkpoint config #45301 (#45328) add fb310423274 branch-2.1: [fix](hive)Clear processed tasks #45309 (#45339) add 416f3cf2bad branch-2.1: [fix](iceberg)Fill in the detailed error information #45285 (#45338) add 855e9a508c6 [fix](catalog) opt the count pushdown rule for iceberg/paimon/hive scan node (#44038) (#45564) add ca24cd1cd97 branch-2.1 [Fix](Insert-Job)Resolving Task Status Discrepancies in Query Execution Records (#45342) (#45451) add bc048a3e369 [bugfix](fe) should use equals not == to get failed to select replica message (#45602) add 02feb16530e branch-2.1: [bug](s3) fix S3 file system gets absolute path #44965 (#45529) add 59b3760fdd8 branch-2.1: [opt](join) Check the property of nullable from intermediate row #45017 (#45476) add 1980e098bcf branch-2.1: [Test](binlog) Check correctly for show create database #45472 (#45551) add 0c86a955064 [fix](export) Make enable_outfile_to_local available for export command (#45241) This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (40f1bfa4cde) \ N -- N -- N refs/heads/auto-pick-45241-branch-2.1 (0c86a955064) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: be/src/io/fs/s3_file_system.h | 2 +- be/src/olap/rowset/segment_v2/column_reader.cpp| 42 +++- be/src/pipeline/exec/hashjoin_build_sink.cpp | 57 +++--- be/src/pipeline/exec/hashjoin_probe_operator.cpp | 29 ++- be/src/pipeline/pipeline.cpp | 7 +- be/src/pipeline/pipeline_task.cpp | 6 +- be/src/pipeline/pipeline_task.h| 5 +- .../pipeline_x/pipeline_x_fragment_context.cpp | 2 +- be/src/pipeline/pipeline_x/pipeline_x_task.cpp | 107 +- be/src/pipeline/pipeline_x/pipeline_x_task.h | 12 +- be/src/util/mysql_row_buffer.cpp | 6 +- be/src/vec/exec/format/table/iceberg_reader.cpp| 18 +- be/src/vec/exec/format/table/iceberg_reader.h | 4 +- be/src/vec/exec/format/table/paimon_reader.cpp | 3 + be/src/vec/exec/scan/vfile_scanner.cpp | 4 + .../main/java/org/apache/doris/common/Config.java | 6 + .../doris/analysis/TableValuedFunctionRef.java | 5 +- .../doris/common/proc/PartitionsProcDir.java | 9 +- .../apache/doris/datasource/FileQueryScanNode.java | 27 ++- .../org/apache/doris/datasource/FileScanNode.java | 79 .../org/apache/doris/datasource/FileSplit.java | 5 +- .../org/apache/doris/datasource/FileSplitter.java | 104 ++ .../apache/doris/datasource/SplitAssignment.java | 2 +- .../org/apache/doris/datasource/SplitCreator.java | 2 +- .../apache/doris/datasource/SplitGenerator.java| 5 +- .../doris/datasource/hive/HMSTransaction.java | 12 ++ .../doris/datasource/hive/source/HiveScanNode.java | 52 +++-- .../doris/datasource/hive/source/HiveSplit.java| 6 +- .../doris/datasource/hudi/source/HudiScanNode.java | 8 +- .../datasource/iceberg/IcebergMetadataOps.java | 3 +- .../datasource/iceberg/source/IcebergScanNode.java | 42 ++-- .../datasource/iceberg/source/IcebergSplit.java| 11 +- .../maxcompute/source/MaxComputeScanNode.java | 28 +-- .../datasource/paimon/source/PaimonScanNode.java | 65 --- .../da
(doris) branch branch-2.1 updated: [fix](restore) Add restore_reset_index_id config #45283 (#45574)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new fca9c91193f [fix](restore) Add restore_reset_index_id config #45283
(#45574)
fca9c91193f is described below
commit fca9c91193ffd917d1157e562ae584bde963ff03
Author: walter
AuthorDate: Wed Dec 18 22:45:53 2024 +0800
[fix](restore) Add restore_reset_index_id config #45283 (#45574)
cherry pick from #45283
---
.../main/java/org/apache/doris/common/Config.java | 9 ++
.../apache/doris/alter/SchemaChangeHandler.java| 10 ++
.../java/org/apache/doris/catalog/OlapTable.java | 14 +-
.../test_backup_restore_inverted_idx.groovy| 174 +
4 files changed, 206 insertions(+), 1 deletion(-)
diff --git a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
index 1b5e412437b..9fead6e6f6e 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
@@ -1574,6 +1574,15 @@ public class Config extends ConfigBase {
@ConfField(mutable = false)
public static boolean enable_restore_snapshot_rpc_compression = true;
+/**
+ * A internal config, to indicate whether to reset the index id when
restore olap table.
+ *
+ * The inverted index saves the index id in the file path/header, so the
index id between
+ * two clusters must be the same.
+ */
+@ConfField(mutable = true, masterOnly = true)
+public static boolean restore_reset_index_id = true;
+
/**
* Control the max num of tablets per backup job involved.
*/
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java
b/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java
index ebad5f24f4f..117297b0a79 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java
@@ -2569,6 +2569,7 @@ public class SchemaChangeHandler extends AlterHandler {
IndexDef indexDef = alterClause.getIndexDef();
Set newColset = Sets.newTreeSet(String.CASE_INSENSITIVE_ORDER);
newColset.addAll(indexDef.getColumns());
+Set existedIndexIdSet = Sets.newHashSet();
for (Index existedIdx : existedIndexes) {
if
(existedIdx.getIndexName().equalsIgnoreCase(indexDef.getIndexName())) {
if (indexDef.isSetIfNotExists()) {
@@ -2585,7 +2586,16 @@ public class SchemaChangeHandler extends AlterHandler {
indexDef.getIndexType() + " index for columns (" +
String.join(",", indexDef.getColumns())
+ " ) already exist.");
}
+existedIndexIdSet.add(existedIdx.getIndexId());
}
+
+// The restored olap table may not reset the index id, which comes
from the upstream,
+// so we need to check and reset the index id here, to avoid
confliction.
+// See OlapTable.resetIdsForRestore for details.
+while (existedIndexIdSet.contains(alterIndex.getIndexId())) {
+alterIndex.setIndexId(Env.getCurrentEnv().getNextId());
+}
+
boolean disableInvertedIndexV1ForVariant =
olapTable.getInvertedIndexStorageFormat()
== TInvertedIndexStorageFormat.V1 &&
ConnectContext.get().getSessionVariable()
.getDisableInvertedIndexV1ForVaraint();
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
index d64b8febc7d..4f85a989e46 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
@@ -833,7 +833,19 @@ public class OlapTable extends Table implements
MTMVRelatedTableIf {
if (this.indexes != null) {
List indexes = this.indexes.getIndexes();
for (Index idx : indexes) {
-idx.setIndexId(env.getNextId());
+long newIdxId;
+if (Config.restore_reset_index_id) {
+newIdxId = env.getNextId();
+} else {
+// The index id from the upstream is used, if
restore_reset_index_id is not set.
+//
+// This is because the index id is used as a part of
inverted file name/header
+// in BE. During restore, the inverted file is copied from
the upstream to the
+// downstream. If the index id is changed, it might cause
the BE to fail to find
+// the inverted files.
+
Re: [PR] [opt](Nereids) lock table in ascending order of table IDs [doris]
morrySnow commented on PR #45045: URL: https://github.com/apache/doris/pull/45045#issuecomment-2551504321 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](restore) Add restore_reset_index_id config #45283 [doris]
yiguolei merged PR #45574: URL: https://github.com/apache/doris/pull/45574 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [Fix](ccr) Wrong result of tosql for add column with comment #45319 [doris]
yiguolei commented on PR #45552: URL: https://github.com/apache/doris/pull/45552#issuecomment-2551506406 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enchement](iceberg)support read iceberg partition evolution table. (#45367) [doris]
yiguolei commented on PR #45569: URL: https://github.com/apache/doris/pull/45569#issuecomment-2551508501 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [fix](external)fix split and get the schema #45408 [doris]
yiguolei commented on PR #45566: URL: https://github.com/apache/doris/pull/45566#issuecomment-2551507147 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [fix](tvf) Tvf supports to parse the enclose character in csv files #45407 [doris]
yiguolei commented on PR #45570: URL: https://github.com/apache/doris/pull/45570#issuecomment-2551510947 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](function) fix wrong result on group_concat with distinct+order_… [doris]
yiguolei merged PR #45513: URL: https://github.com/apache/doris/pull/45513 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [Bug](function) fix wrong result on group_concat with distinct+order_… (#45513)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new fe0cc289ded [Bug](function) fix wrong result on group_concat with
distinct+order_… (#45513)
fe0cc289ded is described below
commit fe0cc289ded85a00d217795f388c0cb3e3aae16a
Author: Pxl
AuthorDate: Wed Dec 18 22:49:18 2024 +0800
[Bug](function) fix wrong result on group_concat with distinct+order_…
(#45513)
---
.../vec/aggregate_functions/aggregate_function.h | 8 ++-
.../aggregate_function_distinct.h | 16 --
.../aggregate_functions/aggregate_function_null.h | 37 +++--
.../aggregate_functions/aggregate_function_sort.h | 4 +-
.../data/nereids_p0/aggregate/agg_group_concat.out | 4 ++
.../nereids_p0/aggregate/agg_group_concat.groovy | 61 +-
6 files changed, 92 insertions(+), 38 deletions(-)
diff --git a/be/src/vec/aggregate_functions/aggregate_function.h
b/be/src/vec/aggregate_functions/aggregate_function.h
index 2aa687c38bb..a1bc13c15a3 100644
--- a/be/src/vec/aggregate_functions/aggregate_function.h
+++ b/be/src/vec/aggregate_functions/aggregate_function.h
@@ -20,6 +20,10 @@
#pragma once
+#include
+
+#include "common/exception.h"
+#include "common/status.h"
#include "util/defer_op.h"
#include "vec/columns/column_complex.h"
#include "vec/columns/column_string.h"
@@ -76,7 +80,7 @@ using ConstAggregateDataPtr = const char*;
*/
class IAggregateFunction {
public:
-IAggregateFunction(const DataTypes& argument_types_) :
argument_types(argument_types_) {}
+IAggregateFunction(DataTypes argument_types_) :
argument_types(std::move(argument_types_)) {}
/// Get main function name.
virtual String get_name() const = 0;
@@ -221,7 +225,7 @@ public:
virtual void set_version(const int version_) { version = version_; }
-virtual AggregateFunctionPtr transmit_to_stable() { return nullptr; }
+virtual IAggregateFunction* transmit_to_stable() { return nullptr; }
protected:
DataTypes argument_types;
diff --git a/be/src/vec/aggregate_functions/aggregate_function_distinct.h
b/be/src/vec/aggregate_functions/aggregate_function_distinct.h
index 455640f8214..33ea4e17dc0 100644
--- a/be/src/vec/aggregate_functions/aggregate_function_distinct.h
+++ b/be/src/vec/aggregate_functions/aggregate_function_distinct.h
@@ -336,10 +336,20 @@ public:
DataTypePtr get_return_type() const override { return
nested_func->get_return_type(); }
-AggregateFunctionPtr transmit_to_stable() override {
-return AggregateFunctionPtr(new AggregateFunctionDistinct(
-nested_func, IAggregateFunction::argument_types));
+IAggregateFunction* transmit_to_stable() override {
+return new AggregateFunctionDistinct(nested_func,
+
IAggregateFunction::argument_types);
}
};
+template
+struct FunctionStableTransfer {
+using FunctionStable = T;
+};
+
+template typename Data>
+struct FunctionStableTransfer> {
+using FunctionStable = AggregateFunctionDistinct;
+};
+
} // namespace doris::vectorized
diff --git a/be/src/vec/aggregate_functions/aggregate_function_null.h
b/be/src/vec/aggregate_functions/aggregate_function_null.h
index b2fbe1b2e7f..f7c97f5b821 100644
--- a/be/src/vec/aggregate_functions/aggregate_function_null.h
+++ b/be/src/vec/aggregate_functions/aggregate_function_null.h
@@ -25,6 +25,7 @@
#include "common/logging.h"
#include "common/status.h"
#include "vec/aggregate_functions/aggregate_function.h"
+#include "vec/aggregate_functions/aggregate_function_distinct.h"
#include "vec/columns/column_nullable.h"
#include "vec/common/assert_cast.h"
#include "vec/data_types/data_type_nullable.h"
@@ -164,7 +165,7 @@ public:
void insert_result_into(ConstAggregateDataPtr __restrict place, IColumn&
to) const override {
if constexpr (result_is_nullable) {
-ColumnNullable& to_concrete = assert_cast(to);
+auto& to_concrete = assert_cast(to);
if (get_flag(place)) {
nested_function->insert_result_into(nested_place(place),
to_concrete.get_nested_column());
@@ -204,6 +205,19 @@ public:
}
}
+IAggregateFunction* transmit_to_stable() override {
+auto f = AggregateFunctionNullBaseInline<
+ NestFuction, result_is_nullable,
+ AggregateFunctionNullUnaryInline>::
+ nested_function->transmit_to_stable();
+if (!f) {
+return nullptr;
+}
+return new AggregateFunctionNullUnaryInline<
+typename FunctionStableTransfer::FunctionStable,
result_is_nullable>(
+f, IAggregateFunction::argument_types);
+}
(doris) branch branch-3.0 updated: branch-3.0: [Bug](function) fix wrong result on group_concat with distinct+order_by+nullable #45313 (#45500)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 2b44d1bdee1 branch-3.0: [Bug](function) fix wrong result on
group_concat with distinct+order_by+nullable #45313 (#45500)
2b44d1bdee1 is described below
commit 2b44d1bdee16a284755fe7e5fbccf4640191a572
Author: github-actions[bot]
<41898282+github-actions[bot]@users.noreply.github.com>
AuthorDate: Wed Dec 18 22:49:48 2024 +0800
branch-3.0: [Bug](function) fix wrong result on group_concat with
distinct+order_by+nullable #45313 (#45500)
Cherry-picked from #45313
Co-authored-by: Pxl
---
.../vec/aggregate_functions/aggregate_function.h | 6 ++-
.../aggregate_function_distinct.h | 16 --
.../aggregate_functions/aggregate_function_null.h | 39 +++---
.../aggregate_functions/aggregate_function_sort.h | 4 +-
.../data/nereids_p0/aggregate/agg_group_concat.out | 4 ++
.../nereids_p0/aggregate/agg_group_concat.groovy | 61 +-
6 files changed, 91 insertions(+), 39 deletions(-)
diff --git a/be/src/vec/aggregate_functions/aggregate_function.h
b/be/src/vec/aggregate_functions/aggregate_function.h
index 32fc9d5efce..253e992b129 100644
--- a/be/src/vec/aggregate_functions/aggregate_function.h
+++ b/be/src/vec/aggregate_functions/aggregate_function.h
@@ -20,6 +20,8 @@
#pragma once
+#include
+
#include "common/exception.h"
#include "common/status.h"
#include "util/defer_op.h"
@@ -80,7 +82,7 @@ using ConstAggregateDataPtr = const char*;
*/
class IAggregateFunction {
public:
-IAggregateFunction(const DataTypes& argument_types_) :
argument_types(argument_types_) {}
+IAggregateFunction(DataTypes argument_types_) :
argument_types(std::move(argument_types_)) {}
/// Get main function name.
virtual String get_name() const = 0;
@@ -224,7 +226,7 @@ public:
virtual void set_version(const int version_) { version = version_; }
-virtual AggregateFunctionPtr transmit_to_stable() { return nullptr; }
+virtual IAggregateFunction* transmit_to_stable() { return nullptr; }
/// Verify function signature
virtual Status verify_result_type(const bool without_key, const DataTypes&
argument_types,
diff --git a/be/src/vec/aggregate_functions/aggregate_function_distinct.h
b/be/src/vec/aggregate_functions/aggregate_function_distinct.h
index ec6936a128c..3d634dec89c 100644
--- a/be/src/vec/aggregate_functions/aggregate_function_distinct.h
+++ b/be/src/vec/aggregate_functions/aggregate_function_distinct.h
@@ -338,10 +338,20 @@ public:
DataTypePtr get_return_type() const override { return
nested_func->get_return_type(); }
-AggregateFunctionPtr transmit_to_stable() override {
-return AggregateFunctionPtr(new AggregateFunctionDistinct(
-nested_func, IAggregateFunction::argument_types));
+IAggregateFunction* transmit_to_stable() override {
+return new AggregateFunctionDistinct(nested_func,
+
IAggregateFunction::argument_types);
}
};
+template
+struct FunctionStableTransfer {
+using FunctionStable = T;
+};
+
+template typename Data>
+struct FunctionStableTransfer> {
+using FunctionStable = AggregateFunctionDistinct;
+};
+
} // namespace doris::vectorized
diff --git a/be/src/vec/aggregate_functions/aggregate_function_null.h
b/be/src/vec/aggregate_functions/aggregate_function_null.h
index 014a3e9c603..e1edb51d1db 100644
--- a/be/src/vec/aggregate_functions/aggregate_function_null.h
+++ b/be/src/vec/aggregate_functions/aggregate_function_null.h
@@ -25,6 +25,7 @@
#include "common/logging.h"
#include "common/status.h"
#include "vec/aggregate_functions/aggregate_function.h"
+#include "vec/aggregate_functions/aggregate_function_distinct.h"
#include "vec/columns/column_nullable.h"
#include "vec/common/assert_cast.h"
#include "vec/data_types/data_type_nullable.h"
@@ -165,7 +166,7 @@ public:
void insert_result_into(ConstAggregateDataPtr __restrict place, IColumn&
to) const override {
if constexpr (result_is_nullable) {
-ColumnNullable& to_concrete = assert_cast(to);
+auto& to_concrete = assert_cast(to);
if (get_flag(place)) {
nested_function->insert_result_into(nested_place(place),
to_concrete.get_nested_column());
@@ -197,7 +198,7 @@ public:
void add(AggregateDataPtr __restrict place, const IColumn** columns,
ssize_t row_num,
Arena* arena) const override {
-const ColumnNullable* column =
+const auto* column =
assert_cast(columns[0]);
if (!column->is_null_at(row_num)) {
this->set_flag(place);
@@ -206,6 +207,19 @@ public:
}
[PR] Update Installation [doris-website]
wangtianyi2004 opened a new pull request, #1543: URL: https://github.com/apache/doris-website/pull/1543 ## Versions - [ ] dev - [ ] 3.0 - [ ] 2.1 - [ ] 2.0 ## Languages - [ ] Chinese - [ ] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [cases](array)first commit for array cases [doris]
amorynan commented on code in PR #45435: URL: https://github.com/apache/doris/pull/45435#discussion_r1890367785 ## regression-test/data/nereids_function_p0/array/test_array_char(255).csv: ## @@ -0,0 +1,7 @@ +// this is data case file for array type, contains normal, corner, exceptional data cases +[] +[null] +NULL +// single element +['0'] Review Comment: done! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [cases](array)first commit for array cases [doris]
amorynan commented on code in PR #45435: URL: https://github.com/apache/doris/pull/45435#discussion_r1890368216 ## regression-test/data/nereids_function_p0/array/test_array_double.csv: ## @@ -0,0 +1,17 @@ +// this is data case file for array type, contains normal, corner, exceptional data cases +[] +[null] +NULL +// single element +[0.0] +// double min/max value (IEEE 754) +[-1.79769e+308, 1.79769e+308] +// double min/max value + 1 (异常) +[-1.79769e+307, 1.79769e+307] +// double -1-16 +[-1.0, 0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0] +// double 0-16 with null +[0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, null, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0] +// all data contains below +[-1.79769e+308, -1.79769e+307, -1.0, 0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0, 1.79769e+307, 1.79769e+308] +[1,1.1,-1,0,0.00,0.01,0.0001,-0.7827311820927767,-0.6997222019775602,0.6547571144385542,-0.9249108129619033,-0.27161961813236046,-0.8509822016865269,-0.025144193739938148,-0.7160429045285339,0.8250354873082724,0.8756039443198336,45.123456,-123.456789,179.99,89.99,'1.7976931348623157e+308','2.2250738585072014e-308','-1.7976931348623157e+308','-2.2250738585072014e-308','1.3407807929942596e+154',NULL] Review Comment: done! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45313-branch-3.0 deleted (was f2d31aa68eb)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45313-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git was f2d31aa68eb [Bug](function) fix wrong result on group_concat with distinct+order_by+nullable (#45313) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [Bug](function) fix wrong result on group_concat with distinct+order_by+nullable #45313 [doris]
yiguolei merged PR #45500: URL: https://github.com/apache/doris/pull/45500 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45000-branch-2.1 updated (947105e9a5b -> 159020f21c2)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch auto-pick-45000-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git discard 947105e9a5b [Fix] Add column with default value string throw exception (#45000) add 36695e871ad [feature](statistics)Support auto analyze columns that haven't been analyzed for a long time. #42399 (#45250) add 9a4f63f180b branch-2.1: [fix](Outfile) fix the unclear error code #45119 (#45277) add 5901e94f29e branch-2.1: [Job](Fix)Improve Event Publishing with Timeout #45103 (#45300) add da885691ca0 branch-2.1: [fix](column_array)fix filter_generic in column_array func #41990 (#44983) add 0c55edd3bf6 branch-2.1: [opt](compaction) Don't check missed rows in cumu compaction if input rowsets are not in tablet (#45279) (#45304) add 8217371d156 branch-2.1: [chore](regression-test) move some variant cases to directory #45169 (#45259) add 667f5e6e6a8 [feat](iceberg)Supports using rest type catalog to read tables in unity catalog for 2.1 (#43525) (#45217) add 95ffc7a65fc branch-2.1: [fix](job)Fix millisecond offset issue in time window scheduling trigger time calculation #45176 (#45353) add 4d6d9a6c8c2 branch-2.1: [fix](persist) Fix TableAddOrDropInvertedIndicesInfo.equals #45335 (#45364) add 384a1a4bd70 [branch-2.1]log more detail when query hits policy (#44685) (#45366) add 6dabdd6a8ce branch-2.1: [fix](txn insert) txn insert show error url #45254 (#45381) add 249ffa44c05 branch-2.1 [Chore](dependencies)Upgrade Fe dependencies #44872 (#45362) add 1db78d44966 branch-2.1: [fix](hive) fix block decompressor bug #45289 (#45379) add b881157d528 branch-2.1: [regression](paimon) test Paimon's connectivity of cloud filesystems #45268 (#45378) add d13241115db [branch-2.1]downgrade resource tag when there is not queryable repli… (#45387) add 4df7069e59e branch-2.1: [fix](ip)fix default value for ip #45194 (#45392) add 00e69022127 branch-2.1: [chore](arrow-flight-sql) Add Arrow Flight Sql demo for Java #45306 (#45389) add 4d1f4fb5d2b branch-2.1: [fix](nereids)EliminateGroupBy rule should keep output's datatype unchanged #45359 (#45399) add 23c5d52b04a branch-2.1: [fix](s3) improve error msg #45360 (#45432) add 9e5c53d5762 branch-2.1: [Fix](job-case)Fix status typo #45326 (#45341) add 17f4c1a17ac branch-2.1: [enhance](nereids) remove first_value and second_value second parameter if the second parameter is false #45264 (#45454) add 7e9f6951c04 [fix](statistics)Fix skip analyze empty table case. (#45441) (#45455) add f3f9cba6bd3 [improvement](statistics)Set enableQueryCache to false for statistics queries. (#45474) (#45493) add c561bdb9069 [cherry-pick](branch-21) fix first/last value return error with ignore null (#44996) (#45486) add 79662fcc943 [branch-2.1](functions) clean some ip functions code and make IS_IP_ADDRESS_IN_RANGE DEPENDS_ON_ARGUMENT (#45358) add 8faf010c640 branch-2.1: [fix](function) The parameters after the first of the mask function need to be restricted to constants #45348 (#45498) add b6a18031270 [improve](backup) Add config ignore_backup_tmp_partitions #45240 (#45331) add 8dc845671f3 [test](load) injection cases should check Exception is thrown (#44713) (#45321) add 191ef9b8b01 branch-2.1: [fix](schema-change) fix array/map/struct in schema-change not-null to null will make core #45305 (#45482) add 7856662ecf4 [Bug](pipeline) make sink operator process eos signals after wake_up_early #45207 (#45400) add 900086667ff [Chore](pipeline) catch exception on task::prepare to avoid exception make backend coredump #45479 (#45516) add 01684ce3b13 branch-2.1: [fix](mysql-buffer) fix special buffer size with nested type #45126 (#45458) add a8e172ea5d3 branch-2.1: [Fix](show-stmt) Raise an error when show partitions with complex predicate #45257 (#45519) add 748648ef1eb branch-2.1: [fix](nereids) fix bug when sum0 distinct appear in cte #45447 (#45511) add 55170455982 branch-2.1: [chore](checkpoint) add enable_checkpoint config #45301 (#45328) add fb310423274 branch-2.1: [fix](hive)Clear processed tasks #45309 (#45339) add 416f3cf2bad branch-2.1: [fix](iceberg)Fill in the detailed error information #45285 (#45338) add 855e9a508c6 [fix](catalog) opt the count pushdown rule for iceberg/paimon/hive scan node (#44038) (#45564) add ca24cd1cd97 branch-2.1 [Fix](Insert-Job)Resolving Task Status Discrepancies in Query Execution Records (#45342) (#45451) add bc048a3e369 [bugfix](fe) should use equals not == to get failed to select replica message (#45602) add 02feb16530e branch-2.1: [bug](s3) fix S3 file system gets absolute path #44965 (#45529) add 59b3760fdd8 branch-2.1: [opt](join) Check the property of nullable from intermediate row #45017 (#45476) add 1980e098bcf bra
Re: [PR] branch-2.1: [Fix] Add column with default value string throw exception #45000 [doris]
yiguolei commented on PR #45316: URL: https://github.com/apache/doris/pull/45316#issuecomment-2551518823 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [feat](binlog) Add Support recover binlog #44818 [doris]
yiguolei merged PR #45294: URL: https://github.com/apache/doris/pull/45294 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [cases](array)first commit for array cases [doris]
github-actions[bot] commented on PR #45435: URL: https://github.com/apache/doris/pull/45435#issuecomment-2551525632 ## Possible file(s) that should be tracked in LFS detected: 🚨 The following file(s) exceeds the file size limit: `1048576` bytes, as set in the .yml configuration files: - regression-test/data/nereids_function_p0/array/test_array_array_varchar(65535).csv Consider using `git-lfs` to manage large files. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-44818-branch-2.1 deleted (was ca089b094f9)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-44818-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git was ca089b094f9 [feat](binlog) Add Support recover binlog (#44818) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
