Re: [PR] [feature](workloadgroup) use slot num to control memory distribution among one workload group [doris]
github-actions[bot] commented on PR #38237: URL: https://github.com/apache/doris/pull/38237#issuecomment-2247051724 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pick](Row store) pick #38236 [doris]
github-actions[bot] commented on PR #38240: URL: https://github.com/apache/doris/pull/38240#issuecomment-2247052807 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[I] [Bug] Data Visible in Flink UI but Not Reaching Downstream Database [doris-flink-connector]
HeyChiang opened a new issue, #442: URL: https://github.com/apache/doris-flink-connector/issues/442 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version flink-1.19.1 flink-doris-connector-1.19-1.6.2.jar flink-cdc-common-3.0.1.jar flink-sql-connector-mysql-cdc-3.0.1.jar flink-sql-connector-oracle-cdc-3.0.1.jar ### What's Wrong?  I'm observing a discrepancy between the data shown in the Flink UI and what's actually being written to the downstream database. The Flink UI shows a significant amount of data being processed and sent (as indicated by the colored blocks representing different tasks), but when I check the downstream database, no records are present. This suggests that while data is being read from the source and processed within Flink, it's not successfully being written to the target database. ### What You Expected? I expected that the data shown as processed in the Flink UI would correspond to actual records written to the downstream database. ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Row store) get the correct value of `row_store_page_size` after FE restarts [doris]
lxr599 commented on PR #38236: URL: https://github.com/apache/doris/pull/38236#issuecomment-2247057552 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [spill](regression) add regression test cases for spill [doris]
jacktengg commented on PR #37319: URL: https://github.com/apache/doris/pull/37319#issuecomment-2247061346 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](RateLimit) Add bvar to monitor OSS's rate limit sleep time and failure time [doris]
github-actions[bot] commented on PR #38294: URL: https://github.com/apache/doris/pull/38294#issuecomment-2247060339 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Row store) get the correct value of `row_store_page_size` after FE restarts [doris]
github-actions[bot] commented on PR #38236: URL: https://github.com/apache/doris/pull/38236#issuecomment-2247064646 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pick](Row store) pick #38236 [doris]
github-actions[bot] commented on PR #38240: URL: https://github.com/apache/doris/pull/38240#issuecomment-2247066143 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Variant) cast from jsonb with string value when enable `jsonb_string_as_string` should consider others types include date,datateime,array ... [doris]
doris-robot commented on PR #38291: URL: https://github.com/apache/doris/pull/38291#issuecomment-2247067605 TPC-DS: Total hot run time: 172971 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 0cfd6268d6d006d1f26eca127cbf794331081133, data reload: false query1 924 368 365 365 query2 6357186917871787 query3 6651200 211 200 query4 27817 17418 17281 17281 query5 3594487 478 478 query6 269 163 168 163 query7 4574290 285 285 query8 239 186 194 186 query9 8539241423922392 query10 433 291 282 282 query11 10796 996210081 9962 query12 117 96 82 82 query13 1647376 368 368 query14 8662798068416841 query15 223 172 166 166 query16 7596431 491 431 query17 1588551 547 547 query18 1740288 283 283 query19 201 156 161 156 query20 92 83 87 83 query21 211 130 127 127 query22 4305423642334233 query23 34149 33797 33636 33636 query24 10787 297528652865 query25 639 400 414 400 query26 1088155 155 155 query27 2479278 279 278 query28 7275206320742063 query29 931 656 648 648 query30 254 165 232 165 query31 976 762 744 744 query32 92 62 54 54 query33 752 326 324 324 query34 939 500 497 497 query35 871 736 750 736 query36 1149999 983 983 query37 149 83 88 83 query38 2967289928382838 query39 897 831 869 831 query40 201 117 116 116 query41 44 47 42 42 query42 118 93 96 93 query43 488 473 465 465 query44 1207717 725 717 query45 192 161 162 161 query46 1079721 708 708 query47 1862176417911764 query48 368 293 285 285 query49 864 396 405 396 query50 783 378 388 378 query51 6790670666246624 query52 99 90 93 90 query53 361 288 276 276 query54 874 443 434 434 query55 72 75 91 75 query56 291 270 260 260 query57 1141105410841054 query58 251 244 257 244 query59 2777265626922656 query60 296 277 266 266 query61 93 92 93 92 query62 794 645 647 645 query63 319 284 285 284 query64 9513221616271627 query65 3165308031053080 query66 781 327 320 320 query67 15675 15072 15024 15024 query68 4638536 545 536 query69 633 433 352 352 query70 1172113611551136 query71 438 271 273 271 query72 8567537354715373 query73 753 320 322 320 query74 6192563557325635 query75 4239266426612661 query76 3359895 929 895 query77 696 299 290 290 query78 11120 925690729072 query79 8132529 535 529 query80 1178485 490 485 query81 596 219 229 219 query82 883 139 133 133 query83 402 164 161 161 query84 274 82 84 82 query85 997 299 354 299 query86 446 327 310 310 query87 3343312930703070 query88 5217239323762376 query89 537 379 378 378 query90 2014194 198 194 query91 129 100 99 99 query92 64 47 51 47 query93 5442518 513 513 query94 1289294 293 293 query95 409 321 314 314 query96 615 272 276 272 query97 3155304230463042 query98 226 194 196 194 query99 1512129412461246 Total cold run time: 292134 ms Total hot run time: 172971 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s
Re: [I] [Bug] Data Visible in Flink UI but Not Reaching Downstream Database [doris-flink-connector]
HeyChiang commented on issue #442: URL: https://github.com/apache/doris-flink-connector/issues/442#issuecomment-2247069387  ` "Process -> (BMS_BATCH_DEF: Writer -> BMS_BATCH_DEF: Committer, BMS_LOT_DEF: Writer -> BMS_LOT_DEF: Committer, BMS_STORER_POS: Writer -> BMS_STORER_POS: Committer, BMS_ST_IO_DOC: Writer -> BMS_ST_IO_DOC: Committer, BMS_ST_IO_DTL: Writer -> BMS_ST_IO_DTL: Committer, BMS_SU_SET_DOC: Writer -> BMS_SU_SET_DOC: Committer, BMS_SU_SET_DTL: Writer -> BMS_SU_SET_DTL: Committer, GPCS_INSIDER: Writer -> GPCS_INSIDER: Committer, GPCS_PLACEPOINT: Writer -> GPCS_PLACEPOINT: Committer, GPCS_PLACEPOINT_CLASS: Writer -> GPCS_PLACEPOINT_CLASS: Committer, GPCS_PLACEPOINT_CLASS_DTL: Writer -> GPCS_PLACEPOINT_CLASS_DTL: Committer, GRESA_SA_DOC: Writer -> GRESA_SA_DOC: Committer, GRESA_SA_DTL: Writer -> GRESA_SA_DTL: Committer, NGPCS_ALL_PRICE: Writer -> NGPCS_ALL_PRICE: Committer, NP_EFILES_OP_FILE: Writer -> NP_EFILES_OP_FILE: Committer, PUB_BRAND: Writer -> PUB_BRAND: Committer, PUB_EMPLOYEE: Writer -> PUB_EMPLOYEE: Committer, PUB_FACTORY: Writer -> PUB_FACTORY: Committer, PUB_GOODS: Writer -> PUB_G OODS: Committer, PUB_GOODS_CLASS: Writer -> PUB_GOODS_CLASS: Committer, PUB_GOODS_CLASS_DTL: Writer -> PUB_GOODS_CLASS_DTL: Committer) (2/3)#0" Id=12810 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@24273330 at java.base@11.0.22/jdk.internal.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@24273330 at java.base@11.0.22/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.22/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.22/java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:417) at org.apache.doris.flink.sink.writer.RecordBuffer.write(RecordBuffer.java:93) at org.apache.doris.flink.sink.writer.RecordStream.write(RecordStream.java:63) at org.apache.doris.flink.sink.writer.DorisStreamLoad.writeRecord(DorisStreamLoad.java:251) at org.apache.doris.flink.sink.writer.DorisWriter.writeOneDorisRecord(DorisWriter.java:216) at org.apache.doris.flink.sink.writer.DorisWriter.write(DorisWriter.java:186) at app//org.apache.flink.streaming.runtime.operators.sink.SinkWriterOperator.processElement(SinkWriterOperator.java:160) at app//org.apache.flink.streaming.runtime.io.RecordProcessorUtils$$Lambda$825/0x0008007eec40.accept(Unknown Source) at app//org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:75) at app//org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:61) at app//org.apache.flink.streaming.runtime.tasks.ChainingOutput.collectAndCheckIfChained(ChainingOutput.java:96) at app//org.apache.flink.streaming.runtime.tasks.BroadcastingOutputCollector.collect(BroadcastingOutputCollector.java:94) at app//org.apache.flink.streaming.api.operators.ProcessOperator$ContextImpl.output(ProcessOperator.java:103) at org.apache.doris.flink.tools.cdc.ParsingProcessFunction.processElement(ParsingProcessFunction.java:51) at org.apache.doris.flink.tools.cdc.ParsingProcessFunction.processElement(ParsingProcessFunction.java:31) at app//org.apache.flink.streaming.api.operators.ProcessOperator.processElement(ProcessOperator.java:66) at app//org.apache.flink.streaming.runtime.io.RecordProcessorUtils$$Lambda$825/0x0008007eec40.accept(Unknown Source) at app//org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:238) at app//org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:157) at app//org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:114) at app//org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) at app//org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:579) at app//org.apache.flink.streaming.runtime.tasks.StreamTask$$Lambda$746/0x0008006ff440.runDefaultAction(Unknown Source) at app//org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) at app//org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:909) at app//org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:858) at app//org.apache.flink.runtime.taskmanager.Task
Re: [PR] [fix](multi table) fix single stream multi table memory leak [doris]
doris-robot commented on PR #38255: URL: https://github.com/apache/doris/pull/38255#issuecomment-2247070886 TPC-DS: Total hot run time: 173558 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit b23b3901295a8bde756c30247c436c65152ce54e, data reload: false query1 914 382 368 368 query2 6440189318621862 query3 6654205 214 205 query4 28353 17640 17276 17276 query5 3770489 470 470 query6 280 160 163 160 query7 4595298 301 298 query8 254 203 197 197 query9 8499241923852385 query10 435 292 272 272 query11 11639 997910017 9979 query12 114 82 88 82 query13 1655381 379 379 query14 10325 775375547554 query15 215 166 175 166 query16 7670474 471 471 query17 1355542 534 534 query18 1851277 274 274 query19 194 144 145 144 query20 89 82 81 81 query21 210 129 129 129 query22 4309414139573957 query23 34162 34021 33647 33647 query24 10904 287028452845 query25 597 399 378 378 query26 702 146 150 146 query27 2291269 279 269 query28 6066212821222122 query29 905 646 649 646 query30 252 158 154 154 query31 1006780 798 780 query32 98 57 53 53 query33 656 334 342 334 query34 952 526 508 508 query35 893 731 750 731 query36 1186980 978 978 query37 142 82 82 82 query38 2931276427542754 query39 894 807 806 806 query40 200 117 115 115 query41 46 43 44 43 query42 119 102 98 98 query43 511 467 463 463 query44 1082729 719 719 query45 191 155 185 155 query46 1090722 705 705 query47 1849176617601760 query48 357 312 284 284 query49 837 394 404 394 query50 774 379 386 379 query51 6753674166956695 query52 105 94 88 88 query53 365 290 288 288 query54 856 446 435 435 query55 75 70 72 70 query56 280 267 269 267 query57 1113105110851051 query58 236 251 247 247 query59 2855277826072607 query60 284 276 276 276 query61 95 92 93 92 query62 799 650 648 648 query63 318 288 284 284 query64 9134228617071707 query65 3185310531353105 query66 729 331 330 330 query67 15415 15167 14961 14961 query68 4514549 557 549 query69 475 327 339 327 query70 1170115411431143 query71 378 286 283 283 query72 6932595654465446 query73 741 321 323 321 query74 6146569057445690 query75 3368270026572657 query76 2124980 925 925 query77 421 306 299 299 query78 9702897997418979 query79 2326515 522 515 query80 2511469 532 469 query81 599 220 228 220 query82 807 141 135 135 query83 306 164 171 164 query84 252 89 81 81 query85 2122306 289 289 query86 360 313 330 313 query87 3281312930913091 query88 4031236223772362 query89 486 380 378 378 query90 1641190 183 183 query91 127 100 98 98 query92 60 49 49 49 query93 2550514 512 512 query94 708 288 286 286 query95 417 327 312 312 query96 609 269 266 266 query97 3232301730193017 query98 229 199 189 189 query99 1533126812761268 Total cold run time: 278268 ms Total hot run time: 173558 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s
Re: [PR] [fix] Broker already had baidu afs support but can't really use, so i fix it [doris]
github-actions[bot] commented on PR #38275: URL: https://github.com/apache/doris/pull/38275#issuecomment-2247076874 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Variant) cast from jsonb with string value when enable `jsonb_string_as_string` should consider others types include date,datateime,array ... [doris]
doris-robot commented on PR #38291: URL: https://github.com/apache/doris/pull/38291#issuecomment-2247076528 ClickBench: Total hot run time: 30.57 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 0cfd6268d6d006d1f26eca127cbf794331081133, data reload: false query1 0.040.030.03 query2 0.070.040.04 query3 0.230.060.05 query4 1.680.070.08 query5 0.500.490.48 query6 1.130.740.73 query7 0.020.010.01 query8 0.060.050.05 query9 0.550.490.49 query10 0.550.550.54 query11 0.160.110.11 query12 0.160.130.12 query13 0.600.590.59 query14 0.760.780.83 query15 0.880.830.83 query16 0.370.370.36 query17 0.980.970.95 query18 0.220.210.21 query19 1.931.741.71 query20 0.010.000.02 query21 15.43 0.780.67 query22 4.137.641.71 query23 18.28 1.391.29 query24 2.200.230.21 query25 0.160.090.08 query26 0.310.210.21 query27 0.460.230.23 query28 13.25 1.061.02 query29 12.64 3.293.32 query30 0.250.070.05 query31 2.860.410.40 query32 3.240.500.48 query33 2.912.952.92 query34 17.09 4.414.35 query35 4.384.494.45 query36 0.670.470.47 query37 0.190.150.15 query38 0.150.160.15 query39 0.040.040.04 query40 0.160.120.13 query41 0.090.040.05 query42 0.050.050.05 query43 0.050.040.04 Total cold run time: 109.89 s Total hot run time: 30.57 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix] Broker already had baidu afs support but can't really use, so i fix it [doris]
github-actions[bot] commented on PR #38275: URL: https://github.com/apache/doris/pull/38275#issuecomment-2247076938 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.1](arrow-flight-sql) Add config arrow_flight_result_sink_buffer_size_rows [doris]
yiguolei merged PR #38223: URL: https://github.com/apache/doris/pull/38223 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](multi table) fix single stream multi table memory leak [doris]
doris-robot commented on PR #38255: URL: https://github.com/apache/doris/pull/38255#issuecomment-2247079809 ClickBench: Total hot run time: 30.79 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit b23b3901295a8bde756c30247c436c65152ce54e, data reload: false query1 0.040.040.04 query2 0.080.040.04 query3 0.230.050.05 query4 1.680.080.08 query5 0.470.470.47 query6 1.120.730.72 query7 0.020.010.01 query8 0.050.050.04 query9 0.550.490.48 query10 0.550.540.51 query11 0.170.120.11 query12 0.140.120.13 query13 0.610.640.59 query14 0.760.790.78 query15 0.860.830.82 query16 0.350.370.38 query17 0.950.991.01 query18 0.240.230.22 query19 1.851.751.87 query20 0.020.010.01 query21 15.46 0.790.65 query22 3.857.601.93 query23 18.34 1.361.28 query24 2.060.230.24 query25 0.140.080.08 query26 0.290.220.21 query27 0.450.230.24 query28 13.27 1.041.01 query29 12.64 3.353.32 query30 0.250.060.06 query31 2.870.400.41 query32 3.250.500.49 query33 2.912.982.94 query34 17.19 4.304.39 query35 4.414.424.40 query36 0.670.500.47 query37 0.200.160.15 query38 0.160.150.15 query39 0.040.040.04 query40 0.160.120.13 query41 0.090.050.05 query42 0.070.040.05 query43 0.050.040.04 Total cold run time: 109.56 s Total hot run time: 30.79 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [branch-2.1](arrow-flight-sql) Add config arrow_flight_result_sink_buffer_size_rows (#38223)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-2.1 by this push: new 10c5c336d8e [branch-2.1](arrow-flight-sql) Add config arrow_flight_result_sink_buffer_size_rows (#38223) 10c5c336d8e is described below commit 10c5c336d8e9e9436bae5a65735173ed58fe16e1 Author: Xinyi Zou AuthorDate: Wed Jul 24 15:15:39 2024 +0800 [branch-2.1](arrow-flight-sql) Add config arrow_flight_result_sink_buffer_size_rows (#38223) pick #38221 --- be/src/common/config.cpp | 2 ++ be/src/common/config.h| 3 +++ be/src/exec/data_sink.cpp | 22 -- be/src/pipeline/exec/result_sink_operator.cpp | 9 - be/src/pipeline/exec/result_sink_operator.h | 1 + 5 files changed, 30 insertions(+), 7 deletions(-) diff --git a/be/src/common/config.cpp b/be/src/common/config.cpp index 863d69338bc..76ce00097b0 100644 --- a/be/src/common/config.cpp +++ b/be/src/common/config.cpp @@ -617,6 +617,8 @@ DEFINE_Int32(load_process_safe_mem_permit_percent, "5"); // result buffer cancelled time (unit: second) DEFINE_mInt32(result_buffer_cancelled_interval_time, "300"); +DEFINE_mInt32(arrow_flight_result_sink_buffer_size_rows, "32768"); + // the increased frequency of priority for remaining tasks in BlockingPriorityQueue DEFINE_mInt32(priority_queue_remaining_tasks_increased_frequency, "512"); diff --git a/be/src/common/config.h b/be/src/common/config.h index 9050701261c..447473e4fdd 100644 --- a/be/src/common/config.h +++ b/be/src/common/config.h @@ -675,6 +675,9 @@ DECLARE_Int32(load_process_safe_mem_permit_percent); // result buffer cancelled time (unit: second) DECLARE_mInt32(result_buffer_cancelled_interval_time); +// arrow flight result sink buffer rows size, default 4096 * 8 +DECLARE_mInt32(arrow_flight_result_sink_buffer_size_rows); + // the increased frequency of priority for remaining tasks in BlockingPriorityQueue DECLARE_mInt32(priority_queue_remaining_tasks_increased_frequency); diff --git a/be/src/exec/data_sink.cpp b/be/src/exec/data_sink.cpp index 054021439c5..dc651080298 100644 --- a/be/src/exec/data_sink.cpp +++ b/be/src/exec/data_sink.cpp @@ -78,10 +78,15 @@ Status DataSink::create_data_sink(ObjectPool* pool, const TDataSink& thrift_sink return Status::InternalError("Missing data buffer sink."); } +int result_sink_buffer_size_rows = vectorized::RESULT_SINK_BUFFER_SIZE; +if (!thrift_sink.result_sink.__isset.type || +thrift_sink.result_sink.type == TResultSinkType::ARROW_FLIGHT_PROTOCAL) { +result_sink_buffer_size_rows = config::arrow_flight_result_sink_buffer_size_rows; +} + // TODO: figure out good buffer size based on size of output row -sink->reset(new doris::vectorized::VResultSink(row_desc, output_exprs, - thrift_sink.result_sink, - vectorized::RESULT_SINK_BUFFER_SIZE)); +sink->reset(new doris::vectorized::VResultSink( +row_desc, output_exprs, thrift_sink.result_sink, result_sink_buffer_size_rows)); break; } case TDataSinkType::RESULT_FILE_SINK: { @@ -233,10 +238,15 @@ Status DataSink::create_data_sink(ObjectPool* pool, const TDataSink& thrift_sink return Status::InternalError("Missing data buffer sink."); } +int result_sink_buffer_size_rows = vectorized::RESULT_SINK_BUFFER_SIZE; +if (!thrift_sink.result_sink.__isset.type || +thrift_sink.result_sink.type == TResultSinkType::ARROW_FLIGHT_PROTOCAL) { +result_sink_buffer_size_rows = config::arrow_flight_result_sink_buffer_size_rows; +} + // TODO: figure out good buffer size based on size of output row -sink->reset(new doris::vectorized::VResultSink(row_desc, output_exprs, - thrift_sink.result_sink, - vectorized::RESULT_SINK_BUFFER_SIZE)); +sink->reset(new doris::vectorized::VResultSink( +row_desc, output_exprs, thrift_sink.result_sink, result_sink_buffer_size_rows)); break; } case TDataSinkType::RESULT_FILE_SINK: { diff --git a/be/src/pipeline/exec/result_sink_operator.cpp b/be/src/pipeline/exec/result_sink_operator.cpp index 624b9ca192d..1aa7f37c1fe 100644 --- a/be/src/pipeline/exec/result_sink_operator.cpp +++ b/be/src/pipeline/exec/result_sink_operator.cpp @@ -19,6 +19,7 @@ #include +#include "common/config.h" #include "common/object_pool.h" #include "exec/rowid_fetcher.h" #include "pipeline/exec/operator.h" @@ -64,8 +65,9 @@ Status ResultSinkLocalState::init(RuntimeState* state,
Re: [PR] [Fix](parquet-reader) Fix and optimize parquet min-max filtering. [doris]
doris-robot commented on PR #38277: URL: https://github.com/apache/doris/pull/38277#issuecomment-2247082312 TPC-H: Total hot run time: 39838 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit b42b73aef9328bb05957f36f485d49557d2f, data reload: false -- Round 1 -- q1 17617 436242614261 q2 2006188 188 188 q3 10450 114810911091 q4 10183 819 790 790 q5 7537268226392639 q6 220 138 134 134 q7 951 592 602 592 q8 9227206920532053 q9 8794653265216521 q10 8743373937343734 q11 455 240 235 235 q12 405 218 223 218 q13 17757 297029722970 q14 276 238 249 238 q15 519 483 491 483 q16 507 375 374 374 q17 960 655 705 655 q18 8040746474067406 q19 8040145914601459 q20 660 312 328 312 q21 4928320132343201 q22 351 295 284 284 Total cold run time: 118626 ms Total hot run time: 39838 ms - Round 2, with runtime_filter_mode=off - q1 4397420542414205 q2 370 263 259 259 q3 3018291928932893 q4 1992167716401640 q5 5619549754655465 q6 228 141 129 129 q7 2188189618041804 q8 3285340834053405 q9 8667878987868786 q10 4103381737783778 q11 573 504 513 504 q12 802 639 642 639 q13 15887 317531853175 q14 315 277 273 273 q15 523 488 496 488 q16 489 441 447 441 q17 1790151015081508 q18 8034786477487748 q19 1718161414661466 q20 2195189518441844 q21 9580479446544654 q22 609 508 519 508 Total cold run time: 76382 ms Total hot run time: 55612 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](function) the bucket number arg of width_bucket should be a positive integer value (#37892) [doris]
doris-robot commented on PR #38295: URL: https://github.com/apache/doris/pull/38295#issuecomment-2247092825 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](function) the bucket number arg of width_bucket should be a positive integer value (#37892) [doris]
mrhhsg opened a new pull request, #38295: URL: https://github.com/apache/doris/pull/38295 ## Proposed changes pick #37892 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] Data Visible in Flink UI but Not Reaching Downstream Database [doris-flink-connector]
HeyChiang commented on issue #442: URL: https://github.com/apache/doris-flink-connector/issues/442#issuecomment-2247094059 > Is checkpoint enabled for the flink job? Yes. Here is the Flink command I'm using: ```bash bin/flink run \ -Dexecution.checkpointing.interval=2min \ -Dexecution.checkpointing.timeout=60min \ -Dexecution.checkpointing.tolerable-failed-checkpoints=3 \ -Dparallelism.default=3 \ -c org.apache.doris.flink.tools.cdc.CdcTools \ lib/flink-doris-connector-1.19-1.6.2.jar \ oracle-sync-database \ --database djwkerp \ --oracle-conf hostname=192.168.191.210 \ --oracle-conf port=1521 \ --oracle-conf username=cdc_sync_user \ --oracle-conf password="xx" \ --oracle-conf database-name=data \ --oracle-conf schema-name=DJWKERP \ --including-tables "NGPCS_ALL_PRICE|GPCS_PLACEPOINT|PUB_EMPLOYEE|BMS_BATCH_DEF|BMS_LOT_DEF|BMS_STORER_POS|GRESA_SA_DOC|GRESA_SA_DTL|GPCS_INSIDER|BMS_ST_IO_DOC|BMS_ST_IO_DTL|GPCS_PLACEPOINT_CLASS|GPCS_PLACEPOINT_CLASS_DTL|PUB_GOODS_CLASS|PUB_GOODS_CLASS_DTL|BMS_SU_SET_DOC|BMS_SU_SET_DTL|PUB_GOODS|PUB_FACTORY|PUB_BRAND|NP_EFILES_OP_FILE" \ --sink-conf fenodes=192.168.191.93:8030 \ --sink-conf username=cdc_sync_user \ --sink-conf password="xx" \ --sink-conf jdbc-url=jdbc:mysql://192.168.191.93:9030?allowPublicKeyRetrieval=true \ --sink-conf sink.label-prefix=label \ --table-conf replication_num=3 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](function) the bucket number arg of width_bucket should be a positive integer value (#37892) [doris]
mrhhsg commented on PR #38295: URL: https://github.com/apache/doris/pull/38295#issuecomment-2247092902 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] rename delete-recover to recover-from-recyclebin [doris-website]
KassieZ commented on code in PR #910: URL: https://github.com/apache/doris-website/pull/910#discussion_r1689271593 ## i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/recyclebin.md: ## Review Comment: Title is equal to #, the level 2 title . the body start with the level 2 title ## . { "title": "Here is level 1 title", "language": "en" } ## Level 2 title ### Level 3 title ## i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/data-admin/recyclebin.md: ## Review Comment: Title is equal to #, the level 2 title . the body start with the level 2 title ## . { "title": "Here is level 1 title", "language": "en" } ## Level 2 title ### Level 3 title ## versioned_docs/version-2.0/admin-manual/data-admin/recyclebin.md: ## Review Comment: Title is equal to #, the level 2 title . the body start with the level 2 title ## . { "title": "Here is level 1 title", "language": "en" } ## Level 2 title ### Level 3 title ## i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/admin-manual/data-admin/recyclebin.md: ## Review Comment: Title is equal to #, the level 2 title . the body start with the level 2 title ## . { "title": "Here is level 1 title", "language": "en" } ## Level 2 title ### Level 3 title ## docs/admin-manual/data-admin/recyclebin.md: ## Review Comment: 1. The body start with the level 2 title ## . 2. please change the relative path to sql statements : [RECOVER](../../sql-manual/sql-statements/Database-Administration-Statements/RECOVER.md) ## versioned_docs/version-2.1/admin-manual/data-admin/recyclebin.md: ## Review Comment: Title is equal to #, the level 2 title . the body start with the level 2 title ## . { "title": "Here is level 1 title", "language": "en" } ## Level 2 title ### Level 3 title please change the relative path to sql statements :[RECOVER](../../sql-manual/sql-statements/Database-Administration-Statements/RECOVER.md) ## versioned_docs/version-3.0/admin-manual/data-admin/recyclebin.md: ## Review Comment: 1. title without q https://github.com/user-attachments/assets/116652b8-a7d6-44c9-b90e-5f6ca95e4768";> 2. body start with level 2 title 3. please change relative path to sql statements: [RECOVER](../../sql-manual/sql-statements/Database-Administration-Statements/RECOVER.md) ## i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/admin-manual/data-admin/recyclebin.md: ## Review Comment: Title is equal to #, the level 2 title . the body start with the level 2 title ## . { "title": "Here is level 1 title", "language": "en" } ## Level 2 title ### Level 3 title -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression](test) check load channel and load stream before quit [doris]
github-actions[bot] commented on PR #38293: URL: https://github.com/apache/doris/pull/38293#issuecomment-2247098423 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression](test) check load channel and load stream before quit [doris]
github-actions[bot] commented on PR #38293: URL: https://github.com/apache/doris/pull/38293#issuecomment-2247098476 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](inverted index) fixed in_list condition not indexed on pipelinex [doris]
zzzxl1993 commented on PR #38178: URL: https://github.com/apache/doris/pull/38178#issuecomment-2247101967 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](Nereids) add where Null rule to create empty relation as where false [doris]
LiBinfeng-01 commented on PR #38135: URL: https://github.com/apache/doris/pull/38135#issuecomment-2247100244 ./run external -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](parquet-reader) Fix and optimize parquet min-max filtering. [doris]
doris-robot commented on PR #38277: URL: https://github.com/apache/doris/pull/38277#issuecomment-2247100928 TPC-DS: Total hot run time: 173837 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit b42b73aef9328bb05957f36f485d49557d2f, data reload: false query1 911 377 366 366 query2 6437192619071907 query3 6645201 216 201 query4 28565 17394 17305 17305 query5 3597475 476 475 query6 254 187 167 167 query7 4586291 282 282 query8 235 200 188 188 query9 8539239223702370 query10 427 322 283 283 query11 10723 10102 10124 10102 query12 115 83 81 81 query13 1647383 394 383 query14 10095 768476257625 query15 221 167 164 164 query16 7633512 481 481 query17 1553566 513 513 query18 1796270 267 267 query19 195 153 167 153 query20 84 80 81 80 query21 212 133 123 123 query22 4464425640314031 query23 34070 33602 33655 33602 query24 10818 287729042877 query25 613 409 403 403 query26 698 153 149 149 query27 2318283 280 280 query28 5802210520872087 query29 878 638 616 616 query30 251 149 160 149 query31 985 748 776 748 query32 95 51 52 51 query33 659 328 345 328 query34 913 498 506 498 query35 900 792 793 792 query36 1194955 965 955 query37 136 80 81 80 query38 2947286828402840 query39 866 806 816 806 query40 201 123 119 119 query41 48 43 43 43 query42 110 97 96 96 query43 510 487 481 481 query44 1049717 722 717 query45 199 167 172 167 query46 1079757 722 722 query47 1905180317791779 query48 377 289 281 281 query49 843 400 406 400 query50 762 384 380 380 query51 6850669266066606 query52 104 95 87 87 query53 353 295 289 289 query54 877 458 439 439 query55 74 75 70 70 query56 298 258 274 258 query57 1142105410411041 query58 245 253 274 253 query59 2820258528342585 query60 299 285 272 272 query61 97 93 96 93 query62 802 655 650 650 query63 316 280 286 280 query64 9121219416571657 query65 3152309631013096 query66 756 337 390 337 query67 15349 15095 15050 15050 query68 4532549 542 542 query69 481 342 343 342 query70 1186112311241123 query71 416 274 280 274 query72 7153551558805515 query73 739 319 321 319 query74 6124573456605660 query75 4199268426772677 query76 2762929 883 883 query77 613 309 290 290 query78 9667959089408940 query79 3207520 522 520 query80 1625465 465 465 query81 598 220 216 216 query82 832 133 142 133 query83 347 164 170 164 query84 272 89 85 85 query85 1914316 305 305 query86 485 311 316 311 query87 3259311631323116 query88 4563233623592336 query89 479 377 374 374 query90 1850192 188 188 query91 126 170 103 103 query92 56 49 47 47 query93 4743515 505 505 query94 1225295 275 275 query95 407 316 317 316 query96 613 273 271 271 query97 3153302930663029 query98 232 205 197 197 query99 1562127012601260 Total cold run time: 282235 ms Total hot run time: 173837 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [fix](function) the bucket number arg of width_bucket should be a positive integer value (#37892) [doris]
github-actions[bot] commented on PR #38295: URL: https://github.com/apache/doris/pull/38295#issuecomment-2247103181 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](inverted index) opt value extraction from column to string [doris]
github-actions[bot] commented on PR #38179: URL: https://github.com/apache/doris/pull/38179#issuecomment-2247105891 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix])(function) add function regexp_extract_or_null [doris]
doris-robot commented on PR #38296: URL: https://github.com/apache/doris/pull/38296#issuecomment-2247108395 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] adjust table design dir to match create table [doris-website]
KassieZ commented on code in PR #906: URL: https://github.com/apache/doris-website/pull/906#discussion_r1689282464 ## docs/table-design/overview.md: ## Review Comment: 1. the language is EN docs, not zh-CN 2. body start with level 2 title -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](inverted index) opt value extraction from column to string [doris]
github-actions[bot] commented on PR #38179: URL: https://github.com/apache/doris/pull/38179#issuecomment-2247105818 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix])(function) add function regexp_extract_or_null [doris]
suxiaogang223 opened a new pull request, #38296: URL: https://github.com/apache/doris/pull/38296 ## Proposed changes Add function regexp_extract_or_null to be compatible with presto. The function is same as regexp_extract, except that it returns null when no match is found. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](parquet-reader) Fix and optimize parquet min-max filtering. [doris]
doris-robot commented on PR #38277: URL: https://github.com/apache/doris/pull/38277#issuecomment-2247110788 ClickBench: Total hot run time: 30.21 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit b42b73aef9328bb05957f36f485d49557d2f, data reload: false query1 0.040.040.04 query2 0.090.040.03 query3 0.220.060.06 query4 1.660.090.08 query5 0.500.470.48 query6 1.130.730.72 query7 0.020.020.02 query8 0.060.040.05 query9 0.550.500.47 query10 0.540.530.54 query11 0.150.110.12 query12 0.150.120.13 query13 0.600.580.58 query14 0.770.770.76 query15 0.850.800.81 query16 0.350.370.37 query17 1.010.980.96 query18 0.240.220.22 query19 1.791.731.70 query20 0.010.010.01 query21 15.40 0.770.66 query22 4.347.791.53 query23 18.18 1.361.25 query24 2.080.210.24 query25 0.160.080.09 query26 0.300.210.20 query27 0.450.230.22 query28 13.26 1.011.00 query29 12.61 3.283.30 query30 0.250.060.06 query31 2.880.390.38 query32 3.270.490.47 query33 2.842.922.87 query34 17.10 4.404.40 query35 4.434.514.46 query36 0.650.470.47 query37 0.180.150.16 query38 0.150.150.14 query39 0.040.030.03 query40 0.150.130.13 query41 0.100.050.05 query42 0.060.040.05 query43 0.040.040.04 Total cold run time: 109.65 s Total hot run time: 30.21 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix])(function) add function regexp_extract_or_null [doris]
suxiaogang223 commented on PR #38296: URL: https://github.com/apache/doris/pull/38296#issuecomment-2247112237 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] Data Visible in Flink UI but Not Reaching Downstream Database [doris-flink-connector]
HeyChiang commented on issue #442: URL: https://github.com/apache/doris-flink-connector/issues/442#issuecomment-2247116534  FAILED -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [opt](inverted index) opt value extraction from column to string (#38179)
This is an automated email from the ASF dual-hosted git repository. airborne pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-2.1 by this push: new c6cd1e2e3d4 [opt](inverted index) opt value extraction from column to string (#38179) c6cd1e2e3d4 is described below commit c6cd1e2e3d49092960ca0d674a0b677e7043266f Author: zzzxl <33418555+zzzxl1...@users.noreply.github.com> AuthorDate: Wed Jul 24 15:38:41 2024 +0800 [opt](inverted index) opt value extraction from column to string (#38179) ## Proposed changes https://github.com/apache/doris/pull/37395 --- be/src/vec/exprs/vexpr.cpp | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/be/src/vec/exprs/vexpr.cpp b/be/src/vec/exprs/vexpr.cpp index 3a965c27c92..d788c8dc518 100644 --- a/be/src/vec/exprs/vexpr.cpp +++ b/be/src/vec/exprs/vexpr.cpp @@ -631,11 +631,13 @@ std::string VExpr::gen_predicate_result_sign(Block& block, const ColumnNumbers& // Generating 'result_sign' from 'inlist' requires sorting the values. std::set values; for (size_t i = 1; i < arguments.size(); i++) { -values.insert(block.get_by_position(arguments[i]).to_string(0)); +const auto& entry = block.get_by_position(arguments[i]); +values.insert(entry.type->to_string(*entry.column, 0)); } pred_result_sign += boost::join(values, ","); } else { -pred_result_sign += block.get_by_position(arguments[1]).to_string(0); +const auto& entry = block.get_by_position(arguments[1]); +pred_result_sign += entry.type->to_string(*entry.column, 0); } return pred_result_sign; } - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](nereids) refine row count estimation for mark join [doris]
xzj7019 opened a new pull request, #38297: URL: https://github.com/apache/doris/pull/38297 ## Proposed changes pick from https://github.com/apache/doris/pull/38270 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids) refine row count estimation for mark join [doris]
doris-robot commented on PR #38297: URL: https://github.com/apache/doris/pull/38297#issuecomment-2247120795 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix])(function) add function regexp_extract_or_null [doris]
github-actions[bot] commented on PR #38296: URL: https://github.com/apache/doris/pull/38296#issuecomment-2247121466 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](inverted index) opt value extraction from column to string [doris]
airborne12 merged PR #38179: URL: https://github.com/apache/doris/pull/38179 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix])(function) add function regexp_extract_or_null [doris]
github-actions[bot] commented on PR #38296: URL: https://github.com/apache/doris/pull/38296#issuecomment-2247119031 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](group commit) group commit show error url [doris]
doris-robot commented on PR #38292: URL: https://github.com/apache/doris/pull/38292#issuecomment-2247124778 TPC-H: Total hot run time: 40027 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 1f6c62f606fbcfd0ac0e2c51986f1231765fa2d6, data reload: false -- Round 1 -- q1 18679 471443084308 q2 2023190 189 189 q3 10554 116811711168 q4 10249 885 765 765 q5 7554271026442644 q6 229 141 140 140 q7 957 595 605 595 q8 9218213820842084 q9 8754658865776577 q10 8722383737873787 q11 450 233 231 231 q12 403 222 227 222 q13 17753 296630122966 q14 268 248 241 241 q15 523 480 473 473 q16 498 386 378 378 q17 1035724 695 695 q18 8360764773737373 q19 7082139114571391 q20 688 318 308 308 q21 4932321539713215 q22 352 277 281 277 Total cold run time: 119283 ms Total hot run time: 40027 ms - Round 2, with runtime_filter_mode=off - q1 4422429442824282 q2 371 259 275 259 q3 3189294929552949 q4 1991175617211721 q5 5576554655895546 q6 239 135 135 135 q7 2269190519151905 q8 3281344134223422 q9 8822891187878787 q10 3977376038473760 q11 592 515 518 515 q12 837 636 647 636 q13 17046 317031853170 q14 313 300 271 271 q15 529 495 500 495 q16 483 439 436 436 q17 1824154115241524 q18 8179789577117711 q19 1774169316361636 q20 2966188818331833 q21 9602472648914726 q22 602 509 518 509 Total cold run time: 78884 ms Total hot run time: 56228 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Chore](fuzzy) enable some fuzzy variables [doris]
BiteThet merged PR #38155: URL: https://github.com/apache/doris/pull/38155 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Improvement](profile) add catalog info in profile [doris]
Yulei-Yang opened a new pull request, #38298: URL: https://github.com/apache/doris/pull/38298 bp #38283 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](profile) add catalog info in profile [doris]
doris-robot commented on PR #38298: URL: https://github.com/apache/doris/pull/38298#issuecomment-2247129127 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](profile) add catalog info in profile [doris]
Yulei-Yang commented on PR #38298: URL: https://github.com/apache/doris/pull/38298#issuecomment-2247129446 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](profile) add catalog info in profile [doris]
Yulei-Yang commented on PR #38283: URL: https://github.com/apache/doris/pull/38283#issuecomment-2247129206 run cloud_p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [Chore](fuzzy) enable some fuzzy variables (#38155)
This is an automated email from the ASF dual-hosted git repository. panxiaolei pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 501270d99e5 [Chore](fuzzy) enable some fuzzy variables (#38155) 501270d99e5 is described below commit 501270d99e5652ce41851186f894aa83fcc89f3f Author: Pxl AuthorDate: Wed Jul 24 15:44:24 2024 +0800 [Chore](fuzzy) enable some fuzzy variables (#38155) ## Proposed changes enable fuzzy topnOptLimitThreshold, runtimeFilterMaxInNum, runtimeFilterWaitTimeMs, enableSyncRuntimeFilterSize --- fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java | 7 +++ 1 file changed, 7 insertions(+) diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java b/fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java index 12455e31c76..32212ca9d45 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java +++ b/fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java @@ -2113,10 +2113,14 @@ public class SessionVariable implements Serializable, Writable { this.rewriteOrToInPredicateThreshold = 10; this.enableFunctionPushdown = false; this.enableDeleteSubPredicateV2 = false; +this.topnOptLimitThreshold = 0; +this.enableSyncRuntimeFilterSize = true; } else { this.rewriteOrToInPredicateThreshold = 2; this.enableFunctionPushdown = true; this.enableDeleteSubPredicateV2 = true; +this.topnOptLimitThreshold = 1024; +this.enableSyncRuntimeFilterSize = false; } /* @@ -2166,6 +2170,7 @@ public class SessionVariable implements Serializable, Writable { switch (Config.pull_request_id % 3) { case 0: this.fragmentTransmissionCompressionCodec = "snappy"; +this.runtimeFilterWaitTimeMs = 10; break; case 1: this.fragmentTransmissionCompressionCodec = "lz4"; @@ -2180,10 +2185,12 @@ public class SessionVariable implements Serializable, Writable { case 0: this.parallelScanMaxScannersCount = 32; this.parallelScanMinRowsPerScanner = 64; +this.runtimeFilterMaxInNum = 10; break; case 1: this.parallelScanMaxScannersCount = 16; this.parallelScanMinRowsPerScanner = 128; +this.runtimeFilterMaxInNum = 0; break; case 2: this.parallelScanMaxScannersCount = 8; - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction (#38224) [doris]
luwei16 opened a new pull request, #38299: URL: https://github.com/apache/doris/pull/38299 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction (#38224) [doris]
doris-robot commented on PR #38299: URL: https://github.com/apache/doris/pull/38299#issuecomment-2247130579 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids) refine row count estimation for mark join [doris]
xzj7019 commented on PR #38297: URL: https://github.com/apache/doris/pull/38297#issuecomment-2247126655 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](profile) Provide more info for schedule time [doris]
doris-robot commented on PR #38290: URL: https://github.com/apache/doris/pull/38290#issuecomment-2247132398 TPC-H: Total hot run time: 40441 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 6e91ded334a0537d243ba5aad67977b275d9abde, data reload: false -- Round 1 -- q1 18618 455643914391 q2 2023191 193 191 q3 10504 117210681068 q4 10201 839 837 837 q5 7562275926732673 q6 229 145 144 144 q7 976 608 613 608 q8 9228212221152115 q9 8787662366026602 q10 8756379838203798 q11 452 241 243 241 q12 402 234 226 226 q13 17773 305230273027 q14 290 236 242 236 q15 524 486 493 486 q16 513 395 390 390 q17 991 737 742 737 q18 8219752475877524 q19 6533136112771277 q20 707 329 337 329 q21 5093324333163243 q22 371 298 309 298 Total cold run time: 118752 ms Total hot run time: 40441 ms - Round 2, with runtime_filter_mode=off - q1 4421435742584258 q2 372 270 266 266 q3 3125296929602960 q4 1950175417801754 q5 5630553655385536 q6 232 139 138 138 q7 2297181718531817 q8 3353345034263426 q9 8806891287798779 q10 4166380739143807 q11 606 516 500 500 q12 800 626 630 626 q13 16323 320032133200 q14 319 329 299 299 q15 549 488 505 488 q16 518 449 445 445 q17 1854155215211521 q18 8168801579547954 q19 1787157615981576 q20 2119190118801880 q21 5162495749474947 q22 613 551 518 518 Total cold run time: 73170 ms Total hot run time: 56695 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](runtime-filter) adjust always true judge rule of runtime filter apply [doris]
BiteThet commented on PR #38300: URL: https://github.com/apache/doris/pull/38300#issuecomment-2247135949 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](runtime-filter) adjust always true judge rule of runtime filter apply [doris]
BiteThet commented on PR #38300: URL: https://github.com/apache/doris/pull/38300#issuecomment-2247138433 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction (#38224) [doris]
luwei16 commented on PR #38299: URL: https://github.com/apache/doris/pull/38299#issuecomment-2247138920 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Row store) get the correct value of `row_store_page_size` after FE restarts [doris]
lxr599 commented on PR #38236: URL: https://github.com/apache/doris/pull/38236#issuecomment-2247139534 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction (#38224) [doris]
github-actions[bot] commented on PR #38299: URL: https://github.com/apache/doris/pull/38299#issuecomment-2247141633 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](group commit) group commit show error url [doris]
doris-robot commented on PR #38292: URL: https://github.com/apache/doris/pull/38292#issuecomment-2247144170 TPC-DS: Total hot run time: 174329 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 1f6c62f606fbcfd0ac0e2c51986f1231765fa2d6, data reload: false query1 914 370 363 363 query2 6394204818431843 query3 6652210 216 210 query4 28257 17280 17418 17280 query5 3663482 493 482 query6 264 173 167 167 query7 4576283 281 281 query8 240 187 190 187 query9 8393241723712371 query10 420 290 267 267 query11 12224 10246 10087 10087 query12 121 92 83 83 query13 1634382 370 370 query14 10250 830976227622 query15 220 170 166 166 query16 7721424 465 424 query17 1279537 521 521 query18 1855273 272 272 query19 191 154 147 147 query20 87 83 89 83 query21 203 132 132 132 query22 4398435842124212 query23 34110 33748 33753 33748 query24 11229 290928622862 query25 603 397 406 397 query26 974 155 152 152 query27 2438274 280 274 query28 7015210120812081 query29 864 641 622 622 query30 269 158 160 158 query31 954 759 748 748 query32 90 52 57 52 query33 741 353 331 331 query34 913 498 519 498 query35 925 769 764 764 query36 1151989 994 989 query37 136 82 89 82 query38 2932277927752775 query39 864 825 815 815 query40 194 117 121 117 query41 45 43 43 43 query42 109 98 98 98 query43 509 489 495 489 query44 1163721 734 721 query45 194 162 164 162 query46 1090755 700 700 query47 1861176117951761 query48 364 298 297 297 query49 835 403 413 403 query50 778 384 395 384 query51 6740671266886688 query52 107 91 95 91 query53 351 298 283 283 query54 860 445 443 443 query55 75 74 73 73 query56 291 262 294 262 query57 1133103210871032 query58 239 240 258 240 query59 3003267728962677 query60 309 289 275 275 query61 94 92 92 92 query62 802 643 658 643 query63 330 293 283 283 query64 9514219716811681 query65 3189311031133110 query66 826 325 324 324 query67 15517 15191 15083 15083 query68 4532539 545 539 query69 526 342 340 340 query70 1192111411001100 query71 408 281 271 271 query72 7355575756735673 query73 756 323 321 321 query74 6230568356175617 query75 3396268026812680 query76 2226951 941 941 query77 506 293 305 293 query78 9622910190059005 query79 2482534 522 522 query80 2525467 470 467 query81 596 220 220 220 query82 1041139 136 136 query83 298 164 164 164 query84 252 87 87 87 query85 1251319 304 304 query86 457 337 272 272 query87 3258311031393110 query88 3363242523492349 query89 466 375 397 375 query90 1723187 189 187 query91 131 101 103 101 query92 60 50 50 50 query93 2102524 512 512 query94 1164277 295 277 query95 416 318 306 306 query96 606 267 272 267 query97 3221307930043004 query98 221 206 203 203 query99 1508126512631263 Total cold run time: 280576 ms Total hot run time: 174329 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [fix])(function) add function regexp_extract_or_null [doris]
doris-robot commented on PR #38296: URL: https://github.com/apache/doris/pull/38296#issuecomment-2247145957 TPC-H: Total hot run time: 40242 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 8e4da4816b00e4ad44cd101cc9a7868c49671ed7, data reload: false -- Round 1 -- q1 18176 444943704370 q2 2576217 187 187 q3 11909 118411461146 q4 10626 944 789 789 q5 8069272527092709 q6 225 138 143 138 q7 960 589 589 589 q8 9215204320792043 q9 8734655865536553 q10 8808376937523752 q11 446 245 236 236 q12 399 237 234 234 q13 17781 300129802980 q14 266 241 225 225 q15 533 485 485 485 q16 481 394 376 376 q17 961 748 667 667 q18 8113753374917491 q19 3982140314231403 q20 709 330 330 330 q21 4910328632533253 q22 350 294 286 286 Total cold run time: 118229 ms Total hot run time: 40242 ms - Round 2, with runtime_filter_mode=off - q1 4415426642414241 q2 367 282 268 268 q3 3004276327412741 q4 1904162516241624 q5 5285530152705270 q6 218 132 129 129 q7 2107171217011701 q8 3188334433173317 q9 8366834583648345 q10 3929369537313695 q11 558 473 486 473 q12 765 602 618 602 q13 16408 295829522952 q14 310 275 272 272 q15 515 474 470 470 q16 474 418 423 418 q17 1776147114591459 q18 7635761072767276 q19 1694147215941472 q20 1965176117581758 q21 4825464446234623 q22 585 491 509 491 Total cold run time: 70293 ms Total hot run time: 53597 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](workloadgroup) use slot num to control memory distribution among one workload group [doris]
doris-robot commented on PR #38237: URL: https://github.com/apache/doris/pull/38237#issuecomment-2247146801 TPC-H: Total hot run time: 40076 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 94fe659308739e065aa346c1b461ad59cc8e8b36, data reload: false -- Round 1 -- q1 19371 561743124312 q2 2024191 199 191 q3 10546 116111221122 q4 10245 850 834 834 q5 7584281327202720 q6 225 138 138 138 q7 971 605 617 605 q8 9219211720752075 q9 8742658265416541 q10 8749380237663766 q11 520 237 251 237 q12 401 231 227 227 q13 18834 294129882941 q14 281 233 244 233 q15 523 489 496 489 q16 494 407 406 406 q17 985 673 689 673 q18 8127749374757475 q19 7048144712991299 q20 680 324 330 324 q21 4960318232973182 q22 360 287 286 286 Total cold run time: 120889 ms Total hot run time: 40076 ms - Round 2, with runtime_filter_mode=off - q1 4395424742654247 q2 388 287 285 285 q3 3183292328952895 q4 2041168416611661 q5 5590550555805505 q6 231 137 136 136 q7 2229182419011824 q8 3277344434233423 q9 8866881187918791 q10 3973389539393895 q11 595 519 502 502 q12 819 649 623 623 q13 16322 321531523152 q14 305 297 281 281 q15 521 495 481 481 q16 512 452 444 444 q17 1852154015321532 q18 8230790778027802 q19 1763154516411545 q20 2142187918801879 q21 9952467848254678 q22 584 498 492 492 Total cold run time: 0 ms Total hot run time: 56073 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](runtime-filter) adjust always true judge rule of runtime filter apply [doris]
github-actions[bot] commented on code in PR #38300: URL: https://github.com/apache/doris/pull/38300#discussion_r1689316626 ## be/src/olap/comparison_predicate.h: ## @@ -246,14 +246,17 @@ class ComparisonPredicateBase : public ColumnPredicate { template __attribute__((flatten)) void _evaluate_vec_internal(const vectorized::IColumn& column, Review Comment: warning: function '_evaluate_vec_internal' has cognitive complexity of 59 (threshold 50) [readability-function-cognitive-complexity] ```cpp __attribute__((flatten)) void _evaluate_vec_internal(const vectorized::IColumn& column, ^ ``` Additional context **be/src/olap/comparison_predicate.h:250:** +1, including nesting penalty of 0, nesting level increased to 1 ```cpp if (_can_ignore()) { ^ ``` **be/src/olap/comparison_predicate.h:251:** +2, including nesting penalty of 1, nesting level increased to 2 ```cpp if (is_and) { ^ ``` **be/src/olap/comparison_predicate.h:252:** +3, including nesting penalty of 2, nesting level increased to 3 ```cpp for (uint16_t i = 0; i < size; i++) { ^ ``` **be/src/olap/comparison_predicate.h:255:** +1, nesting level increased to 2 ```cpp } else { ^ ``` **be/src/olap/comparison_predicate.h:261:** +1, including nesting penalty of 0, nesting level increased to 1 ```cpp if (column.is_nullable()) { ^ ``` **be/src/olap/comparison_predicate.h:269:** +2, including nesting penalty of 1, nesting level increased to 2 ```cpp if (nested_column.is_column_dictionary()) { ^ ``` **be/src/olap/comparison_predicate.h:270:** +3, including nesting penalty of 2, nesting level increased to 3 ```cpp if constexpr (std::is_same_v) { ^ ``` **be/src/olap/comparison_predicate.h:276:** +4, including nesting penalty of 3, nesting level increased to 4 ```cpp do { ^ ``` **be/src/olap/comparison_predicate.h:277:** +5, including nesting penalty of 4, nesting level increased to 5 ```cpp if constexpr (PT == PredicateType::EQ) { ^ ``` **be/src/olap/comparison_predicate.h:278:** +6, including nesting penalty of 5, nesting level increased to 6 ```cpp if (dict_code == -2) { ^ ``` **be/src/olap/comparison_predicate.h:288:** +1, nesting level increased to 3 ```cpp } else { ^ ``` **be/src/olap/comparison_predicate.h:292:** +1, nesting level increased to 2 ```cpp } else { ^ ``` **be/src/olap/comparison_predicate.h:302:** +1, nesting level increased to 1 ```cpp } else { ^ ``` **be/src/olap/comparison_predicate.h:303:** +2, including nesting penalty of 1, nesting level increased to 2 ```cpp if (column.is_column_dictionary()) { ^ ``` **be/src/olap/comparison_predicate.h:304:** +3, including nesting penalty of 2, nesting level increased to 3 ```cpp if constexpr (std::is_same_v) { ^ ``` **be/src/olap/comparison_predicate.h:308:** +4, including nesting penalty of 3, nesting level increased to 4 ```cpp do { ^ ``` **be/src/olap/comparison_predicate.h:309:** +5, including nesting penalty of 4, nesting level increased to 5 ```cpp if constexpr (PT == PredicateType::EQ) { ^ ``` **be/src/olap/comparison_predicate.h:310:** +6, including nesting penalty of 5, nesting level increased to 6 ```cpp if (dict_code == -2) { ^ ``` **be/src/olap/comparison_predicate.h:319:** +1, nesting level increased to 3 ```cpp } else { ^ ``` **be/src/olap/comparison_predicate.h:323:** +1, nesting level increased to 2 ```cpp } else { ^ ``` **be/src/olap/comparison_predicate.h:335:** +1, including nesting penalty of 0, nesting level increased to 1 ```cpp if (_opposite) { ^ ``` **be/src/olap/comparison_predicate.h:336:** +2, including nesting penalty of 1, nesting level increased to 2 ```cpp for (uint16_t i = 0; i < size; i++) { ^ ``` **be/src/olap/comparison_predicate.h:341:** +1, including nesting penalty of 0, nesting level increased to 1 ```cpp if (_can_ignore()) { ^ ``` **be/src/olap/comparison_predicate.h:342:** +2, including nesting penalty of 1, nesting
Re: [PR] [fix](cloud tvf) Fix tvf query run in cloud multi cluster [doris]
deardeng commented on PR #37157: URL: https://github.com/apache/doris/pull/37157#issuecomment-2247152254 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](group commit) group commit show error url [doris]
doris-robot commented on PR #38292: URL: https://github.com/apache/doris/pull/38292#issuecomment-2247154398 ClickBench: Total hot run time: 30.77 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 1f6c62f606fbcfd0ac0e2c51986f1231765fa2d6, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.220.040.05 query4 1.680.070.07 query5 0.610.470.48 query6 1.130.720.72 query7 0.020.010.01 query8 0.060.040.04 query9 0.550.500.49 query10 0.540.560.54 query11 0.150.120.11 query12 0.140.110.13 query13 0.600.580.58 query14 0.760.780.76 query15 0.850.810.82 query16 0.350.350.37 query17 0.980.980.95 query18 0.220.210.22 query19 1.751.721.66 query20 0.010.010.01 query21 15.44 0.790.67 query22 3.967.122.26 query23 18.29 1.351.26 query24 2.160.230.22 query25 0.150.100.09 query26 0.300.210.21 query27 0.460.220.23 query28 13.26 1.021.00 query29 12.56 3.283.26 query30 0.250.060.05 query31 2.870.380.37 query32 3.290.480.48 query33 2.882.882.92 query34 17.02 4.354.35 query35 4.424.424.39 query36 0.650.460.48 query37 0.190.160.16 query38 0.160.150.15 query39 0.040.040.04 query40 0.160.120.12 query41 0.090.050.05 query42 0.060.050.04 query43 0.050.040.04 Total cold run time: 109.45 s Total hot run time: 30.77 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Row store) get the correct value of `row_store_page_size` after FE restarts [doris]
doris-robot commented on PR #38236: URL: https://github.com/apache/doris/pull/38236#issuecomment-2247153285 TPC-H: Total hot run time: 39743 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 26e18ac623c12ffa189e17ac6897901ce8faa0c0, data reload: false -- Round 1 -- q1 17650 432742804280 q2 2023188 189 188 q3 10401 116911321132 q4 10187 815 708 708 q5 7533279426472647 q6 218 139 139 139 q7 952 598 596 596 q8 9216208120812081 q9 8777658365596559 q10 8789377837833778 q11 491 235 237 235 q12 397 227 222 222 q13 17873 295529882955 q14 273 234 234 234 q15 523 481 497 481 q16 521 368 375 368 q17 965 641 763 641 q18 8064759273587358 q19 7370145613111311 q20 661 329 326 326 q21 4915321933093219 q22 351 286 285 285 Total cold run time: 118150 ms Total hot run time: 39743 ms - Round 2, with runtime_filter_mode=off - q1 4431423142384231 q2 376 277 257 257 q3 2964288429762884 q4 1972167017621670 q5 5647553054335433 q6 229 138 130 130 q7 2183184218381838 q8 3246342534533425 q9 8800882588978825 q10 4138387837083708 q11 592 516 502 502 q12 807 661 618 618 q13 16907 316731813167 q14 321 301 290 290 q15 518 500 478 478 q16 505 436 437 436 q17 1819152015151515 q18 8131805778287828 q19 1736166615751575 q20 2936191118551855 q21 8157489446294629 q22 607 501 537 501 Total cold run time: 77022 ms Total hot run time: 55795 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](profile) Provide more info for schedule time [doris]
doris-robot commented on PR #38290: URL: https://github.com/apache/doris/pull/38290#issuecomment-2247153472 TPC-DS: Total hot run time: 175791 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 6e91ded334a0537d243ba5aad67977b275d9abde, data reload: false query1 927 385 379 379 query2 6456187618311831 query3 6647211 219 211 query4 25874 17556 17404 17404 query5 3777496 498 496 query6 276 185 183 183 query7 4609296 289 289 query8 236 187 193 187 query9 8497242123982398 query10 445 306 272 272 query11 12167 10221 10169 10169 query12 126 86 85 85 query13 1638382 368 368 query14 10365 768679027686 query15 214 168 171 168 query16 7764515 485 485 query17 1607596 555 555 query18 1986289 289 289 query19 196 155 161 155 query20 91 85 83 83 query21 205 142 134 134 query22 4571403240014001 query23 34351 34048 33708 33708 query24 11036 303530473035 query25 648 425 416 416 query26 1139172 171 171 query27 2309286 297 286 query28 6987209320902090 query29 942 698 676 676 query30 259 157 164 157 query31 981 788 763 763 query32 97 60 57 57 query33 780 377 368 368 query34 914 511 515 511 query35 912 777 789 777 query36 11651007981 981 query37 150 91 92 91 query38 3082285628812856 query39 960 842 810 810 query40 212 126 127 126 query41 51 46 50 46 query42 118 107 104 104 query43 508 462 457 457 query44 1177733 749 733 query45 203 165 165 165 query46 1110758 839 758 query47 1820177317431743 query48 377 300 287 287 query49 860 408 456 408 query50 790 414 401 401 query51 6684667366846673 query52 107 96 92 92 query53 370 300 298 298 query54 868 455 455 455 query55 76 79 75 75 query56 303 273 288 273 query57 1143104110581041 query58 252 256 270 256 query59 2884253525902535 query60 304 280 291 280 query61 95 93 98 93 query62 795 644 662 644 query63 321 298 299 298 query64 9511223017431743 query65 3179313031263126 query66 813 329 332 329 query67 15752 15051 15206 15051 query68 8637558 559 558 query69 791 441 371 371 query70 1431116710841084 query71 530 289 293 289 query72 9096601557365736 query73 2200332 330 330 query74 6257569756745674 query75 5563269727382697 query76 5143926 1031926 query77 794 318 316 316 query78 983310780 92939293 query79 11063 557 542 542 query80 865 491 493 491 query81 578 222 220 220 query82 640 149 142 142 query83 395 173 170 170 query84 285 87 86 86 query85 727 301 306 301 query86 370 327 321 321 query87 3352314731503147 query88 4149237623662366 query89 507 395 398 395 query90 2152202 201 201 query91 133 103 104 103 query92 68 50 52 50 query93 5637536 539 536 query94 1345344 284 284 query95 433 328 326 326 query96 609 278 276 276 query97 3203305930353035 query98 228 212 196 196 query99 1560126212581258 Total cold run time: 304643 ms Total hot run time: 175791 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th
Re: [PR] [Feature](multi-catalog) Add memory tracker for orc reader/writer and arrow parquet writer。 [doris]
doris-robot commented on PR #37257: URL: https://github.com/apache/doris/pull/37257#issuecomment-2247159711 TeamCity be ut coverage result: Function Coverage: 36.40% (9243/25396) Line Coverage: 27.97% (75534/270071) Region Coverage: 26.79% (38830/144960) Branch Coverage: 23.53% (19712/83764) Coverage Report: http://coverage.selectdb-in.cc/coverage/8f3272d573f85eac5bf4ac38c04a24caaa92e5bf_8f3272d573f85eac5bf4ac38c04a24caaa92e5bf/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [spill](regression) add regression test cases for spill [doris]
doris-robot commented on PR #37319: URL: https://github.com/apache/doris/pull/37319#issuecomment-2247161323 TPC-H: Total hot run time: 39632 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 8790744140c53fe64a3ff8dc9f6a4c533ddbc5e4, data reload: false -- Round 1 -- q1 17661 457942524252 q2 2015189 194 189 q3 10504 123810691069 q4 10188 708 859 708 q5 7563271926952695 q6 225 145 140 140 q7 959 596 597 596 q8 9218209620962096 q9 8732658665776577 q10 8857382137603760 q11 453 232 228 228 q12 417 214 218 214 q13 17779 294929602949 q14 267 242 229 229 q15 532 488 489 488 q16 511 383 377 377 q17 997 612 703 612 q18 8168747774297429 q19 6698130113871301 q20 684 317 330 317 q21 4987312632733126 q22 359 287 280 280 Total cold run time: 117774 ms Total hot run time: 39632 ms - Round 2, with runtime_filter_mode=off - q1 4441429642654265 q2 361 272 262 262 q3 2972292629312926 q4 1971172616971697 q5 5622553554995499 q6 226 138 140 138 q7 2252187218201820 q8 3315345134273427 q9 8786887088578857 q10 4102382938653829 q11 583 487 498 487 q12 816 624 643 624 q13 16526 317631793176 q14 331 282 290 282 q15 522 513 511 511 q16 495 451 439 439 q17 1815153515121512 q18 8085789880007898 q19 1773158515571557 q20 2762191218271827 q21 7826478246584658 q22 606 491 491 491 Total cold run time: 76188 ms Total hot run time: 56182 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature] support data mask in doris internal auth [doris]
doris-robot commented on PR #38301: URL: https://github.com/apache/doris/pull/38301#issuecomment-2247162335 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Feature] support data mask in doris internal auth [doris]
qzsee opened a new pull request, #38301: URL: https://github.com/apache/doris/pull/38301 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](profile) Provide more info for schedule time [doris]
doris-robot commented on PR #38290: URL: https://github.com/apache/doris/pull/38290#issuecomment-2247164158 ClickBench: Total hot run time: 31.03 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 6e91ded334a0537d243ba5aad67977b275d9abde, data reload: false query1 0.040.030.03 query2 0.090.040.04 query3 0.230.070.06 query4 1.640.090.10 query5 0.520.490.49 query6 1.130.740.73 query7 0.020.010.01 query8 0.050.040.04 query9 0.550.480.50 query10 0.550.530.54 query11 0.160.120.11 query12 0.150.130.12 query13 0.620.590.59 query14 0.780.790.77 query15 0.860.820.82 query16 0.360.370.37 query17 0.960.951.04 query18 0.230.210.22 query19 1.811.701.72 query20 0.020.010.01 query21 15.67 0.770.64 query22 4.316.852.27 query23 18.28 1.421.26 query24 2.130.220.22 query25 0.150.090.09 query26 0.290.220.22 query27 0.450.230.25 query28 13.35 1.051.00 query29 12.62 3.303.30 query30 0.240.060.05 query31 2.880.400.39 query32 3.240.480.46 query33 2.922.892.93 query34 17.09 4.364.36 query35 4.414.464.42 query36 0.650.490.49 query37 0.200.150.16 query38 0.160.150.15 query39 0.050.040.04 query40 0.150.120.14 query41 0.090.040.05 query42 0.060.050.05 query43 0.040.050.04 Total cold run time: 110.2 s Total hot run time: 31.03 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](group commit) group commit show error url [doris]
mymeiyi commented on PR #38292: URL: https://github.com/apache/doris/pull/38292#issuecomment-2247166611 run cloud_p1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix])(function) add function regexp_extract_or_null [doris]
doris-robot commented on PR #38296: URL: https://github.com/apache/doris/pull/38296#issuecomment-2247167371 TPC-DS: Total hot run time: 172205 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 8e4da4816b00e4ad44cd101cc9a7868c49671ed7, data reload: false query1 922 371 360 360 query2 6458184817791779 query3 6658206 219 206 query4 26394 17596 17162 17162 query5 4182477 483 477 query6 268 174 181 174 query7 4582299 285 285 query8 231 182 186 182 query9 8518243024162416 query10 451 295 272 272 query11 12533 997998649864 query12 135 87 82 82 query13 1635381 386 381 query14 10243 749676247496 query15 220 165 168 165 query16 7891501 479 479 query17 1574558 515 515 query18 1992273 271 271 query19 198 142 151 142 query20 91 82 89 82 query21 208 127 125 125 query22 4361394342353943 query23 33717 32894 33092 32894 query24 12040 288028602860 query25 668 363 384 363 query26 1763152 151 151 query27 3025268 273 268 query28 7754203020012001 query29 1126624 614 614 query30 286 145 148 145 query31 959 719 729 719 query32 94 55 56 55 query33 762 328 323 323 query34 916 475 474 474 query35 847 734 714 714 query36 1071966 938 938 query37 283 80 80 80 query38 2861273227432732 query39 856 798 803 798 query40 270 116 123 116 query41 47 44 46 44 query42 119 98 101 98 query43 490 460 478 460 query44 1261731 724 724 query45 188 193 159 159 query46 1098718 728 718 query47 1848175517701755 query48 379 287 291 287 query49 1176414 412 412 query50 795 400 395 395 query51 6697657866086578 query52 108 90 95 90 query53 355 299 290 290 query54 974 450 445 445 query55 75 74 74 74 query56 287 263 265 263 query57 1182105210811052 query58 269 246 250 246 query59 2795269125452545 query60 308 286 285 285 query61 94 95 95 95 query62 829 647 665 647 query63 322 289 285 285 query64 10451 220016781678 query65 3143310431093104 query66 1376355 360 355 query67 15643 15143 14873 14873 query68 4600548 538 538 query69 462 328 336 328 query70 1148115611431143 query71 402 277 293 277 query72 7170620058785878 query73 744 331 326 326 query74 6062563656315631 query75 3398261627012616 query76 28111003935 935 query77 458 312 308 308 query78 9722895389138913 query79 2717534 532 532 query80 2080482 488 482 query81 574 224 219 219 query82 907 136 136 136 query83 273 169 170 169 query84 274 95 85 85 query85 2095314 365 314 query86 500 322 297 297 query87 3288305030723050 query88 3941238523802380 query89 483 381 387 381 query90 1851200 199 199 query91 129 98 102 98 query92 58 51 51 51 query93 3041520 510 510 query94 1283300 288 288 query95 405 319 315 315 query96 597 276 270 270 query97 3206302230243022 query98 216 198 192 192 query99 1784126212731262 Total cold run time: 288031 ms Total hot run time: 172205 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s
Re: [PR] [feature](workloadgroup) use slot num to control memory distribution among one workload group [doris]
doris-robot commented on PR #38237: URL: https://github.com/apache/doris/pull/38237#issuecomment-2247168850 TPC-DS: Total hot run time: 173897 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 94fe659308739e065aa346c1b461ad59cc8e8b36, data reload: false query1 917 374 367 367 query2 6419184818071807 query3 6657204 219 204 query4 28616 17354 17308 17308 query5 3752480 483 480 query6 283 166 160 160 query7 4581294 285 285 query8 231 190 190 190 query9 8638240423812381 query10 418 274 278 274 query11 12501 10099 10053 10053 query12 115 88 85 85 query13 1644373 357 357 query14 10170 751583197515 query15 220 170 164 164 query16 7404468 471 468 query17 1140576 518 518 query18 1821276 279 276 query19 195 147 154 147 query20 91 82 85 82 query21 204 124 123 123 query22 4347399541553995 query23 34024 33744 33785 33744 query24 11032 294529062906 query25 588 382 418 382 query26 706 152 153 152 query27 2412280 283 280 query28 6075208820852085 query29 912 628 663 628 query30 259 160 150 150 query31 950 782 736 736 query32 95 53 58 53 query33 664 335 346 335 query34 909 511 513 511 query35 922 764 788 764 query36 11621007978 978 query37 149 88 87 87 query38 3016291327892789 query39 902 885 856 856 query40 210 121 127 121 query41 47 44 45 44 query42 115 104 101 101 query43 507 457 473 457 query44 1083730 735 730 query45 191 165 163 163 query46 1101715 733 715 query47 1825177017871770 query48 379 296 292 292 query49 856 444 428 428 query50 788 396 409 396 query51 6859668364986498 query52 111 93 99 93 query53 365 300 300 300 query54 888 457 459 457 query55 82 75 76 75 query56 309 295 290 290 query57 1133106210611061 query58 264 260 278 260 query59 2811252527642525 query60 322 300 307 300 query61 121 119 114 114 query62 792 646 669 646 query63 327 292 294 292 query64 9251231417081708 query65 3204315031243124 query66 777 337 341 337 query67 15632 14915 14879 14879 query68 6269562 566 562 query69 764 467 384 384 query70 1138110411651104 query71 533 296 288 288 query72 9040551057685510 query73 829 329 324 324 query74 6079571457685714 query75 4733267027192670 query76 4373930 976 930 query77 758 308 311 308 query78 9725917689978997 query79 10528 541 525 525 query80 1085501 475 475 query81 576 223 218 218 query82 850 171 130 130 query83 329 166 167 166 query84 279 86 86 86 query85 1398317 301 301 query86 452 301 327 301 query87 3299314431273127 query88 5001237923882379 query89 550 389 390 389 query90 1994192 195 192 query91 130 100 105 100 query92 71 53 49 49 query93 7160523 518 518 query94 1286287 321 287 query95 413 316 329 316 query96 620 283 275 275 query97 3241296430262964 query98 220 202 206 202 query99 1579125512831255 Total cold run time: 300793 ms Total hot run time: 173897 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t
Re: [PR] [Improvement](profile) add catalog info in profile [doris]
Yulei-Yang commented on PR #38302: URL: https://github.com/apache/doris/pull/38302#issuecomment-2247170313 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Improvement](profile) add catalog info in profile [doris]
Yulei-Yang opened a new pull request, #38302: URL: https://github.com/apache/doris/pull/38302 bp #38283 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](profile) add catalog info in profile [doris]
doris-robot commented on PR #38302: URL: https://github.com/apache/doris/pull/38302#issuecomment-2247169862 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
zclllyybb commented on code in PR #38103: URL: https://github.com/apache/doris/pull/38103#discussion_r1689333607 ## fe/fe-core/src/main/java/org/apache/doris/insertoverwrite/InsertOverwriteManager.java: ## @@ -107,44 +107,82 @@ public void registerTaskInGroup(long groupId, long taskId) { taskGroups.get(groupId).add(taskId); } -public List tryReplacePartitionIds(long groupId, List oldPartitionIds) { +/** + * this func should in lock scope of getLock(groupId) + * + * @param newIds if have replaced, replace with new. otherwise itself. + * @return + */ +public boolean tryReplacePartitionIds(long groupId, List oldPartitionIds, List newIds) { Map relations = partitionPairs.get(groupId); -List newIds = new ArrayList(); -for (Long id : oldPartitionIds) { +boolean needReplace = false; +for (int i = 0; i < oldPartitionIds.size(); i++) { +long id = oldPartitionIds.get(i); if (relations.containsKey(id)) { // if we replaced it. then return new one. newIds.add(relations.get(id)); } else { -// otherwise itself. we will deal it soon. -newIds.add(id); +newIds.add(oldPartitionIds.get(i)); Review Comment: well they're same. will fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](function) optimization for substr with ascii string [doris]
yanyxin commented on PR #29799: URL: https://github.com/apache/doris/pull/29799#issuecomment-2247174047 @BiteThet Will the modifications be synchronized to 2.1.5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Row store) get the correct value of `row_store_page_size` after FE restarts [doris]
doris-robot commented on PR #38236: URL: https://github.com/apache/doris/pull/38236#issuecomment-2247175447 TPC-DS: Total hot run time: 174056 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 26e18ac623c12ffa189e17ac6897901ce8faa0c0, data reload: false query1 908 368 378 368 query2 6432196318481848 query3 6641205 215 205 query4 26767 17694 17447 17447 query5 3811491 503 491 query6 264 176 169 169 query7 4594287 277 277 query8 227 190 184 184 query9 8534240023902390 query10 432 318 280 280 query11 11798 10086 10112 10086 query12 111 81 80 80 query13 1635362 357 357 query14 9445754276987542 query15 217 175 173 173 query16 7738499 473 473 query17 1364538 535 535 query18 1907276 272 272 query19 199 151 155 151 query20 92 86 82 82 query21 211 127 128 127 query22 4499431039983998 query23 34172 33622 33669 33622 query24 10877 293828722872 query25 605 403 398 398 query26 705 149 148 148 query27 2252276 289 276 query28 6193208320552055 query29 867 634 633 633 query30 268 155 153 153 query31 986 779 770 770 query32 91 53 62 53 query33 640 356 347 347 query34 920 497 515 497 query35 865 733 763 733 query36 1141978 948 948 query37 137 81 84 81 query38 2952283027522752 query39 877 813 831 813 query40 200 121 117 117 query41 46 43 45 43 query42 112 99 97 97 query43 501 470 463 463 query44 1112722 723 722 query45 190 159 160 159 query46 1101728 756 728 query47 1844179617981796 query48 353 291 296 291 query49 856 405 436 405 query50 772 381 389 381 query51 6776667867636678 query52 98 95 91 91 query53 358 289 285 285 query54 861 456 445 445 query55 76 74 76 74 query56 285 265 268 265 query57 1146104410441044 query58 237 252 273 252 query59 2938267025592559 query60 322 273 265 265 query61 98 95 95 95 query62 798 640 651 640 query63 321 285 287 285 query64 9249232417851785 query65 3167310731093107 query66 740 321 319 319 query67 15583 15091 14853 14853 query68 4622549 539 539 query69 730 426 349 349 query70 1154115810611061 query71 411 278 276 276 query72 7692569654845484 query73 775 342 333 333 query74 6432600357215721 query75 3448269327162693 query76 2684953 933 933 query77 636 312 311 311 query78 9653920210144 9202 query79 2993517 536 517 query80 1360464 467 464 query81 583 217 218 217 query82 736 137 132 132 query83 332 164 168 164 query84 271 86 84 84 query85 1666321 300 300 query86 473 325 321 321 query87 3268310631173106 query88 4060237723852377 query89 472 395 382 382 query90 1729193 196 193 query91 132 102 101 101 query92 58 48 49 48 query93 2387512 519 512 query94 1132278 304 278 query95 416 317 315 315 query96 600 274 276 274 query97 3170302430023002 query98 230 199 205 199 query99 1637123812701238 Total cold run time: 278456 ms Total hot run time: 174056 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [fix])(function) add function regexp_extract_or_null [doris]
doris-robot commented on PR #38296: URL: https://github.com/apache/doris/pull/38296#issuecomment-2247178680 ClickBench: Total hot run time: 30.56 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 8e4da4816b00e4ad44cd101cc9a7868c49671ed7, data reload: false query1 0.040.040.03 query2 0.090.040.03 query3 0.220.050.05 query4 1.680.070.07 query5 0.500.500.47 query6 1.120.720.72 query7 0.020.020.01 query8 0.040.040.04 query9 0.540.500.50 query10 0.560.560.53 query11 0.150.110.12 query12 0.140.120.12 query13 0.600.580.58 query14 0.750.780.77 query15 0.840.800.81 query16 0.350.360.35 query17 0.961.001.00 query18 0.220.220.21 query19 1.821.751.76 query20 0.010.010.01 query21 15.41 0.780.66 query22 4.287.221.89 query23 18.33 1.391.22 query24 2.120.230.22 query25 0.140.080.09 query26 0.310.210.21 query27 0.450.230.23 query28 13.26 1.010.99 query29 12.56 3.323.33 query30 0.260.060.06 query31 2.880.380.39 query32 3.280.490.47 query33 2.892.922.87 query34 17.30 4.384.42 query35 4.474.424.38 query36 0.660.490.51 query37 0.180.150.15 query38 0.150.160.14 query39 0.040.030.04 query40 0.150.130.12 query41 0.080.040.05 query42 0.060.040.04 query43 0.040.040.04 Total cold run time: 109.95 s Total hot run time: 30.56 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](workloadgroup) use slot num to control memory distribution among one workload group [doris]
doris-robot commented on PR #38237: URL: https://github.com/apache/doris/pull/38237#issuecomment-2247180109 ClickBench: Total hot run time: 30.8 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 94fe659308739e065aa346c1b461ad59cc8e8b36, data reload: false query1 0.040.040.03 query2 0.070.040.04 query3 0.220.050.04 query4 1.670.070.09 query5 0.490.490.47 query6 1.130.740.72 query7 0.020.010.01 query8 0.050.040.04 query9 0.560.500.51 query10 0.540.550.56 query11 0.140.110.12 query12 0.150.130.13 query13 0.590.580.57 query14 0.760.790.77 query15 0.850.830.80 query16 0.370.360.37 query17 1.040.970.98 query18 0.240.220.21 query19 1.771.681.71 query20 0.010.010.01 query21 15.40 0.780.65 query22 3.187.852.06 query23 18.29 1.381.29 query24 2.080.230.23 query25 0.170.090.09 query26 0.280.200.21 query27 0.450.230.23 query28 13.26 1.021.02 query29 12.62 3.343.31 query30 0.260.060.06 query31 2.860.410.39 query32 3.270.470.46 query33 2.892.962.91 query34 16.99 4.394.36 query35 4.414.414.43 query36 0.640.460.47 query37 0.200.150.17 query38 0.150.150.15 query39 0.040.040.03 query40 0.140.120.13 query41 0.100.050.06 query42 0.060.050.06 query43 0.060.050.04 Total cold run time: 108.51 s Total hot run time: 30.8 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [spill](regression) add regression test cases for spill [doris]
doris-robot commented on PR #37319: URL: https://github.com/apache/doris/pull/37319#issuecomment-2247183964 TPC-DS: Total hot run time: 174786 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 8790744140c53fe64a3ff8dc9f6a4c533ddbc5e4, data reload: false query1 896 368 364 364 query2 6440181217391739 query3 6637206 219 206 query4 28170 17594 17395 17395 query5 3765475 475 475 query6 291 176 157 157 query7 4573287 286 286 query8 228 187 193 187 query9 8459238223732373 query10 428 292 261 261 query11 12251 10157 10165 10157 query12 112 83 88 83 query13 1632361 359 359 query14 9212776879397768 query15 239 162 163 162 query16 7586473 474 473 query17 1292569 519 519 query18 1356271 271 271 query19 197 148 147 147 query20 85 81 81 81 query21 206 128 131 128 query22 4564420542724205 query23 34039 33761 33535 33535 query24 11229 296829982968 query25 599 414 380 380 query26 1220148 147 147 query27 2746278 282 278 query28 7953208620582058 query29 883 652 613 613 query30 252 155 151 151 query31 986 770 753 753 query32 92 51 56 51 query33 732 336 318 318 query34 1010505 498 498 query35 911 764 783 764 query36 1159961 954 954 query37 145 85 86 85 query38 2952283428192819 query39 892 805 807 805 query40 207 115 118 115 query41 46 42 47 42 query42 123 102 99 99 query43 500 465 466 465 query44 1218721 717 717 query45 198 159 158 158 query46 1084766 748 748 query47 1859176217901762 query48 371 280 292 280 query49 831 416 409 409 query50 780 384 386 384 query51 6791656867056568 query52 99 92 91 91 query53 356 282 289 282 query54 884 452 449 449 query55 72 73 73 73 query56 290 273 281 273 query57 1112104310491043 query58 276 251 259 251 query59 2788265626062606 query60 335 288 302 288 query61 119 111 114 111 query62 790 642 661 642 query63 314 284 283 283 query64 9553229417671767 query65 3217310231043102 query66 766 340 333 333 query67 15459 15139 15090 15090 query68 4631553 547 547 query69 648 438 365 365 query70 1210115311711153 query71 472 287 284 284 query72 8073580460275804 query73 743 324 324 324 query74 6288566956945669 query75 4257268227062682 query76 3292919 986 919 query77 690 312 311 311 query78 9581921991469146 query79 6904515 518 515 query80 2827478 474 474 query81 592 227 236 227 query82 1054136 133 133 query83 309 164 169 164 query84 277 86 84 84 query85 2152311 294 294 query86 443 312 322 312 query87 3324309831383098 query88 5436238123912381 query89 506 377 370 370 query90 2009194 189 189 query91 127 96 101 96 query92 58 50 46 46 query93 5825527 519 519 query94 1410308 262 262 query95 409 313 307 307 query96 616 268 269 268 query97 3216299630482996 query98 235 200 196 196 query99 1531124712641247 Total cold run time: 296002 ms Total hot run time: 174786 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [Fix](multi-catalog) Fix some undefined behaviors. [doris]
morningman merged PR #38274: URL: https://github.com/apache/doris/pull/38274 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Row store) get the correct value of `row_store_page_size` after FE restarts [doris]
doris-robot commented on PR #38236: URL: https://github.com/apache/doris/pull/38236#issuecomment-2247185958 ClickBench: Total hot run time: 30.53 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 26e18ac623c12ffa189e17ac6897901ce8faa0c0, data reload: false query1 0.040.030.03 query2 0.070.040.04 query3 0.220.040.05 query4 1.680.080.08 query5 0.500.490.48 query6 1.130.730.73 query7 0.020.020.01 query8 0.050.040.05 query9 0.550.500.48 query10 0.540.540.54 query11 0.150.120.11 query12 0.140.130.12 query13 0.590.600.59 query14 0.750.770.78 query15 0.860.810.82 query16 0.360.360.37 query17 0.950.940.95 query18 0.230.230.22 query19 1.811.681.69 query20 0.010.010.01 query21 15.42 0.740.65 query22 4.666.961.99 query23 18.29 1.431.26 query24 2.090.220.22 query25 0.150.100.08 query26 0.290.210.21 query27 0.450.220.23 query28 13.32 1.021.00 query29 12.60 3.323.31 query30 0.250.050.06 query31 2.880.390.39 query32 3.260.490.46 query33 2.892.932.86 query34 16.96 4.374.33 query35 4.434.424.38 query36 0.650.500.46 query37 0.190.160.15 query38 0.160.150.15 query39 0.040.030.04 query40 0.140.120.13 query41 0.090.050.04 query42 0.060.050.05 query43 0.050.040.04 Total cold run time: 109.97 s Total hot run time: 30.53 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [Fix](multi-catalog) Fix some undefined behaviors. (#38274)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-2.1 by this push: new ef00dad680d [Fix](multi-catalog) Fix some undefined behaviors. (#38274) ef00dad680d is described below commit ef00dad680d1ec6da5e482c4c1a9503046b0ae2f Author: Qi Chen AuthorDate: Wed Jul 24 16:14:34 2024 +0800 [Fix](multi-catalog) Fix some undefined behaviors. (#38274) ## Proposed changes backport #37845 --- be/src/vec/exec/format/orc/vorc_reader.cpp | 4 ++-- .../exec/format/parquet/parquet_column_convert.h | 26 -- 2 files changed, 16 insertions(+), 14 deletions(-) diff --git a/be/src/vec/exec/format/orc/vorc_reader.cpp b/be/src/vec/exec/format/orc/vorc_reader.cpp index 16909f0023a..49430471c9d 100644 --- a/be/src/vec/exec/format/orc/vorc_reader.cpp +++ b/be/src/vec/exec/format/orc/vorc_reader.cpp @@ -1586,7 +1586,7 @@ Status OrcReader::get_next_block_impl(Block* block, size_t* read_rows, bool* eof _decimal_scale_params_index = 0; try { rr = _row_reader->nextBatch(*_batch, block); -if (rr == 0) { +if (rr == 0 || _batch->numElements == 0) { *eof = true; *read_rows = 0; return Status::OK(); @@ -1656,7 +1656,7 @@ Status OrcReader::get_next_block_impl(Block* block, size_t* read_rows, bool* eof _decimal_scale_params_index = 0; try { rr = _row_reader->nextBatch(*_batch, block); -if (rr == 0) { +if (rr == 0 || _batch->numElements == 0) { *eof = true; *read_rows = 0; return Status::OK(); diff --git a/be/src/vec/exec/format/parquet/parquet_column_convert.h b/be/src/vec/exec/format/parquet/parquet_column_convert.h index bc6bc232327..551bf7e14ed 100644 --- a/be/src/vec/exec/format/parquet/parquet_column_convert.h +++ b/be/src/vec/exec/format/parquet/parquet_column_convert.h @@ -408,18 +408,20 @@ class StringToDecimal : public PhysicalToLogicalConverter { // When Decimal in parquet is stored in byte arrays, binary and fixed, // the unscaled number must be encoded as two's complement using big-endian byte order. ValueCopyType value = 0; -memcpy(reinterpret_cast(&value), buf + offset[i - 1], len); -value = BitUtil::big_endian_to_host(value); -value = value >> ((sizeof(value) - len) * 8); -if constexpr (ScaleType == DecimalScaleParams::SCALE_UP) { -value *= scale_params.scale_factor; -} else if constexpr (ScaleType == DecimalScaleParams::SCALE_DOWN) { -value /= scale_params.scale_factor; -} else if constexpr (ScaleType == DecimalScaleParams::NO_SCALE) { -// do nothing -} else { -LOG(FATAL) << "__builtin_unreachable"; -__builtin_unreachable(); +if (len > 0) { +memcpy(reinterpret_cast(&value), buf + offset[i - 1], len); +value = BitUtil::big_endian_to_host(value); +value = value >> ((sizeof(value) - len) * 8); +if constexpr (ScaleType == DecimalScaleParams::SCALE_UP) { +value *= scale_params.scale_factor; +} else if constexpr (ScaleType == DecimalScaleParams::SCALE_DOWN) { +value /= scale_params.scale_factor; +} else if constexpr (ScaleType == DecimalScaleParams::NO_SCALE) { +// do nothing +} else { +LOG(FATAL) << "__builtin_unreachable"; +__builtin_unreachable(); +} } auto& v = reinterpret_cast(data[start_idx + i]); v = (DecimalType)value; - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](profile) add catalog info in profile [doris]
doris-robot commented on PR #38298: URL: https://github.com/apache/doris/pull/38298#issuecomment-2247190003 TPC-H: Total hot run time: 49948 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 1d785a78e1f0fb948f448ab50eac873f7152550e, data reload: false -- Round 1 -- q1 17641 436443534353 q2 2094156 144 144 q3 10469 191919501919 q4 10327 123613251236 q5 8404386238423842 q6 235 125 124 124 q7 2045162416081608 q8 9289270926752675 q9 10517 10255 10313 10255 q10 8624350434963496 q11 418 250 242 242 q12 472 307 304 304 q13 18334 398740073987 q14 356 324 320 320 q15 494 460 454 454 q16 683 569 568 568 q17 1127957 987 957 q18 7272705369836983 q19 1787164315971597 q20 538 322 311 311 q21 4523422740964096 q22 539 477 484 477 Total cold run time: 116188 ms Total hot run time: 49948 ms - Round 2, with runtime_filter_mode=off - q1 4427446044204420 q2 330 236 234 234 q3 4204417241394139 q4 2749274927452745 q5 7126708871177088 q6 241 122 118 118 q7 3163282028472820 q8 4368445744794457 q9 16784 16884 16706 16706 q10 4257422942984229 q11 751 709 689 689 q12 1011852 854 852 q13 7046373337183718 q14 448 424 431 424 q15 491 456 456 456 q16 734 687 681 681 q17 3860383437933793 q18 8710870287288702 q19 1721167916071607 q20 2363212520972097 q21 8548842185478421 q22 1041967 961 961 Total cold run time: 84373 ms Total hot run time: 79357 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [Bug](agg) fix collect_set function core dump without arena pool (#38234)
This is an automated email from the ASF dual-hosted git repository. lihaopeng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new c16ea419b27 [Bug](agg) fix collect_set function core dump without arena pool (#38234) c16ea419b27 is described below commit c16ea419b27095a37d54005815f441512ef5b782 Author: zhangstar333 <87313068+zhangstar...@users.noreply.github.com> AuthorDate: Wed Jul 24 16:18:07 2024 +0800 [Bug](agg) fix collect_set function core dump without arena pool (#38234) before the add_range_single_place pass nullptr as arena object, but collect_set function need save data in arena, so will core dump without arena pool. --- be/src/pipeline/exec/analytic_source_operator.cpp| 4 +--- .../data/nereids_function_p0/agg_function/group_unique_array.out | 9 + .../nereids_function_p0/agg_function/group_unique_array.groovy | 3 +++ 3 files changed, 13 insertions(+), 3 deletions(-) diff --git a/be/src/pipeline/exec/analytic_source_operator.cpp b/be/src/pipeline/exec/analytic_source_operator.cpp index a036481d727..406108fbc4f 100644 --- a/be/src/pipeline/exec/analytic_source_operator.cpp +++ b/be/src/pipeline/exec/analytic_source_operator.cpp @@ -279,8 +279,6 @@ void AnalyticLocalState::_destroy_agg_status() { } } -//now is execute for lead/lag row_number/rank/dense_rank/ntile functions -//sum min max count avg first_value last_value functions void AnalyticLocalState::_execute_for_win_func(int64_t partition_start, int64_t partition_end, int64_t frame_start, int64_t frame_end) { for (size_t i = 0; i < _agg_functions_size; ++i) { @@ -292,7 +290,7 @@ void AnalyticLocalState::_execute_for_win_func(int64_t partition_start, int64_t partition_start, partition_end, frame_start, frame_end, _fn_place_ptr + _parent->cast()._offsets_of_aggregate_states[i], -agg_columns.data(), nullptr); +agg_columns.data(), _agg_arena_pool.get()); // If the end is not greater than the start, the current window should be empty. _current_window_empty = diff --git a/regression-test/data/nereids_function_p0/agg_function/group_unique_array.out b/regression-test/data/nereids_function_p0/agg_function/group_unique_array.out index 036ac5ce57f..74c053e38f6 100644 --- a/regression-test/data/nereids_function_p0/agg_function/group_unique_array.out +++ b/regression-test/data/nereids_function_p0/agg_function/group_unique_array.out @@ -8,3 +8,12 @@ 3 ["2023-01-02"] ["hello"] 4 ["2023-01-02", "2023-01-03"]["sql"] +-- !3 -- +["doris", "world", "hello", "sql"] +["doris", "world", "hello", "sql"] +["doris", "world", "hello", "sql"] +["doris", "world", "hello", "sql"] +["doris", "world", "hello", "sql"] +["doris", "world", "hello", "sql"] +["doris", "world", "hello", "sql"] + diff --git a/regression-test/suites/nereids_function_p0/agg_function/group_unique_array.groovy b/regression-test/suites/nereids_function_p0/agg_function/group_unique_array.groovy index f17eadf73a5..f110f1a50c9 100644 --- a/regression-test/suites/nereids_function_p0/agg_function/group_unique_array.groovy +++ b/regression-test/suites/nereids_function_p0/agg_function/group_unique_array.groovy @@ -48,4 +48,7 @@ suite("group_unique_array") { qt_2 """ select k1,collect_set(k2),collect_set(k3,1) from test_group_unique_array_table group by k1 order by k1; """ +qt_3 """ +select collect_set(k3) over() from test_group_unique_array_table; +""" } \ No newline at end of file - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [spill](regression) add regression test cases for spill [doris]
doris-robot commented on PR #37319: URL: https://github.com/apache/doris/pull/37319#issuecomment-2247195231 ClickBench: Total hot run time: 31.44 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 8790744140c53fe64a3ff8dc9f6a4c533ddbc5e4, data reload: false query1 0.040.040.04 query2 0.080.040.04 query3 0.220.040.04 query4 1.690.070.07 query5 0.500.500.49 query6 1.130.730.73 query7 0.020.020.01 query8 0.050.050.04 query9 0.540.470.47 query10 0.530.550.54 query11 0.140.110.12 query12 0.140.110.12 query13 0.610.590.58 query14 0.750.770.77 query15 0.850.810.81 query16 0.350.350.37 query17 0.931.010.96 query18 0.220.210.21 query19 1.771.751.83 query20 0.010.000.00 query21 15.39 0.780.66 query22 3.577.202.84 query23 18.30 1.401.31 query24 2.090.250.23 query25 0.150.080.08 query26 0.310.210.21 query27 0.450.230.23 query28 13.24 1.011.00 query29 12.58 3.273.31 query30 0.260.050.05 query31 2.890.390.37 query32 3.280.470.46 query33 2.862.852.94 query34 17.00 4.324.42 query35 4.394.504.39 query36 0.660.460.47 query37 0.190.150.17 query38 0.150.150.15 query39 0.040.040.04 query40 0.150.120.13 query41 0.100.050.05 query42 0.060.040.05 query43 0.050.040.04 Total cold run time: 108.73 s Total hot run time: 31.44 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](agg) fix collect_set function core dump without arena pool [doris]
HappenLee merged PR #38234: URL: https://github.com/apache/doris/pull/38234 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Refactor](exec) remove the unless file and refactor the agg code [doris]
github-actions[bot] commented on PR #38193: URL: https://github.com/apache/doris/pull/38193#issuecomment-2247198464 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Refactor](exec) remove the unless file and refactor the agg code [doris]
github-actions[bot] commented on PR #38193: URL: https://github.com/apache/doris/pull/38193#issuecomment-2247198528 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](multi-catalog) Add memory tracker for orc reader/writer and arrow parquet writer。 [doris]
kaka11chen commented on PR #37234: URL: https://github.com/apache/doris/pull/37234#issuecomment-2247211729 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](backup) Log create restore job failed [doris]
github-actions[bot] commented on PR #38267: URL: https://github.com/apache/doris/pull/38267#issuecomment-2247212687 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](backup) Log create restore job failed [doris]
github-actions[bot] commented on PR #38267: URL: https://github.com/apache/doris/pull/38267#issuecomment-2247212611 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](profile) add catalog info in profile [doris]
doris-robot commented on PR #38298: URL: https://github.com/apache/doris/pull/38298#issuecomment-2247216329 TPC-DS: Total hot run time: 204763 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 1d785a78e1f0fb948f448ab50eac873f7152550e, data reload: false query1 936 421 385 385 query2 6554300429402940 query3 6914211 214 211 query4 20195 17999 18012 17999 query5 19740 651664766476 query6 274 212 225 212 query7 4153293 310 293 query8 420 462 439 439 query9 3152272226652665 query10 408 316 294 294 query11 11420 10674 10736 10674 query12 131 72 80 72 query13 5602701 697 697 query14 17579 13370 13814 13370 query15 372 254 230 230 query16 6489282 260 260 query17 17121478880 880 query18 2312415 411 411 query19 216 157 154 154 query20 82 83 83 83 query21 193 98 94 94 query22 5337501450655014 query23 32527 32105 31977 31977 query24 7039645264956452 query25 517 423 433 423 query26 534 167 162 162 query27 1870301 299 299 query28 6143238623182318 query29 2920272829452728 query30 244 174 171 171 query31 932 730 764 730 query32 70 63 61 61 query33 398 247 253 247 query34 849 467 491 467 query35 1093920 894 894 query36 1333103911061039 query37 89 59 62 59 query38 3043293629142914 query39 1365132613041304 query40 203 99 92 92 query41 49 43 45 43 query42 91 81 86 81 query43 734 767 837 767 query44 1126715 711 711 query45 247 233 226 226 query46 1239968 985 968 query47 2262183716891689 query48 1028703 731 703 query49 636 372 365 365 query50 847 618 644 618 query51 4745469947604699 query52 93 79 82 79 query53 449 320 319 319 query54 2652243124642431 query55 86 75 81 75 query56 236 213 222 213 query57 1185122410761076 query58 223 210 206 206 query59 4229419342694193 query60 217 222 206 206 query61 95 91 95 91 query62 843 471 491 471 query63 489 348 351 348 query64 2584154715251525 query65 3626356135713561 query66 777 394 372 372 query67 17296 15616 15673 15616 query68 9099641 676 641 query69 591 352 369 352 query70 1635144415761444 query71 419 316 318 316 query72 345634633456 query73 742 331 322 322 query74 6260599157845784 query75 5489373936793679 query76 5473114512321145 query77 950 261 261 261 query78 12455 11720 11760 11720 query79 7008641 622 622 query80 989 403 401 401 query81 493 237 234 234 query82 1498104 99 99 query83 163 135 132 132 query84 253 69 72 69 query85 885 333 340 333 query86 336 290 300 290 query87 3302304230153015 query88 4868235323472347 query89 360 288 315 288 query90 1949214 226 214 query91 175 138 145 138 query92 58 54 54 54 query93 3680588 564 564 query94 707 210 217 210 query95 1106108110501050 query96 659 340 332 332 query97 6390633863466338 query98 200 178 167 167 query99 2980926 904 904 Total cold run time: 311282 ms Total hot run time: 204763 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [fix](nereids) refine row count estimation for mark join [doris]
morrySnow merged PR #38270: URL: https://github.com/apache/doris/pull/38270 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Bug](dry-run) make dryrun result set collect multi batch's QueryStatistics [doris]
BiteThet opened a new pull request, #38303: URL: https://github.com/apache/doris/pull/38303 ## Proposed changes make dryrun result set collect multi batch's QueryStatistics -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](dry-run) make dryrun result set collect multi batch's QueryStatistics [doris]
BiteThet commented on PR #38303: URL: https://github.com/apache/doris/pull/38303#issuecomment-2247218669 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org