Re: [PR] [fix](csv reader) fix core dump when parsing csv with enclose [doris]

2024-12-19 Thread via GitHub


sollhui commented on PR #45485:
URL: https://github.com/apache/doris/pull/45485#issuecomment-2553008866

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhance](nerieds) date_format, to_monday, last_day, to_date, year implement Monotonic [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #44789:
URL: https://github.com/apache/doris/pull/44789#issuecomment-2553009295

   
   
   ClickBench: Total hot run time: 33.2 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 29e6ca9e1fb09f56396cbfe4ddbc6e313322b31a, 
data reload: false
   
   query1   0.030.030.05
   query2   0.070.030.03
   query3   0.230.060.07
   query4   1.610.110.10
   query5   0.410.410.43
   query6   1.150.650.65
   query7   0.020.020.01
   query8   0.040.030.04
   query9   0.580.520.49
   query10  0.550.590.56
   query11  0.150.110.10
   query12  0.150.120.11
   query13  0.600.610.60
   query14  2.842.882.72
   query15  0.890.820.81
   query16  0.390.400.37
   query17  1.070.981.00
   query18  0.230.210.21
   query19  1.971.751.97
   query20  0.010.020.01
   query21  15.36   0.600.57
   query22  2.702.152.99
   query23  16.94   0.960.90
   query24  3.691.291.28
   query25  0.200.110.11
   query26  0.460.140.14
   query27  0.040.050.04
   query28  9.901.121.09
   query29  12.58   3.283.21
   query30  0.240.060.06
   query31  2.870.390.38
   query32  3.250.470.45
   query33  3.123.333.10
   query34  17.13   4.454.55
   query35  4.534.474.50
   query36  0.680.480.49
   query37  0.090.060.06
   query38  0.040.030.04
   query39  0.040.020.02
   query40  0.170.130.12
   query41  0.080.020.02
   query42  0.040.020.02
   query43  0.040.030.02
   Total cold run time: 107.18 s
   Total hot run time: 33.2 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [BugFix](StoragePolicy) fix modify partition's storage policy with different resource issue succeed with execption [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #35421:
URL: https://github.com/apache/doris/pull/35421#issuecomment-2553009194

   
   
   TPC-H: Total hot run time: 39731 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit a23fb90c8c9afdbc2c7ed0bd9608569c3f1ab9b6, 
data reload: false
   
   -- Round 1 --
   q1   17618   742772347234
   q2   2053181 186 181
   q3   10891   110312171103
   q4   10536   747 756 747
   q5   7576267326922673
   q6   233 148 151 148
   q7   995 613 614 613
   q8   9235181119191811
   q9   6770637464516374
   q10  6994226423202264
   q11  461 299 259 259
   q12  437 225 225 225
   q13  17810   292629642926
   q14  251 205 210 205
   q15  595 512 482 482
   q16  663 597 587 587
   q17  973 585 552 552
   q18  7208662766256625
   q19  1336100410621004
   q20  450 187 179 179
   q21  4009339832293229
   q22  376 318 310 310
   Total cold run time: 107470 ms
   Total hot run time: 39731 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7258719572157195
   q2   326 235 225 225
   q3   2878281329492813
   q4   2094187118911871
   q5   5684565056385638
   q6   218 149 143 143
   q7   2242181118241811
   q8   3424355834693469
   q9   8779910589138913
   q10  3582354035453540
   q11  604 519 525 519
   q12  814 610 593 593
   q13  12173   315131603151
   q14  319 281 275 275
   q15  559 500 516 500
   q16  716 671 637 637
   q17  1886162815991599
   q18  8202777176417641
   q19  1757156015311531
   q20  2075187118751871
   q21  5646553054365436
   q22  648 599 578 578
   Total cold run time: 71884 ms
   Total hot run time: 59949 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feat](nereids) implement showConvertLscCommand in nereids [doris]

2024-12-19 Thread via GitHub


vinlee19 commented on PR #45631:
URL: https://github.com/apache/doris/pull/45631#issuecomment-2553009920

   run external


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](sql cache) fix prepare statement with sql cache throw NullPointerException [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45640:
URL: https://github.com/apache/doris/pull/45640#issuecomment-2553013561

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](sql cache) fix prepare statement with sql cache throw NullPointerException [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45640:
URL: https://github.com/apache/doris/pull/45640#issuecomment-2553013417

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feat](nereids) implement showConvertLscCommand in nereids [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45631:
URL: https://github.com/apache/doris/pull/45631#issuecomment-2553013961

   
   
   ClickBench: Total hot run time: 32.3 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 6d4f834ad3e121c76f29963b3d873499d20f8578, 
data reload: false
   
   query1   0.030.030.03
   query2   0.070.030.04
   query3   0.240.070.07
   query4   1.610.110.10
   query5   0.420.420.44
   query6   1.160.640.64
   query7   0.020.020.02
   query8   0.040.040.03
   query9   0.590.510.50
   query10  0.570.600.57
   query11  0.130.100.10
   query12  0.140.120.11
   query13  0.600.600.60
   query14  2.722.872.87
   query15  0.890.830.84
   query16  0.390.390.39
   query17  1.071.051.04
   query18  0.230.220.21
   query19  1.951.882.02
   query20  0.020.010.01
   query21  15.37   0.600.58
   query22  3.092.721.26
   query23  16.94   0.960.76
   query24  3.191.141.06
   query25  0.220.190.08
   query26  0.440.140.14
   query27  0.050.050.05
   query28  10.38   1.111.07
   query29  12.60   3.263.25
   query30  0.240.070.06
   query31  3.090.390.38
   query32  3.640.450.46
   query33  3.083.263.08
   query34  16.98   4.414.51
   query35  4.444.434.44
   query36  0.670.480.48
   query37  0.090.060.07
   query38  0.050.040.03
   query39  0.030.020.03
   query40  0.160.130.13
   query41  0.080.030.02
   query42  0.030.020.02
   query43  0.030.040.03
   Total cold run time: 107.78 s
   Total hot run time: 32.3 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](csv reader) fix core dump when parsing csv with enclose [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45485:
URL: https://github.com/apache/doris/pull/45485#issuecomment-2553015356

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](Job)Fix redundant job scheduling by preventing same state transitions (e.g., RUNNING → RUNNING) [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45495:
URL: https://github.com/apache/doris/pull/45495#issuecomment-2553016671

   
   
   ClickBench: Total hot run time: 33.09 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 80363dcab4af779a0f4eb82b080ef941cc2ecd7c, 
data reload: false
   
   query1   0.030.030.03
   query2   0.080.030.03
   query3   0.230.070.07
   query4   1.600.100.11
   query5   0.440.400.41
   query6   1.140.660.65
   query7   0.020.010.01
   query8   0.040.030.04
   query9   0.590.500.51
   query10  0.560.590.57
   query11  0.150.100.11
   query12  0.140.110.10
   query13  0.610.610.60
   query14  2.742.782.79
   query15  0.900.820.84
   query16  0.370.390.38
   query17  1.021.001.06
   query18  0.230.210.20
   query19  1.831.802.00
   query20  0.010.010.02
   query21  15.35   0.590.58
   query22  2.992.592.38
   query23  17.02   0.930.87
   query24  3.061.350.80
   query25  0.380.140.10
   query26  0.360.140.13
   query27  0.040.040.04
   query28  10.48   1.101.07
   query29  12.58   3.253.26
   query30  0.250.060.06
   query31  2.830.400.39
   query32  3.240.460.46
   query33  3.113.033.09
   query34  16.88   4.494.51
   query35  4.524.594.47
   query36  0.660.480.48
   query37  0.100.060.06
   query38  0.040.040.04
   query39  0.030.020.02
   query40  0.150.140.12
   query41  0.070.020.02
   query42  0.030.020.02
   query43  0.030.030.04
   Total cold run time: 106.93 s
   Total hot run time: 33.09 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feat](nereids) when check shape failed, print all plan with memo [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45627:
URL: https://github.com/apache/doris/pull/45627#issuecomment-2553022783

   
   
   TPC-DS: Total hot run time: 191595 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 4dc24cddb00bfc0146974133e473528d337e9091, 
data reload: false
   
   query1   975 383 386 383
   query2   6516238522672267
   query3   6716219 215 215
   query4   33910   23417   23456   23417
   query5   4362494 458 458
   query6   275 195 200 195
   query7   4633313 301 301
   query8   307 251 230 230
   query9   9650272027132713
   query10  479 255 253 253
   query11  17959   15210   15261   15210
   query12  158 104 112 104
   query13  1664439 407 407
   query14  10700   709777797097
   query15  294 182 182 182
   query16  8299407 381 381
   query17  1757628 556 556
   query18  2050288 295 288
   query19  379 161 148 148
   query20  128 110 114 110
   query21  210 103 101 101
   query22  4557432143114311
   query23  34496   35325   34697   34697
   query24  11329   246424542454
   query25  533 379 390 379
   query26  1161152 153 152
   query27  2678336 335 335
   query28  7720241024222410
   query29  651 399 403 399
   query30  312 150 148 148
   query31  1009813 819 813
   query32  102 60  61  60
   query33  770 303 290 290
   query34  938 522 527 522
   query35  860 746 744 744
   query36  1082957 944 944
   query37  121 73  78  73
   query38  4179430542214221
   query39  1504144814151415
   query40  282 106 107 106
   query41  51  45  48  45
   query42  113 106 105 105
   query43  535 495 484 484
   query44  1243837 808 808
   query45  183 177 172 172
   query46  1175725 715 715
   query47  1953184318421842
   query48  429 318 309 309
   query49  1265407 404 404
   query50  813 388 400 388
   query51  7077713070007000
   query52  104 92  93  92
   query53  261 179 185 179
   query54  1061412 413 412
   query55  78  77  90  77
   query56  268 257 236 236
   query57  1275110711171107
   query58  238 224 253 224
   query59  3131292929772929
   query60  285 257 262 257
   query61  110 106 103 103
   query62  897 671 680 671
   query63  217 191 191 191
   query64  5040649 643 643
   query65  3240322232123212
   query66  1229320 335 320
   query67  15547   15464   15562   15464
   query68  5628565 558 558
   query69  433 263 261 261
   query70  1211116411481148
   query71  355 259 258 258
   query72  6170412540294029
   query73  789 356 356 356
   query74  10401   891288458845
   query75  3410264127002641
   query76  3603110510531053
   query77  521 283 301 283
   query78  10160   942293529352
   query79  1434603 606 603
   query80  856 442 437 437
   query81  540 234 237 234
   query82  715 123 125 123
   query83  267 144 149 144
   query84  238 122 70  70
   query85  1659306 314 306
   query86  480 303 307 303
   query87  4650432144944321
   query88  4138221221922192
   query89  404 296 295 295
   query90  2109191 188 188
   query91  142 103 110 103
   query92  68  53  54  53
   query93  1923549 542 542
   query94  1029284 298 284
   query95  373 253 250 250
   query96  621 276 281 276
   query97  2871271726982698
   query98  219 194 197 194
   query99  1550131713171317
   Total cold run time: 302490 ms
   Total hot run time: 191595 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t

Re: [PR] [feat](nereids) when check shape failed, print all plan with memo [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45627:
URL: https://github.com/apache/doris/pull/45627#issuecomment-2553032762

   
   
   ClickBench: Total hot run time: 33.38 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 4dc24cddb00bfc0146974133e473528d337e9091, 
data reload: false
   
   query1   0.040.040.03
   query2   0.060.030.04
   query3   0.220.070.07
   query4   1.610.100.10
   query5   0.440.400.43
   query6   1.190.650.65
   query7   0.020.020.01
   query8   0.030.030.03
   query9   0.560.530.49
   query10  0.550.590.55
   query11  0.140.120.11
   query12  0.140.110.12
   query13  0.620.600.61
   query14  2.892.772.87
   query15  0.900.830.82
   query16  0.370.370.40
   query17  1.051.061.04
   query18  0.220.210.21
   query19  1.921.861.98
   query20  0.010.010.01
   query21  15.36   0.590.60
   query22  2.392.922.10
   query23  17.04   0.960.89
   query24  2.951.641.19
   query25  0.290.340.11
   query26  0.410.140.13
   query27  0.050.040.05
   query28  9.991.111.07
   query29  12.60   3.253.25
   query30  0.250.070.07
   query31  2.850.380.38
   query32  3.250.460.46
   query33  3.173.083.16
   query34  17.20   4.544.50
   query35  4.444.464.48
   query36  0.670.480.49
   query37  0.100.060.06
   query38  0.050.040.04
   query39  0.030.020.02
   query40  0.160.120.12
   query41  0.080.030.03
   query42  0.030.020.02
   query43  0.040.030.03
   Total cold run time: 106.38 s
   Total hot run time: 33.38 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [branch-2.1](fix) fix incorrect result of hash join with const column [doris]

2024-12-19 Thread via GitHub


yiguolei commented on PR #45630:
URL: https://github.com/apache/doris/pull/45630#issuecomment-2553053745

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch branch-2.1 updated: branch-2.1: [fix](tvf) Tvf supports to parse the enclose character in csv files #45407 (#45570)

2024-12-19 Thread yiguolei
This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-2.1 by this push:
 new d6c629d293a branch-2.1: [fix](tvf) Tvf supports to parse the enclose 
character in csv files #45407 (#45570)
d6c629d293a is described below

commit d6c629d293a302d5b6f887188e2f4f88c5ed7d08
Author: github-actions[bot] 
<41898282+github-actions[bot]@users.noreply.github.com>
AuthorDate: Thu Dec 19 16:23:44 2024 +0800

branch-2.1: [fix](tvf) Tvf supports to parse the enclose character in csv 
files #45407 (#45570)

Cherry-picked from #45407

Co-authored-by: Tiewei Fang 
---
 be/src/vec/exec/format/csv/csv_reader.cpp  |  6 +-
 .../doris/common/util/FileFormatConstants.java |  1 +
 .../ExternalFileTableValuedFunction.java   | 15 +
 .../data/external_table_p0/tvf/enclose_csv.csv |  6 ++
 .../tvf/test_local_tvf_enclose.out | 15 +
 .../tvf/test_local_tvf_enclose.groovy  | 72 ++
 6 files changed, 114 insertions(+), 1 deletion(-)

diff --git a/be/src/vec/exec/format/csv/csv_reader.cpp 
b/be/src/vec/exec/format/csv/csv_reader.cpp
index 41ad9ad6019..1fc3bbad294 100644
--- a/be/src/vec/exec/format/csv/csv_reader.cpp
+++ b/be/src/vec/exec/format/csv/csv_reader.cpp
@@ -925,9 +925,13 @@ Status CsvReader::_prepare_parse(size_t* read_line, bool* 
is_parse_name) {
 _trim_tailing_spaces, _trim_double_quotes, _value_separator,
 _value_separator_length);
 } else {
+// If we pass `_file_slot_descs.size() - 1` to 
EncloseCsvTextFieldSplitter, it will cause BE core dump
+// because currently _file_slot_descs is an empty vector.
+// The _file_slot_descs.size() is only used to reserve space,
+// so it's ok to pass 0 to EncloseCsvLineReaderContext
 text_line_reader_ctx = std::make_shared(
 _line_delimiter, _line_delimiter_length, _value_separator, 
_value_separator_length,
-_file_slot_descs.size() - 1, _enclose, _escape, _keep_cr);
+0, _enclose, _escape, _keep_cr);
 _fields_splitter = std::make_unique(
 _trim_tailing_spaces, false,
 
std::static_pointer_cast(text_line_reader_ctx),
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/common/util/FileFormatConstants.java
 
b/fe/fe-core/src/main/java/org/apache/doris/common/util/FileFormatConstants.java
index e86c06fcb61..b4f313c7822 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/common/util/FileFormatConstants.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/common/util/FileFormatConstants.java
@@ -49,6 +49,7 @@ public class FileFormatConstants {
 public static final String PROP_CSV_SCHEMA = "csv_schema";
 public static final String PROP_COMPRESS_TYPE = "compress_type";
 public static final String PROP_PATH_PARTITION_KEYS = 
"path_partition_keys";
+public static final String PROP_ENCLOSE = "enclose";
 
 // decimal(p,s)
 public static final Pattern DECIMAL_TYPE_PATTERN = 
Pattern.compile("decimal\\((\\d+),(\\d+)\\)");
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/tablefunction/ExternalFileTableValuedFunction.java
 
b/fe/fe-core/src/main/java/org/apache/doris/tablefunction/ExternalFileTableValuedFunction.java
index 967838d34f0..1f65921832b 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/tablefunction/ExternalFileTableValuedFunction.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/tablefunction/ExternalFileTableValuedFunction.java
@@ -119,6 +119,7 @@ public abstract class ExternalFileTableValuedFunction 
extends TableValuedFunctio
 private TTextSerdeType textSerdeType = TTextSerdeType.JSON_TEXT_SERDE;
 private String columnSeparator = 
FileFormatConstants.DEFAULT_COLUMN_SEPARATOR;
 private String lineDelimiter = FileFormatConstants.DEFAULT_LINE_DELIMITER;
+private byte enclose = 0;
 private String jsonRoot = "";
 private String jsonPaths = "";
 private boolean stripOuterArray;
@@ -230,6 +231,17 @@ public abstract class ExternalFileTableValuedFunction 
extends TableValuedFunctio
 }
 lineDelimiter = Separator.convertSeparator(lineDelimiter);
 
+String enclosedString = getOrDefaultAndRemove(copiedProps, 
FileFormatConstants.PROP_ENCLOSE, "");
+if (!Strings.isNullOrEmpty(enclosedString)) {
+if (enclosedString.length() > 1) {
+throw new AnalysisException("enclose should not be longer than 
one byte.");
+}
+enclose = (byte) enclosedString.charAt(0);
+if (enclose == 0) {
+throw new AnalysisException("enclose should not be byte [0].");
+}
+}
+
 jsonRoot = getOrDefaultAndRemove(copiedProps, 
FileFormatConstants.PROP_JSON_ROOT, "");
 jsonPaths = get

Re: [PR] branch-2.1: [fix](tvf) Tvf supports to parse the enclose character in csv files #45407 [doris]

2024-12-19 Thread via GitHub


yiguolei merged PR #45570:
URL: https://github.com/apache/doris/pull/45570


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch auto-pick-45407-branch-2.1 deleted (was e10128b2b16)

2024-12-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch auto-pick-45407-branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git


 was e10128b2b16 [fix](tvf) Tvf supports to parse the enclose character in 
csv files (#45407)

The revisions that were on this branch are still contained in
other references; therefore, this change does not discard any commits
from the repository.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] branch-2.1: [fix](txn insert) txn insert show error url #45254 [doris]

2024-12-19 Thread via GitHub


yiguolei closed pull request #45318: branch-2.1: [fix](txn insert) txn insert 
show error url #45254
URL: https://github.com/apache/doris/pull/45318


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Improvement](scheduler) Use a separate eager queue to execute cancel… [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45614:
URL: https://github.com/apache/doris/pull/45614#issuecomment-2553031894

   TeamCity be ut coverage result:
Function Coverage: 38.87% (10133/26067) 
Line Coverage: 29.78% (85171/285998)
Region Coverage: 28.83% (43691/151545)
Branch Coverage: 25.37% (22195/87496)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/42532ad7b23925dc5c5ec8051beca939e2ce4b0d_42532ad7b23925dc5c5ec8051beca939e2ce4b0d/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [BugFix](StoragePolicy) fix modify partition's storage policy with different resource issue succeed with execption [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #35421:
URL: https://github.com/apache/doris/pull/35421#issuecomment-2553030455

   
   
   TPC-DS: Total hot run time: 197098 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit a23fb90c8c9afdbc2c7ed0bd9608569c3f1ab9b6, 
data reload: false
   
   query1   1294973 946 946
   query2   6444233222832283
   query3   8   495948314831
   query4   33504   23472   23649   23472
   query5   4748484 478 478
   query6   300 194 180 180
   query7   4016310 309 309
   query8   302 246 238 238
   query9   9530272127102710
   query10  497 257 257 257
   query11  17924   15230   15141   15141
   query12  162 106 105 105
   query13  1576428 423 423
   query14  10784   694479976944
   query15  264 199 213 199
   query16  8064476 456 456
   query17  1734623 578 578
   query18  2048323 321 321
   query19  354 169 156 156
   query20  122 114 114 114
   query21  203 103 110 103
   query22  4975474147234723
   query23  34838   33643   34683   33643
   query24  10476   248625072486
   query25  644 395 391 391
   query26  1197156 160 156
   query27  2639335 333 333
   query28  7594251324502450
   query29  837 425 417 417
   query30  230 149 147 147
   query31  1054850 859 850
   query32  97  59  54  54
   query33  780 301 315 301
   query34  977 534 533 533
   query35  889 750 760 750
   query36  1121958 971 958
   query37  152 79  84  79
   query38  4140446642234223
   query39  1506146314701463
   query40  216 116 116 116
   query41  48  46  46  46
   query42  119 114 110 110
   query43  547 483 532 483
   query44  1250830 841 830
   query45  195 169 174 169
   query46  1175732 767 732
   query47  2061190919281909
   query48  435 326 332 326
   query49  879 405 394 394
   query50  824 434 397 397
   query51  7369721772127212
   query52  105 95  95  95
   query53  270 189 191 189
   query54  1121434 436 434
   query55  86  86  83  83
   query56  284 254 256 254
   query57  1278118611761176
   query58  236 233 230 230
   query59  3390315430193019
   query60  272 249 245 245
   query61  105 103 124 103
   query62  850 747 652 652
   query63  226 198 191 191
   query64  3993698 643 643
   query65  3231323031873187
   query66  709 294 311 294
   query67  15978   15664   15646   15646
   query68  5401560 541 541
   query69  481 266 263 263
   query70  1227115210721072
   query71  455 255 264 255
   query72  7045413640734073
   query73  800 366 359 359
   query74  10155   900988328832
   query75  3582266826812668
   query76  3626108811481088
   query77  605 271 277 271
   query78  10393   935294459352
   query79  2558616 622 616
   query80  1486432 447 432
   query81  542 237 235 235
   query82  1330119 129 119
   query83  263 154 143 143
   query84  284 75  80  75
   query85  1513328 309 309
   query86  479 299 304 299
   query87  4498446043664366
   query88  4075222621962196
   query89  431 288 302 288
   query90  2041191 201 191
   query91  153 140 103 103
   query92  71  50  53  50
   query93  2722545 541 541
   query94  749 294 283 283
   query95  358 254 256 254
   query96  630 278 279 278
   query97  2899265827282658
   query98  225 200 198 198
   query99  1636132413261324
   Total cold run time: 309099 ms
   Total hot run time: 197098 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go 

Re: [PR] [fix](csv reader) fix core dump when parsing csv with enclose [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45485:
URL: https://github.com/apache/doris/pull/45485#issuecomment-2553048513

   
   
   TPC-H: Total hot run time: 39972 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 0dd081551b2fc92fcc4ca3bb4049a591ec1001db, 
data reload: false
   
   -- Round 1 --
   q1   17610   745372807280
   q2   2048176 180 176
   q3   10540   112312101123
   q4   10213   684 712 684
   q5   7599268027012680
   q6   243 148 146 146
   q7   995 629 628 628
   q8   9257187519581875
   q9   6620644963956395
   q10  6955231723432317
   q11  463 262 264 262
   q12  429 227 225 225
   q13  17744   301029952995
   q14  251 222 217 217
   q15  559 504 494 494
   q16  624 589 597 589
   q17  978 552 586 552
   q18  7355667867466678
   q19  13461076959 959
   q20  483 184 189 184
   q21  3993320832863208
   q22  376 310 305 305
   Total cold run time: 106681 ms
   Total hot run time: 39972 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7218720272547202
   q2   326 226 233 226
   q3   2918277729952777
   q4   2065176818021768
   q5   5633567156355635
   q6   216 135 137 135
   q7   2221180218101802
   q8   3355358435103510
   q9   8941894089898940
   q10  3605360135163516
   q11  600 504 501 501
   q12  803 609 605 605
   q13  12164   310830903090
   q14  299 286 279 279
   q15  545 507 529 507
   q16  680 664 633 633
   q17  1846158115661566
   q18  7846740674437406
   q19  1675150814581458
   q20  2021182218441822
   q21  5494533753825337
   q22  641 563 566 563
   Total cold run time: 71112 ms
   Total hot run time: 59278 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](tools) Support automatic profile fetching under the tools directory. [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45612:
URL: https://github.com/apache/doris/pull/45612#issuecomment-2553038517

    `sh-checker report`
   
   To get the full details, please check in the 
[job]("https://github.com/apache/doris/actions/runs/12408788429";) output.
   
   
   shellcheck errors
   
   ```
   
   'shellcheck ' returned error 1 finding the following syntactical issues:
   
   --
   
   In gensrc/script/gen_build_version.sh line 38:
   if [[ ${build_version_hotfix} > 0 ]]; then
 ^-- SC2071 (error): > is for string 
comparisons. Use -gt instead.
   
   
   In gensrc/script/gen_build_version.sh line 228:
   if [[ ${build_version_hotfix} > 0 ]]; then
 ^-- SC2071 (error): > is for string 
comparisons. Use -gt instead.
   
   
   In tools/ssb-tools/bin/create-ssb-tables.sh line 103:
   echo "DB: ${DB}"
 ^---^ SC2154 (warning): DB is referenced but not assigned.
   
   
   In tools/ssb-tools/bin/load-ssb-data.sh line 115:
   mysql -h"${FE_HOST}" -u"${USER}" -P"${FE_QUERY_PORT}" -D"${DB}" -e "$@"
^---^ SC2154 
(warning): DB is referenced but not assigned.
   
   
   In tools/ssb-tools/bin/run-ssb-flat-queries.sh line 97:
   local query_id=$(curl -s -X GET -u "${USER}:${PASSWORD}"  
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/lastqueryid)
 ^--^ SC2155 (warning): Declare and assign separately to avoid 
masking return values.
   
   
   In tools/ssb-tools/bin/run-ssb-flat-queries.sh line 109:
   IFS="|" read -r name queryid <<< "$entry"
 ^^ SC2250 (style): Prefer 
putting braces around variable references even when not strictly required.
   
   Did you mean: 
   IFS="|" read -r name queryid <<< "${entry}"
   
   
   In tools/ssb-tools/bin/run-ssb-flat-queries.sh line 111:
   curl -s -X GET -u "${USER}:${PASSWORD}" 
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/text?query_id=${queryid} >> 
"${PROFILE_DIR}/${profile_name}"

   ^^ SC2086 (info): Double quote to 
prevent globbing and word splitting.
   
   Did you mean: 
   curl -s -X GET -u "${USER}:${PASSWORD}" 
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/text?query_id="${queryid}"; >> 
"${PROFILE_DIR}/${profile_name}"
   
   
   In tools/ssb-tools/bin/run-ssb-queries.sh line 97:
   local query_id=$(curl -s -X GET -u "${USER}:${PASSWORD}"  
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/lastqueryid)
 ^--^ SC2155 (warning): Declare and assign separately to avoid 
masking return values.
   
   
   In tools/ssb-tools/bin/run-ssb-queries.sh line 109:
   IFS="|" read -r name queryid <<< "$entry"
 ^^ SC2250 (style): Prefer 
putting braces around variable references even when not strictly required.
   
   Did you mean: 
   IFS="|" read -r name queryid <<< "${entry}"
   
   
   In tools/ssb-tools/bin/run-ssb-queries.sh line 111:
   curl -s -X GET -u "${USER}:${PASSWORD}" 
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/text?query_id=${queryid} >> 
"${PROFILE_DIR}/${profile_name}"

   ^^ SC2086 (info): Double quote to 
prevent globbing and word splitting.
   
   Did you mean: 
   curl -s -X GET -u "${USER}:${PASSWORD}" 
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/text?query_id="${queryid}"; >> 
"${PROFILE_DIR}/${profile_name}"
   
   
   In tools/tpcds-tools/bin/create-tpcds-tables.sh line 103:
   echo "DB: ${DB}"
 ^---^ SC2154 (warning): DB is referenced but not assigned.
   
   
   In tools/tpcds-tools/bin/load-tpcds-data.sh line 120:
   echo "DB: ${DB}"
 ^---^ SC2154 (warning): DB is referenced but not assigned.
   
   
   In tools/tpcds-tools/bin/run-tpcds-queries.sh line 118:
   local query_id=$(curl -s -X GET -u "${USER}:${PASSWORD}"  
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/lastqueryid)
 ^--^ SC2155 (warning): Declare and assign separately to avoid 
masking return values.
   
   
   In tools/tpcds-tools/bin/run-tpcds-queries.sh line 130:
   IFS="|" read -r name queryid <<< "$entry"
 ^^ SC2250 (style): Prefer 
putting braces around variable references even when not strictly required.
   
   Did you mean: 
   IFS="|" read -r name queryid <<< "${entry}"
   
   
   In tools/tpcds-tools/bin/run-tpcds-queries.sh line 132:
   curl -s -X GET -u "${USER}:${PASSWORD}" 
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/text?query_id=${queryid} >> 
"${PROFILE_DIR}/${profile_name}"
 

Re: [PR] [BugFix](StoragePolicy) fix modify partition's storage policy with different resource issue succeed with execption [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #35421:
URL: https://github.com/apache/doris/pull/35421#issuecomment-2553041022

   
   
   ClickBench: Total hot run time: 33.3 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit a23fb90c8c9afdbc2c7ed0bd9608569c3f1ab9b6, 
data reload: false
   
   query1   0.030.030.03
   query2   0.070.030.04
   query3   0.240.070.07
   query4   1.610.100.10
   query5   0.430.390.40
   query6   1.180.650.65
   query7   0.020.020.02
   query8   0.040.030.03
   query9   0.580.520.50
   query10  0.520.580.55
   query11  0.140.100.11
   query12  0.140.110.12
   query13  0.610.600.59
   query14  2.842.852.89
   query15  0.890.820.83
   query16  0.400.380.39
   query17  1.081.061.04
   query18  0.230.220.21
   query19  1.971.772.03
   query20  0.010.010.01
   query21  15.36   0.620.58
   query22  2.902.362.13
   query23  16.89   0.910.92
   query24  3.111.551.13
   query25  0.150.170.10
   query26  0.580.130.14
   query27  0.050.050.04
   query28  9.921.111.07
   query29  12.58   3.293.23
   query30  0.240.060.06
   query31  2.850.390.38
   query32  3.240.470.47
   query33  3.073.113.24
   query34  16.96   4.464.50
   query35  4.504.474.53
   query36  0.660.500.47
   query37  0.090.060.06
   query38  0.050.040.03
   query39  0.030.030.02
   query40  0.170.130.13
   query41  0.080.030.02
   query42  0.030.020.02
   query43  0.030.030.03
   Total cold run time: 106.57 s
   Total hot run time: 33.3 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](Job)Fix redundant job scheduling by preventing same state transitions (e.g., RUNNING → RUNNING) [doris]

2024-12-19 Thread via GitHub


CalvinKirs commented on PR #45495:
URL: https://github.com/apache/doris/pull/45495#issuecomment-2553044848

   run cloud_p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](Nereids) lock table in ascending order of table IDs [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45045:
URL: https://github.com/apache/doris/pull/45045#issuecomment-2553044415

   
   
   TPC-H: Total hot run time: 39749 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit ce4e8c61950260a5a1e863c4152d0e5fca6f9055, 
data reload: false
   
   -- Round 1 --
   q1   17604   739073097309
   q2   2048177 163 163
   q3   10874   109112091091
   q4   10554   737 685 685
   q5   7559267526492649
   q6   237 153 146 146
   q7   998 627 608 608
   q8   9248186119351861
   q9   6650653063866386
   q10  7021229522912291
   q11  482 262 259 259
   q12  465 225 228 225
   q13  17780   290029482900
   q14  258 210 216 210
   q15  575 509 498 498
   q16  675 587 610 587
   q17  1003611 539 539
   q18  7334660667476606
   q19  1359104410421042
   q20  459 185 180 180
   q21  4075333532013201
   q22  379 313 313 313
   Total cold run time: 107637 ms
   Total hot run time: 39749 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7301726572277227
   q2   329 237 229 229
   q3   2862282030062820
   q4   2096183218611832
   q5   5644565856875658
   q6   227 146 151 146
   q7   2219182418121812
   q8   3417353434823482
   q9   8915901288968896
   q10  3620354635633546
   q11  606 529 536 529
   q12  855 617 612 612
   q13  11236   313730943094
   q14  323 278 270 270
   q15  569 505 518 505
   q16  685 663 659 659
   q17  1843162816431628
   q18  8307792177187718
   q19  1744155416071554
   q20  2098186118761861
   q21  5570553154975497
   q22  643 573 572 572
   Total cold run time: 71109 ms
   Total hot run time: 60147 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix] Fix title of small file mananger [doris-website]

2024-12-19 Thread via GitHub


KassieZ opened a new pull request, #1545:
URL: https://github.com/apache/doris-website/pull/1545

   ## Versions 
   
   - [ ] dev
   - [ ] 3.0
   - [ ] 2.1
   - [ ] 2.0
   
   ## Languages
   
   - [ ] Chinese
   - [ ] English
   
   ## Docs Checklist
   
   - [ ] Checked by AI
   - [ ] Test Cases Built
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](tools) Support automatic profile fetching under the tools directory. [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45612:
URL: https://github.com/apache/doris/pull/45612#issuecomment-2553063885

    `sh-checker report`
   
   To get the full details, please check in the 
[job]("https://github.com/apache/doris/actions/runs/12408964676";) output.
   
   
   shellcheck errors
   
   ```
   
   'shellcheck ' returned error 1 finding the following syntactical issues:
   
   --
   
   In gensrc/script/gen_build_version.sh line 38:
   if [[ ${build_version_hotfix} > 0 ]]; then
 ^-- SC2071 (error): > is for string 
comparisons. Use -gt instead.
   
   
   In gensrc/script/gen_build_version.sh line 228:
   if [[ ${build_version_hotfix} > 0 ]]; then
 ^-- SC2071 (error): > is for string 
comparisons. Use -gt instead.
   
   
   In tools/ssb-tools/bin/create-ssb-tables.sh line 103:
   echo "DB: ${DB}"
 ^---^ SC2154 (warning): DB is referenced but not assigned.
   
   
   In tools/ssb-tools/bin/load-ssb-data.sh line 115:
   mysql -h"${FE_HOST}" -u"${USER}" -P"${FE_QUERY_PORT}" -D"${DB}" -e "$@"
^---^ SC2154 
(warning): DB is referenced but not assigned.
   
   
   In tools/ssb-tools/bin/run-ssb-flat-queries.sh line 97:
   local query_id=$(curl -s -X GET -u "${USER}:${PASSWORD}" 
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/lastqueryid)
 ^--^ SC2155 (warning): Declare and assign separately to avoid 
masking return values.
   
   
   In tools/ssb-tools/bin/run-ssb-flat-queries.sh line 109:
   IFS="|" read -r name queryid <<<"$entry"
^^ SC2250 (style): Prefer 
putting braces around variable references even when not strictly required.
   
   Did you mean: 
   IFS="|" read -r name queryid <<<"${entry}"
   
   
   In tools/ssb-tools/bin/run-ssb-flat-queries.sh line 111:
   curl -s -X GET -u "${USER}:${PASSWORD}" 
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/text?query_id=${queryid} 
>>"${PROFILE_DIR}/${profile_name}"

   ^^ SC2086 (info): Double quote to 
prevent globbing and word splitting.
   
   Did you mean: 
   curl -s -X GET -u "${USER}:${PASSWORD}" 
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/text?query_id="${queryid}"; 
>>"${PROFILE_DIR}/${profile_name}"
   
   
   In tools/ssb-tools/bin/run-ssb-queries.sh line 97:
   local query_id=$(curl -s -X GET -u "${USER}:${PASSWORD}" 
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/lastqueryid)
 ^--^ SC2155 (warning): Declare and assign separately to avoid 
masking return values.
   
   
   In tools/ssb-tools/bin/run-ssb-queries.sh line 109:
   IFS="|" read -r name queryid <<<"$entry"
^^ SC2250 (style): Prefer 
putting braces around variable references even when not strictly required.
   
   Did you mean: 
   IFS="|" read -r name queryid <<<"${entry}"
   
   
   In tools/ssb-tools/bin/run-ssb-queries.sh line 111:
   curl -s -X GET -u "${USER}:${PASSWORD}" 
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/text?query_id=${queryid} 
>>"${PROFILE_DIR}/${profile_name}"

   ^^ SC2086 (info): Double quote to 
prevent globbing and word splitting.
   
   Did you mean: 
   curl -s -X GET -u "${USER}:${PASSWORD}" 
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/text?query_id="${queryid}"; 
>>"${PROFILE_DIR}/${profile_name}"
   
   
   In tools/tpcds-tools/bin/create-tpcds-tables.sh line 103:
   echo "DB: ${DB}"
 ^---^ SC2154 (warning): DB is referenced but not assigned.
   
   
   In tools/tpcds-tools/bin/load-tpcds-data.sh line 120:
   echo "DB: ${DB}"
 ^---^ SC2154 (warning): DB is referenced but not assigned.
   
   
   In tools/tpcds-tools/bin/run-tpcds-queries.sh line 118:
   local query_id=$(curl -s -X GET -u "${USER}:${PASSWORD}" 
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/lastqueryid)
 ^--^ SC2155 (warning): Declare and assign separately to avoid 
masking return values.
   
   
   In tools/tpcds-tools/bin/run-tpcds-queries.sh line 130:
   IFS="|" read -r name queryid <<<"$entry"
^^ SC2250 (style): Prefer 
putting braces around variable references even when not strictly required.
   
   Did you mean: 
   IFS="|" read -r name queryid <<<"${entry}"
   
   
   In tools/tpcds-tools/bin/run-tpcds-queries.sh line 132:
   curl -s -X GET -u "${USER}:${PASSWORD}" 
http://"${FE_HOST}":"${FE_HTTP_PORT}"/api/profile/text?query_id=${queryid} 
>>"${PROFILE_DIR}/${profile_name}"
  

[PR] fix:admonition style [doris-website]

2024-12-19 Thread via GitHub


yang1666204 opened a new pull request, #1546:
URL: https://github.com/apache/doris-website/pull/1546

   
![62af2a86-d27c-4e81-ab79-d2be8db94197](https://github.com/user-attachments/assets/8ab8c39e-7709-46ff-a118-7af76ff57827)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](csv reader) fix core dump when parsing csv with enclose [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45485:
URL: https://github.com/apache/doris/pull/45485#issuecomment-2553068461

   TeamCity be ut coverage result:
Function Coverage: 38.88% (10134/26066) 
Line Coverage: 29.80% (85205/285970)
Region Coverage: 28.84% (43694/151524)
Branch Coverage: 25.38% (22203/87482)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/0dd081551b2fc92fcc4ca3bb4049a591ec1001db_0dd081551b2fc92fcc4ca3bb4049a591ec1001db/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Improvement](scheduler) Use a separate eager queue to execute cancel… [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45614:
URL: https://github.com/apache/doris/pull/45614#issuecomment-2553072796

    `sh-checker report`
   
   To get the full details, please check in the 
[job]("https://github.com/apache/doris/actions/runs/12409031692";) output.
   
   
   shellcheck errors
   
   ```
   
   'shellcheck ' returned error 1 finding the following syntactical issues:
   
   --
   
   In gensrc/script/gen_build_version.sh line 38:
   if [[ ${build_version_hotfix} > 0 ]]; then
 ^-- SC2071 (error): > is for string 
comparisons. Use -gt instead.
   
   
   In gensrc/script/gen_build_version.sh line 228:
   if [[ ${build_version_hotfix} > 0 ]]; then
 ^-- SC2071 (error): > is for string 
comparisons. Use -gt instead.
   
   For more information:
 https://www.shellcheck.net/wiki/SC2071 -- > is for string comparisons. Use 
...
   --
   
   You can address the above issues in one of three ways:
   1. Manually correct the issue in the offending shell script;
   2. Disable specific issues by adding the comment:
 # shellcheck disable=
   above the line that contains the issue, where  is the error code;
   3. Add '-e ' to the SHELLCHECK_OPTS setting in your .yml action file.
   
   
   
   ```
   
   
   
   shfmt errors
   
   ```
   'shfmt ' found no issues.
   
   ```
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris-website) branch master updated: [fix] Fix title of small file mananger (#1545)

2024-12-19 Thread kassiez
This is an automated email from the ASF dual-hosted git repository.

kassiez pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git


The following commit(s) were added to refs/heads/master by this push:
 new 46c91065498 [fix] Fix title of small file mananger (#1545)
46c91065498 is described below

commit 46c91065498fd690036a16cb9a172e657facef81
Author: KassieZ <139741991+kass...@users.noreply.github.com>
AuthorDate: Thu Dec 19 16:32:11 2024 +0800

[fix] Fix title of small file mananger (#1545)

## Versions

- [ ] dev
- [ ] 3.0
- [x] 2.1
- [ ] 2.0

## Languages

- [x] Chinese
- [ ] English

## Docs Checklist

- [ ] Checked by AI
- [ ] Test Cases Built
---
 .../version-2.1/admin-manual/small-file-mgr.md  | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git 
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/small-file-mgr.md
 
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/small-file-mgr.md
index 41674d04595..d7bcb4af53a 100644
--- 
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/small-file-mgr.md
+++ 
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/small-file-mgr.md
@@ -1,6 +1,6 @@
 ---
 {
-"title": "Workload Group",
+"title": "文件管理器",
 "language": "zh-CN"
 }
 ---


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix] Fix title of small file mananger [doris-website]

2024-12-19 Thread via GitHub


KassieZ merged PR #1545:
URL: https://github.com/apache/doris-website/pull/1545


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](Nereids) lock table in ascending order of table IDs [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45045:
URL: https://github.com/apache/doris/pull/45045#issuecomment-2553079791

   
   
   ClickBench: Total hot run time: 32.49 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit ce4e8c61950260a5a1e863c4152d0e5fca6f9055, 
data reload: false
   
   query1   0.040.030.03
   query2   0.070.030.03
   query3   0.240.070.07
   query4   1.600.110.11
   query5   0.440.420.40
   query6   1.160.660.65
   query7   0.020.020.02
   query8   0.040.030.03
   query9   0.590.500.51
   query10  0.550.560.57
   query11  0.140.110.11
   query12  0.140.110.11
   query13  0.600.610.60
   query14  2.732.842.84
   query15  0.900.830.82
   query16  0.380.370.37
   query17  1.091.071.04
   query18  0.230.220.21
   query19  1.981.882.01
   query20  0.020.010.01
   query21  15.38   0.600.58
   query22  2.452.751.56
   query23  16.92   1.000.79
   query24  2.890.821.56
   query25  0.160.210.11
   query26  0.470.130.13
   query27  0.050.050.04
   query28  10.59   1.111.08
   query29  12.60   3.233.24
   query30  0.250.070.06
   query31  2.860.390.38
   query32  3.240.460.48
   query33  3.083.153.11
   query34  17.15   4.474.54
   query35  4.514.524.50
   query36  0.670.470.49
   query37  0.100.060.07
   query38  0.050.030.03
   query39  0.040.020.03
   query40  0.170.130.13
   query41  0.080.020.03
   query42  0.030.020.02
   query43  0.030.030.03
   Total cold run time: 106.73 s
   Total hot run time: 32.49 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](Nereids) lock table in ascending order of table IDs [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45045:
URL: https://github.com/apache/doris/pull/45045#issuecomment-2553069420

   
   
   TPC-DS: Total hot run time: 196065 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit ce4e8c61950260a5a1e863c4152d0e5fca6f9055, 
data reload: false
   
   query1   1308980 912 912
   query2   6248226922372237
   query3   11125   492347024702
   query4   33213   23509   23498   23498
   query5   4068480 446 446
   query6   296 205 194 194
   query7   3989305 299 299
   query8   314 243 249 243
   query9   9356270727162707
   query10  480 255 250 250
   query11  17924   15144   15197   15144
   query12  165 107 106 106
   query13  1583431 416 416
   query14  10416   687872356878
   query15  277 203 192 192
   query16  8104455 467 455
   query17  1771606 571 571
   query18  2170303 307 303
   query19  395 152 160 152
   query20  126 115 122 115
   query21  205 104 107 104
   query22  4856449745664497
   query23  34538   33653   33906   33653
   query24  9935241324312413
   query25  605 388 386 386
   query26  729 154 149 149
   query27  2168357 337 337
   query28  6279245324872453
   query29  746 426 462 426
   query30  227 155 156 155
   query31  1053829 849 829
   query32  88  55  54  54
   query33  683 319 299 299
   query34  985 514 533 514
   query35  903 761 746 746
   query36  1116945 981 945
   query37  121 72  97  72
   query38  4257421441004100
   query39  1505148314631463
   query40  205 104 101 101
   query41  44  45  42  42
   query42  111 102 103 102
   query43  546 501 503 501
   query44  1207855 828 828
   query45  197 177 179 177
   query46  1159746 705 705
   query47  2039192319511923
   query48  426 317 331 317
   query49  893 380 398 380
   query50  831 409 413 409
   query51  7390712072947120
   query52  105 89  108 89
   query53  259 188 187 187
   query54  1160438 418 418
   query55  81  77  84  77
   query56  270 262 243 243
   query57  1288113111771131
   query58  237 221 224 221
   query59  3336310230823082
   query60  293 252 244 244
   query61  122 103 103 103
   query62  884 663 670 663
   query63  212 182 186 182
   query64  3651748 639 639
   query65  3306320031573157
   query66  809 298 297 297
   query67  15945   15626   15468   15468
   query68  4421556 549 549
   query69  427 250 255 250
   query70  1226115911101110
   query71  388 249 263 249
   query72  6424416641814166
   query73  761 355 364 355
   query74  10090   892288538853
   query75  3418267726502650
   query76  2811989 1054989
   query77  379 268 280 268
   query78  10190   948495019484
   query79  1658610 594 594
   query80  1102424 413 413
   query81  541 231 234 231
   query82  993 120 126 120
   query83  235 136 146 136
   query84  233 73  75  73
   query85  1216304 294 294
   query86  389 301 293 293
   query87  4490440245194402
   query88  3283223321892189
   query89  453 295 295 295
   query90  1942183 177 177
   query91  139 106 107 106
   query92  65  48  49  48
   query93  1961533 529 529
   query94  854 284 296 284
   query95  352 253 248 248
   query96  603 276 282 276
   query97  2889269226992692
   query98  216 194 202 194
   query99  1585134013271327
   Total cold run time: 297067 ms
   Total hot run time: 196065 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t

Re: [PR] [feature](tools) Support automatic profile fetching under the tools directory. [doris]

2024-12-19 Thread via GitHub


Mryange commented on PR #45612:
URL: https://github.com/apache/doris/pull/45612#issuecomment-2553069726

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](csv reader) fix core dump when parsing csv with enclose [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45485:
URL: https://github.com/apache/doris/pull/45485#issuecomment-2553073316

   
   
   TPC-DS: Total hot run time: 190951 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 0dd081551b2fc92fcc4ca3bb4049a591ec1001db, 
data reload: false
   
   query1   973 386 374 374
   query2   6516244223282328
   query3   6716225 223 223
   query4   33544   23597   23373   23373
   query5   4363483 472 472
   query6   275 194 196 194
   query7   4658304 306 304
   query8   313 237 233 233
   query9   9791271027012701
   query10  484 262 248 248
   query11  18169   15277   15109   15109
   query12  158 108 106 106
   query13  1673423 415 415
   query14  9877784271457145
   query15  315 182 181 181
   query16  8312457 416 416
   query17  1829593 576 576
   query18  2143305 321 305
   query19  400 157 149 149
   query20  119 108 110 108
   query21  208 107 100 100
   query22  4595413542004135
   query23  34451   33659   33824   33659
   query24  11582   252525222522
   query25  684 383 382 382
   query26  1855153 151 151
   query27  2857320 335 320
   query28  7804239023902390
   query29  1055412 407 407
   query30  304 147 147 147
   query31  1071788 804 788
   query32  98  58  61  58
   query33  788 321 288 288
   query34  1026518 525 518
   query35  895 747 753 747
   query36  1137928 963 928
   query37  290 79  79  79
   query38  4172418641414141
   query39  1487144814091409
   query40  288 105 103 103
   query41  46  44  47  44
   query42  121 101 101 101
   query43  546 498 497 497
   query44  1249809 805 805
   query45  186 173 173 173
   query46  1205734 747 734
   query47  1988182718641827
   query48  438 313 312 312
   query49  1289389 388 388
   query50  874 382 390 382
   query51  7214705570347034
   query52  103 100 91  91
   query53  272 186 186 186
   query54  1204407 404 404
   query55  82  78  77  77
   query56  251 242 220 220
   query57  1273112311011101
   query58  247 219 239 219
   query59  3307311330603060
   query60  285 243 249 243
   query61  108 109 112 109
   query62  919 665 670 665
   query63  220 197 191 191
   query64  5094663 630 630
   query65  3282318132173181
   query66  1442318 320 318
   query67  15847   15692   15668   15668
   query68  5883542 551 542
   query69  449 256 261 256
   query70  1223113710781078
   query71  373 258 304 258
   query72  6612437742864286
   query73  822 360 378 360
   query74  10273   884388468843
   query75  3547261527292615
   query76  3568127911401140
   query77  566 269 275 269
   query78  10363   941799469417
   query79  2196600 611 600
   query80  965 432 458 432
   query81  531 241 227 227
   query82  708 126 122 122
   query83  251 148 158 148
   query84  233 76  78  76
   query85  1354310 301 301
   query86  439 310 279 279
   query87  4535445645874456
   query88  3764222021892189
   query89  429 294 296 294
   query90  2212193 189 189
   query91  143 106 111 106
   query92  65  54  52  52
   query93  1143538 528 528
   query94  1198295 294 294
   query95  359 245 251 245
   query96  678 279 288 279
   query97  2890265826932658
   query98  224 198 193 193
   query99  1579132713321327
   Total cold run time: 305942 ms
   Total hot run time: 190951 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t

Re: [PR] [Improvement](scheduler) Use a separate eager queue to execute cancel… [doris]

2024-12-19 Thread via GitHub


Gabriel39 commented on PR #45614:
URL: https://github.com/apache/doris/pull/45614#issuecomment-2553073711

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] update example [doris-website]

2024-12-19 Thread via GitHub


intelligentfu8 opened a new pull request, #1547:
URL: https://github.com/apache/doris-website/pull/1547

   ## Versions 
   
   - [ ] dev
   - [ ] 3.0
   - [x] 2.1
   - [ ] 2.0
   
   ## Languages
   
   - [x] Chinese
   - [ ] English
   
   ## Docs Checklist
   
   - [ ] Checked by AI
   - [ ] Test Cases Built
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](csv reader) fix core dump when parsing csv with enclose [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45485:
URL: https://github.com/apache/doris/pull/45485#issuecomment-2553083476

   
   
   ClickBench: Total hot run time: 32.41 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 0dd081551b2fc92fcc4ca3bb4049a591ec1001db, 
data reload: false
   
   query1   0.040.030.03
   query2   0.070.030.03
   query3   0.240.070.06
   query4   1.610.100.11
   query5   0.430.390.41
   query6   1.170.660.65
   query7   0.020.020.02
   query8   0.040.030.03
   query9   0.580.510.50
   query10  0.550.580.57
   query11  0.150.110.10
   query12  0.140.110.11
   query13  0.620.620.60
   query14  2.732.722.74
   query15  0.900.820.82
   query16  0.390.360.39
   query17  0.960.991.05
   query18  0.230.220.20
   query19  1.861.831.98
   query20  0.010.010.01
   query21  15.38   0.590.58
   query22  3.101.622.19
   query23  17.12   0.930.98
   query24  3.530.902.00
   query25  0.180.080.15
   query26  0.590.130.14
   query27  0.050.050.05
   query28  9.661.131.07
   query29  12.55   3.193.21
   query30  0.240.060.06
   query31  2.860.380.39
   query32  3.250.460.45
   query33  3.183.033.08
   query34  16.64   4.514.55
   query35  4.524.544.57
   query36  0.650.480.47
   query37  0.100.060.06
   query38  0.050.030.03
   query39  0.040.030.03
   query40  0.160.130.12
   query41  0.080.030.02
   query42  0.040.020.03
   query43  0.040.030.02
   Total cold run time: 106.75 s
   Total hot run time: 32.41 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](rowset meta) truncate segments key bounds if too large to avoid `RowsetMetaCloudPB` exceeds fdb's 100KB limits [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45287:
URL: https://github.com/apache/doris/pull/45287#issuecomment-2553088302

   
   
   TPC-H: Total hot run time: 39949 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 51fe72230ebc59097729fd83ae68ce087c5a9ccd, 
data reload: false
   
   -- Round 1 --
   q1   17959   857673027302
   q2   2046176 165 165
   q3   10719   106512081065
   q4   10541   726 660 660
   q5   7627269826792679
   q6   240 148 149 148
   q7   980 621 599 599
   q8   9250181719771817
   q9   6745647864436443
   q10  6997233823302330
   q11  461 255 252 252
   q12  430 229 223 223
   q13  17763   296430052964
   q14  237 209 207 207
   q15  556 494 496 494
   q16  669 582 571 571
   q17  987 686 583 583
   q18  7410673066576657
   q19  1346108911281089
   q20  474 180 189 180
   q21  4315338332193219
   q22  375 322 302 302
   Total cold run time: 108127 ms
   Total hot run time: 39949 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7315726672577257
   q2   333 229 226 226
   q3   2896297929402940
   q4   2140194818671867
   q5   5639567056415641
   q6   227 149 145 145
   q7   2256184318221822
   q8   3388358335513551
   q9   8891894489938944
   q10  3588354735493547
   q11  608 494 491 491
   q12  785 639 589 589
   q13  11633   312130433043
   q14  315 270 264 264
   q15  613 501 511 501
   q16  661 638 642 638
   q17  1874167716441644
   q18  8477764276957642
   q19  1734169216201620
   q20  2095187618841876
   q21  5631546953675367
   q22  647 625 577 577
   Total cold run time: 71746 ms
   Total hot run time: 60192 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] branch-3.0: [fix](orc) ignore null values when the literals of in_predicate contains #45104 [doris]

2024-12-19 Thread via GitHub


yiguolei merged PR #45586:
URL: https://github.com/apache/doris/pull/45586


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch branch-3.0 updated: branch-3.0: [fix](orc) ignore null values when the literals of in_predicate contains #45104 (#45586)

2024-12-19 Thread yiguolei
This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-3.0 by this push:
 new 55a51dc6606 branch-3.0: [fix](orc) ignore null values when the 
literals of in_predicate contains #45104 (#45586)
55a51dc6606 is described below

commit 55a51dc6606202fff1d934bc5f1f8f0a71c4e53b
Author: github-actions[bot] 
<41898282+github-actions[bot]@users.noreply.github.com>
AuthorDate: Thu Dec 19 16:46:21 2024 +0800

branch-3.0: [fix](orc) ignore null values when the literals of in_predicate 
contains #45104 (#45586)

Cherry-picked from #45104

Co-authored-by: Socrates 
---
 be/src/apache-orc  |   2 +-
 be/src/vec/exec/format/orc/vorc_reader.cpp |  24 +++--
 be/test/vec/exec/orc_reader_test.cpp   |   3 +
 .../data/external_table_p0/hive/test_hive_orc.out  | 102 +
 .../external_table_p0/hive/test_hive_orc.groovy|  22 +
 5 files changed, 145 insertions(+), 8 deletions(-)

diff --git a/be/src/apache-orc b/be/src/apache-orc
index db01184f765..2f937bdc764 16
--- a/be/src/apache-orc
+++ b/be/src/apache-orc
@@ -1 +1 @@
-Subproject commit db01184f765c03496e4107bd3ac37c077ac4bc5f
+Subproject commit 2f937bdc76406f150b484b6e57629aa8a03d48b6
diff --git a/be/src/vec/exec/format/orc/vorc_reader.cpp 
b/be/src/vec/exec/format/orc/vorc_reader.cpp
index df8ec52efc2..a1ecb1ae0dc 100644
--- a/be/src/vec/exec/format/orc/vorc_reader.cpp
+++ b/be/src/vec/exec/format/orc/vorc_reader.cpp
@@ -586,6 +586,10 @@ std::tuple 
OrcReader::_make_orc_lite
 // only get the predicate_type
 return std::make_tuple(true, orc::Literal(true), predicate_type);
 }
+// this only happens when the literals of in_predicate contains null 
value, like in (1, null)
+if (literal->get_column_ptr()->is_null_at(0)) {
+return std::make_tuple(false, orc::Literal(false), predicate_type);
+}
 auto literal_data = literal->get_column_ptr()->get_data_at(0);
 auto* slot = _tuple_descriptor->slots()[slot_ref->column_id()];
 auto slot_type = slot->type();
@@ -666,12 +670,13 @@ bool OrcReader::_check_rest_children_can_push_down(const 
VExprSPtr& expr) {
 return false;
 }
 
+bool at_least_one_child_can_push_down = false;
 for (size_t i = 1; i < expr->children().size(); ++i) {
-if (!_check_literal_can_push_down(expr, i)) {
-return false;
+if (_check_literal_can_push_down(expr, i)) {
+at_least_one_child_can_push_down = true;
 }
 }
-return true;
+return at_least_one_child_can_push_down;
 }
 
 // check if the expr can be pushed down to orc reader
@@ -774,12 +779,17 @@ void OrcReader::_build_filter_in(const VExprSPtr& expr,
 for (size_t i = 1; i < expr->children().size(); ++i) {
 DCHECK(expr->children()[i]->is_literal());
 const auto* literal = static_cast(expr->children()[i].get());
-DCHECK(_vliteral_to_orc_literal.contains(literal));
-auto orc_literal = _vliteral_to_orc_literal.find(literal)->second;
-literals.emplace_back(orc_literal);
+if (_vliteral_to_orc_literal.contains(literal)) {
+auto orc_literal = _vliteral_to_orc_literal.find(literal)->second;
+literals.emplace_back(orc_literal);
+}
 }
 DCHECK(!literals.empty());
-builder->in(slot_ref->expr_name(), predicate_type, literals);
+if (literals.size() == 1) {
+builder->equals(slot_ref->expr_name(), predicate_type, literals[0]);
+} else {
+builder->in(slot_ref->expr_name(), predicate_type, literals);
+}
 }
 
 void OrcReader::_build_is_null(const VExprSPtr& expr,
diff --git a/be/test/vec/exec/orc_reader_test.cpp 
b/be/test/vec/exec/orc_reader_test.cpp
index f2bba434368..ff7452ae625 100644
--- a/be/test/vec/exec/orc_reader_test.cpp
+++ b/be/test/vec/exec/orc_reader_test.cpp
@@ -111,6 +111,8 @@ TEST_F(OrcReaderTest, test_build_search_argument) {
 
R"|({"1":{"lst":["rec",11,{"1":{"i32":6},"2":{"rec":{"1":{"lst":["rec",1,{"1":{"i32":0},"2":{"rec":{"1":{"i32":2]},"3":{"i64":-1}}},"3":{"i32":3},"4":{"i32":2},"20":{"i32":-1},"29":{"tf":1}},{"1":{"i32":2},"2":{"rec":{"1":{"lst":["rec",1,{"1":{"i32":0},"2":{"rec":{"1":{"i32":2]},"3":{"i64":-1}}},"3":{"i32":11},"4":{"i32":2},"20":{"i32":-1},"26":{"rec":{"1":{"rec":{"2":{"str":"lt"}}},"2":{"i32":0},"3":{"lst":["rec",2,{"1":{"lst":["rec",1,{"1":{"i32":0},
 [...]
 // select count(o_orderkey) from tpch1_orc.orders 
where o_orderkey < 1 + 1;
 
R"|({"1":{"lst":["rec",3,{"1":{"i32":2},"2":{"rec":{"1":{"lst":["rec",1,{"1":{"i32":0},"2":{"rec":{"1":{"i32":2]},"3":{"i64":-1}}},"3":{"i32":11},"4":{"i32":2},"20":{"i32":-1},"26":{"rec":{"1":{"rec":{"2":{"str":"lt"}}},"2":{"i32":0},"3":{"ls

(doris) branch auto-pick-45104-branch-3.0 deleted (was fe2f617ed55)

2024-12-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch auto-pick-45104-branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git


 was fe2f617ed55 [fix](orc) ignore null values when the literals of 
in_predicate contains (#45104)

The revisions that were on this branch are still contained in
other references; therefore, this change does not discard any commits
from the repository.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](rowset meta) truncate segments key bounds if too large to avoid `RowsetMetaCloudPB` exceeds fdb's 100KB limits [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45287:
URL: https://github.com/apache/doris/pull/45287#issuecomment-2553111811

   
   
   TPC-DS: Total hot run time: 196816 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 51fe72230ebc59097729fd83ae68ce087c5a9ccd, 
data reload: false
   
   query1   1322958 893 893
   query2   6248244123152315
   query3   11123   463346884633
   query4   3   23258   23105   23105
   query5   3594472 467 467
   query6   280 200 193 193
   query7   4015305 311 305
   query8   316 257 225 225
   query9   9589271227042704
   query10  459 260 248 248
   query11  17752   15118   15124   15118
   query12  159 105 106 105
   query13  1562421 407 407
   query14  8785760472517251
   query15  266 189 209 189
   query16  8171456 446 446
   query17  1540595 587 587
   query18  2159309 313 309
   query19  281 164 156 156
   query20  115 119 118 118
   query21  204 105 105 105
   query22  4919453745334533
   query23  35053   33716   34903   33716
   query24  10444   241625102416
   query25  632 397 401 397
   query26  1235156 151 151
   query27  2498336 335 335
   query28  7836251724752475
   query29  857 436 424 424
   query30  238 170 162 162
   query31  1064844 852 844
   query32  106 60  61  60
   query33  806 303 316 303
   query34  1129516 499 499
   query35  877 761 761 761
   query36  1143991 961 961
   query37  144 86  84  84
   query38  4452426541674167
   query39  1540146814401440
   query40  211 107 105 105
   query41  44  44  45  44
   query42  124 109 117 109
   query43  549 513 512 512
   query44  1361842 851 842
   query45  186 170 174 170
   query46  1198733 749 733
   query47  2084192619931926
   query48  439 330 321 321
   query49  895 402 421 402
   query50  844 391 392 391
   query51  7367729672507250
   query52  110 101 104 101
   query53  266 189 193 189
   query54  1116438 452 438
   query55  96  86  87  86
   query56  285 261 262 261
   query57  1294118911881188
   query58  261 249 248 248
   query59  3336321330743074
   query60  290 258 259 258
   query61  110 102 133 102
   query62  862 690 709 690
   query63  229 190 192 190
   query64  3996745 693 693
   query65  3286324132213221
   query66  741 311 321 311
   query67  16526   15683   15519   15519
   query68  5703552 549 549
   query69  479 270 265 265
   query70  1269113411571134
   query71  455 264 256 256
   query72  7126412441024102
   query73  802 355 373 355
   query74  10203   905888388838
   query75  3469263626612636
   query76  3793108910761076
   query77  576 292 282 282
   query78  10262   943593999399
   query79  1653608 593 593
   query80  801 438 426 426
   query81  512 239 224 224
   query82  671 121 119 119
   query83  172 147 145 145
   query84  243 75  70  70
   query85  1282307 307 307
   query86  503 290 272 272
   query87  4492443444714434
   query88  3677223722182218
   query89  441 290 289 289
   query90  1865190 195 190
   query91  141 109 106 106
   query92  65  58  62  58
   query93  1905547 543 543
   query94  675 288 289 288
   query95  355 262 256 256
   query96  622 286 283 283
   query97  2882268626352635
   query98  215 194 201 194
   query99  1622131412931293
   Total cold run time: 303283 ms
   Total hot run time: 196816 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go

Re: [PR] [feat](nereids) when check shape failed, print all plan with memo [doris]

2024-12-19 Thread via GitHub


englefly commented on PR #45627:
URL: https://github.com/apache/doris/pull/45627#issuecomment-2553112836

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feat](nereids) when check shape failed, print all plan with memo [doris]

2024-12-19 Thread via GitHub


englefly commented on PR #45627:
URL: https://github.com/apache/doris/pull/45627#issuecomment-2553113033

   run cloud_p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feat](nereids) when check shape failed, print all plan with memo [doris]

2024-12-19 Thread via GitHub


englefly commented on PR #45627:
URL: https://github.com/apache/doris/pull/45627#issuecomment-2553113227

   run external


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch auto-pick-45542-branch-2.1 created (now d6c629d293a)

2024-12-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch auto-pick-45542-branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git


  at d6c629d293a branch-2.1: [fix](tvf) Tvf supports to parse the enclose 
character in csv files #45407 (#45570)

No new revisions were added by this update.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch auto-pick-45542-branch-3.0 created (now 55a51dc6606)

2024-12-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch auto-pick-45542-branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git


  at 55a51dc6606 branch-3.0: [fix](orc) ignore null values when the 
literals of in_predicate contains #45104 (#45586)

No new revisions were added by this update.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [branch-2.1](fix) fix incorrect result of hash join with const column [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45630:
URL: https://github.com/apache/doris/pull/45630#issuecomment-2553115477

   TeamCity be ut coverage result:
Function Coverage: 36.46% (9569/26243) 
Line Coverage: 27.93% (78660/281681)
Region Coverage: 26.59% (40375/151821)
Branch Coverage: 23.35% (20448/87570)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/9ac71a9d3a9b8cc7925f18d38dbecc14d5076468_9ac71a9d3a9b8cc7925f18d38dbecc14d5076468/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](rowset meta) truncate segments key bounds if too large to avoid `RowsetMetaCloudPB` exceeds fdb's 100KB limits [doris]

2024-12-19 Thread via GitHub


bobhan1 commented on PR #45287:
URL: https://github.com/apache/doris/pull/45287#issuecomment-2553114256

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [I] [Bug] ${LIMIT:+${LIMIT}} "${DORIS_HOME}/lib/doris_be" "$@" 2>&1 < /dev/null doris be莫名其妙停止 [doris]

2024-12-19 Thread via GitHub


wangjinxiang0522 commented on issue #34934:
URL: https://github.com/apache/doris/issues/34934#issuecomment-2553114661

   问题解决了吗? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](Job)Fix redundant job scheduling by preventing same state transitions (e.g., RUNNING → RUNNING) [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45495:
URL: https://github.com/apache/doris/pull/45495#issuecomment-2553119415

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] fix 6 [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45633:
URL: https://github.com/apache/doris/pull/45633#issuecomment-2553118897

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris-website) branch master updated: update example (#1547)

2024-12-19 Thread kassiez
This is an automated email from the ASF dual-hosted git repository.

kassiez pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git


The following commit(s) were added to refs/heads/master by this push:
 new e49416f4454 update example (#1547)
e49416f4454 is described below

commit e49416f44545786f579fe9ab0b244fc55a905472
Author: smiletan 
AuthorDate: Thu Dec 19 16:56:58 2024 +0800

update example (#1547)

## Versions

- [ ] dev
- [ ] 3.0
- [x] 2.1
- [ ] 2.0

## Languages

- [x] Chinese
- [ ] English

## Docs Checklist

- [ ] Checked by AI
- [ ] Test Cases Built
---
 .../k8s-deploy/install-config-cluster.md   | 44 --
 1 file changed, 33 insertions(+), 11 deletions(-)

diff --git 
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/install/cluster-deployment/k8s-deploy/install-config-cluster.md
 
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/install/cluster-deployment/k8s-deploy/install-config-cluster.md
index 4136845d1e9..331ffbceacf 100644
--- 
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/install/cluster-deployment/k8s-deploy/install-config-cluster.md
+++ 
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/install/cluster-deployment/k8s-deploy/install-config-cluster.md
@@ -165,17 +165,39 @@ Kubernetes 部署中,建议使用 FQDN 模式,启动配置中应添加 enabl
  第 1 步:配置并部署 ConfigMap   
 以下定义了名为 `be-conf` ConfigMap,该配置可供 Doris BE 使用:
 ```yaml
-beSpec:
-  persistentVolumes:
-  - mountPath: /opt/apache-doris/be/log
-name: belog
-persistentVolumeClaimSpec:
-  storageClassName: ${your_storageclass}
-  accessModes:
-  - ReadWriteOnce
-  resources:
-requests:
-  storage: ${storageSize}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: be-conf
+  labels:
+app.kubernetes.io/component: be
+data:
+  be.conf: |
+CUR_DATE=`date +%Y%m%d-%H%M%S`
+
+PPROF_TMPDIR="$DORIS_HOME/log/"
+
+JAVA_OPTS="-Xmx1024m -DlogPath=$DORIS_HOME/log/jni.log 
-Xloggc:$DORIS_HOME/log/be.gc.log.$CUR_DATE 
-Djavax.security.auth.useSubjectCredsOnly=false -Dsun.java.command=DorisBE 
-XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100 
-DJDBC_MAX_IDLE_TIME=30 -DJDBC_MAX_WAIT_TIME=5000"
+
+# For jdk 9+, this JAVA_OPTS will be used as default JVM options
+JAVA_OPTS_FOR_JDK_9="-Xmx1024m -DlogPath=$DORIS_HOME/log/jni.log 
-Xlog:gc:$DORIS_HOME/log/be.gc.log.$CUR_DATE 
-Djavax.security.auth.useSubjectCredsOnly=false -Dsun.java.command=DorisBE 
-XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100 
-DJDBC_MAX_IDLE_TIME=30 -DJDBC_MAX_WAIT_TIME=5000"
+
+# since 1.2, the JAVA_HOME need to be set to run BE process.
+# JAVA_HOME=/path/to/jdk/
+
+# 
https://github.com/apache/doris/blob/master/docs/zh-CN/community/developer-guide/debug-tool.md#jemalloc-heap-profile
+# https://jemalloc.net/jemalloc.3.html
+
JEMALLOC_CONF="percpu_arena:percpu,background_thread:true,metadata_thp:auto,muzzy_decay_ms:15000,dirty_decay_ms:15000,oversize_threshold:0,lg_tcache_max:20,prof:false,lg_prof_interval:32,lg_prof_sample:19,prof_gdump:false,prof_accum:false,prof_leak:false,prof_final:false"
+JEMALLOC_PROF_PRFIX=""
+
+# INFO, WARNING, ERROR, FATAL
+sys_log_level = INFO
+
+# ports for admin, web, heartbeat service
+be_port = 9060
+webserver_port = 8040
+heartbeat_service_port = 9050
+brpc_port = 8060
 ```
 使用 ConfigMap 挂载 BE 启动配置信息时,配置信息对应的 key 必须为 `be.conf` 。完成配置文件后,将其部署到目标 
`DorisCluster` 资源需要部署的命名空间。
 ```shell


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] update example [doris-website]

2024-12-19 Thread via GitHub


KassieZ merged PR #1547:
URL: https://github.com/apache/doris-website/pull/1547


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](Job)Fix redundant job scheduling by preventing same state transitions (e.g., RUNNING → RUNNING) [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45495:
URL: https://github.com/apache/doris/pull/45495#issuecomment-2553119542

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](rowset meta) truncate segments key bounds if too large to avoid `RowsetMetaCloudPB` exceeds fdb's 100KB limits [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45287:
URL: https://github.com/apache/doris/pull/45287#issuecomment-2553123027

   
   
   ClickBench: Total hot run time: 33.6 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 51fe72230ebc59097729fd83ae68ce087c5a9ccd, 
data reload: false
   
   query1   0.040.030.02
   query2   0.090.050.04
   query3   0.230.060.06
   query4   1.640.090.09
   query5   0.430.400.40
   query6   1.180.670.66
   query7   0.020.030.01
   query8   0.060.050.05
   query9   0.540.500.49
   query10  0.560.570.54
   query11  0.160.110.12
   query12  0.170.130.13
   query13  0.610.610.60
   query14  2.892.742.75
   query15  0.900.830.83
   query16  0.370.400.37
   query17  1.071.011.07
   query18  0.180.180.19
   query19  1.901.731.99
   query20  0.020.010.02
   query21  15.36   0.650.65
   query22  4.037.062.39
   query23  18.25   1.471.25
   query24  2.270.220.23
   query25  0.150.080.09
   query26  0.280.180.18
   query27  0.080.080.09
   query28  13.20   1.171.14
   query29  12.65   3.373.31
   query30  0.240.060.06
   query31  2.870.400.40
   query32  3.230.490.48
   query33  3.063.203.13
   query34  16.82   4.514.54
   query35  4.574.574.54
   query36  0.670.480.50
   query37  0.210.160.15
   query38  0.160.150.15
   query39  0.050.050.05
   query40  0.160.130.12
   query41  0.100.060.06
   query42  0.070.050.05
   query43  0.060.050.05
   Total cold run time: 111.6 s
   Total hot run time: 33.6 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris-website) branch master updated: fix:admonition style (#1546)

2024-12-19 Thread jeffreyh
This is an automated email from the ASF dual-hosted git repository.

jeffreyh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git


The following commit(s) were added to refs/heads/master by this push:
 new 90a6b123ac7 fix:admonition style (#1546)
90a6b123ac7 is described below

commit 90a6b123ac7c05d26de361ecf4879c5a49b24719
Author: yangon <2689991...@qq.com>
AuthorDate: Thu Dec 19 17:01:30 2024 +0800

fix:admonition style (#1546)


![62af2a86-d27c-4e81-ab79-d2be8db94197](https://github.com/user-attachments/assets/8ab8c39e-7709-46ff-a118-7af76ff57827)

Co-authored-by: liyang 
---
 src/theme/Admonition/Layout/styles.module.css | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/theme/Admonition/Layout/styles.module.css 
b/src/theme/Admonition/Layout/styles.module.css
index 3f5fe3be285..26af2fdb158 100644
--- a/src/theme/Admonition/Layout/styles.module.css
+++ b/src/theme/Admonition/Layout/styles.module.css
@@ -34,5 +34,5 @@
 }
 
 .admonitionContent> :last-child {
-  margin-bottom: 0;
+  margin-bottom: 0 !important;
 }
\ No newline at end of file


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] fix:admonition style [doris-website]

2024-12-19 Thread via GitHub


jeffreys-cat merged PR #1546:
URL: https://github.com/apache/doris-website/pull/1546


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Improvement](scheduler) Use a separate eager queue to execute cancel… [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45614:
URL: https://github.com/apache/doris/pull/45614#issuecomment-2553129387

   TeamCity be ut coverage result:
Function Coverage: 38.88% (10134/26067) 
Line Coverage: 29.78% (85173/285997)
Region Coverage: 28.83% (43692/151545)
Branch Coverage: 25.37% (22195/87496)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/6e2fe8b6325172cb7a79b2b3a6c829d20f225962_6e2fe8b6325172cb7a79b2b3a6c829d20f225962/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] fix 7 [doris]

2024-12-19 Thread via GitHub


eldenmoon merged PR #45680:
URL: https://github.com/apache/doris/pull/45680


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] fix 7 [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on code in PR #45680:
URL: https://github.com/apache/doris/pull/45680#discussion_r1892752108


##
be/src/olap/rowset/segment_v2/variant_column_writer_impl.cpp:
##
@@ -333,10 +344,10 @@ uint64_t VariantColumnWriterImpl::estimate_buffer_size() {
 return _column->byte_size();
 }
 uint64_t size = 0;
+size += _root_writer->estimate_buffer_size();

Review Comment:
   warning: method 'estimate_buffer_size' can be made const 
[readability-make-member-function-const]
   
   ```suggestion
   uint64_t VariantColumnWriterImpl::estimate_buffer_size() const {
   ```
   
   be/src/olap/rowset/segment_v2/variant_column_writer_impl.h:63:
   ```diff
   - uint64_t estimate_buffer_size();
   + uint64_t estimate_buffer_size() const;
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [opt](Nereids) lock table in ascending order of table IDs (#45045) [doris]

2024-12-19 Thread via GitHub


morrySnow opened a new pull request, #45681:
URL: https://github.com/apache/doris/pull/45681

   pick from master #45045
   
   Problem Summary:
   
   Doris's table locks are fair read-write locks. If two threads acquire read 
locks on tables in different orders and simultaneously a third thread attempts 
to acquire a write lock on one of these tables, a deadlock can form between the 
two threads trying to acquire read locks. This PR changes the lock acquisition 
order for queries to follow the order of table IDs, ensuring that the lock 
acquisition order for tables is consistent among different threads.
   
   Execute table locking operations in ascending order of table IDs
   
   ### What problem does this PR solve?
   
   Issue Number: close #xxx
   
   Related PR: #xxx
   
   Problem Summary:
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test 
   - [ ] Regression test
   - [ ] Unit Test
   - [ ] Manual test (add detailed scripts or steps below)
   - [ ] No need to test or manual test. Explain why:
   - [ ] This is a refactor/code format and no logic has been changed.
   - [ ] Previous test can cover this change.
   - [ ] No code files have been changed.
   - [ ] Other reason 
   
   - Behavior changed:
   - [ ] No.
   - [ ] Yes. 
   
   - Does this need documentation?
   - [ ] No.
   - [ ] Yes. 
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](Nereids) lock table in ascending order of table IDs (#45045) [doris]

2024-12-19 Thread via GitHub


Thearas commented on PR #45681:
URL: https://github.com/apache/doris/pull/45681#issuecomment-2555180708

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting 
information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it 
now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and 
after the optimization?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](Nereids) lock table in ascending order of table IDs (#45045) [doris]

2024-12-19 Thread via GitHub


morrySnow commented on PR #45681:
URL: https://github.com/apache/doris/pull/45681#issuecomment-2555180759

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [test](p0) force replica = 1 in p0 regression tests [doris]

2024-12-19 Thread via GitHub


kaijchen opened a new pull request, #45682:
URL: https://github.com/apache/doris/pull/45682

   ### What problem does this PR solve?
   
   Some tests in p0 does not set replica = 1, and will fail when BE number is 
not enough.
   
   This PR fixed this problem by set `force_olap_table_replication_allocation` 
in p0 fe.conf 
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test 
   - [ ] Regression test
   - [ ] Unit Test
   - [ ] Manual test (add detailed scripts or steps below)
   - [ ] No need to test or manual test. Explain why:
   - [ ] This is a refactor/code format and no logic has been changed.
   - [ ] Previous test can cover this change.
   - [ ] No code files have been changed.
   - [ ] Other reason 
   
   - Behavior changed:
   - [ ] No.
   - [ ] Yes. 
   
   - Does this need documentation?
   - [ ] No.
   - [ ] Yes. 
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] fix 7 [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45680:
URL: https://github.com/apache/doris/pull/45680#issuecomment-2555138540

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch variant-sparse updated: [fix](serialize) fix column serialize and deserialize (#45667)

2024-12-19 Thread eldenmoon
This is an automated email from the ASF dual-hosted git repository.

eldenmoon pushed a commit to branch variant-sparse
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/variant-sparse by this push:
 new c15122c043d [fix](serialize) fix column serialize and deserialize 
(#45667)
c15122c043d is described below

commit c15122c043defa4c417663bf34f5e48d98dc2f42
Author: Sun Chenyang 
AuthorDate: Fri Dec 20 01:01:55 2024 +0800

[fix](serialize) fix column serialize and deserialize (#45667)
---
 be/src/vec/columns/column_object.cpp   | 13 +++---
 .../vec/data_types/serde/data_type_array_serde.cpp |  7 +++---
 .../vec/data_types/serde/data_type_array_serde.h   |  2 +-
 .../vec/data_types/serde/data_type_jsonb_serde.cpp |  5 ++--
 .../vec/data_types/serde/data_type_jsonb_serde.h   |  2 +-
 .../data_types/serde/data_type_nullable_serde.cpp  | 29 +++---
 .../data_types/serde/data_type_nullable_serde.h|  4 +--
 .../data_types/serde/data_type_number_serde.cpp|  4 +--
 .../vec/data_types/serde/data_type_number_serde.h  |  2 +-
 be/src/vec/data_types/serde/data_type_serde.h  |  2 +-
 .../vec/data_types/serde/data_type_string_serde.h  |  4 +--
 11 files changed, 38 insertions(+), 36 deletions(-)

diff --git a/be/src/vec/columns/column_object.cpp 
b/be/src/vec/columns/column_object.cpp
index 91a0936673f..17c02172da8 100644
--- a/be/src/vec/columns/column_object.cpp
+++ b/be/src/vec/columns/column_object.cpp
@@ -1053,8 +1053,10 @@ void 
ColumnObject::Subcolumn::serialize_to_sparse_column(ColumnString* key, std:
 auto& nullable_col = assert_cast(*part);
 
 // insert value
+ColumnString::Chars& chars = value->get_chars();
 nullable_serde->get_nested_serde()->write_one_cell_to_binary(
-nullable_col.get_nested_column(), value, row);
+nullable_col.get_nested_column(), chars, row);
+value->get_offsets().push_back(chars.size());
 }
 return;
 }
@@ -1114,6 +1116,11 @@ const char* parse_binary_from_sparse_column(TypeIndex 
type, const char* data, Fi
 end = data + size;
 break;
 }
+case TypeIndex::Nothing: {
+res = Null();
+end = data;
+break;
+}
 case TypeIndex::Array: {
 const size_t size = *reinterpret_cast(data);
 data += sizeof(size_t);
@@ -1123,9 +1130,9 @@ const char* parse_binary_from_sparse_column(TypeIndex 
type, const char* data, Fi
 for (size_t i = 0; i < size; ++i) {
 Field nested_field;
 const auto nested_type =
-assert_cast(*reinterpret_cast(data++));
+static_cast(*reinterpret_cast(data++));
 data = parse_binary_from_sparse_column(nested_type, data, 
nested_field, info_res);
-array.emplace_back(std::move(nested_field));
+array[i] = std::move(nested_field);
 }
 end = data;
 break;
diff --git a/be/src/vec/data_types/serde/data_type_array_serde.cpp 
b/be/src/vec/data_types/serde/data_type_array_serde.cpp
index 57a43fbb381..2c906ce4c49 100644
--- a/be/src/vec/data_types/serde/data_type_array_serde.cpp
+++ b/be/src/vec/data_types/serde/data_type_array_serde.cpp
@@ -436,9 +436,9 @@ Status DataTypeArraySerDe::read_column_from_pb(IColumn& 
column, const PValues& a
 }
 
 void DataTypeArraySerDe::write_one_cell_to_binary(const IColumn& src_column,
-  ColumnString* dst_column, 
int64_t row_num) const {
+  ColumnString::Chars& chars,
+  int64_t row_num) const {
 const uint8_t type = static_cast(TypeIndex::Array);
-ColumnString::Chars& chars = dst_column->get_chars();
 const size_t old_size = chars.size();
 const size_t new_size = old_size + sizeof(uint8_t) + sizeof(size_t);
 chars.resize(new_size);
@@ -453,9 +453,8 @@ void DataTypeArraySerDe::write_one_cell_to_binary(const 
IColumn& src_column,
 memcpy(chars.data() + old_size + sizeof(uint8_t), reinterpret_cast(&size),
sizeof(size_t));
 for (size_t offset = start; offset != end; ++offset) {
-nested_serde->write_one_cell_to_binary(nested_column, dst_column, 
offset);
+nested_serde->write_one_cell_to_binary(nested_column, chars, offset);
 }
-dst_column->get_offsets().push_back(chars.size());
 }
 
 } // namespace vectorized
diff --git a/be/src/vec/data_types/serde/data_type_array_serde.h 
b/be/src/vec/data_types/serde/data_type_array_serde.h
index aaf1a425512..25da83f2cff 100644
--- a/be/src/vec/data_types/serde/data_type_array_serde.h
+++ b/be/src/vec/data_types/serde/data_type_array_serde.h
@@ -101,7 +101,7 @@ public:
 nested_serde->set_return_object_as_string(value);
 }
 
-void write_one_cell_t

(doris) branch variant-sparse updated: fix 7 (#45680)

2024-12-19 Thread eldenmoon
This is an automated email from the ASF dual-hosted git repository.

eldenmoon pushed a commit to branch variant-sparse
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/variant-sparse by this push:
 new 89b3d4178f7 fix 7 (#45680)
89b3d4178f7 is described below

commit 89b3d4178f7004fa5c67a8cabd14f9e4e1b8cc35
Author: lihangyu 
AuthorDate: Fri Dec 20 01:00:55 2024 +0800

fix 7 (#45680)
---
 .../rowset/segment_v2/hierarchical_data_reader.cpp |  8 +
 be/src/olap/rowset/segment_v2/segment.cpp  |  6 +++-
 be/src/olap/rowset/segment_v2/segment.h|  2 +-
 be/src/olap/rowset/segment_v2/segment_writer.cpp   |  1 +
 .../segment_v2/variant_column_writer_impl.cpp  | 39 ++
 .../rowset/segment_v2/vertical_segment_writer.cpp  |  2 +-
 be/src/vec/columns/column_object.cpp   |  7 +++-
 be/src/vec/columns/column_object.h |  4 +--
 be/src/vec/functions/function_cast.h   |  1 -
 be/src/vec/json/parse2column.cpp   |  7 
 10 files changed, 55 insertions(+), 22 deletions(-)

diff --git a/be/src/olap/rowset/segment_v2/hierarchical_data_reader.cpp 
b/be/src/olap/rowset/segment_v2/hierarchical_data_reader.cpp
index a0e8b3fd0ee..de0123a330a 100644
--- a/be/src/olap/rowset/segment_v2/hierarchical_data_reader.cpp
+++ b/be/src/olap/rowset/segment_v2/hierarchical_data_reader.cpp
@@ -104,6 +104,10 @@ Status HierarchicalDataReader::seek_to_ordinal(ordinal_t 
ord) {
 DCHECK(_root_reader->inited);
 RETURN_IF_ERROR(_root_reader->iterator->seek_to_ordinal(ord));
 }
+if (_sparse_column_reader) {
+DCHECK(_sparse_column_reader->inited);
+RETURN_IF_ERROR(_sparse_column_reader->iterator->seek_to_ordinal(ord));
+}
 return Status::OK();
 }
 
@@ -424,6 +428,10 @@ void 
SparseColumnExtractReader::_fill_path_column(vectorized::MutableColumnPtr&
 *var.get_subcolumn({}) /*root*/, null_map, StringRef 
{_path.data(), _path.size()},
 _sparse_column->get_ptr(), 0, _sparse_column->size());
 var.incr_num_rows(_sparse_column->size());
+
var.get_sparse_column()->assume_mutable()->insert_many_defaults(_sparse_column->size());
+#ifndef NDEBUG
+var.check_consistency();
+#endif
 _sparse_column->clear();
 }
 
diff --git a/be/src/olap/rowset/segment_v2/segment.cpp 
b/be/src/olap/rowset/segment_v2/segment.cpp
index 238898a74ec..9b505e4a4a5 100644
--- a/be/src/olap/rowset/segment_v2/segment.cpp
+++ b/be/src/olap/rowset/segment_v2/segment.cpp
@@ -812,7 +812,11 @@ Status Segment::new_column_iterator(const TabletColumn& 
tablet_column,
 return Status::OK();
 }
 
-ColumnReader* Segment::get_column_reader(int32_t col_unique_id) {
+Result Segment::get_column_reader(int32_t col_unique_id) {
+auto status = _create_column_readers_once();
+if (!status) {
+return ResultError(std::move(status));
+}
 if (_column_readers.contains(col_unique_id)) {
 return _column_readers[col_unique_id].get();
 }
diff --git a/be/src/olap/rowset/segment_v2/segment.h 
b/be/src/olap/rowset/segment_v2/segment.h
index 5b88e60e37a..9fe545006e3 100644
--- a/be/src/olap/rowset/segment_v2/segment.h
+++ b/be/src/olap/rowset/segment_v2/segment.h
@@ -209,7 +209,7 @@ public:
 
 const TabletSchemaSPtr& tablet_schema() { return _tablet_schema; }
 
-ColumnReader* get_column_reader(int32_t col_unique_id);
+Result get_column_reader(int32_t col_unique_id);
 
 private:
 DISALLOW_COPY_AND_ASSIGN(Segment);
diff --git a/be/src/olap/rowset/segment_v2/segment_writer.cpp 
b/be/src/olap/rowset/segment_v2/segment_writer.cpp
index b76acf68978..60fd7cea28a 100644
--- a/be/src/olap/rowset/segment_v2/segment_writer.cpp
+++ b/be/src/olap/rowset/segment_v2/segment_writer.cpp
@@ -1151,6 +1151,7 @@ Status SegmentWriter::_write_footer() {
 
 // Footer := SegmentFooterPB, FooterPBSize(4), FooterPBChecksum(4), 
MagicNumber(4)
 std::string footer_buf;
+VLOG_DEBUG << "footer " << _footer.DebugString();
 if (!_footer.SerializeToString(&footer_buf)) {
 return Status::InternalError("failed to serialize segment footer");
 }
diff --git a/be/src/olap/rowset/segment_v2/variant_column_writer_impl.cpp 
b/be/src/olap/rowset/segment_v2/variant_column_writer_impl.cpp
index 6588e7dbe4f..33499a8e7e2 100644
--- a/be/src/olap/rowset/segment_v2/variant_column_writer_impl.cpp
+++ b/be/src/olap/rowset/segment_v2/variant_column_writer_impl.cpp
@@ -30,6 +30,7 @@
 #include "vec/columns/column_object.h"
 #include "vec/columns/columns_number.h"
 #include "vec/common/schema_util.h"
+#include "vec/json/path_in_data.h"
 #include "vec/olap/olap_data_convertor.h"
 
 namespace doris::segment_v2 {
@@ -47,12 +48,12 @@ Status VariantColumnWriterImpl::init() {
 if (dynamic_paths.empty()) {
 _column = vectorized::ColumnObject::create(true, false);
 } else {
-vectorized::ColumnObject::Subcolumns dynamic_subcolum

Re: [PR] [fix](serialize) fix column serialize and deserialize [doris]

2024-12-19 Thread via GitHub


eldenmoon merged PR #45667:
URL: https://github.com/apache/doris/pull/45667


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] branch-3.0: [opt](cloud) reduce cache hotspot table write amplification #45557 [doris]

2024-12-19 Thread via GitHub


doris-robot commented on PR #45677:
URL: https://github.com/apache/doris/pull/45677#issuecomment-2555126888

   
   
   TPC-DS: Total hot run time: 195466 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 708df297eb3ec20f8ef33f743d41df1154436c91, 
data reload: false
   
   query1   1271927 908 908
   query2   6235207320722072
   query3   10835   405041034050
   query4   65707   29474   23546   23546
   query5   5430445 439 439
   query6   472 173 178 173
   query7   5670310 310 310
   query8   316 231 230 230
   query9   9290268326722672
   query10  494 275 265 265
   query11  17708   15117   15640   15117
   query12  169 97  97  97
   query13  1558440 416 416
   query14  10809   709672977096
   query15  211 184 180 180
   query16  7342518 495 495
   query17  1170578 580 578
   query18  1880336 318 318
   query19  221 159 152 152
   query20  119 116 110 110
   query21  71  46  45  45
   query22  4724447643264326
   query23  34924   34031   33948   33948
   query24  6105288329872883
   query25  519 432 422 422
   query26  689 179 172 172
   query27  1782310 298 298
   query28  4566255725432543
   query29  699 467 458 458
   query30  247 166 179 166
   query31  1015816 816 816
   query32  70  54  60  54
   query33  449 291 303 291
   query34  927 512 512 512
   query35  815 759 760 759
   query36  1085934 933 933
   query37  118 73  72  72
   query38  4069400239573957
   query39  1506145114691451
   query40  141 79  78  78
   query41  49  45  45  45
   query42  112 96  96  96
   query43  514 486 488 486
   query44  1120792 792 792
   query45  184 168 173 168
   query46  1142715 716 715
   query47  1965191619041904
   query48  486 362 366 362
   query49  731 382 368 368
   query50  827 414 416 414
   query51  7129707270947072
   query52  96  90  85  85
   query53  272 185 183 183
   query54  556 453 439 439
   query55  79  72  72  72
   query56  263 227 243 227
   query57  1153106910921069
   query58  199 207 212 207
   query59  2973297628092809
   query60  270 253 250 250
   query61  109 108 108 108
   query62  757 646 660 646
   query63  214 196 188 188
   query64  1781667 626 626
   query65  3240315131593151
   query66  640 280 313 280
   query67  15663   15270   15260   15260
   query68  4459563 539 539
   query69  411 249 255 249
   query70  1154109311521093
   query71  393 251 268 251
   query72  6556399439293929
   query73  754 338 339 338
   query74  10228   891389098909
   query75  3334259526612595
   query76  2411100310901003
   query77  526 257 269 257
   query78  10749   972094849484
   query79  8225604 600 600
   query80  2138464 421 421
   query81  552 249 246 246
   query82  1321128 117 117
   query83  259 145 150 145
   query84  280 80  74  74
   query85  1731308 296 296
   query86  467 318 305 305
   query87  4505426342424242
   query88  5575236923882369
   query89  423 284 289 284
   query90  2113179 182 179
   query91  182 143 144 143
   query92  65  49  50  49
   query93  6674528 532 528
   query94  944 275 308 275
   query95  351 256 252 252
   query96  631 272 282 272
   query97  3280312131013101
   query98  217 199 206 199
   query99  1623129412921292
   Total cold run time: 336483 ms
   Total hot run time: 195466 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to t

(doris) branch revert-45667-seralize-variant created (now f47a7297142)

2024-12-19 Thread eldenmoon
This is an automated email from the ASF dual-hosted git repository.

eldenmoon pushed a change to branch revert-45667-seralize-variant
in repository https://gitbox.apache.org/repos/asf/doris.git


  at f47a7297142 Revert "[fix](serialize) fix column serialize and 
deserialize (#45667)"

This branch includes the following new commits:

 new f47a7297142 Revert "[fix](serialize) fix column serialize and 
deserialize (#45667)"

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] Revert "[fix](serialize) fix column serialize and deserialize" [doris]

2024-12-19 Thread via GitHub


eldenmoon opened a new pull request, #45683:
URL: https://github.com/apache/doris/pull/45683

   Reverts apache/doris#45667


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) 01/01: Revert "[fix](serialize) fix column serialize and deserialize (#45667)"

2024-12-19 Thread eldenmoon
This is an automated email from the ASF dual-hosted git repository.

eldenmoon pushed a commit to branch revert-45667-seralize-variant
in repository https://gitbox.apache.org/repos/asf/doris.git

commit f47a7297142b4ca2d7b959bb696a73bb4f1089b1
Author: lihangyu 
AuthorDate: Fri Dec 20 01:21:23 2024 +0800

Revert "[fix](serialize) fix column serialize and deserialize (#45667)"

This reverts commit c15122c043defa4c417663bf34f5e48d98dc2f42.
---
 be/src/vec/columns/column_object.cpp   | 13 +++---
 .../vec/data_types/serde/data_type_array_serde.cpp |  7 +++---
 .../vec/data_types/serde/data_type_array_serde.h   |  2 +-
 .../vec/data_types/serde/data_type_jsonb_serde.cpp |  5 ++--
 .../vec/data_types/serde/data_type_jsonb_serde.h   |  2 +-
 .../data_types/serde/data_type_nullable_serde.cpp  | 29 +++---
 .../data_types/serde/data_type_nullable_serde.h|  4 +--
 .../data_types/serde/data_type_number_serde.cpp|  4 ++-
 .../vec/data_types/serde/data_type_number_serde.h  |  2 +-
 be/src/vec/data_types/serde/data_type_serde.h  |  2 +-
 .../vec/data_types/serde/data_type_string_serde.h  |  4 ++-
 11 files changed, 36 insertions(+), 38 deletions(-)

diff --git a/be/src/vec/columns/column_object.cpp 
b/be/src/vec/columns/column_object.cpp
index 17c02172da8..91a0936673f 100644
--- a/be/src/vec/columns/column_object.cpp
+++ b/be/src/vec/columns/column_object.cpp
@@ -1053,10 +1053,8 @@ void 
ColumnObject::Subcolumn::serialize_to_sparse_column(ColumnString* key, std:
 auto& nullable_col = assert_cast(*part);
 
 // insert value
-ColumnString::Chars& chars = value->get_chars();
 nullable_serde->get_nested_serde()->write_one_cell_to_binary(
-nullable_col.get_nested_column(), chars, row);
-value->get_offsets().push_back(chars.size());
+nullable_col.get_nested_column(), value, row);
 }
 return;
 }
@@ -1116,11 +1114,6 @@ const char* parse_binary_from_sparse_column(TypeIndex 
type, const char* data, Fi
 end = data + size;
 break;
 }
-case TypeIndex::Nothing: {
-res = Null();
-end = data;
-break;
-}
 case TypeIndex::Array: {
 const size_t size = *reinterpret_cast(data);
 data += sizeof(size_t);
@@ -1130,9 +1123,9 @@ const char* parse_binary_from_sparse_column(TypeIndex 
type, const char* data, Fi
 for (size_t i = 0; i < size; ++i) {
 Field nested_field;
 const auto nested_type =
-static_cast(*reinterpret_cast(data++));
+assert_cast(*reinterpret_cast(data++));
 data = parse_binary_from_sparse_column(nested_type, data, 
nested_field, info_res);
-array[i] = std::move(nested_field);
+array.emplace_back(std::move(nested_field));
 }
 end = data;
 break;
diff --git a/be/src/vec/data_types/serde/data_type_array_serde.cpp 
b/be/src/vec/data_types/serde/data_type_array_serde.cpp
index 2c906ce4c49..57a43fbb381 100644
--- a/be/src/vec/data_types/serde/data_type_array_serde.cpp
+++ b/be/src/vec/data_types/serde/data_type_array_serde.cpp
@@ -436,9 +436,9 @@ Status DataTypeArraySerDe::read_column_from_pb(IColumn& 
column, const PValues& a
 }
 
 void DataTypeArraySerDe::write_one_cell_to_binary(const IColumn& src_column,
-  ColumnString::Chars& chars,
-  int64_t row_num) const {
+  ColumnString* dst_column, 
int64_t row_num) const {
 const uint8_t type = static_cast(TypeIndex::Array);
+ColumnString::Chars& chars = dst_column->get_chars();
 const size_t old_size = chars.size();
 const size_t new_size = old_size + sizeof(uint8_t) + sizeof(size_t);
 chars.resize(new_size);
@@ -453,8 +453,9 @@ void DataTypeArraySerDe::write_one_cell_to_binary(const 
IColumn& src_column,
 memcpy(chars.data() + old_size + sizeof(uint8_t), reinterpret_cast(&size),
sizeof(size_t));
 for (size_t offset = start; offset != end; ++offset) {
-nested_serde->write_one_cell_to_binary(nested_column, chars, offset);
+nested_serde->write_one_cell_to_binary(nested_column, dst_column, 
offset);
 }
+dst_column->get_offsets().push_back(chars.size());
 }
 
 } // namespace vectorized
diff --git a/be/src/vec/data_types/serde/data_type_array_serde.h 
b/be/src/vec/data_types/serde/data_type_array_serde.h
index 25da83f2cff..aaf1a425512 100644
--- a/be/src/vec/data_types/serde/data_type_array_serde.h
+++ b/be/src/vec/data_types/serde/data_type_array_serde.h
@@ -101,7 +101,7 @@ public:
 nested_serde->set_return_object_as_string(value);
 }
 
-void write_one_cell_to_binary(const IColumn& src_column, 
ColumnString::Chars& chars,
+void write_one_cell_to_binary(const

Re: [PR] [test](p0) force replica = 1 in p0 regression tests [doris]

2024-12-19 Thread via GitHub


hello-stephen commented on PR #45682:
URL: https://github.com/apache/doris/pull/45682#issuecomment-2555222102

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting 
information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it 
now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and 
after the optimization?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] Revert "[fix](serialize) fix column serialize and deserialize" [doris]

2024-12-19 Thread via GitHub


hello-stephen commented on PR #45683:
URL: https://github.com/apache/doris/pull/45683#issuecomment-2555233314

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting 
information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it 
now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and 
after the optimization?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [test](p0) force replica = 1 in p0 regression tests [doris]

2024-12-19 Thread via GitHub


kaijchen commented on PR #45682:
URL: https://github.com/apache/doris/pull/45682#issuecomment-2555236280

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [test](p0) force replica = 1 in p0 regression tests [doris]

2024-12-19 Thread via GitHub


kaijchen commented on PR #45684:
URL: https://github.com/apache/doris/pull/45684#issuecomment-2555240079

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [test](p0) force replica = 1 in p0 regression tests [doris]

2024-12-19 Thread via GitHub


hello-stephen commented on PR #45684:
URL: https://github.com/apache/doris/pull/45684#issuecomment-2555236241

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting 
information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it 
now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and 
after the optimization?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [test](p0) force replica = 1 in p0 regression tests [doris]

2024-12-19 Thread via GitHub


kaijchen opened a new pull request, #45684:
URL: https://github.com/apache/doris/pull/45684

   ### What problem does this PR solve?
   
   Some tests in p0 does not set replica = 1, and will fail when BE number is 
not enough.
   
   This PR fixed this problem by set `force_olap_table_replication_allocation` 
in p0 fe.conf
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test 
   - [ ] Regression test
   - [ ] Unit Test
   - [ ] Manual test (add detailed scripts or steps below)
   - [ ] No need to test or manual test. Explain why:
   - [ ] This is a refactor/code format and no logic has been changed.
   - [ ] Previous test can cover this change.
   - [ ] No code files have been changed.
   - [ ] Other reason 
   
   - Behavior changed:
   - [ ] No.
   - [ ] Yes. 
   
   - Does this need documentation?
   - [ ] No.
   - [ ] Yes. 
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] Revert "[fix](serialize) fix column serialize and deserialize" [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45683:
URL: https://github.com/apache/doris/pull/45683#issuecomment-2555265319

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [enhancement](tablet-meta) Avoid be coredump due to potential race condition when updating tablet cumu point [doris]

2024-12-19 Thread via GitHub


TangSiyang2001 opened a new pull request, #45643:
URL: https://github.com/apache/doris/pull/45643

   
   ### What problem does this PR solve?
   
   Issue Number: close #xxx
   
   Related PR: #xxx
   
   Problem Summary:
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test 
   - [ ] Regression test
   - [ ] Unit Test
   - [ ] Manual test (add detailed scripts or steps below)
   - [x] No need to test or manual test. Explain why:
   - [ ] This is a refactor/code format and no logic has been changed.
   - [x] Previous test can cover this change.
   - [ ] No code files have been changed.
   - [ ] Other reason 
   
   - Behavior changed:
   - [x] No.
   - [ ] Yes. 
   
   - Does this need documentation?
   - [x] No.
   - [ ] Yes. 
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhancement](tablet-meta) Avoid be coredump due to potential race condition when updating tablet cumu point [doris]

2024-12-19 Thread via GitHub


hello-stephen commented on PR #45643:
URL: https://github.com/apache/doris/pull/45643#issuecomment-2553168216

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting 
information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it 
now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and 
after the optimization?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated (ea6958cb40e -> 549abf4a56e)

2024-12-19 Thread kirs
This is an automated email from the ASF dual-hosted git repository.

kirs pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


from ea6958cb40e [Enhancement](nereids)implement showStatusCommand in 
nereids (#45427)
 add 549abf4a56e [Fix](Job)Fix redundant job scheduling by preventing same 
state transitions (e.g., RUNNING → RUNNING) (#45495)

No new revisions were added by this update.

Summary of changes:
 .../src/main/java/org/apache/doris/job/manager/JobManager.java   | 6 +-
 regression-test/suites/job_p0/test_base_insert_job.groovy| 9 ++---
 2 files changed, 11 insertions(+), 4 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](Job)Fix redundant job scheduling by preventing same state transitions (e.g., RUNNING → RUNNING) [doris]

2024-12-19 Thread via GitHub


CalvinKirs merged PR #45495:
URL: https://github.com/apache/doris/pull/45495


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhancement](tablet-meta) Avoid be coredump due to potential race condition when updating tablet cumu point [doris]

2024-12-19 Thread via GitHub


TangSiyang2001 commented on PR #45643:
URL: https://github.com/apache/doris/pull/45643#issuecomment-2553202866

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](binlog) Avoid clear binlog dir #45581 [doris]

2024-12-19 Thread via GitHub


w41ter closed pull request #45620: [fix](binlog) Avoid clear binlog dir #45581
URL: https://github.com/apache/doris/pull/45620


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](binlog) Avoid clear binlog dir #45581 [doris]

2024-12-19 Thread via GitHub


w41ter commented on PR #45620:
URL: https://github.com/apache/doris/pull/45620#issuecomment-2553234454

   run feut


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](binlog) Avoid clear binlog dir #45581 [doris]

2024-12-19 Thread via GitHub


w41ter commented on PR #45620:
URL: https://github.com/apache/doris/pull/45620#issuecomment-2553234957

   run beut


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](test) Make the case stable. #44195 [doris]

2024-12-19 Thread via GitHub


yiguolei merged PR #45638:
URL: https://github.com/apache/doris/pull/45638


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch branch-2.1 updated: [opt](test) Make the case stable. #44195 (#45638)

2024-12-19 Thread yiguolei
This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-2.1 by this push:
 new 299bb8735cb [opt](test) Make the case stable. #44195 (#45638)
299bb8735cb is described below

commit 299bb8735cb2e9ea8ff813d6cc28e9a684b0c602
Author: Sun Chenyang 
AuthorDate: Thu Dec 19 17:44:45 2024 +0800

[opt](test) Make the case stable. #44195 (#45638)
---
 .../test_single_compaction_with_variant_inverted_index.groovy  |  2 +-
 .../suites/fault_injection_p0/test_variant_bloom_filter.groovy | 10 ++
 .../inverted_index_p2/test_variant_index_format_v1.groovy  |  2 +-
 regression-test/suites/load_p2/test_single_replica_load.groovy |  2 +-
 regression-test/suites/variant_p0/with_index/var_index.groovy  |  3 ++-
 5 files changed, 11 insertions(+), 8 deletions(-)

diff --git 
a/regression-test/suites/compaction/test_single_compaction_with_variant_inverted_index.groovy
 
b/regression-test/suites/compaction/test_single_compaction_with_variant_inverted_index.groovy
index fce01e48ddd..49a90af8e43 100644
--- 
a/regression-test/suites/compaction/test_single_compaction_with_variant_inverted_index.groovy
+++ 
b/regression-test/suites/compaction/test_single_compaction_with_variant_inverted_index.groovy
@@ -17,7 +17,7 @@
 
 import org.codehaus.groovy.runtime.IOGroovyMethods
 
-suite("test_single_compaction_with_variant_inverted", "p2") {
+suite("test_single_compaction_with_variant_inverted", "p2, nonConcurrent") {
 def tableName = "test_single_compaction_with_variant_inverted"
 
 def calc_file_crc_on_tablet = { ip, port, tablet ->
diff --git 
a/regression-test/suites/fault_injection_p0/test_variant_bloom_filter.groovy 
b/regression-test/suites/fault_injection_p0/test_variant_bloom_filter.groovy
index 33f93d1bef0..88c529d685d 100644
--- a/regression-test/suites/fault_injection_p0/test_variant_bloom_filter.groovy
+++ b/regression-test/suites/fault_injection_p0/test_variant_bloom_filter.groovy
@@ -72,8 +72,9 @@ suite("test_variant_bloom_filter", "nonConcurrent") {
 getBackendIpHttpPort(backendId_to_backendIP, backendId_to_backendHttpPort);
 def tablets = sql_return_maparray """ show tablets from ${index_table}; """
 
-int beforeSegmentCount = 0
+
 for (def tablet in tablets) {
+int beforeSegmentCount = 0
 String tablet_id = tablet.TabletId
 (code, out, err) = curl("GET", tablet.CompactionStatus)
 logger.info("Show tablets status: code=" + code + ", out=" + out + ", 
err=" + err)
@@ -83,8 +84,8 @@ suite("test_variant_bloom_filter", "nonConcurrent") {
 for (String rowset in (List) tabletJson.rowsets) {
 beforeSegmentCount += Integer.parseInt(rowset.split(" ")[1])
 }
+assertEquals(beforeSegmentCount, 5)
 }
-assertEquals(beforeSegmentCount, 5)
 
 // trigger compactions for all tablets in ${tableName}
 for (def tablet in tablets) {
@@ -111,8 +112,8 @@ suite("test_variant_bloom_filter", "nonConcurrent") {
 });
 }
 
-int afterSegmentCount = 0
 for (def tablet in tablets) {
+int afterSegmentCount = 0
 String tablet_id = tablet.TabletId
 (code, out, err) = curl("GET", tablet.CompactionStatus)
 logger.info("Show tablets status: code=" + code + ", out=" + out + ", 
err=" + err)
@@ -123,8 +124,9 @@ suite("test_variant_bloom_filter", "nonConcurrent") {
 logger.info("rowset is: " + rowset)
 afterSegmentCount += Integer.parseInt(rowset.split(" ")[1])
 }
+ assertEquals(afterSegmentCount, 1)
 }
-assertEquals(afterSegmentCount, 1)
+   
 
 try {
 
GetDebugPoint().enableDebugPointForAllBEs("bloom_filter_must_filter_data")
diff --git 
a/regression-test/suites/inverted_index_p2/test_variant_index_format_v1.groovy 
b/regression-test/suites/inverted_index_p2/test_variant_index_format_v1.groovy
index 5f262c8bf07..2bf1d20b106 100644
--- 
a/regression-test/suites/inverted_index_p2/test_variant_index_format_v1.groovy
+++ 
b/regression-test/suites/inverted_index_p2/test_variant_index_format_v1.groovy
@@ -15,7 +15,7 @@
 // specific language governing permissions and limitations
 // under the License.
 
-suite("test_variant_index_format_v1", "p2") {
+suite("test_variant_index_format_v1", "p2, nonConcurrent") {
 def calc_file_crc_on_tablet = { ip, port, tablet ->
 return curl("GET", 
String.format("http://%s:%s/api/calc_crc?tablet_id=%s";, ip, port, tablet))
 }
diff --git a/regression-test/suites/load_p2/test_single_replica_load.groovy 
b/regression-test/suites/load_p2/test_single_replica_load.groovy
index cb4f970ac66..67f0b4639f8 100644
--- a/regression-test/suites/load_p2/test_single_replica_load.groovy
+++ b/regression-test/suites/load_p2/test_single_replica_load.groovy
@@ -19,7 +19,7 @@
 // /testing/trino-product-tests/src/main/resources/sql-

Re: [PR] [fix](create table) fix create table fail msg #45623 [doris]

2024-12-19 Thread via GitHub


yujun777 commented on PR #45651:
URL: https://github.com/apache/doris/pull/45651#issuecomment-2553229193

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](create table) fix create table fail msg #45623 [doris]

2024-12-19 Thread via GitHub


yujun777 opened a new pull request, #45651:
URL: https://github.com/apache/doris/pull/45651

   cherry pick from #45623


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](create table) fix create table fail msg #45623 [doris]

2024-12-19 Thread via GitHub


Thearas commented on PR #45651:
URL: https://github.com/apache/doris/pull/45651#issuecomment-2553229456

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting 
information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it 
now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and 
after the optimization?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] branch-3.0: [fix](nereids) fix bug when sum0 distinct appear in cte #45447 [doris]

2024-12-19 Thread via GitHub


feiniaofeiafei commented on PR #45510:
URL: https://github.com/apache/doris/pull/45510#issuecomment-2553176161

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Enhancement](nereids)implement showStatusCommand in nereids [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45427:
URL: https://github.com/apache/doris/pull/45427#issuecomment-2553176424

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Enhancement] (nereids)implement alterCatalogRenameCommand in nereids [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45163:
URL: https://github.com/apache/doris/pull/45163#issuecomment-2553176679

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Enhancement] (nereids)implement alterCatalogRenameCommand in nereids [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45163:
URL: https://github.com/apache/doris/pull/45163#issuecomment-2553176757

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](csv reader) fix core dump when parsing csv with enclose [doris]

2024-12-19 Thread via GitHub


sollhui commented on PR #45485:
URL: https://github.com/apache/doris/pull/45485#issuecomment-2553174374

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](Nereids) lock table in ascending order of table IDs [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45045:
URL: https://github.com/apache/doris/pull/45045#issuecomment-2553173445

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhancement](tablet-meta) Avoid be coredump due to potential race condition when updating tablet cumu point [doris]

2024-12-19 Thread via GitHub


github-actions[bot] commented on PR #45643:
URL: https://github.com/apache/doris/pull/45643#issuecomment-2553177979

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



  1   2   3   4   5   6   7   8   9   10   >