Re: [PR] branch-3.0: [fix](cloud) fix abort transaction in runningTxns list when show routine load #45629 [doris]
dataroaring merged PR #45721: URL: https://github.com/apache/doris/pull/45721 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet complex types. [doris]
github-actions[bot] commented on PR #44098: URL: https://github.com/apache/doris/pull/44098#issuecomment-2562060036 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](jdbc catalog) add some jdbc query tvf test [doris]
morningman merged PR #45938: URL: https://github.com/apache/doris/pull/45938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [test](jdbc catalog) add some jdbc query tvf test #45938 [doris]
hello-stephen commented on PR #45973: URL: https://github.com/apache/doris/pull/45973#issuecomment-2562072822 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45938-branch-3.0 updated (9f2097c882e -> 2c4b4268845)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45938-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git from 9f2097c882e branch-3.0: [fix](cloud) fix abort transaction in runningTxns list when show routine load #45629 (#45721) add 2c4b4268845 [test](jdbc catalog) add some jdbc query tvf test (#45938) No new revisions were added by this update. Summary of changes: .../jdbc/client/JdbcPostgreSQLClient.java | 4 ++ .../datasource/jdbc/util/JdbcFieldSchema.java | 1 + .../jdbc/test_db2_jdbc_catalog.out | 50 +- .../jdbc/test_oracle_jdbc_catalog.out | 4 ++ .../jdbc/test_pg_jdbc_catalog.out | 4 ++ .../jdbc/test_sqlserver_jdbc_catalog.out | 8 ++-- .../jdbc/test_db2_jdbc_catalog.groovy | 6 ++- .../jdbc/test_oracle_jdbc_catalog.groovy | 2 + .../jdbc/test_pg_jdbc_catalog.groovy | 2 + .../jdbc/test_sqlserver_jdbc_catalog.groovy| 1 + 10 files changed, 56 insertions(+), 26 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [improvement](http) improve table schema api for catalog table #45933 [doris]
Thearas commented on PR #45982: URL: https://github.com/apache/doris/pull/45982#issuecomment-2562102950 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [improvement](http) improve table schema api for catalog table #45933 [doris]
Thearas commented on PR #45981: URL: https://github.com/apache/doris/pull/45981#issuecomment-2562102611 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45933-branch-2.1 created (now 879bb796556)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45933-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git at 879bb796556 [regression-test](fix) fix two case bug (#45974) No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45933-branch-2.1 updated (879bb796556 -> 2378fb141f2)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45933-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git from 879bb796556 [regression-test](fix) fix two case bug (#45974) add 2378fb141f2 [improvement](http) improve table schema api for catalog table (#45933) No new revisions were added by this update. Summary of changes: .../doris/httpv2/controller/BaseController.java| 10 +++- .../doris/httpv2/rest/RestBaseController.java | 1 + .../doris/httpv2/rest/TableSchemaAction.java | 39 ++ .../http_rest_api/get/test_schema_api.groovy | 60 ++ 4 files changed, 98 insertions(+), 12 deletions(-) create mode 100644 regression-test/suites/http_rest_api/get/test_schema_api.groovy - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [improvement](http) improve table schema api for catalog table #45933 [doris]
dataroaring closed pull request #45982: branch-2.1: [improvement](http) improve table schema api for catalog table #45933 URL: https://github.com/apache/doris/pull/45982 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [improvement](http) improve table schema api for catalog table #45933 [doris]
dataroaring closed pull request #45981: branch-3.0: [improvement](http) improve table schema api for catalog table #45933 URL: https://github.com/apache/doris/pull/45981 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [improvement](http) improve table schema api for catalog table #45933 [doris]
Thearas commented on PR #45982: URL: https://github.com/apache/doris/pull/45982#issuecomment-2562102971 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] branch-2.1: [improvement](http) improve table schema api for catalog table #45933 [doris]
github-actions[bot] opened a new pull request, #45982: URL: https://github.com/apache/doris/pull/45982 Cherry-picked from #45933 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](nereids) add rule MultiDistinctSplit [doris]
github-actions[bot] commented on PR #45209: URL: https://github.com/apache/doris/pull/45209#issuecomment-2562092570 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45806-branch-2.1 updated (879bb796556 -> 730c50c72b7)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45806-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git from 879bb796556 [regression-test](fix) fix two case bug (#45974) add 730c50c72b7 [fix](mem leak) fe non_heap mem leak while use jdbc catalog (#45806) No new revisions were added by this update. Summary of changes: .../apache/doris/datasource/jdbc/client/JdbcClient.java| 14 +++--- 1 file changed, 11 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [fix](mem leak) fe non_heap mem leak while use jdbc catalog #45806 [doris]
Thearas commented on PR #45980: URL: https://github.com/apache/doris/pull/45980#issuecomment-2562099033 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [fix](meta-mgr) Real-time parsing meta service endpoint to avoid rpc failed after config muted #45877 [doris]
dataroaring merged PR #45899: URL: https://github.com/apache/doris/pull/45899 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45877-branch-3.0 deleted (was 3d42f9b9a1f)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45877-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git was 3d42f9b9a1f [fix](meta-mgr) Real-time parsing meta service endpoint to avoid rpc failed after config muted (#45877) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-3.0 updated (9f2097c882e -> 4ea5b16105e)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git from 9f2097c882e branch-3.0: [fix](cloud) fix abort transaction in runningTxns list when show routine load #45629 (#45721) add 4ea5b16105e branch-3.0: [fix](meta-mgr) Real-time parsing meta service endpoint to avoid rpc failed after config muted #45877 (#45899) No new revisions were added by this update. Summary of changes: be/src/cloud/cloud_meta_mgr.cpp | 17 +++-- 1 file changed, 7 insertions(+), 10 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Iceberg)Fix HDFS FileSystem Leak Caused by Frequent Refresh of Iceberg Catalog [doris]
doris-robot commented on PR #45956: URL: https://github.com/apache/doris/pull/45956#issuecomment-2562106346 TPC-H: Total hot run time: 32859 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 3ec2f54c9aab694b4a0d571e19962ac26373ba54, data reload: false -- Round 1 -- q1 18050 624961336133 q2 2047311 170 170 q3 10744 1222759 759 q4 10620 871 429 429 q5 7587219319891989 q6 215 189 154 154 q7 929 787 609 609 q8 9264141911951195 q9 6218509251615092 q10 6939233918691869 q11 480 290 254 254 q12 352 364 219 219 q13 18036 365130083008 q14 244 245 216 216 q15 575 515 514 514 q16 641 615 590 590 q17 566 853 322 322 q18 6873644363426342 q19 2170979 569 569 q20 298 321 183 183 q21 2791215619361936 q22 359 335 307 307 Total cold run time: 105998 ms Total hot run time: 32859 ms - Round 2, with runtime_filter_mode=off - q1 6320622162326221 q2 230 325 233 233 q3 2221265023052305 q4 1381183013281328 q5 4347478248924782 q6 178 179 142 142 q7 2129200118371837 q8 2598276327262726 q9 7318728573507285 q10 3049335827342734 q11 593 513 526 513 q12 679 773 592 592 q13 3443377731353135 q14 305 300 294 294 q15 579 498 505 498 q16 666 671 661 661 q17 1186171412421242 q18 7640750173957395 q19 799 116310651065 q20 1990202318951895 q21 5706510949714971 q22 607 608 615 608 Total cold run time: 53964 ms Total hot run time: 52462 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression-test](fix) forbid test_partial_update_delete_sign_with_conflict in cloud mode. [doris]
hello-stephen commented on PR #45983: URL: https://github.com/apache/doris/pull/45983#issuecomment-2562106234 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [regression-test](fix) forbid test_partial_update_delete_sign_with_conflict in cloud mode. [doris]
shuke987 opened a new pull request, #45983: URL: https://github.com/apache/doris/pull/45983 ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: test_partial_update_delete_sign_with_conflict is not suitable for cloud mode. ### Release note None ### Check List (For Author) - Test - [x] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [x] No. - [ ] Yes. - Does this need documentation? - [x] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression-test](fix) forbid test_partial_update_delete_sign_with_conflict in cloud mode. [doris]
shuke987 commented on PR #45983: URL: https://github.com/apache/doris/pull/45983#issuecomment-2562106440 skip buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](nerieds) date_format, to_monday, last_day, to_date, year implement Monotonic [doris]
github-actions[bot] commented on PR #44789: URL: https://github.com/apache/doris/pull/44789#issuecomment-2562092958 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [test](jdbc catalog) add some jdbc query tvf test #45938 [doris]
doris-robot commented on PR #45973: URL: https://github.com/apache/doris/pull/45973#issuecomment-2562092625 TPC-H: Total hot run time: 40978 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 2c4b42688452b919fd8eab9ba7954fd51b2d678f, data reload: false -- Round 1 -- q1 17577 749473167316 q2 2072175 170 170 q3 10558 106212261062 q4 10577 793 746 746 q5 7759283127312731 q6 237 147 142 142 q7 972 625 615 615 q8 9353194720071947 q9 6568639263956392 q10 6971228122972281 q11 460 264 272 264 q12 406 223 222 222 q13 17787 299930392999 q14 232 212 208 208 q15 549 513 532 513 q16 694 627 613 613 q17 965 570 592 570 q18 7230669367516693 q19 1436108310511051 q20 488 209 203 203 q21 4020348632503250 q22 10821013990 990 Total cold run time: 107993 ms Total hot run time: 40978 ms - Round 2, with runtime_filter_mode=off - q1 7227721372197213 q2 322 231 235 231 q3 2929290829562908 q4 2000183117821782 q5 5703577157265726 q6 221 141 141 141 q7 2284176718681767 q8 3325358235173517 q9 8858883788488837 q10 3581354435293529 q11 607 495 492 492 q12 807 643 654 643 q13 10650 318031643164 q14 299 268 273 268 q15 577 527 522 522 q16 715 674 680 674 q17 1868162316001600 q18 8272769376687668 q19 1649165216741652 q20 2093187418871874 q21 5551541453655365 q22 1135108010461046 Total cold run time: 70673 ms Total hot run time: 60619 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](nereids) add rule MultiDistinctSplit [doris]
github-actions[bot] commented on PR #45209: URL: https://github.com/apache/doris/pull/45209#issuecomment-2562092585 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] Workload management [doris-website]
wangtianyi2004 opened a new pull request, #1621: URL: https://github.com/apache/doris-website/pull/1621 ## Versions - [ ] dev - [ ] 3.0 - [ ] 2.1 - [ ] 2.0 ## Languages - [ ] Chinese - [ ] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [fix](nereids) fix compare with date like overflow #45868 [doris]
yujun777 commented on PR #45929: URL: https://github.com/apache/doris/pull/45929#issuecomment-2562091785 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (b6a48f22397 -> 5c978567eca)
This is an automated email from the ASF dual-hosted git repository.
starocean999 pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
from b6a48f22397 [fix](regression test) fix stream load 2pc case fail
(#45919)
add 5c978567eca [fix](auth) fix use database stmt access unauthorized
catalog (#45720)
No new revisions were added by this update.
Summary of changes:
.../java/org/apache/doris/analysis/UseStmt.java| 9 ++-
.../nereids_commands/test_use_database_stmt.out} | 4 +-
.../nereids_commands/test_use_database_stmt.groovy | 76 ++
3 files changed, 82 insertions(+), 7 deletions(-)
copy regression-test/data/{correctness_p0/test_group_by_constant.out =>
external_table_p0/nereids_commands/test_use_database_stmt.out} (86%)
create mode 100644
regression-test/suites/external_table_p0/nereids_commands/test_use_database_stmt.groovy
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](auth) fix use database stmt access unauthorized catalog [doris]
starocean999 merged PR #45720: URL: https://github.com/apache/doris/pull/45720 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45720-branch-3.0 created (now 9f2097c882e)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45720-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git at 9f2097c882e branch-3.0: [fix](cloud) fix abort transaction in runningTxns list when show routine load #45629 (#45721) No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](json)just for test [doris]
hello-stephen commented on PR #45976: URL: https://github.com/apache/doris/pull/45976#issuecomment-2562095823 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [test](json)just for test [doris]
amorynan opened a new pull request, #45976: URL: https://github.com/apache/doris/pull/45976 ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [ ] No. - [ ] Yes. - Does this need documentation? - [ ] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] branch-3.0: [fix](auth) fix use database stmt access unauthorized catalog #45720 [doris]
github-actions[bot] opened a new pull request, #45977: URL: https://github.com/apache/doris/pull/45977 Cherry-picked from #45720 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [fix](auth) fix use database stmt access unauthorized catalog #45720 [doris]
hello-stephen commented on PR #45977: URL: https://github.com/apache/doris/pull/45977#issuecomment-2562097219 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [fix](mem leak) fe non_heap mem leak while use jdbc catalog #45806 [doris]
dataroaring closed pull request #45979: branch-3.0: [fix](mem leak) fe non_heap mem leak while use jdbc catalog #45806 URL: https://github.com/apache/doris/pull/45979 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch asf-site updated (a745b5684ae -> 929171ab98f)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/doris-website.git discard a745b5684ae Automated deployment with doris branch @ ca123a2e6f591a1e160c84c80dadfe111befe150 new 929171ab98f Automated deployment with doris branch @ ca123a2e6f591a1e160c84c80dadfe111befe150 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (a745b5684ae) \ N -- N -- N refs/heads/asf-site (929171ab98f) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/1.2/search-index.json | 2 +- docs/2.0/search-index.json | 2 +- docs/3.0/search-index.json | 2 +- docs/dev/search-index.json | 2 +- search-index.json| 2 +- zh-CN/docs/1.2/search-index.json | 2 +- zh-CN/docs/2.0/search-index.json | 2 +- zh-CN/docs/3.0/search-index.json | 2 +- zh-CN/docs/dev/search-index.json | 2 +- zh-CN/search-index.json | 2 +- 10 files changed, 10 insertions(+), 10 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45806-branch-2.1 created (now 879bb796556)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45806-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git at 879bb796556 [regression-test](fix) fix two case bug (#45974) No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](plugin) logstash: add retry queue without blocking tasks [doris]
xiaokang commented on code in PR #44999:
URL: https://github.com/apache/doris/pull/44999#discussion_r1897568480
##
extension/logstash/lib/logstash/outputs/doris.rb:
##
@@ -131,22 +132,35 @@ def register
end
end
+ # Run named Timer as daemon thread
+ @timer = java.util.Timer.new("Doris Output #{self.params['id']}", true)
+ # The queue in Timer is unbounded and uncontrollable, so use a new queue
to control the amount
+ @count_block_queue = java.util.concurrent.ArrayBlockingQueue.new(128)
Review Comment:
I think you should use LinkedBlockingQueue here since it's very offen to do
insert and delete.
##
extension/logstash/lib/logstash/outputs/doris.rb:
##
@@ -131,22 +132,35 @@ def register
end
end
+ # Run named Timer as daemon thread
+ @timer = java.util.Timer.new("Doris Output #{self.params['id']}", true)
+ # The queue in Timer is unbounded and uncontrollable, so use a new queue
to control the amount
+ @count_block_queue = java.util.concurrent.ArrayBlockingQueue.new(128)
+
+ @retry_queue = Queue.new
+ retry_thread = Thread.new do
+ while popped = @retry_queue.pop
+documents, http_headers, event_num, req_count = popped
+handle_request(documents, http_headers, event_num, req_count)
+ end
+ end
+
print_plugin_info()
end # def register
+ def close
+ @timer.cancel
+ end
+
def multi_receive(events)
return if events.empty?
send_events(events)
end
private
def send_events(events)
- documents = ""
- event_num = 0
- events.each do |event|
- documents << event_body(event) << "\n"
- event_num += 1
- end
+ documents = events.map { |event| event_body(event) }.join("\n")
Review Comment:
just refactor?
##
extension/logstash/lib/logstash/outputs/doris.rb:
##
@@ -157,50 +171,56 @@ def send_events(events)
http_headers["label"] = @label_prefix + "_" + @db + "_" + @table +
"_" + Time.now.strftime('%Y%m%d_%H%M%S_%L_' + SecureRandom.uuid)
end
- req_count = 0
- sleep_for = 1
- while true
- response = make_request(documents, http_headers, @http_query,
@http_hosts.sample)
-
- req_count += 1
- response_json = {}
- begin
-response_json = JSON.parse(response.body)
- rescue => e
-@logger.warn("doris stream load response: #{response} is not a
valid JSON")
- end
+ handle_request(documents, http_headers, event_num, 1)
+ end
+
+ def sleep_for_attempt(attempt)
+ sleep_for = attempt**2
+ sleep_for = sleep_for <= 60 ? sleep_for : 60
+ (sleep_for/2) + (rand(0..sleep_for)/2)
+ end
- status = response_json["Status"]
+ private
+ def handle_request(documents, http_headers, event_num, req_count)
+ response = make_request(documents, http_headers, @http_query,
@http_hosts.sample)
+ response_json = {}
+ begin
+ response_json = JSON.parse(response.body)
+ rescue => _
+ @logger.warn("doris stream load response: #{response} is not a valid
JSON")
+ end
- if status == 'Label Already Exists'
- @logger.warn("Label already exists: #{response_json['Label']}, skip
#{event_num} records.")
- break
- end
+ status = response_json["Status"]
- if status == "Success" || status == "Publish Timeout"
-@total_bytes.addAndGet(documents.size)
-@total_rows.addAndGet(event_num)
-break
- else
-@logger.warn("FAILED doris stream load response:\n#{response}")
-
-if @max_retries >= 0 && req_count > @max_retries
- @logger.warn("DROP this batch after failed #{req_count} times.")
- if @save_on_failure
- @logger.warn("Try save to disk.Disk file path :
#{@save_dir}/#{@table}_#{@save_file}")
- save_to_disk(documents)
- end
- break
-end
-
-# sleep and then retry
-sleep_for = sleep_for * 2
-sleep_for = sleep_for <= 60 ? sleep_for : 60
-sleep_rand = (sleep_for / 2) + (rand(0..sleep_for) / 2)
-@logger.warn("Will do retry #{req_count} after sleep #{sleep_rand}
secs.")
-sleep(sleep_rand)
+ if status == 'Label Already Exists'
+ @logger.warn("Label already exists: #{response_json['Label']}, skip
#{event_num} records.")
+ return
+ end
+
+ if status == "Success" || status == "Publish Timeout"
+ @total_bytes.addAndGet(documents.size)
+ @total_rows.addAndGet(event_num)
+ return
+ end
+
+ @logger.warn("FAILED doris stream load response:\n#{response}")
+ # if there are data quality issues, we do not retry
+ if (status == 'Fail' &&
response_json['Message'].sta
Re: [PR] branch-3.0: [fix](mem leak) fe non_heap mem leak while use jdbc catalog #45806 [doris]
Thearas commented on PR #45979: URL: https://github.com/apache/doris/pull/45979#issuecomment-2562098736 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] branch-2.1: [fix](mem leak) fe non_heap mem leak while use jdbc catalog #45806 [doris]
github-actions[bot] opened a new pull request, #45980: URL: https://github.com/apache/doris/pull/45980 Cherry-picked from #45806 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] branch-3.0: [fix](mem leak) fe non_heap mem leak while use jdbc catalog #45806 [doris]
github-actions[bot] opened a new pull request, #45979: URL: https://github.com/apache/doris/pull/45979 Cherry-picked from #45806 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [fix](mem leak) fe non_heap mem leak while use jdbc catalog #45806 [doris]
Thearas commented on PR #45980: URL: https://github.com/apache/doris/pull/45980#issuecomment-2562099055 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [fix](mem leak) fe non_heap mem leak while use jdbc catalog #45806 [doris]
Thearas commented on PR #45979: URL: https://github.com/apache/doris/pull/45979#issuecomment-2562098766 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45806-branch-3.0 updated (9f2097c882e -> 3da9accfc22)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45806-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git from 9f2097c882e branch-3.0: [fix](cloud) fix abort transaction in runningTxns list when show routine load #45629 (#45721) add 3da9accfc22 [fix](mem leak) fe non_heap mem leak while use jdbc catalog (#45806) No new revisions were added by this update. Summary of changes: .../apache/doris/datasource/jdbc/client/JdbcClient.java| 14 +++--- 1 file changed, 11 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45806-branch-3.0 created (now 9f2097c882e)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45806-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git at 9f2097c882e branch-3.0: [fix](cloud) fix abort transaction in runningTxns list when show routine load #45629 (#45721) No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] (test)[regression-test]: trigger compaction with table_name instead of tablets [doris]
py023 commented on PR #45957: URL: https://github.com/apache/doris/pull/45957#issuecomment-2562099761 run external -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [test](jdbc catalog) add some jdbc query tvf test #45938 [doris]
doris-robot commented on PR #45973: URL: https://github.com/apache/doris/pull/45973#issuecomment-2562099681 TPC-DS: Total hot run time: 197854 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 2c4b42688452b919fd8eab9ba7954fd51b2d678f, data reload: false query1 1271954 961 954 query2 6243208920262026 query3 10825 433242144214 query4 66101 29094 23735 23735 query5 4971470 449 449 query6 425 185 179 179 query7 5534313 315 313 query8 320 232 246 232 query9 8737271626812681 query10 436 268 257 257 query11 17159 15235 15765 15235 query12 160 103 99 99 query13 1503457 427 427 query14 10361 703474717034 query15 201 181 184 181 query16 7202484 487 484 query17 1093585 588 585 query18 1963331 334 331 query19 215 166 163 163 query20 120 112 114 112 query21 61 45 44 44 query22 4627456045634560 query23 34882 34252 34341 34252 query24 6559289029552890 query25 516 415 416 415 query26 657 169 169 169 query27 1842316 316 316 query28 4022250425032503 query29 710 453 427 427 query30 246 161 167 161 query31 970 809 833 809 query32 69 59 57 57 query33 416 289 287 287 query34 918 510 515 510 query35 851 736 770 736 query36 1101941 973 941 query37 120 72 73 72 query38 4088396240373962 query39 1497149714601460 query40 142 83 82 82 query41 48 52 45 45 query42 110 110 95 95 query43 530 500 482 482 query44 1181843 830 830 query45 192 175 170 170 query46 1156739 739 739 query47 2067196219301930 query48 482 396 374 374 query49 723 380 387 380 query50 847 432 435 432 query51 7298726573697265 query52 100 91 83 83 query53 253 182 178 178 query54 551 443 447 443 query55 79 70 74 70 query56 249 251 227 227 query57 1248109211241092 query58 202 191 205 191 query59 3176302228932893 query60 278 254 249 249 query61 113 123 106 106 query62 777 654 664 654 query63 206 188 195 188 query64 1410690 644 644 query65 3256319732143197 query66 702 303 301 301 query67 16071 15741 15793 15741 query68 4020586 574 574 query69 423 265 261 261 query70 1188113610631063 query71 360 271 255 255 query72 6481398640663986 query73 759 344 344 344 query74 10266 905790849057 query75 3364264126842641 query76 1842104810411041 query77 499 262 264 262 query78 10513 970495649564 query79 1212604 599 599 query80 828 420 418 418 query81 511 245 238 238 query82 1334119 119 119 query83 225 146 140 140 query84 287 85 78 78 query85 885 316 295 295 query86 335 297 308 297 query87 4399424444734244 query88 3235239523392339 query89 408 284 285 284 query90 1991185 185 185 query91 187 149 150 149 query92 72 48 47 47 query93 1440561 565 561 query94 779 301 290 290 query95 353 262 256 256 query96 611 282 282 282 query97 3357315331693153 query98 212 210 198 198 query99 1713129113091291 Total cold run time: 316528 ms Total hot run time: 197854 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t
Re: [PR] branch-2.1: [fix](mem leak) fe non_heap mem leak while use jdbc catalog #45806 [doris]
dataroaring closed pull request #45980: branch-2.1: [fix](mem leak) fe non_heap mem leak while use jdbc catalog #45806 URL: https://github.com/apache/doris/pull/45980 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] fix some typo error according to AI [doris-website]
yiguolei merged PR #1620: URL: https://github.com/apache/doris-website/pull/1620 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [fix](mem leak) fe non_heap mem leak while use jdbc catalog (#45806)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 529586c7781 [fix](mem leak) fe non_heap mem leak while use jdbc
catalog (#45806)
529586c7781 is described below
commit 529586c778114956658b8d7aca8b2b9bd17bd2f7
Author: camby
AuthorDate: Thu Dec 26 10:48:20 2024 +0800
[fix](mem leak) fe non_heap mem leak while use jdbc catalog (#45806)
### What problem does this PR solve?
Issue Number: close https://github.com/apache/doris/issues/45609
---
.../apache/doris/datasource/jdbc/client/JdbcClient.java| 14 +++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/client/JdbcClient.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/client/JdbcClient.java
index 51f55ea24ba..f139dde5af2 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/client/JdbcClient.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/client/JdbcClient.java
@@ -48,6 +48,7 @@ import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Consumer;
@Getter
@@ -56,6 +57,8 @@ public abstract class JdbcClient {
private static final int HTTP_TIMEOUT_MS = 1;
protected static final int JDBC_DATETIME_SCALE = 6;
+private static final Map classLoaderMap = new
ConcurrentHashMap<>();
+
private String catalogName;
protected String dbType;
protected String jdbcUser;
@@ -147,11 +150,16 @@ public abstract class JdbcClient {
}
}
-private void initializeClassLoader(JdbcClientConfig config) {
+private synchronized void initializeClassLoader(JdbcClientConfig config) {
try {
URL[] urls = {new
URL(JdbcResource.getFullDriverUrl(config.getDriverUrl()))};
-ClassLoader parent = getClass().getClassLoader();
-this.classLoader = URLClassLoader.newInstance(urls, parent);
+if (classLoaderMap.containsKey(urls[0])) {
+this.classLoader = classLoaderMap.get(urls[0]);
+} else {
+ClassLoader parent = getClass().getClassLoader();
+this.classLoader = URLClassLoader.newInstance(urls, parent);
+classLoaderMap.put(urls[0], this.classLoader);
+}
} catch (MalformedURLException e) {
throw new RuntimeException("Error loading JDBC driver.", e);
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud-mow) FE should release delete bitmap lock when calculating delete bitmap failed [doris]
zhannngchen commented on code in PR #45673: URL: https://github.com/apache/doris/pull/45673#discussion_r1897579669 ## fe/fe-core/src/main/java/org/apache/doris/cloud/transaction/CloudGlobalTransactionMgr.java: ## @@ -465,16 +482,14 @@ private Set getBaseTabletsFromTables(List tableList, List tableList, long transactionId, Review Comment: rename the method to commitTransactionWithoutLock -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mem leak) fe non_heap mem leak while use jdbc catalog [doris]
morningman merged PR #45806: URL: https://github.com/apache/doris/pull/45806 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] FE memory leak [doris]
morningman closed issue #45609: [Bug] FE memory leak URL: https://github.com/apache/doris/issues/45609 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](auth) fix use database stmt access unauthorized catalog [doris]
github-actions[bot] commented on PR #45720: URL: https://github.com/apache/doris/pull/45720#issuecomment-2562094370 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [fix](auth) fix use database stmt access unauthorized catalog #45720 [doris]
dataroaring closed pull request #45977: branch-3.0: [fix](auth) fix use database stmt access unauthorized catalog #45720 URL: https://github.com/apache/doris/pull/45977 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mem leak) fe non_heap mem leak while use jdbc catalog [doris]
github-actions[bot] commented on PR #45806: URL: https://github.com/apache/doris/pull/45806#issuecomment-2562097245 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45720-branch-3.0 updated (9f2097c882e -> 90bc4402c73)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch auto-pick-45720-branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
from 9f2097c882e branch-3.0: [fix](cloud) fix abort transaction in
runningTxns list when show routine load #45629 (#45721)
add 90bc4402c73 [fix](auth) fix use database stmt access unauthorized
catalog (#45720)
No new revisions were added by this update.
Summary of changes:
.../java/org/apache/doris/analysis/UseStmt.java| 9 ++-
.../nereids_commands/test_use_database_stmt.out} | 4 +-
.../nereids_commands/test_use_database_stmt.groovy | 76 ++
3 files changed, 82 insertions(+), 7 deletions(-)
copy regression-test/data/{correctness_p0/test_group_by_constant.out =>
external_table_p0/nereids_commands/test_use_database_stmt.out} (86%)
create mode 100644
regression-test/suites/external_table_p0/nereids_commands/test_use_database_stmt.groovy
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mem leak) fe non_heap mem leak while use jdbc catalog [doris]
github-actions[bot] commented on PR #45806: URL: https://github.com/apache/doris/pull/45806#issuecomment-2562097264 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [fix](auth) fix use database stmt access unauthorized catalog #45720 [doris]
Thearas commented on PR #45978: URL: https://github.com/apache/doris/pull/45978#issuecomment-2562097593 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45720-branch-2.1 updated (879bb796556 -> aeb1b3d28f0)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch auto-pick-45720-branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
from 879bb796556 [regression-test](fix) fix two case bug (#45974)
add aeb1b3d28f0 [fix](auth) fix use database stmt access unauthorized
catalog (#45720)
No new revisions were added by this update.
Summary of changes:
.../java/org/apache/doris/analysis/UseStmt.java| 9 ++-
.../nereids_commands/test_use_database_stmt.out} | 4 +-
.../nereids_commands/test_use_database_stmt.groovy | 76 ++
3 files changed, 82 insertions(+), 7 deletions(-)
copy regression-test/data/{correctness_p0/test_group_by_constant.out =>
external_table_p0/nereids_commands/test_use_database_stmt.out} (86%)
create mode 100644
regression-test/suites/external_table_p0/nereids_commands/test_use_database_stmt.groovy
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [fix](auth) fix use database stmt access unauthorized catalog #45720 [doris]
dataroaring closed pull request #45978: branch-2.1: [fix](auth) fix use database stmt access unauthorized catalog #45720 URL: https://github.com/apache/doris/pull/45978 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] branch-2.1: [fix](auth) fix use database stmt access unauthorized catalog #45720 [doris]
github-actions[bot] opened a new pull request, #45978: URL: https://github.com/apache/doris/pull/45978 Cherry-picked from #45720 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [fix](auth) fix use database stmt access unauthorized catalog #45720 [doris]
Thearas commented on PR #45978: URL: https://github.com/apache/doris/pull/45978#issuecomment-2562097631 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45720-branch-2.1 created (now 879bb796556)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45720-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git at 879bb796556 [regression-test](fix) fix two case bug (#45974) No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [fix](auth) fix use database stmt access unauthorized catalog #45720 [doris]
hello-stephen commented on PR #45977: URL: https://github.com/apache/doris/pull/45977#issuecomment-2562097251 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] Stream Load http接口偶尔报乱码问题 [doris]
helxm closed issue #45969: [Bug] Stream Load http接口偶尔报乱码问题 URL: https://github.com/apache/doris/issues/45969 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] Stream Load http接口偶尔报乱码问题 [doris]
helxm commented on issue #45969: URL: https://github.com/apache/doris/issues/45969#issuecomment-2562100388 已解决 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] FE memory leak [doris]
morningman closed issue #45609: [Bug] FE memory leak URL: https://github.com/apache/doris/issues/45609 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45933-branch-3.0 created (now 9f2097c882e)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45933-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git at 9f2097c882e branch-3.0: [fix](cloud) fix abort transaction in runningTxns list when show routine load #45629 (#45721) No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45933-branch-3.0 updated (9f2097c882e -> 6cea9dd32ef)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45933-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git from 9f2097c882e branch-3.0: [fix](cloud) fix abort transaction in runningTxns list when show routine load #45629 (#45721) add 6cea9dd32ef [improvement](http) improve table schema api for catalog table (#45933) No new revisions were added by this update. Summary of changes: .../doris/httpv2/controller/BaseController.java| 10 +++- .../doris/httpv2/rest/RestBaseController.java | 1 + .../doris/httpv2/rest/TableSchemaAction.java | 39 ++ .../http_rest_api/get/test_schema_api.groovy | 60 ++ 4 files changed, 98 insertions(+), 12 deletions(-) create mode 100644 regression-test/suites/http_rest_api/get/test_schema_api.groovy - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [improvement](http) improve table schema api for catalog table #45933 [doris]
Thearas commented on PR #45981: URL: https://github.com/apache/doris/pull/45981#issuecomment-2562102587 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [test](jdbc catalog) add some jdbc query tvf test #45938 [doris]
doris-robot commented on PR #45973: URL: https://github.com/apache/doris/pull/45973#issuecomment-2562102035 ClickBench: Total hot run time: 33.9 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 2c4b42688452b919fd8eab9ba7954fd51b2d678f, data reload: false query1 0.030.030.03 query2 0.070.040.02 query3 0.230.060.06 query4 1.630.110.10 query5 0.520.500.50 query6 1.140.730.73 query7 0.020.020.02 query8 0.040.040.03 query9 0.590.520.49 query10 0.550.560.55 query11 0.150.100.10 query12 0.140.120.12 query13 0.600.590.60 query14 3.062.922.93 query15 0.920.840.83 query16 0.380.370.37 query17 0.981.011.03 query18 0.240.220.22 query19 1.931.852.04 query20 0.020.010.00 query21 15.37 0.560.55 query22 3.372.182.10 query23 16.93 0.940.75 query24 3.481.841.71 query25 0.190.210.09 query26 0.510.140.15 query27 0.050.050.05 query28 9.351.121.08 query29 12.53 3.323.30 query30 0.250.060.06 query31 2.860.390.39 query32 3.260.450.47 query33 3.023.013.03 query34 17.04 4.414.44 query35 4.544.494.52 query36 0.690.500.48 query37 0.090.060.06 query38 0.040.030.04 query39 0.040.020.02 query40 0.160.120.13 query41 0.080.030.02 query42 0.040.020.02 query43 0.030.030.03 Total cold run time: 107.16 s Total hot run time: 33.9 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (529586c7781 -> e4e38f3fee8)
This is an automated email from the ASF dual-hosted git repository. diwu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 529586c7781 [fix](mem leak) fe non_heap mem leak while use jdbc catalog (#45806) add e4e38f3fee8 [improvement](http) improve table schema api for catalog table (#45933) No new revisions were added by this update. Summary of changes: .../doris/httpv2/controller/BaseController.java| 10 +++- .../doris/httpv2/rest/RestBaseController.java | 1 + .../doris/httpv2/rest/TableSchemaAction.java | 39 ++ .../http_rest_api/get/test_schema_api.groovy | 60 ++ 4 files changed, 98 insertions(+), 12 deletions(-) create mode 100644 regression-test/suites/http_rest_api/get/test_schema_api.groovy - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] branch-3.0: [improvement](http) improve table schema api for catalog table #45933 [doris]
github-actions[bot] opened a new pull request, #45981: URL: https://github.com/apache/doris/pull/45981 Cherry-picked from #45933 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](http) improve table schema api for catalog table [doris]
JNSimba merged PR #45933: URL: https://github.com/apache/doris/pull/45933 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](hms table)Some optimizations for hms external table [doris]
wuwenchi commented on PR #44909: URL: https://github.com/apache/doris/pull/44909#issuecomment-2562111324 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Iceberg)Fix HDFS FileSystem Leak Caused by Frequent Refresh of Iceberg Catalog [doris]
doris-robot commented on PR #45956: URL: https://github.com/apache/doris/pull/45956#issuecomment-2562111829 TPC-DS: Total hot run time: 196901 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 3ec2f54c9aab694b4a0d571e19962ac26373ba54, data reload: false query1 1293946 878 878 query2 6455225422282228 query3 10981 463547024635 query4 33031 23750 23681 23681 query5 4432623 491 491 query6 290 211 189 189 query7 4005486 304 304 query8 294 246 226 226 query9 9632275327492749 query10 474 299 253 253 query11 18111 15287 15070 15070 query12 148 110 104 104 query13 1558539 410 410 query14 9875742670157015 query15 279 205 182 182 query16 8088631 432 432 query17 1522748 594 594 query18 2192410 309 309 query19 192 193 162 162 query20 122 117 116 116 query21 210 132 109 109 query22 4709466945614561 query23 34559 33729 33512 33512 query24 6566231923252319 query25 503 451 426 426 query26 790 280 152 152 query27 2040485 334 334 query28 6052251425082508 query29 676 551 426 426 query30 208 182 147 147 query31 1015947 845 845 query32 94 59 63 59 query33 478 356 304 304 query34 764 865 509 509 query35 820 825 775 775 query36 10521101973 973 query37 113 107 77 77 query38 4302429643354296 query39 1599148414611461 query40 211 114 102 102 query41 46 44 45 44 query42 126 109 143 109 query43 535 529 501 501 query44 1336831 836 831 query45 180 170 171 170 query46 877 1055669 669 query47 1984198719531953 query48 377 406 330 330 query49 721 486 395 395 query50 651 678 418 418 query51 7339733571197119 query52 100 105 96 96 query53 249 272 193 193 query54 506 526 453 453 query55 81 78 80 78 query56 256 260 245 245 query57 1284122811481148 query58 229 230 226 226 query59 3203308730183018 query60 280 290 261 261 query61 109 109 120 109 query62 861 791 773 773 query63 231 201 198 198 query64 37191126759 759 query65 3300325632513251 query66 827 445 323 323 query67 16691 15919 15554 15554 query68 9853759 518 518 query69 483 290 258 258 query70 1238111611381116 query71 443 302 256 256 query72 6324382638483826 query73 661 748 364 364 query74 10329 902589878987 query75 4602316026702670 query76 47231197804 804 query77 991 374 272 272 query78 10132 10300 94159415 query79 2978905 604 604 query80 748 516 430 430 query81 486 270 225 225 query82 625 150 131 131 query83 205 157 144 144 query84 280 92 75 75 query85 778 417 300 300 query86 340 321 305 305 query87 4456447746774477 query88 3279227322492249 query89 448 329 291 291 query90 1933184 190 184 query91 135 136 103 103 query92 63 52 54 52 query93 1910915 534 534 query94 667 399 254 254 query95 328 263 255 255 query96 484 610 281 281 query97 2768281327132713 query98 220 201 197 197 query99 1698151614641464 Total cold run time: 300441 ms Total hot run time: 196901 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] branch-3.0: [fix](mem leak) fe non_heap mem leak while use jdbc catalog #45806 [doris]
doris-robot commented on PR #45979: URL: https://github.com/apache/doris/pull/45979#issuecomment-2562111941 TPC-H: Total hot run time: 40895 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 3da9accfc2246f12373c269949b9c5ab57c9b5b7, data reload: false -- Round 1 -- q1 18062 751473917391 q2 2629170 167 167 q3 12510 113111741131 q4 10665 782 753 753 q5 8018293327912791 q6 247 149 144 144 q7 980 629 612 612 q8 9682190819771908 q9 6638637364046373 q10 6980227523012275 q11 462 266 256 256 q12 393 214 210 210 q13 17761 298930192989 q14 238 209 206 206 q15 566 518 506 506 q16 676 620 619 619 q17 960 636 520 520 q18 7220666767026667 q19 1415105810401040 q20 470 207 201 201 q21 3962325931443144 q22 1098995 992 992 Total cold run time: 111632 ms Total hot run time: 40895 ms - Round 2, with runtime_filter_mode=off - q1 7361720172337201 q2 326 234 244 234 q3 3196304529192919 q4 2101182018061806 q5 5655568156515651 q6 217 137 136 136 q7 2256178417821782 q8 3306349334583458 q9 8797884487918791 q10 3585358235383538 q11 598 512 482 482 q12 809 587 575 575 q13 6053318530263026 q14 283 254 259 254 q15 563 503 522 503 q16 694 659 655 655 q17 1797158515491549 q18 7823748673077307 q19 1652159616051596 q20 2078178317831783 q21 5337519351295129 q22 11141019999 999 Total cold run time: 65601 ms Total hot run time: 59374 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](docs) refine query acceleration doc on 2.1&3.0 [doris-website]
morrySnow commented on code in PR #1613:
URL: https://github.com/apache/doris-website/pull/1613#discussion_r1897572845
##
i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/query-acceleration/materialized-view/sync-materialized-view.md:
##
@@ -558,3 +614,13 @@ show alter table materialized view from test_db;
```
如果查询结果显示`status`字段不是`FINISHED`，那么需要等待，直到状态变为`FINISHED`后，物化视图才会变得可用。
+
+
+
+## 附录
+
+### 1. 物化视图相关开关介绍
+
+| 开关 | 说明 |
+| --- | --- |
+| `SET enable_sync_mv_cost_based_rewrite = true;` |
是否使用基于结构信息的方式对同步物化视图进行透明改写，默认为 `true`。此属性自 3.0.0 版本起开始支持。 |
Review Comment:
这段需要删掉，2.1 没有这个开关
##
i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/query-acceleration/materialized-view/sync-materialized-view.md:
##
@@ -47,7 +47,7 @@ under the License.
- SELECT 列表中，不能包含自增列，不能包含常量，不能有重复表达式，也不支持窗口函数。
-- 如果 SELECT 列表包含聚合函数，则聚合函数必须是根表达式（不支持 `sum(a) + 1`，支持 `sum(a +
1)`），且聚合函数之后不能有其他非聚合函数表达式（例如，`SELECT x, sum(a)` 可以，而 `SELECT sum(a), x` 不行）。
+- 如果 SELECT 列表包含聚合函数，则聚合函数必须是根表达式（不支持 `sum(a) + 1`，支持 `sum(a +
1)`），且聚合函数之后不能有其他非聚合函数表达式（例如，`SELECT x, sum(a)` 可以，而 `SELECT sum(a)`, x 不行）。
Review Comment:
```suggestion
- 如果 SELECT 列表包含聚合函数，则聚合函数必须是根表达式（不支持 `sum(a) + 1`，支持 `sum(a +
1)`），且聚合函数之后不能有其他非聚合函数表达式（例如，`SELECT x, sum(a)` 可以，而 `SELECT sum(a), x` 不行）。
```
##
i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/query-acceleration/tuning/tuning-plan/transparent-rewriting-with-async-mv.md:
##
@@ -24,23 +24,22 @@ specific language governing permissions and limitations
under the License.
-->
-## 工作原理
+## 概述
-多表物化视图采用的是基于 SPJG（SELECT-PROJECT-JOIN-GROUP-BY）模式的透明改写算法。该算法能够分析查询 SQL
的结构信息，自动寻找合适的物化视图，并尝试进行透明改写，以利用最优的物化视图来表达查询
SQL。通过使用预计算的物化视图结果，可以显著提高查询性能，并降低计算成本。
+[多表物化视图](../../materialized-view/async-materialized-view/overview.md)采用的是基于
SPJG（SELECT-PROJECT-JOIN-GROUP-BY）模式的透明改写算法。该算法能够分析查询 SQL
的结构信息，自动寻找合适的物化视图，并尝试进行透明改写，以利用最优的物化视图来表达查询
SQL。通过使用预计算的物化视图结果，可以显著提高查询性能，并降低计算成本。
Review Comment:
```suggestion
[异步物化视图](../../materialized-view/async-materialized-view/overview.md)采用的是基于
SPJG（SELECT-PROJECT-JOIN-GROUP-BY）模式的透明改写算法。该算法能够分析查询 SQL
的结构信息，自动寻找合适的物化视图，并尝试进行透明改写，以利用最优的物化视图来表达查询
SQL。通过使用预计算的物化视图结果，可以显著提高查询性能，并降低计算成本。
```
##
i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/query-acceleration/tuning/tuning-plan/transparent-rewriting-with-async-mv.md:
##
@@ -118,108 +114,55 @@ l_partkey,
l_suppkey;
```
-### 3 查看物化视图元信息
+### 使用物化视图进行透明改写
```sql
-SELECT * FROM mv_infos("database"="tpch") WHERE Name="mv1";
+mysql> explain shape plan SELECT l_shipdate, SUM(o_totalprice) AS total_price
+-> FROM lineitem
+-> LEFT JOIN orders ON lineitem.l_orderkey = orders.o_orderkey AND
l_shipdate = o_orderdate
+-> WHERE l_partkey = 2 AND l_suppkey = 3
+-> GROUP BY l_shipdate;
++---+
+| Explain String(Nereids Planner) |
++---+
+| PhysicalResultSink|
+| --PhysicalDistribute[DistributionSpecGather] |
+| PhysicalProject |
+| --hashAgg[GLOBAL] |
+| PhysicalDistribute[DistributionSpecHash] |
+| --hashAgg[LOCAL] |
+| PhysicalProject |
+| --filter((mv1.l_partkey = 2) and (mv1.l_suppkey = 3)) |
+| PhysicalOlapScan[mv1] |
++---+
```
-### 4 刷新物化视图
-
-首先查看分区列表：
-
-```sql
-SHOW PARTITIONS FROM mv1;
-```
-
-然后刷新特定分区：
-
-```sql
-REFRESH MATERIALIZED VIEW mv1 PARTITIONS(p_20231017_20231018);
-```
-
-### 5 任务管理
-
-管理物化视图的作业，包括查看作业、暂停定时调度、恢复定时调度以及查看和取消任务。
-
-- 查看物化视图 Job
-
-```sql
-SELECT * FROM jobs("type"="mv") ORDER BY CreateTime;
-```
-
-- 暂停物化视图 Job 定时调度
-
-```sql
-PAUSE MATERIALIZED VIEW JOB ON mv1;
-```
-
-- 恢复物化视图 Job 定时调度
-
-```sql
-RESUME MATERIALIZED VIEW JOB ON mv1;
-```
-
-- 查看物化视图的 Task
-
-```sql
-SELECT * FROM tasks("type"="mv");
-```
-
-- 取消物化视图的 Task：假设 `realTaskId` 为 123
-
-```sql
-CANCEL MATERIALIZED VIEW TASK 123 ON mv1;
-```
-
-### 6 修改物化视图
+通过 explain shape plan 可见经过 mv1 透明改写后的计划已经命中 mv1。通过 explain 也可以查看当前计划经过 mv
改写的状态，包括是否命中以及命中的 mv 等信息，如下所示：
```sql
-ALTER MATERIALIZED VIEW mv1 SET("grace_period"="");
+| == MATERIALIZATIONS ==
|
+|
|
+| MaterializedView
|
+| MaterializedViewRewriteSuccessAndChose:
|
+|
(doris) branch master updated (7a332517474 -> 3cc11d59972)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
from 7a332517474 [enhance](nerieds) date_format, to_monday, last_day,
to_date, year implement Monotonic (#44789)
add 3cc11d59972 [opt](parquet-reader)Implement late materialization of
parquet complex types. (#44098)
No new revisions were added by this update.
Summary of changes:
be/src/vec/exec/format/parquet/parquet_common.cpp | 152 ++--
be/src/vec/exec/format/parquet/parquet_common.h| 156 +++-
.../exec/format/parquet/vparquet_column_reader.cpp | 170 +++-
.../exec/format/parquet/vparquet_column_reader.h | 60 +-
.../exec/format/parquet/vparquet_group_reader.cpp | 58 +-
.../exec/format/parquet/vparquet_group_reader.h| 6 +-
be/src/vec/exec/format/parquet/vparquet_reader.cpp | 3 +-
be/test/vec/exec/parquet/parquet_common_test.cpp | 457 ++
.../parquet_nested_type_cross_page_test.cpp| 179
be/test/vec/exec/parquet/parquet_thrift_test.cpp | 8 +-
.../parquet_nested_types/create_table.hql | 58 ++
.../multi_catalog/parquet_nested_types/data.tar.gz | Bin 0 -> 36976 bytes
.../data_gen_scripts/nested_cross_page_test1.py| 192
.../data_gen_scripts/nested_cross_page_test2.py| 287 ++
.../data_gen_scripts/nested_cross_page_test3.py| 196
.../run.sh | 0
.../hive/test_parquet_nested_types.out | 985 +
.../hive/test_parquet_nested_types.groovy | 209 +
18 files changed, 2953 insertions(+), 223 deletions(-)
create mode 100644 be/test/vec/exec/parquet/parquet_common_test.cpp
create mode 100644
be/test/vec/exec/parquet/parquet_nested_type_cross_page_test.cpp
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/parquet_nested_types/create_table.hql
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/parquet_nested_types/data.tar.gz
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/parquet_nested_types/data_gen_scripts/nested_cross_page_test1.py
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/parquet_nested_types/data_gen_scripts/nested_cross_page_test2.py
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/parquet_nested_types/data_gen_scripts/nested_cross_page_test3.py
copy
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/{hive_text_complex_type
=> parquet_nested_types}/run.sh (100%)
create mode 100644
regression-test/data/external_table_p0/hive/test_parquet_nested_types.out
create mode 100644
regression-test/suites/external_table_p0/hive/test_parquet_nested_types.groovy
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet complex types. [doris]
morningman merged PR #44098: URL: https://github.com/apache/doris/pull/44098 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] refine description of rg,wg,cg [doris-website]
yiguolei opened a new pull request, #1623: URL: https://github.com/apache/doris-website/pull/1623 ## Versions - [ ] dev - [ ] 3.0 - [ ] 2.1 - [ ] 2.0 ## Languages - [ ] Chinese - [ ] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](statistics)Async drop table stats while doing truncate and schema change. [doris]
github-actions[bot] commented on PR #45923: URL: https://github.com/apache/doris/pull/45923#issuecomment-2562113896 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-44098-branch-3.0 created (now 44cbeaca577)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-44098-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git at 44cbeaca577 branch-3.0: [enhancement](meta) Sync tablet meta even if local state is not running #45821 (#45962) No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](profile) add summary metric for file scanner [doris]
github-actions[bot] commented on PR #45941: URL: https://github.com/apache/doris/pull/45941#issuecomment-2562114032 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Iceberg)Fix HDFS FileSystem Leak Caused by Frequent Refresh of Iceberg Catalog [doris]
doris-robot commented on PR #45956: URL: https://github.com/apache/doris/pull/45956#issuecomment-2562114144 ClickBench: Total hot run time: 31.57 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 3ec2f54c9aab694b4a0d571e19962ac26373ba54, data reload: false query1 0.030.030.02 query2 0.070.030.03 query3 0.230.070.08 query4 1.600.110.10 query5 0.420.410.42 query6 1.140.650.65 query7 0.020.020.01 query8 0.040.040.03 query9 0.580.500.51 query10 0.550.560.55 query11 0.140.100.11 query12 0.140.110.11 query13 0.600.600.60 query14 2.732.742.72 query15 0.900.830.82 query16 0.380.390.38 query17 1.051.061.05 query18 0.230.220.21 query19 1.931.721.98 query20 0.010.010.01 query21 15.38 0.940.59 query22 0.740.800.82 query23 15.16 1.410.51 query24 3.221.281.58 query25 0.230.190.09 query26 0.340.150.14 query27 0.050.050.04 query28 13.96 1.491.05 query29 12.57 3.913.28 query30 0.250.090.07 query31 2.820.620.38 query32 3.230.550.47 query33 3.043.103.22 query34 16.63 5.104.46 query35 4.514.444.43 query36 0.690.490.47 query37 0.090.060.06 query38 0.050.040.04 query39 0.040.020.02 query40 0.160.130.14 query41 0.080.030.02 query42 0.030.020.02 query43 0.030.030.04 Total cold run time: 106.09 s Total hot run time: 31.57 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](plugin) unify the plugin dir [doris]
morningman commented on PR #45936: URL: https://github.com/apache/doris/pull/45936#issuecomment-2562114139 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] branch-3.0: [opt](parquet-reader)Implement late materialization of parquet complex types. #44098 [doris]
github-actions[bot] opened a new pull request, #45985: URL: https://github.com/apache/doris/pull/45985 Cherry-picked from #44098 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-44098-branch-3.0 updated (44cbeaca577 -> 3c9b7d5ea42)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch auto-pick-44098-branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
from 44cbeaca577 branch-3.0: [enhancement](meta) Sync tablet meta even if
local state is not running #45821 (#45962)
add 3c9b7d5ea42 [opt](parquet-reader)Implement late materialization of
parquet complex types. (#44098)
No new revisions were added by this update.
Summary of changes:
be/src/vec/exec/format/parquet/parquet_common.cpp | 152 ++--
be/src/vec/exec/format/parquet/parquet_common.h| 156 +++-
.../exec/format/parquet/vparquet_column_reader.cpp | 170 +++-
.../exec/format/parquet/vparquet_column_reader.h | 60 +-
.../exec/format/parquet/vparquet_group_reader.cpp | 58 +-
.../exec/format/parquet/vparquet_group_reader.h| 6 +-
be/src/vec/exec/format/parquet/vparquet_reader.cpp | 3 +-
be/test/vec/exec/parquet/parquet_common_test.cpp | 457 ++
.../parquet_nested_type_cross_page_test.cpp| 179
be/test/vec/exec/parquet/parquet_thrift_test.cpp | 8 +-
.../parquet_nested_types/create_table.hql | 58 ++
.../multi_catalog/parquet_nested_types/data.tar.gz | Bin 0 -> 36976 bytes
.../data_gen_scripts/nested_cross_page_test1.py| 192
.../data_gen_scripts/nested_cross_page_test2.py| 287 ++
.../data_gen_scripts/nested_cross_page_test3.py| 196
.../run.sh | 0
.../hive/test_parquet_nested_types.out | 985 +
.../hive/test_parquet_nested_types.groovy | 209 +
18 files changed, 2953 insertions(+), 223 deletions(-)
create mode 100644 be/test/vec/exec/parquet/parquet_common_test.cpp
create mode 100644
be/test/vec/exec/parquet/parquet_nested_type_cross_page_test.cpp
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/parquet_nested_types/create_table.hql
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/parquet_nested_types/data.tar.gz
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/parquet_nested_types/data_gen_scripts/nested_cross_page_test1.py
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/parquet_nested_types/data_gen_scripts/nested_cross_page_test2.py
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/parquet_nested_types/data_gen_scripts/nested_cross_page_test3.py
copy
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/{hive_text_complex_type
=> parquet_nested_types}/run.sh (100%)
create mode 100644
regression-test/data/external_table_p0/hive/test_parquet_nested_types.out
create mode 100644
regression-test/suites/external_table_p0/hive/test_parquet_nested_types.groovy
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [opt](parquet-reader)Implement late materialization of parquet complex types. #44098 [doris]
dataroaring closed pull request #45985: branch-3.0: [opt](parquet-reader)Implement late materialization of parquet complex types. #44098 URL: https://github.com/apache/doris/pull/45985 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [opt](parquet-reader)Implement late materialization of parquet complex types. #44098 [doris]
hello-stephen commented on PR #45985: URL: https://github.com/apache/doris/pull/45985#issuecomment-2562114227 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud-mow) FE should release delete bitmap lock when calculating delete bitmap failed [doris]
hust-hhb commented on PR #45673: URL: https://github.com/apache/doris/pull/45673#issuecomment-2562114272 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](statistics)Async drop table stats while doing truncate and schema change. [doris]
github-actions[bot] commented on PR #45923: URL: https://github.com/apache/doris/pull/45923#issuecomment-2562113875 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feat](Nereids) Support use mv hint [doris]
LiBinfeng-01 commented on PR #45855: URL: https://github.com/apache/doris/pull/45855#issuecomment-2562114026 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] Fix code inspection workflow check failure [doris-kafka-connector]
DongLiang-0 opened a new pull request, #56: URL: https://github.com/apache/doris-kafka-connector/pull/56 Since the code inspection workflow does not specify the Java version, during the actual permission process, it will fail due to the use of a higher version of Java. ``` SEVERE: Step 'removeUnusedImports' found problem in 'src/main/java/org/apache/doris/kafka/connector/decode/avro/DorisAvroConverter.java': null java.lang.reflect.InvocationTargetException at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at com.diffplug.spotless.java.GoogleJavaFormatStep$State.lambda$constructRemoveUnusedFunction$3(GoogleJavaFormatStep.java:190) at com.diffplug.spotless.java.GoogleJavaFormatStep$State.lambda$createRemoveUnusedImportsOnly$1(GoogleJavaFormatStep.java:167) at com.diffplug.spotless.FormatterFunc.apply(FormatterFunc.java:32) at com.diffplug.spotless.FormatterStepImpl$Standard.format(FormatterStepImpl.java:78) at com.diffplug.spotless.FormatterStep$Strict.format(FormatterStep.java:76) at com.diffplug.spotless.Formatter.compute(Formatter.java:230) at com.diffplug.spotless.PaddedCell.calculateDirtyState(PaddedCell.java:201) at com.diffplug.spotless.PaddedCell.calculateDirtyState(PaddedCell.java:188) at com.diffplug.spotless.maven.SpotlessCheckMojo.process(SpotlessCheckMojo.java:52) at com.diffplug.spotless.maven.AbstractSpotlessMojo.execute(AbstractSpotlessMojo.java:146) at com.diffplug.spotless.maven.AbstractSpotlessMojo.execute(AbstractSpotlessMojo.java:137) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137) at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2(MojoExecutor.java:370) at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:351) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:215) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:171) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:163) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:299) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:963) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:296) at org.apache.maven.cli.MavenCli.main(MavenCli.java:199) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347) Caused by: java.lang.IllegalAccessError: class com.google.googlejavaformat.java.RemoveUnusedImports (in unnamed module @0x5122e4bf) cannot access class com.sun.tools.javac.util.Context (in module jdk.compiler) because module jdk.compiler does not export com.sun.tools.javac.util to unnamed module @0x5122e4bf at com.google.googlejavaformat.java.RemoveUnusedImports.removeUnusedImports(RemoveUnusedImports.java:187) ... 39 more ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org -
(doris) branch auto-pick-45941-branch-3.0 created (now 44cbeaca577)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45941-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git at 44cbeaca577 branch-3.0: [enhancement](meta) Sync tablet meta even if local state is not running #45821 (#45962) No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [opt](profile) add summary metric for file scanner (#45941)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 0cae9789e61 [opt](profile) add summary metric for file scanner (#45941)
0cae9789e61 is described below
commit 0cae9789e61a3dfb34029eefb67fa6de03d345e2
Author: Mingyu Chen (Rayner)
AuthorDate: Thu Dec 26 11:27:27 2024 +0800
[opt](profile) add summary metric for file scanner (#45941)
### What problem does this PR solve?
Add summary metrics in profile for:
- FileScannerGetBlockTime
- FileScannerOpenReaderTime
- FileScannerCastInputBlockTime
- FileScannerFillMissingColumnTime
- FileScannerPreFilterTimer
- FileScannerConvertOuputBlockTime
- EmptyFileNum
- NotFoundFileNum
- FileNumber
---
be/src/vec/exec/scan/vfile_scanner.cpp | 31 ++-
1 file changed, 18 insertions(+), 13 deletions(-)
diff --git a/be/src/vec/exec/scan/vfile_scanner.cpp
b/be/src/vec/exec/scan/vfile_scanner.cpp
index 15b681f5979..b07fbc05721 100644
--- a/be/src/vec/exec/scan/vfile_scanner.cpp
+++ b/be/src/vec/exec/scan/vfile_scanner.cpp
@@ -118,19 +118,24 @@ VFileScanner::VFileScanner(
Status VFileScanner::prepare(RuntimeState* state, const VExprContextSPtrs&
conjuncts) {
RETURN_IF_ERROR(VScanner::prepare(state, conjuncts));
-_get_block_timer = ADD_TIMER(_local_state->scanner_profile(),
"FileScannerGetBlockTime");
-_open_reader_timer = ADD_TIMER(_local_state->scanner_profile(),
"FileScannerOpenReaderTime");
-_cast_to_input_block_timer =
-ADD_TIMER(_local_state->scanner_profile(),
"FileScannerCastInputBlockTime");
-_fill_missing_columns_timer =
-ADD_TIMER(_local_state->scanner_profile(),
"FileScannerFillMissingColumnTime");
-_pre_filter_timer = ADD_TIMER(_local_state->scanner_profile(),
"FileScannerPreFilterTimer");
-_convert_to_output_block_timer =
-ADD_TIMER(_local_state->scanner_profile(),
"FileScannerConvertOuputBlockTime");
-_empty_file_counter = ADD_COUNTER(_local_state->scanner_profile(),
"EmptyFileNum", TUnit::UNIT);
-_not_found_file_counter =
-ADD_COUNTER(_local_state->scanner_profile(), "NotFoundFileNum",
TUnit::UNIT);
-_file_counter = ADD_COUNTER(_local_state->scanner_profile(), "FileNumber",
TUnit::UNIT);
+_get_block_timer =
+ADD_TIMER_WITH_LEVEL(_local_state->scanner_profile(),
"FileScannerGetBlockTime", 1);
+_open_reader_timer =
+ADD_TIMER_WITH_LEVEL(_local_state->scanner_profile(),
"FileScannerOpenReaderTime", 1);
+_cast_to_input_block_timer =
ADD_TIMER_WITH_LEVEL(_local_state->scanner_profile(),
+
"FileScannerCastInputBlockTime", 1);
+_fill_missing_columns_timer =
ADD_TIMER_WITH_LEVEL(_local_state->scanner_profile(),
+
"FileScannerFillMissingColumnTime", 1);
+_pre_filter_timer =
+ADD_TIMER_WITH_LEVEL(_local_state->scanner_profile(),
"FileScannerPreFilterTimer", 1);
+_convert_to_output_block_timer =
ADD_TIMER_WITH_LEVEL(_local_state->scanner_profile(),
+
"FileScannerConvertOuputBlockTime", 1);
+_empty_file_counter =
+ADD_COUNTER_WITH_LEVEL(_local_state->scanner_profile(),
"EmptyFileNum", TUnit::UNIT, 1);
+_not_found_file_counter =
ADD_COUNTER_WITH_LEVEL(_local_state->scanner_profile(),
+ "NotFoundFileNum",
TUnit::UNIT, 1);
+_file_counter =
+ADD_COUNTER_WITH_LEVEL(_local_state->scanner_profile(),
"FileNumber", TUnit::UNIT, 1);
_file_cache_statistics.reset(new io::FileCacheStatistics());
_io_ctx.reset(new io::IOContext());
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [opt](parquet-reader)Implement late materialization of parquet complex types. #44098 [doris]
hello-stephen commented on PR #45985: URL: https://github.com/apache/doris/pull/45985#issuecomment-2562114197 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](metrics) Unify metrics of thread pool [doris]
zhiqiang- commented on PR #43144: URL: https://github.com/apache/doris/pull/43144#issuecomment-2562114638 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
