[incubator-doris] branch master updated: [Improvement] add regression test for outfile (#10058)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new acd62a2137 [Improvement] add regression test for outfile (#10058) acd62a2137 is described below commit acd62a21377ea7c50dcdc84555e66bd0fa3654d1 Author: Gabriel AuthorDate: Mon Jun 13 15:02:30 2022 +0800 [Improvement] add regression test for outfile (#10058) --- regression-test/data/export/test_outfile.out | 1003 + regression-test/suites/export/test_outfile.groovy | 132 +++ 2 files changed, 1135 insertions(+) diff --git a/regression-test/data/export/test_outfile.out b/regression-test/data/export/test_outfile.out new file mode 100644 index 00..2a69933051 --- /dev/null +++ b/regression-test/data/export/test_outfile.out @@ -0,0 +1,1003 @@ +-- This file is automatically generated. You should know what you did if you want to edit this +-- !select_default -- +1 2017-10-01 2017-10-01T00:00Beijing 1 1 true 1 1 1 1.1 1.1 char1 1 +2 2017-10-01 2017-10-01T00:00Beijing 2 2 true 2 2 2 2.2 2.2 char2 2 +3 2017-10-01 2017-10-01T00:00Beijing 3 3 true 3 3 3 3.3 3.3 char3 3 +4 2017-10-01 2017-10-01T00:00Beijing 4 4 true 4 4 4 4.4 4.4 char4 4 +5 2017-10-01 2017-10-01T00:00Beijing 5 5 true 5 5 5 5.5 5.5 char5 5 +6 2017-10-01 2017-10-01T00:00Beijing 6 6 true 6 6 6 6.6 6.6 char6 6 +7 2017-10-01 2017-10-01T00:00Beijing 7 7 true 7 7 7 7.7 7.7 char7 7 +8 2017-10-01 2017-10-01T00:00Beijing 8 8 true 8 8 8 8.8 8.8 char8 8 +9 2017-10-01 2017-10-01T00:00Beijing 9 9 true 9 9 9 9.9 9.9 char9 9 +10 2017-10-01 2017-10-01T00:00Beijing 10 10 true 10 10 10 10.110.1char10 10 +11 2017-10-01 2017-10-01T00:00Beijing 11 11 true 11 11 11 11.11 11.11 char11 11 +12 2017-10-01 2017-10-01T00:00Beijing 12 12 true 12 12 12 12.12 12.12 char12 12 +13 2017-10-01 2017-10-01T00:00Beijing 13 13 true 13 13 13 13.13 13.13 char13 13 +14 2017-10-01 2017-10-01T00:00Beijing 14 14 true 14 14 14 14.14 14.14 char14 14 +15 2017-10-01 2017-10-01T00:00Beijing 15 15 true 15 15 15 15.15 15.15 char15 15 +16 2017-10-01 2017-10-01T00:00Beijing 16 16 true 16 16 16 16.16 16.16 char16 16 +17 2017-10-01 2017-10-01T00:00Beijing 17 17 true 17 17 17 17.17 17.17 char17 17 +18 2017-10-01 2017-10-01T00:00Beijing 18 18 true 18 18 18 18.18 18.18 char18 18 +19 2017-10-01 2017-10-01T00:00Beijing 19 19 true 19 19 19 19.19 19.19 char19 19 +20 2017-10-01 2017-10-01T00:00Beijing 20 20 true 20 20 20 20.220.2char20 20 +21 2017-10-01 2017-10-01T00:00Beijing 21 21 true 21 21 21 21.21 21.21 char21 21 +22 2017-10-01 2017-10-01T00:00Beijing 22 22 true 22 22 22 22.22 22.22 char22 22 +23 2017-10-01 2017-10-01T00:00Beijing 23 23 true 23 23 23 23.23 23.23 char23 23 +24 2017-10-01 2017-10-01T00:00Beijing 24 24 true 24 24 24 24.24 24.24 char24 24 +25 2017-10-01 2017-10-01T00:00Beijing 25 25 true 25 25 25 25.25 25.25 char25 25 +26 2017-10-01 2017-10-01T00:00Beijing 26 26 true 26 26 26 26.26 26.26 char26 26 +27 2017-10-01 2017-10-01T00:00Beijing 27 27 true 27 27 27 27.27 27.27 char27 27 +28 2017-10-01 2017-10-01T00:00Beijing 28 28 true 28 28 28 28.28 28.28 char28 28 +29 2017-10-01 2017-10-01T00:00Beijing 29 29 true 29 29 29 29.29 29.29 char29 29 +30 2017-10-01 2017-10-01T00:00Beijing 30 30 true 30 30 30 30.3
[GitHub] [incubator-doris] yiguolei merged pull request #10058: [Improvement] add regression test for outfile
yiguolei merged PR #10058: URL: https://github.com/apache/incubator-doris/pull/10058 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei closed issue #10057: [Enhancement] add regression test for outfile
yiguolei closed issue #10057: [Enhancement] add regression test for outfile URL: https://github.com/apache/incubator-doris/issues/10057 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] cambyzju commented on pull request #10065: [feature-wip](array-type) Support loading data in vectorized format
cambyzju commented on PR #10065: URL: https://github.com/apache/incubator-doris/pull/10065#issuecomment-1153559754 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] qimumu9406 commented on issue #9992: How to delete data.
qimumu9406 commented on issue #9992: URL: https://github.com/apache/incubator-doris/issues/9992#issuecomment-1153561250 Check whether the delete table has a rollup. If there is a rollup, the key column specified by the delete statement must exist in the rollup. https://user-images.githubusercontent.com/104196507/173299907-f98cbde1-414d-4954-88df-d7efa9eb7981.png";> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] qimumu9406 commented on issue #9839: [Bug]Report No Doris FE is avaliable，actually the fe and be is running
qimumu9406 commented on issue #9839: URL: https://github.com/apache/incubator-doris/issues/9839#issuecomment-1153563936 Is this problem solved? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] mengyejiang commented on issue #9839: [Bug]Report No Doris FE is avaliable，actually the fe and be is running
mengyejiang commented on issue #9839: URL: https://github.com/apache/incubator-doris/issues/9839#issuecomment-1153571934 > Reference no, unable to solve -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt opened a new issue, #10090: [Bug][Compaction] core dump on compaction with largeint
BiteThet opened a new issue, #10090: URL: https://github.com/apache/incubator-doris/issues/10090 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version master ### What's Wrong? ```cpp *** Aborted at 1655104088 (unix time) try "date -d @1655104088" if you are using GNU date *** *** SIGSEGV unkown detail explain (@0x0) received by PID 38416 (TID 0x7fafc6f4a700) from PID 0; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/disk2/pxl/dev/baidu/bdg/doris/core/be/src/common/signal_handler.h:407 1# 0x00318AE32920 in /lib64/libc.so.6 2# doris::BaseFieldtypeTraits<(doris::FieldType)9>::equal(void const*, void const*) at /home/disk2/pxl/dev/baidu/bdg/doris/core/be/src/olap/types.h:541 3# doris::TupleReader::_agg_key_next_row(doris::RowCursor*, doris::MemPool*, doris::ObjectPool*, bool*) at /home/disk2/pxl/dev/baidu/bdg/doris/core/be/src/olap/tuple_reader.cpp:160 4# doris::Merger::merge_rowsets(std::shared_ptr, doris::ReaderType, std::vector, std::allocator > > const&, doris::RowsetWriter*, doris::Merger::Statistics*) at /home/disk2/pxl/dev/baidu/bdg/doris/core/be/src/olap/merger.cpp:59 5# doris::Compaction::do_compaction_impl(long) at /home/disk2/pxl/dev/baidu/bdg/doris/core/be/src/olap/compaction.cpp:104 6# doris::Compaction::do_compaction(long) at /home/disk2/pxl/dev/baidu/bdg/doris/core/be/src/olap/compaction.cpp:63 7# doris::CumulativeCompaction::execute_compact_impl() at /home/disk2/pxl/dev/baidu/bdg/doris/core/be/src/olap/cumulative_compaction.cpp:75 8# doris::Compaction::execute_compact() at /home/disk2/pxl/dev/baidu/bdg/doris/core/be/src/olap/compaction.cpp:52 9# doris::Tablet::execute_compaction(doris::CompactionType) at /home/disk2/pxl/dev/baidu/bdg/doris/core/be/src/olap/tablet.cpp:1469 10# std::_Function_handler, doris::CompactionType)::$_12>::_M_invoke(std::_Any_data const&) at /home/disk2/pxl/dev/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291 11# doris::ThreadPool::dispatch_thread() at /home/disk2/pxl/dev/baidu/bdg/doris/core/be/src/util/threadpool.cpp:548 12# doris::Thread::supervise_thread(void*) at /home/disk2/pxl/dev/baidu/bdg/doris/core/be/src/util/thread.cpp:409 13# start_thread in /lib64/libpthread.so.0 14# clone in /lib64/libc.so.6 ``` ### What You Expected? fix ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris-flink-connector] JNSimba commented on pull request #34: add 2pc config
JNSimba commented on PR #34: URL: https://github.com/apache/incubator-doris-flink-connector/pull/34#issuecomment-1153589257 Is it possible to automatically adapt to non-2pc scenes? There is no need to add configuration parameters. For example, after doris enables 2pc, it will be submitted with 2pc. If 2pc is not enabled, the streamload operation will be performed directly. I feel like this is a better experience -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] wangyf0555 opened a new pull request, #10091: [doc][website]update home page content and add slack button
wangyf0555 opened a new pull request, #10091: URL: https://github.com/apache/incubator-doris/pull/10091 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] SWJTU-ZhangLei commented on issue #10063: [Bug] asan be core doris::NotInListPredicate::evaluate
SWJTU-ZhangLei commented on issue #10063: URL: https://github.com/apache/incubator-doris/issues/10063#issuecomment-1153597220 > Is it same problem like #10047 ? i am not sure, i will try to run regression tests again with this patch to confirm. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] jacktengg commented on pull request #10080: [improvement] send merged runtime filter asynchrously
jacktengg commented on PR #10080: URL: https://github.com/apache/incubator-doris/pull/10080#issuecomment-1153606495 > > Please add ssb 100g performance test. I am afraid there is performance loss. ssb 100G perf test: | master-f26b81e-1replica | Send merged RF asynchronously | Time Ratio -- | -- | -- | -- q1.1 | 87 | 87 | 1.00 q1.2 | 62 | 65.00 | 1.05 q1.3 | 60 | 64.00 | 1.07 q2.1 | 529 | 544.00 | 1.03 q2.2 | 456 | 462.00 | 1.01 q2.3 | 431 | 437.00 | 1.01 q3.1 | 1022 | 1055.00 | 1.03 q3.2 | 463 | 470.00 | 1.02 q3.3 | 387 | 396.00 | 1.02 q3.4 | 122 | 120.00 | 0.98 q4.1 | 1207 | 1221.00 | 1.01 q4.2 | 757 | 773.00 | 1.02 q4.3 | 984 | 1001.00 | 1.02 No perf lost for ssb. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] xinyiZzz commented on pull request #10080: [improvement] send merged runtime filter asynchrously
xinyiZzz commented on PR #10080: URL: https://github.com/apache/incubator-doris/pull/10080#issuecomment-1153640671 @jacktengg hi, why removed the logic that multiple filters only wait 1s at most. Is there a performance loss in the test? This is a TODO for the first implementation of the runtime filter. Currently, `ScanNode` will wait at most n seconds, and n is equal to the number of runtime filters, which will cause serious performance loss in extreme cases. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] chenlinzhong commented on pull request #9804: [Feature] compaction quickly for small data import #9791
chenlinzhong commented on PR #9804: URL: https://github.com/apache/incubator-doris/pull/9804#issuecomment-1153642845 > will there be any side effect for normal compaction performance, if there are many stream load and with not small data? I think it will not , the quick compaciton will do nothing if not with samll data -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg opened a new pull request, #10092: [deps] update libhdfs3 to fix a uuid set problem
yangzhg opened a new pull request, #10092: URL: https://github.com/apache/incubator-doris/pull/10092 # Proposed changes in libhdfs3 2.3.0 when write uuid to rpc head use string not int128 this will cause rpc failed, updated to 2.3.1 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No Need) 3. Has document been added or modified: (No Need) 4. Does it need to update dependencies: (Yes) 5. Are there any changes that cannot be rolled back: (No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] jacktengg commented on pull request #10080: [improvement] send merged runtime filter asynchrously
jacktengg commented on PR #10080: URL: https://github.com/apache/incubator-doris/pull/10080#issuecomment-1153655751 > @jacktengg hi, why removed the logic that multiple filters only wait 1s at most. Is there a performance loss in the test? > > This is a TODO for the first implementation of the runtime filter. Currently, `ScanNode` will wait at most n seconds, and n is equal to the number of runtime filters, which will cause serious performance loss in extreme cases. No enough test have been done for the change of 'multiple filters only wait 1s at most', so the performance impact is not sure for mutiple runtime filters in on ScanNode. E.g., waiting a RF for 1.5 seconds and it can filter a lot of rows, VS waiting for 1 seconds and the RF is missed, I have not tested this situation, not sure which is better. So in this PR will not include the change of RF waiting. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] mrhhsg opened a new pull request, #10093: [improvement](profile) Add table name and predicates
mrhhsg opened a new pull request, #10093:
URL: https://github.com/apache/incubator-doris/pull/10093
# Proposed changes
Issue Number: close #xxx
## Problem Summary:
Add information of table name and column predicates in profile:
```bash
VOLAP_SCAN_NODE (id=0):(Active: 12s858ms, non-child: 99.62%)
- Table: lineorder_flat
- OlapFilters: [{c_city IN [UNITED KI1, UNITED KI5]}, {lo_orderdate
>= [19920101]}, {lo_orderdate <= [19971231]}, {s_city IN [UNITED KI1, UNITED
KI5]}]
```
## Checklist(Required)
1. Does it affect the original behavior: (Yes/No/I Don't know)
2. Has unit tests been added: (Yes/No/No Need)
3. Has document been added or modified: (Yes/No/No Need)
4. Does it need to update dependencies: (Yes/No)
5. Are there any changes that cannot be rolled back: (Yes/No)
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt opened a new pull request, #10094: [Bug][Compaction] core dump on compaction with largeint
BiteThet opened a new pull request, #10094: URL: https://github.com/apache/incubator-doris/pull/10094 # Proposed changes Issue Number: close #10090 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt commented on pull request #10093: [improvement](profile) Add table name and predicates
BiteThet commented on PR #10093: URL: https://github.com/apache/incubator-doris/pull/10093#issuecomment-1153680651 Maybe we can remove the `VconjunctExprTree` in the profile by the way. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10087: [Bug][Build] Build fail with clang
github-actions[bot] commented on PR #10087: URL: https://github.com/apache/incubator-doris/pull/10087#issuecomment-1153689821 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10087: [Bug][Build] Build fail with clang
github-actions[bot] commented on PR #10087: URL: https://github.com/apache/incubator-doris/pull/10087#issuecomment-1153689860 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] Gabriel39 opened a new issue, #10095: [Bug] Error occurs when showing query profile
Gabriel39 opened a new issue, #10095: URL: https://github.com/apache/incubator-doris/issues/10095 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version trunk ### What's Wrong? select * from customer limit 100; show query profile "/56d7cc7abc254ca7-84d884fe4f4d401b"; ERROR 1105 (HY000): errCode = 2, detailMessage = failed to get fragment profile tree. err: errCode = 2, detailMessage = Invalid instance profile, without sender or exec node: Instance 56d7cc7abc254ca7-84d884fe4f4d401d (host=TNetworkAddress(hostname:172.19.0.11, port:9065)): ### What You Expected? works well ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10072: [fix] (mem tracker) Fix some memory leaks, inaccurate statistics, core dump, deadlock bugs
github-actions[bot] commented on PR #10072: URL: https://github.com/apache/incubator-doris/pull/10072#issuecomment-1153694169 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10072: [fix] (mem tracker) Fix some memory leaks, inaccurate statistics, core dump, deadlock bugs
github-actions[bot] commented on PR #10072: URL: https://github.com/apache/incubator-doris/pull/10072#issuecomment-1153694235 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] Gabriel39 opened a new issue, #10096: [Feature] Runtime filter on vectorized engine
Gabriel39 opened a new issue, #10096: URL: https://github.com/apache/incubator-doris/issues/10096 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description VOLAPScanNode will continue to scan data files when runtime filters are not ready at beginning. As expected, runtime filter should be used once it's ready during scan thread running. But now, runtime filter is not used when it's not ready at beginning. ### Use case _No response_ ### Related issues _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10043: [optimize](compaction) optimize cu compaction policy in pick rowset
github-actions[bot] commented on PR #10043: URL: https://github.com/apache/incubator-doris/pull/10043#issuecomment-1153696329 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10043: [optimize](compaction) optimize cu compaction policy in pick rowset
github-actions[bot] commented on PR #10043: URL: https://github.com/apache/incubator-doris/pull/10043#issuecomment-1153696390 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] xinyiZzz commented on pull request #10080: [improvement] send merged runtime filter asynchrously
xinyiZzz commented on PR #10080: URL: https://github.com/apache/incubator-doris/pull/10080#issuecomment-1153711447 > > @jacktengg hi, why removed the logic that multiple filters only wait 1s at most. Is there a performance loss in the test? > > This is a TODO for the first implementation of the runtime filter. Currently, `ScanNode` will wait at most n seconds, and n is equal to the number of runtime filters, which will cause serious performance loss in extreme cases. > > No enough test have been done for the change of 'multiple filters only wait 1s at most', so the performance impact is not sure for mutiple runtime filters in on ScanNode. > > E.g., waiting a RF for 1.5 seconds and it can filter a lot of rows, VS waiting for 1 seconds and the RF is missed, I have not tested this situation, not sure which is better. > > So in this PR will not include the change of RF waiting. If waiting for 1 second causes RF to be missed, `runtime_filter_wait_time_ms` should be increased. Otherwise it will only wait 1.5 seconds if `ScanNode`apply is greater than 1 RF, which is uncontrollable. `Number of RFs` is not directly related to `Waiting time per RF`. I think 'multiple filters only wait 1s at most' is a viable optimization, maybe do some testing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] bbnneemm opened a new issue, #10097: flink-connector-doris. connect timed out
bbnneemm opened a new issue, #10097: URL: https://github.com/apache/incubator-doris/issues/10097 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version 1.0.0 ### What's Wrong?  ### What You Expected? fe.conf 已经配置 http_v2 为true 并重启 web ui可以正常访问 ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei commented on a diff in pull request #10093: [improvement](profile) Add table name and predicates
yiguolei commented on code in PR #10093:
URL: https://github.com/apache/incubator-doris/pull/10093#discussion_r895539468
##
be/src/exec/olap_scan_node.cpp:
##
@@ -663,6 +690,8 @@ Status OlapScanNode::build_olap_filters() {
}
}
+_runtime_profile->add_info_string("OlapFilters",
olap_filters_to_string(_olap_filter));
Review Comment:
"PushdownPredicate"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yixiutt opened a new pull request, #10098: [improvement](compaction) remove pick rowset skip window strategy to reduce compaction score
yixiutt opened a new pull request, #10098: URL: https://github.com/apache/incubator-doris/pull/10098 romove the stategy that will not pick rowset generated in 530 seconds. intruduced in pr #3270, but we now reserve compacted rowset for 30 minutes, so this logic is useless but will lead to a high compaction score when load frequency is high. # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] englefly opened a new pull request, #10099: [Refactor] change the macro style of the InListPredicate
englefly opened a new pull request, #10099: URL: https://github.com/apache/incubator-doris/pull/10099 # Proposed changes refactor in_list_predicate.cpp 1. use normal cpp code instead of macro definition. 2. remvoe NotInListPredicate class. It is almost duplicated to InListPredicate 3. add bool _in_or_not member in InListPredicate. if _in_or_not=false, InListPredicate stands for former NotInListPredicate. Only rewrite vectorization part. The non-vertorization parts are still in Macros style. remove them when we clean non-verctorization code. Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 3. Has unit tests been added: (Yes/No/No Need) 6. Has document been added or modified: (Yes/No/No Need) 7. Does it need to update dependencies: (Yes/No) 8. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] Gabriel39 commented on a diff in pull request #10099: [Refactor] change the macro style of the InListPredicate
Gabriel39 commented on code in PR #10099:
URL: https://github.com/apache/incubator-doris/pull/10099#discussion_r895570045
##
be/src/olap/in_list_predicate.h:
##
@@ -76,33 +76,55 @@ namespace doris {
class VectorizedRowBatch;
// todo(wb) support evaluate_and,evaluate_or
+template
+class InListPredicate : public ColumnPredicate {
+public:
+InListPredicate(uint32_t column_id, phmap::flat_hash_set&& values, bool
in_or_not, bool is_opposite = false);
+PredicateType type() const override { return PredicateType::IN_LIST; }
+virtual void evaluate(VectorizedRowBatch* batch) const override;
+void evaluate(ColumnBlock* block, uint16_t* sel, uint16_t* size) const
override;
+void evaluate_or(ColumnBlock* block, uint16_t* sel, uint16_t size,
+bool* flags) const override;
+void evaluate_and(ColumnBlock* block, uint16_t* sel, uint16_t size,
+bool* flags) const override;
+virtual Status evaluate(const Schema& schema,
+const std::vector& iterators,
+uint32_t num_rows, roaring::Roaring* bitmap) const
override;
+void evaluate(vectorized::IColumn& column, uint16_t* sel, uint16_t* size)
const override;
+void evaluate_and(vectorized::IColumn& column, uint16_t* sel, uint16_t
size,
+bool* flags) const override {}
+void evaluate_or(vectorized::IColumn& column, uint16_t* sel, uint16_t size,
+bool* flags) const override {}
+
+private:
+phmap::flat_hash_set _values;
+bool _in_or_not; //true for in(...), false for not in (...)
+};
+
+// template
Review Comment:
delete these commented codes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei commented on pull request #10099: [Refactor] change the macro style of the InListPredicate
yiguolei commented on PR #10099: URL: https://github.com/apache/incubator-doris/pull/10099#issuecomment-1153761209 very expected work -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt commented on pull request #10099: [Refactor] change the macro style of the InListPredicate
BiteThet commented on PR #10099: URL: https://github.com/apache/incubator-doris/pull/10099#issuecomment-1153762906 Hi, I have done some similar work, maybe you can refer to some designs. https://github.com/BiteThet/incubator-doris/commit/93bdd0582fd6c2890e6f36c1805755712e40553d#diff-1cfbfefc7969508323c47da436eeb1d5788b4ec905166f99b967c248974d9eb1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] englefly commented on a diff in pull request #10099: [Refactor] change the macro style of the InListPredicate
englefly commented on code in PR #10099:
URL: https://github.com/apache/incubator-doris/pull/10099#discussion_r895580596
##
be/src/olap/in_list_predicate.h:
##
@@ -76,33 +76,55 @@ namespace doris {
class VectorizedRowBatch;
// todo(wb) support evaluate_and,evaluate_or
+template
+class InListPredicate : public ColumnPredicate {
+public:
+InListPredicate(uint32_t column_id, phmap::flat_hash_set&& values, bool
in_or_not, bool is_opposite = false);
+PredicateType type() const override { return PredicateType::IN_LIST; }
+virtual void evaluate(VectorizedRowBatch* batch) const override;
+void evaluate(ColumnBlock* block, uint16_t* sel, uint16_t* size) const
override;
+void evaluate_or(ColumnBlock* block, uint16_t* sel, uint16_t size,
+bool* flags) const override;
+void evaluate_and(ColumnBlock* block, uint16_t* sel, uint16_t size,
+bool* flags) const override;
+virtual Status evaluate(const Schema& schema,
+const std::vector& iterators,
+uint32_t num_rows, roaring::Roaring* bitmap) const
override;
+void evaluate(vectorized::IColumn& column, uint16_t* sel, uint16_t* size)
const override;
+void evaluate_and(vectorized::IColumn& column, uint16_t* sel, uint16_t
size,
+bool* flags) const override {}
+void evaluate_or(vectorized::IColumn& column, uint16_t* sel, uint16_t size,
+bool* flags) const override {}
+
+private:
+phmap::flat_hash_set _values;
+bool _in_or_not; //true for in(...), false for not in (...)
+};
+
+// template
Review Comment:
It is better remove these codes when we clean non-vectorized code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] dataroaring commented on pull request #10098: [improvement](compaction) remove pick rowset skip window strategy to reduce compaction score
dataroaring commented on PR #10098: URL: https://github.com/apache/incubator-doris/pull/10098#issuecomment-1153794675 This commit should be merged after https://github.com/apache/incubator-doris/pull/10043. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10092: [deps] update libhdfs3 to fix a uuid set problem
github-actions[bot] commented on PR #10092: URL: https://github.com/apache/incubator-doris/pull/10092#issuecomment-1153806709 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10092: [deps] update libhdfs3 to fix a uuid set problem
github-actions[bot] commented on PR #10092: URL: https://github.com/apache/incubator-doris/pull/10092#issuecomment-1153806734 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] gitbbrid opened a new issue, #10100: [Bug] When the 1.0.5-no-avx2 version queries data, the program crashes on the BE node
gitbbrid opened a new issue, #10100: URL: https://github.com/apache/incubator-doris/issues/10100 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version all version of no-avx2 Tested version:0.15.5-no-avx2 ,1.0.0-preview2-no-avx2 ,1.0.5-no-avx2 所有no-avx2发行的版本 已测试版本：0.15.5-no-avx2 ,1.0.0-preview2-no-avx2 ,1.0.5-no-avx2 ### What's Wrong? When we use the No-Avx2 version for data query, the BE node crashes directly 当我们使用No-Avx2版本进行数据查询时，BE节点直接崩溃 > Query example： MYQL [fintell_poc]> select * from apply_info_v3; ERROR 1064 (HY000): errCode = 2, detailMessage = there is no scanNode Backend. [10003: in black list(io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason)]` > The log shows: start time: Mon Jun 13 18:45:28 CST 2022 *** Aborted at 1655117413 (unix time) try "date -d @1655117413" if you are using GNU date *** PC: @ 0x352bf06 _avx2_run_container_cardinality.isra.0 *** SIGILL (@0x352bf06) received by PID 16929 (TID 0x7fae159fb700) from PID 55754502; stack trace: *** @ 0x2f43f62 google::(anonymous namespace)::FailureSignalHandler() @ 0x7fae19442630 (unknown) @ 0x352bf06 _avx2_run_container_cardinality.isra.0 @ 0x3513a95 roaring_bitmap_get_cardinality @ 0x1fe52d8 doris::segment_v2::SegmentIterator::_get_row_ranges_by_keys() @ 0x1fe6cf1 doris::segment_v2::SegmentIterator::_init() @ 0x1fe751e doris::segment_v2::SegmentIterator::next_batch() @ 0x1805d34 doris::BetaRowsetReader::next_block() @ 0x1764571 doris::CollectIterator::Level0Iterator::_refresh_current_row_v2() @ 0x1765633 doris::CollectIterator::add_child() @ 0x173a90f doris::Reader::_capture_rs_readers() @ 0x1743d54 doris::Reader::init() @ 0x1ea083e doris::OlapScanner::open() @ 0x1e5ab9f doris::OlapScanNode::scanner_thread() @ 0x18681af doris::PriorityThreadPool::work_thread() @ 0x2f10737 thread_proxy @ 0x7fae1943aea5 start_thread @ 0x7fae18e61b0d __clone @0x0 (unknown) Connection closed` ### What You Expected? A quick fix as we are planning to put it into production use. 快速的到解决，因为我们正在计划投入生产使用。 ### How to Reproduce? _No response_ ### Anything Else? no ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10068: [enhancement] support simd instructions on arm cpus through sse2neon
github-actions[bot] commented on PR #10068: URL: https://github.com/apache/incubator-doris/pull/10068#issuecomment-1153814458 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10068: [enhancement] support simd instructions on arm cpus through sse2neon
github-actions[bot] commented on PR #10068: URL: https://github.com/apache/incubator-doris/pull/10068#issuecomment-1153814500 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] englefly commented on a diff in pull request #10099: [Refactor] change the macro style of the InListPredicate
englefly commented on code in PR #10099:
URL: https://github.com/apache/incubator-doris/pull/10099#discussion_r895631456
##
be/src/olap/in_list_predicate.h:
##
@@ -76,33 +76,55 @@ namespace doris {
class VectorizedRowBatch;
// todo(wb) support evaluate_and,evaluate_or
+template
+class InListPredicate : public ColumnPredicate {
+public:
+InListPredicate(uint32_t column_id, phmap::flat_hash_set&& values, bool
in_or_not, bool is_opposite = false);
+PredicateType type() const override { return PredicateType::IN_LIST; }
+virtual void evaluate(VectorizedRowBatch* batch) const override;
+void evaluate(ColumnBlock* block, uint16_t* sel, uint16_t* size) const
override;
+void evaluate_or(ColumnBlock* block, uint16_t* sel, uint16_t size,
+bool* flags) const override;
+void evaluate_and(ColumnBlock* block, uint16_t* sel, uint16_t size,
+bool* flags) const override;
+virtual Status evaluate(const Schema& schema,
+const std::vector& iterators,
+uint32_t num_rows, roaring::Roaring* bitmap) const
override;
+void evaluate(vectorized::IColumn& column, uint16_t* sel, uint16_t* size)
const override;
+void evaluate_and(vectorized::IColumn& column, uint16_t* sel, uint16_t
size,
+bool* flags) const override {}
+void evaluate_or(vectorized::IColumn& column, uint16_t* sel, uint16_t size,
+bool* flags) const override {}
+
+private:
+phmap::flat_hash_set _values;
+bool _in_or_not; //true for in(...), false for not in (...)
+};
+
+// template
Review Comment:
The NotInListPredicate definition is removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] caomingjie-code commented on issue #9992: How to delete data.
caomingjie-code commented on issue #9992: URL: https://github.com/apache/incubator-doris/issues/9992#issuecomment-1153824760 请问 怎么知道这个列是否存在 某个 rollup index 。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman opened a new pull request, #10101: [improvement] Add new session variable `session_context`
morningman opened a new pull request, #10101: URL: https://github.com/apache/incubator-doris/pull/10101 # Proposed changes Issue Number: close #xxx ## Problem Summary: Add a new session variable: `session_context` and it can be set like: `set session_context="k1:v1;k2:v2;..."` Where `k:v` is key-value pair. This is used to pass some user-defined or pre-defined session variables from client to server. For example, I add a pre-defined key: `trace_id`, And I can use the following stmt: ``` select /*+SET_VAR(session_context=trace_id:123456) */ from tbl ; ``` And Doris will get this trace id and saved it in profile. After that, user can use this trace id to get the query id, then get some other info (eg, query profile) by query id. Also, user can pass the trace id to Doris, so that we can monitor Doris's execution in some tracing system. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10080: [improvement] send merged runtime filter asynchrously
github-actions[bot] commented on PR #10080: URL: https://github.com/apache/incubator-doris/pull/10080#issuecomment-1153838803 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10080: [improvement] send merged runtime filter asynchrously
github-actions[bot] commented on PR #10080: URL: https://github.com/apache/incubator-doris/pull/10080#issuecomment-1153838828 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] Gabriel39 opened a new pull request, #10102: [Improvement] set table name in olap scanner
Gabriel39 opened a new pull request, #10102: URL: https://github.com/apache/incubator-doris/pull/10102 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10102: [Improvement] set table name in olap scanner
github-actions[bot] commented on PR #10102: URL: https://github.com/apache/incubator-doris/pull/10102#issuecomment-1153856338 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10102: [Improvement] set table name in olap scanner
github-actions[bot] commented on PR #10102: URL: https://github.com/apache/incubator-doris/pull/10102#issuecomment-1153856373 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #9803: [fix](brpc) Embed serialized request into the attachment and transmit it through http brpc
morningman merged PR #9803: URL: https://github.com/apache/incubator-doris/pull/9803 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [fix](brpc) Embed serialized request into the attachment and transmit it through http brpc (#9803)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new d58e00c49c [fix](brpc) Embed serialized request into the attachment
and transmit it through http brpc (#9803)
d58e00c49c is described below
commit d58e00c49c6424a90209113f40d633d784003829
Author: Xinyi Zou
AuthorDate: Mon Jun 13 20:41:48 2022 +0800
[fix](brpc) Embed serialized request into the attachment and transmit it
through http brpc (#9803)
When the length of `Tuple/Block data` is greater than 2G, serialize the
protoBuf request and embed the
`Tuple/Block data` into the controller attachment and transmit it through
http brpc.
This is to avoid errors when the length of the protoBuf request exceeds 2G:
`Bad request, error_text=[E1003]Fail to compress request`.
In #7164, `Tuple/Block data` was put into attachment and sent via default
`baidu_std brpc`,
but when the attachment exceeds 2G, it will be truncated. There is no 2G
limit for sending via `http brpc`.
Also, in #7921, consider putting `Tuple/Block data` into attachment
transport by default, as this theoretically
reduces one serialization and improves performance. However, the test found
that the performance did not improve,
but the memory peak increased due to the addition of a memory copy.
---
be/src/common/config.h | 8 +-
be/src/exec/tablet_sink.cpp | 39 +++--
be/src/exec/tablet_sink.h| 15 +-
be/src/runtime/data_stream_sender.cpp| 48 +++---
be/src/runtime/data_stream_sender.h | 15 +-
be/src/runtime/row_batch.cpp | 33 ++---
be/src/runtime/row_batch.h | 4 +-
be/src/service/internal_service.cpp | 181 ---
be/src/service/internal_service.h| 39 +
be/src/util/brpc_client_cache.h | 20 ++-
be/src/util/proto_util.h | 110 +-
be/src/vec/core/block.cpp| 21 +--
be/src/vec/core/block.h | 2 +-
be/src/vec/sink/vdata_stream_sender.cpp | 37 -
be/src/vec/sink/vdata_stream_sender.h| 9 +-
be/src/vec/sink/vtablet_sink.cpp | 33 -
be/src/vec/sink/vtablet_sink.h | 5 -
be/test/vec/core/block_test.cpp | 7 +-
docs/en/docs/admin-manual/config/be-config.md| 6 +-
docs/zh-CN/docs/admin-manual/config/be-config.md | 6 +-
gensrc/proto/internal_service.proto | 7 +
21 files changed, 496 insertions(+), 149 deletions(-)
diff --git a/be/src/common/config.h b/be/src/common/config.h
index fd37ca0b48..6fd53e69d5 100644
--- a/be/src/common/config.h
+++ b/be/src/common/config.h
@@ -562,9 +562,11 @@ CONF_String(default_rowset_type, "BETA");
CONF_Int64(brpc_max_body_size, "3147483648");
// Max unwritten bytes in each socket, if the limit is reached, Socket.Write
fails with EOVERCROWDED
CONF_Int64(brpc_socket_max_unwritten_bytes, "1073741824");
-// Whether to transfer RowBatch in ProtoBuf Request to Controller Attachment
and send it
-// through brpc, this will be faster and avoid the error of Request length
overflow.
-CONF_mBool(transfer_data_by_brpc_attachment, "false");
+// TODO(zxy): expect to be true in v1.3
+// Whether to embed the ProtoBuf Request serialized string together with
Tuple/Block data into
+// Controller Attachment and send it through http brpc when the length of the
Tuple/Block data
+// is greater than 1.8G. This is to avoid the error of Request length overflow
(2G).
+CONF_mBool(transfer_large_data_by_brpc, "false");
// max number of txns for every txn_partition_map in txn manager
// this is a self protection to avoid too many txns saving in manager
diff --git a/be/src/exec/tablet_sink.cpp b/be/src/exec/tablet_sink.cpp
index f5567a9165..e1d60f694c 100644
--- a/be/src/exec/tablet_sink.cpp
+++ b/be/src/exec/tablet_sink.cpp
@@ -48,9 +48,6 @@ namespace stream_load {
NodeChannel::NodeChannel(OlapTableSink* parent, IndexChannel* index_channel,
int64_t node_id)
: _parent(parent), _index_channel(index_channel), _node_id(node_id) {
-if (_parent->_transfer_data_by_brpc_attachment) {
-_tuple_data_buffer_ptr = &_tuple_data_buffer;
-}
_node_channel_tracker = MemTracker::create_tracker(
-1, fmt::format("NodeChannel:indexID={}:threadId={}",
std::to_string(_index_channel->_index_id),
tls_ctx()->thread_id_str()));
@@ -79,6 +76,7 @@ NodeChannel::~NodeChannel() noexcept {
Status NodeChannel::init(RuntimeState* state) {
SCOPED_SWITCH_THREAD_LOCAL_MEM_TRACKER(_node_channel_tracker);
_tuple_desc = _parent->_output_tuple_desc;
+_state =
[GitHub] [incubator-doris] Gabriel39 opened a new pull request, #10103: [feature] support runtime filter on vectorized engine
Gabriel39 opened a new pull request, #10103: URL: https://github.com/apache/incubator-doris/pull/10103 # Proposed changes Issue Number: close #10096 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] Gabriel39 commented on pull request #10103: [feature] support runtime filter on vectorized engine
Gabriel39 commented on PR #10103: URL: https://github.com/apache/incubator-doris/pull/10103#issuecomment-1153872445 DO NOT MERGE now. I will continue to test this PR. Feel free to review this PR as I think this is almost done. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on pull request #10103: [feature] support runtime filter on vectorized engine
morningman commented on PR #10103: URL: https://github.com/apache/incubator-doris/pull/10103#issuecomment-1153891268 > DO NOT MERGE now. I will continue to test this PR. Feel free to review this PR as I think this is almost done. You can convert this PR to draft if it is not ready for review. See `convert to draft` at the side bar -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee opened a new pull request, #10104: [Feature] Support outer join in vec exec engine
HappenLee opened a new pull request, #10104: URL: https://github.com/apache/incubator-doris/pull/10104 # Proposed changes Issue Number: close #7901 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 commented on a diff in pull request #9056: [feature-wip](array-type) Add array aggregation functions
adonis0147 commented on code in PR #9056:
URL: https://github.com/apache/incubator-doris/pull/9056#discussion_r895707236
##
be/src/vec/functions/array/function_array_aggregation.cpp:
##
@@ -0,0 +1,300 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+// This file is copied from
+//
https://github.com/ClickHouse/ClickHouse/blob/master/src/Functions/array/arrayAggregation.cpp
+// and modified by Doris
+
+#include
+
+#include "vec/aggregate_functions/aggregate_function_avg.h"
+#include "vec/aggregate_functions/aggregate_function_min_max.h"
+#include "vec/aggregate_functions/aggregate_function_null.h"
+#include "vec/aggregate_functions/aggregate_function_product.h"
+#include "vec/aggregate_functions/aggregate_function_sum.h"
+#include "vec/aggregate_functions/helpers.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/common/arena.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_nullable.h"
+#include "vec/functions/array/function_array_mapped.h"
+#include "vec/functions/simple_function_factory.h"
+
+namespace doris {
+namespace vectorized {
+
+enum class AggregateOperation { MIN, MAX, SUM, AVERAGE, PRODUCT };
+
+template
+struct ArrayAggregateResultImpl;
+
+template
+struct ArrayAggregateResultImpl {
+using Result = Element;
+};
+
+template
+struct ArrayAggregateResultImpl {
+using Result = Element;
+};
+
+template
+struct ArrayAggregateResultImpl {
+using Result = std::conditional_t, Decimal128,
Float64>;
+};
+
+template
+struct ArrayAggregateResultImpl {
+using Result = std::conditional_t, Decimal128,
Float64>;
+};
+
+template
+struct ArrayAggregateResultImpl {
+using Result = std::conditional_t<
+IsDecimalNumber, Decimal128,
+std::conditional_t, Float64,
+ std::conditional_t, Int128, Int64>>>;
+};
+
+template
+using ArrayAggregateResult = typename ArrayAggregateResultImpl::Result;
+
+template
+struct AggregateFunctionImpl;
+
+template <>
+struct AggregateFunctionImpl {
+template
+struct TypeTraits {
+using ResultType = ArrayAggregateResult;
+using AggregateDataType = AggregateFunctionSumData;
+using Function = AggregateFunctionSum;
+};
+};
+
+template <>
+struct AggregateFunctionImpl {
+template
+struct TypeTraits {
+struct AggregateDataType : public AggregateFunctionAvgData {
+using AggregateFunctionAvgData::count;
+using AggregateFunctionAvgData::sum;
+
+template
+ResultT result() const {
+return count ? AggregateFunctionAvgData::template
result()
+ : static_cast(sum);
+}
+};
+
+using ResultType = ArrayAggregateResult;
+using Function = AggregateFunctionAvg;
+static_assert(std::is_same_v,
+ "ResultType doesn't match.");
+};
+};
+
+template <>
+struct AggregateFunctionImpl {
+template
+struct TypeTraits {
+using ResultType = ArrayAggregateResult;
+using AggregateDataType = AggregateFunctionProductData;
+using Function = AggregateFunctionProduct;
+};
+};
+
+template
+struct AggregateFunction {
+template
+using Function = typename Derived::template TypeTraits::Function;
+
+static auto create(const DataTypePtr& data_type) -> AggregateFunctionPtr {
+DataTypes data_types = {data_type};
+AggregateFunctionPtr function;
+
+if (data_type->is_nullable()) {
+const auto& nested_data_type =
+static_cast(*data_type).get_nested_type();
+auto nested_function = create(nested_data_type);
+function.reset(new
AggregateFunctionNullUnary(nested_function, data_types, {}));
+} else {
+if (is_decimal(data_type)) {
+function.reset(
+create_with_decimal_type(*data_type,
*data_type, data_types));
+} else {
+function.reset(create_with_numeric_type(*data_type,
data_types));
+}
+}
+return function;
+}
+};
+
+template
+struct ArrayAggregateImpl {
+
[GitHub] [incubator-doris] adonis0147 commented on a diff in pull request #9056: [feature-wip](array-type) Add array aggregation functions
adonis0147 commented on code in PR #9056:
URL: https://github.com/apache/incubator-doris/pull/9056#discussion_r895713249
##
be/src/vec/functions/array/function_array_mapped.h:
##
@@ -0,0 +1,76 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+// This file is copied from
+//
https://github.com/ClickHouse/ClickHouse/blob/master/src/Functions/array/FunctionArrayMapped.h
+// and modified by Doris
+
+#include "vec/columns/column.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_nullable.h"
+#include "vec/functions/function.h"
+
+namespace doris {
+namespace vectorized {
+
+/** Higher-order functions for arrays.
+ * These functions optionally apply a map (transform) to array (or multiple
arrays of identical size) by lambda function,
+ * and return some result based on that transformation.
+ *
+ * Examples:
+ * arrayMap(x1,...,xn -> expression, array1,...,arrayn) - apply the
expression to each element of the array (or set of parallel arrays).
+ * arrayFilter(x -> predicate, array) - leave in the array only the elements
for which the expression is true.
+ *
+ * For some functions arrayCount, arrayExists, arrayAll, an overload of the
form f(array) is available,
+ * which works in the same way as f(x -> x, array).
+ *
+ * See the example of Impl template parameter in arrayMap.cpp
+ */
+template
+class FunctionArrayMapped : public IFunction {
+public:
+static constexpr auto name = Name::name;
+static FunctionPtr create() { return
std::make_shared(); }
+
+String get_name() const override { return name; }
+Status execute_impl(FunctionContext* context, Block& block, const
ColumnNumbers& arguments,
+size_t result, size_t input_rows_count) override {
+const auto& typed_column = block.get_by_position(arguments[0]);
+const typename Impl::column_type* column_array;
+if (typed_column.column->is_nullable()) {
+column_array = check_and_get_column(
+static_cast(typed_column.column.get())
+->get_nested_column_ptr()
+.get());
+} else {
+column_array = check_and_get_column(
+typed_column.column.get());
+}
+const auto* data_type_array =
+static_cast(remove_nullable(typed_column.type).get());
+return Impl::execute(block, result, data_type_array, *column_array);
+}
+size_t get_number_of_arguments() const override { return 1; }
+DataTypePtr get_return_type_impl(const DataTypes& arguments) const
override {
+const DataTypeArray* data_type_array =
+static_cast(remove_nullable(arguments[0]).get());
+return Impl::get_return_type(data_type_array);
Review Comment:
No need to modify due to
[function.cpp:304](https://github.com/apache/incubator-doris/blob/master/be/src/vec/functions/function.cpp#L304)
will add nullable to the type returned.
##
be/src/vec/functions/array/function_array_aggregation.cpp:
##
@@ -0,0 +1,300 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+// This file is copied from
+//
https://github.com/ClickHouse/ClickHouse/blob/master/src/Functions/array/arrayAggregation.cpp
+// and modified by Doris
+
+#include
+
+#include "vec/aggregate_functions/aggregate_function_avg.h"
+#include "vec/aggregate_fun
[GitHub] [incubator-doris] adonis0147 commented on a diff in pull request #9056: [feature-wip](array-type) Add array aggregation functions
adonis0147 commented on code in PR #9056:
URL: https://github.com/apache/incubator-doris/pull/9056#discussion_r895713917
##
be/src/vec/aggregate_functions/aggregate_function_product.h:
##
@@ -0,0 +1,137 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include
+#include
+
+#include "vec/aggregate_functions/aggregate_function.h"
+#include "vec/columns/column_decimal.h"
+#include "vec/columns/column_vector.h"
+#include "vec/common/arena.h"
+#include "vec/common/string_buffer.hpp"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type_decimal.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/io/io_helper.h"
+
+namespace doris {
+namespace vectorized {
+
+template
+struct AggregateFunctionProductData {
+T product {};
+bool has_data = false;
Review Comment:
Removed.
##
be/src/vec/functions/array/function_array_aggregation.cpp:
##
@@ -0,0 +1,300 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+// This file is copied from
+//
https://github.com/ClickHouse/ClickHouse/blob/master/src/Functions/array/arrayAggregation.cpp
+// and modified by Doris
+
+#include
+
+#include "vec/aggregate_functions/aggregate_function_avg.h"
+#include "vec/aggregate_functions/aggregate_function_min_max.h"
+#include "vec/aggregate_functions/aggregate_function_null.h"
+#include "vec/aggregate_functions/aggregate_function_product.h"
+#include "vec/aggregate_functions/aggregate_function_sum.h"
+#include "vec/aggregate_functions/helpers.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/common/arena.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_nullable.h"
+#include "vec/functions/array/function_array_mapped.h"
+#include "vec/functions/simple_function_factory.h"
+
+namespace doris {
+namespace vectorized {
+
+enum class AggregateOperation { MIN, MAX, SUM, AVERAGE, PRODUCT };
+
+template
+struct ArrayAggregateResultImpl;
+
+template
+struct ArrayAggregateResultImpl {
+using Result = Element;
+};
+
+template
+struct ArrayAggregateResultImpl {
+using Result = Element;
+};
+
+template
+struct ArrayAggregateResultImpl {
+using Result = std::conditional_t, Decimal128,
Float64>;
+};
+
+template
+struct ArrayAggregateResultImpl {
+using Result = std::conditional_t, Decimal128,
Float64>;
+};
+
+template
+struct ArrayAggregateResultImpl {
+using Result = std::conditional_t<
+IsDecimalNumber, Decimal128,
+std::conditional_t, Float64,
+ std::conditional_t, Int128, Int64>>>;
+};
+
+template
+using ArrayAggregateResult = typename ArrayAggregateResultImpl::Result;
+
+template
+struct AggregateFunctionImpl;
+
+template <>
+struct AggregateFunctionImpl {
+template
+struct TypeTraits {
+using ResultType = ArrayAggregateResult;
+using AggregateDataType = AggregateFunctionSumData;
+using Function = AggregateFunctionSum;
+};
+};
+
+template <>
+struct AggregateFunctionImpl {
+template
+struct TypeTraits {
+struct AggregateDataType : public AggregateFunctionAvgData {
Review Comment:
Removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this serv
[GitHub] [incubator-doris] morningman commented on pull request #10069: [test] add cast array regression test
morningman commented on PR #10069: URL: https://github.com/apache/incubator-doris/pull/10069#issuecomment-1153919071 please rebase to pass the BE ut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] cambyzju commented on issue #10100: [Bug] When the 1.0.5-no-avx2 version queries data, the program crashes on the BE node
cambyzju commented on issue #10100: URL: https://github.com/apache/incubator-doris/issues/10100#issuecomment-1153919359 Please give schema of table apply_info_v3, and some test data to reproduce the problem is better. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] cambyzju opened a new pull request, #10105: [doc] Add docs for SHOW TABLETS
cambyzju opened a new pull request, #10105: URL: https://github.com/apache/incubator-doris/pull/10105 # Proposed changes `SHOW TABLETS` command is similar to `SHOW TABLET` command. It is easy to mess up, so add docs for `SHOW TABLETS` command. ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zxealous opened a new pull request, #10106: [typo] Fix typos in comments
zxealous opened a new pull request, #10106: URL: https://github.com/apache/incubator-doris/pull/10106 # Proposed changes Fix typos in comments ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No) 3. Has document been added or modified: (No) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] stalary commented on issue #10024: [Bug] "Show databases" appears "ERROR 2006 (HY000): MySQL server has gone away" when login with a non-exist database
stalary commented on issue #10024: URL: https://github.com/apache/incubator-doris/issues/10024#issuecomment-1154020924 I used the master and did not reproduce it locally. Is there something wrong with the client you used? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] cambyzju opened a new pull request, #10107: [improvement] remove unused codes and docs for `SHOW USER`
cambyzju opened a new pull request, #10107: URL: https://github.com/apache/incubator-doris/pull/10107 # Proposed changes remove unused codes and docs for command `SHOW USER` ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] stalary commented on issue #10024: [Bug] "Show databases" appears "ERROR 2006 (HY000): MySQL server has gone away" when login with a non-exist database
stalary commented on issue #10024: URL: https://github.com/apache/incubator-doris/issues/10024#issuecomment-1154024336 I used the master and did not reproduce it locally. Is there something wrong with the client you used? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 commented on pull request #10106: [typo] Fix typos in comments
adonis0147 commented on PR #10106: URL: https://github.com/apache/incubator-doris/pull/10106#issuecomment-1154033450 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 closed pull request #9056: [feature-wip](array-type) Add array aggregation functions
adonis0147 closed pull request #9056: [feature-wip](array-type) Add array aggregation functions URL: https://github.com/apache/incubator-doris/pull/9056 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 opened a new pull request, #10108: [feature-wip](array-type) Add array aggregation functions
adonis0147 opened a new pull request, #10108: URL: https://github.com/apache/incubator-doris/pull/10108 # Proposed changes Issue Number: close #9055 ## Problem Summary: Please refer to #9055 ## Checklist(Required) 1. Does it affect the original behavior: No 2. Has unit tests been added: Yes 3. Has document been added or modified: No 4. Does it need to update dependencies: No 5. Are there any changes that cannot be rolled back: No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei merged pull request #10080: [improvement] send merged runtime filter asynchrously
yiguolei merged PR #10080: URL: https://github.com/apache/incubator-doris/pull/10080 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [improvement] send merged runtime filter asynchrously (#10080)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new ce730293c0 [improvement] send merged runtime filter asynchrously
(#10080)
ce730293c0 is described below
commit ce730293c07faa4763821abe6f9f073b85e4961f
Author: jacktengg <18241664+jackte...@users.noreply.github.com>
AuthorDate: Tue Jun 14 08:16:25 2022 +0800
[improvement] send merged runtime filter asynchrously (#10080)
---
be/src/runtime/runtime_filter_mgr.cpp | 13 ++---
1 file changed, 10 insertions(+), 3 deletions(-)
diff --git a/be/src/runtime/runtime_filter_mgr.cpp
b/be/src/runtime/runtime_filter_mgr.cpp
index e29609b9b3..654f865729 100644
--- a/be/src/runtime/runtime_filter_mgr.cpp
+++ b/be/src/runtime/runtime_filter_mgr.cpp
@@ -239,7 +239,6 @@ Status RuntimeFilterMergeControllerEntity::merge(const
PMergeFilterRequest* requ
has_attachment = true;
}
-// TODO: async send publish rpc
std::vector& targets = cntVal->target_info;
for (size_t i = 0; i < targets.size(); i++) {
rpc_contexts.emplace_back(new PPublishFilterRpcContext);
@@ -249,6 +248,7 @@ Status RuntimeFilterMergeControllerEntity::merge(const
PMergeFilterRequest* requ
if (has_attachment) {
rpc_contexts[i]->cntl.request_attachment().append(request_attachment);
}
+rpc_contexts[i]->cid = rpc_contexts[i]->cntl.call_id();
// set fragment-id
auto request_fragment_id =
rpc_contexts[i]->request.mutable_fragment_id();
@@ -267,9 +267,16 @@ Status RuntimeFilterMergeControllerEntity::merge(const
PMergeFilterRequest* requ
continue;
}
stub->apply_filter(&rpc_contexts[i]->cntl,
&rpc_contexts[i]->request,
- &rpc_contexts[i]->response, nullptr);
+ &rpc_contexts[i]->response, brpc::DoNothing());
+}
+for (auto& rpc_context : rpc_contexts) {
+brpc::Join(rpc_context->cid);
+if (rpc_context->cntl.Failed()) {
+LOG(WARNING) << "runtimefilter rpc err:" <<
rpc_context->cntl.ErrorText();
+ExecEnv::GetInstance()->brpc_internal_client_cache()->erase(
+rpc_context->cntl.remote_side());
+}
}
-/// TODO: use async and join rpc
}
return Status::OK();
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei merged pull request #10106: [typo] Fix typos in comments
yiguolei merged PR #10106: URL: https://github.com/apache/incubator-doris/pull/10106 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [typo] Fix typos in comments (#10106)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new d4d2e82bdf [typo] Fix typos in comments (#10106) d4d2e82bdf is described below commit d4d2e82bdf9967a5d7f4e1c9dbdc11ae1703c4a4 Author: zxealous AuthorDate: Tue Jun 14 08:17:19 2022 +0800 [typo] Fix typos in comments (#10106) --- be/src/exec/exchange_node.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/be/src/exec/exchange_node.h b/be/src/exec/exchange_node.h index 83df46230f..75ecdf1c0f 100644 --- a/be/src/exec/exchange_node.h +++ b/be/src/exec/exchange_node.h @@ -35,7 +35,7 @@ class RuntimeProfile; // according to the sort parameters in _sort_exec_exprs. (It is assumed that the rows // received from the senders themselves are sorted.) // If _is_merging is true, the exchange node creates a DataStreamRecvr with the -// _is_merging flag and retrieves retrieves rows from the receiver via calls to +// _is_merging flag and retrieves rows from the receiver via calls to // DataStreamRecvr::GetNext(). It also prepares, opens and closes the ordering exprs in // its SortExecExprs member that are used to compare rows. // If _is_merging is false, the exchange node directly retrieves batches from the row - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei merged pull request #10102: [Improvement] set table name in olap scanner
yiguolei merged PR #10102: URL: https://github.com/apache/incubator-doris/pull/10102 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Improvement] set table name in olap scanner (#10102)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new bdcf2e7ed2 [Improvement] set table name in olap scanner (#10102)
bdcf2e7ed2 is described below
commit bdcf2e7ed2c2c3c012c0cac06bb6772d1d211c7a
Author: Gabriel
AuthorDate: Tue Jun 14 08:18:18 2022 +0800
[Improvement] set table name in olap scanner (#10102)
---
fe/fe-core/src/main/java/org/apache/doris/planner/OlapScanNode.java | 1 +
1 file changed, 1 insertion(+)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/planner/OlapScanNode.java
b/fe/fe-core/src/main/java/org/apache/doris/planner/OlapScanNode.java
index ee02af4b44..fff10e95ad 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/planner/OlapScanNode.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/planner/OlapScanNode.java
@@ -775,6 +775,7 @@ public class OlapScanNode extends ScanNode {
msg.olap_scan_node.setSortColumn(sortColumn);
}
msg.olap_scan_node.setKeyType(olapTable.getKeysType().toThrift());
+msg.olap_scan_node.setTableName(olapTable.getName());
}
// export some tablets
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #10092: [deps] update libhdfs3 to fix a uuid set problem
yangzhg merged PR #10092: URL: https://github.com/apache/incubator-doris/pull/10092 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (bdcf2e7ed2 -> 7cf0cc7dd6)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git from bdcf2e7ed2 [Improvement] set table name in olap scanner (#10102) add 7cf0cc7dd6 [deps] update libhdfs3 to fix a uuid set problem (#10092) No new revisions were added by this update. Summary of changes: thirdparty/CHANGELOG.md | 3 +++ thirdparty/vars.sh | 8 2 files changed, 7 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #10068: [enhancement] support simd instructions on arm cpus through sse2neon
yangzhg merged PR #10068: URL: https://github.com/apache/incubator-doris/pull/10068 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg closed issue #10086: [Bug] [Build] Build fail with clang
yangzhg closed issue #10086: [Bug] [Build] Build fail with clang URL: https://github.com/apache/incubator-doris/issues/10086 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #10087: [Bug][Build] Build fail with clang
yangzhg merged PR #10087: URL: https://github.com/apache/incubator-doris/pull/10087 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [build] use inline to replace static (#10087)
This is an automated email from the ASF dual-hosted git repository.
yangzhg pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new e58cac1f00 [build] use inline to replace static (#10087)
e58cac1f00 is described below
commit e58cac1f00860f4414a028a783dbe1a98319f2fd
Author: Pxl <952130...@qq.com>
AuthorDate: Tue Jun 14 09:18:15 2022 +0800
[build] use inline to replace static (#10087)
---
be/src/olap/rowset/segment_v2/bitshuffle_page.h | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/be/src/olap/rowset/segment_v2/bitshuffle_page.h
b/be/src/olap/rowset/segment_v2/bitshuffle_page.h
index 4a6f3d4052..234bda61b6 100644
--- a/be/src/olap/rowset/segment_v2/bitshuffle_page.h
+++ b/be/src/olap/rowset/segment_v2/bitshuffle_page.h
@@ -219,7 +219,7 @@ private:
return _buffer.build();
}
-typedef typename TypeTraits::CppType CppType;
+using CppType = typename TypeTraits::CppType;
CppType cell(int idx) const {
DCHECK_GE(idx, 0);
@@ -239,7 +239,7 @@ private:
CppType _last_value;
};
-static Status parse_bit_shuffle_header(const Slice& data, size_t& num_elements,
+inline Status parse_bit_shuffle_header(const Slice& data, size_t& num_elements,
size_t& compressed_size, size_t&
num_element_after_padding,
int& size_of_element) {
if (data.size < BITSHUFFLE_PAGE_HEADER_SIZE) {
@@ -425,7 +425,7 @@ private:
memcpy(data, get_data(_cur_index), n * SIZE_OF_TYPE);
}
-typedef typename TypeTraits::CppType CppType;
+using CppType = typename TypeTraits::CppType;
enum { SIZE_OF_TYPE = TypeTraits::size };
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #10048: [doc]Add common error messages to doris backup
yangzhg merged PR #10048: URL: https://github.com/apache/incubator-doris/pull/10048 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [docs] Add common error messages to doris backup (#10048)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new dc4761593b [docs] Add common error messages to doris backup (#10048) dc4761593b is described below commit dc4761593b009261fbe6fa475fb60db935485ba4 Author: caoliang-web <71004656+caoliang-...@users.noreply.github.com> AuthorDate: Tue Jun 14 09:20:04 2022 +0800 [docs] Add common error messages to doris backup (#10048) --- docs/en/docs/admin-manual/data-admin/backup.md| 5 - docs/zh-CN/docs/admin-manual/data-admin/backup.md | 3 +++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/en/docs/admin-manual/data-admin/backup.md b/docs/en/docs/admin-manual/data-admin/backup.md index 52e677c3c8..e3339e68a4 100644 --- a/docs/en/docs/admin-manual/data-admin/backup.md +++ b/docs/en/docs/admin-manual/data-admin/backup.md @@ -145,7 +145,10 @@ It is recommended to import the new and old clusters in parallel for a period of 3. Both backup and recovery support operations at the minimum partition (Partition) level. When the amount of data in the table is large, it is recommended to perform operations by partition to reduce the cost of failed retry. 4. Because of the backup and restore operations, the operations are the actual data files. Therefore, when a table has too many shards, or a shard has too many small versions, it may take a long time to backup or restore even if the total amount of data is small. Users can use `SHOW PARTITIONS FROM table_name;` and `SHOW TABLET FROM table_name;` to view the number of shards in each partition and the number of file versions in each shard to estimate job execution time. The number of files [...] 5. When checking job status via `SHOW BACKUP` or `SHOW RESTORE` command. It is possible to see error messages in the `TaskErrMsg` column. But as long as the `State` column is not `CANCELLED`, the job is still continuing. These tasks may retry successfully. Of course, some Task errors will also directly cause the job to fail. -6. If the recovery job is an overwrite operation (specifying the recovery data to an existing table or partition), then from the `COMMIT` phase of the recovery job, the overwritten data on the current cluster may no longer be restored. If the restore job fails or is canceled at this time, the previous data may be damaged and inaccessible. In this case, the only way to do it is to perform the recovery operation again and wait for the job to complete. Therefore, we recommend that if unnece [...] + Common `TaskErrMsg` errors are as follows: + Q1: Backup to HDFS, the status shows UPLOADING, TaskErrMsg error message: [1: Close broker writer failed, broker:TNetworkAddress(hostname=10.10.0.0, port=8000) msg:errors while close file output stream, cause by: DataStreamer Exception : ] + This is generally a network communication problem. Check the broker log to see if a certain ip or port is blocked. If it is a cloud service, you need to check whether is accessed the intranet. If so, you can add hdfs-site.xml in the broker/conf folder, you need to add dfs.client.use.datanode.hostname=true under the hdfs-site.xml configuration file, and configure the hostname mapping of the HADOOP cluster on the broker node. +7. If the recovery job is an overwrite operation (specifying the recovery data to an existing table or partition), then from the `COMMIT` phase of the recovery job, the overwritten data on the current cluster may no longer be restored. If the restore job fails or is canceled at this time, the previous data may be damaged and inaccessible. In this case, the only way to do it is to perform the recovery operation again and wait for the job to complete. Therefore, we recommend that if unnece [...] ## Related Commands diff --git a/docs/zh-CN/docs/admin-manual/data-admin/backup.md b/docs/zh-CN/docs/admin-manual/data-admin/backup.md index 80a80245e1..b1c32056ee 100644 --- a/docs/zh-CN/docs/admin-manual/data-admin/backup.md +++ b/docs/zh-CN/docs/admin-manual/data-admin/backup.md @@ -145,6 +145,9 @@ BACKUP的更多用法可参考 [这里](../../sql-manual/sql-reference/Data-Defi 3. 备份和恢复都支持最小分区（Partition）级别的操作，当表的数据量很大时，建议按分区分别执行，以降低失败重试的代价。 4. 因为备份恢复操作，操作的都是实际的数据文件。所以当一个表的分片过多，或者一个分片有过多的小版本时，可能即使总数据量很小，依然需要备份或恢复很长时间。用户可以通过 `SHOW PARTITIONS FROM table_name;` 和 `SHOW TABLET FROM table_name;` 来查看各个分区的分片数量，以及各个分片的文件版本数量，来预估作业执行时间。文件数量对作业执行的时间影响非常大，所以建议在建表时，合理规划分区分桶，以避免过多的分片。 5. 当通过 `SHOW BACKUP` 或者 `SHOW RESTORE` 命令查看作业状态时。有可能会在 `TaskErrMsg` 一列中看到错误信息。但只要 `State` 列不为 `CANCELLED`，则说明作业依然在继续。这些 Task 有可能会重试成功。当然，有些 Task 错误，也会直接导致作业失败。 + 常见的`TaskErrMsg`错误如下： + Q1：备份到HDFS，状态显示UPLOADING，TaskErrMsg 错误信息：[1: Close broker writer failed, broker:TNetworkAddress(hostname=10.10.0.0,port=8000) msg:errors while close file output stream, cause by: DataStrea
[GitHub] [incubator-doris] qimumu9406 commented on issue #9992: How to delete data.
qimumu9406 commented on issue #9992: URL: https://github.com/apache/incubator-doris/issues/9992#issuecomment-1154611854 https://user-images.githubusercontent.com/104196507/173474046-3295cad7-68ab-418c-8a03-cb8f2b3c5d1a.png";> https://doris.apache.org/zh-CN/sql-manual/sql-reference/Utility-Statements/DESCRIBE.html#describe -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10055: [Enhancement] tpch q3 rewrite, change join order, make lineitem on left side
github-actions[bot] commented on PR #10055: URL: https://github.com/apache/incubator-doris/pull/10055#issuecomment-1154627926 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] qimumu9406 commented on issue #10097: flink-connector-doris. connect timed out
qimumu9406 commented on issue #10097: URL: https://github.com/apache/incubator-doris/issues/10097#issuecomment-1154627739 1. Confirm the firewall status and normal network access 2. Confirm that the Doris cluster is normal -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yinzhijian commented on pull request #10069: [test] add cast array regression test
yinzhijian commented on PR #10069: URL: https://github.com/apache/incubator-doris/pull/10069#issuecomment-1154628557 > please rebase to pass the BE ut done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10055: [Enhancement] tpch q3 rewrite, change join order, make lineitem on left side
github-actions[bot] commented on PR #10055: URL: https://github.com/apache/incubator-doris/pull/10055#issuecomment-1154627944 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] qimumu9406 commented on issue #10095: [Bug] Error occurs when showing query profile
qimumu9406 commented on issue #10095: URL: https://github.com/apache/incubator-doris/issues/10095#issuecomment-1154629198 Check that the instance exists, if so, can you describe the operation process? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] cambyzju commented on pull request #10069: [test] add cast array regression test
cambyzju commented on PR #10069: URL: https://github.com/apache/incubator-doris/pull/10069#issuecomment-1154632143 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10073: [fix] fix bug that show proc "/cluster_balance/history_tablets" return malformat error
github-actions[bot] commented on PR #10073: URL: https://github.com/apache/incubator-doris/pull/10073#issuecomment-1154635704 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #10073: [fix] fix bug that show proc "/cluster_balance/history_tablets" return malformat error
github-actions[bot] commented on PR #10073: URL: https://github.com/apache/incubator-doris/pull/10073#issuecomment-1154635729 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yinzhijian opened a new pull request, #10109: [spell] Fix spell error in row_batch.h
yinzhijian opened a new pull request, #10109: URL: https://github.com/apache/incubator-doris/pull/10109 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No Need) 3. Has document been added or modified: (No Need) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9579: [Feature] Support read hive external table and outfile into HDFS that authenticated by kerberos
github-actions[bot] commented on PR #9579: URL: https://github.com/apache/incubator-doris/pull/9579#issuecomment-1154638016 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9579: [Feature] Support read hive external table and outfile into HDFS that authenticated by kerberos
github-actions[bot] commented on PR #9579: URL: https://github.com/apache/incubator-doris/pull/9579#issuecomment-1154637997 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] JNSimba opened a new pull request, #10110: [improvement][config] change be 2pc config enbale defalut
JNSimba opened a new pull request, #10110: URL: https://github.com/apache/incubator-doris/pull/10110 # Proposed changes Issue Number: close #xxx ## Problem Summary: default disable_stream_load_2pc=false ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yinzhijian opened a new pull request, #10111: [typo] Fix typos in comments
yinzhijian opened a new pull request, #10111: URL: https://github.com/apache/incubator-doris/pull/10111 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No Need) 3. Has document been added or modified: (No Need) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yinzhijian opened a new pull request, #10112: [typo] Fix typos in runtime_state.cpp
yinzhijian opened a new pull request, #10112: URL: https://github.com/apache/incubator-doris/pull/10112 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No Need) 3. Has document been added or modified: (No Need) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 commented on a diff in pull request #10070: [enhancement] Change array offset type from UInt32 to Int64
adonis0147 commented on code in PR #10070:
URL: https://github.com/apache/incubator-doris/pull/10070#discussion_r896319479
##
be/src/runtime/collection_value.h:
##
@@ -123,21 +123,21 @@ class CollectionValue {
const bool* null_signs() const { return _null_signs; }
void* mutable_data() { return _data; }
bool* mutable_null_signs() { return _null_signs; }
-void set_length(uint32_t length) { _length = length; }
+void set_length(int64_t length) { _length = length; }
void set_has_null(bool has_null) { _has_null = has_null; }
void set_data(void* data) { _data = data; }
void set_null_signs(bool* null_signs) { _null_signs = null_signs; }
private:
using AllocateMemFunc = std::function;
static Status init_collection(CollectionValue* value, const
AllocateMemFunc& allocate,
- uint32_t size, PrimitiveType child_type);
+ int64_t size, PrimitiveType child_type);
ArrayIterator internal_iterator(PrimitiveType child_type) const;
private:
// child column data
void* _data;
-uint32_t _length;
+int64_t _length;
Review Comment:
Using `uint64_t` is better. For some places which need signed value, we use
signed value there.
##
be/src/runtime/collection_value.h:
##
@@ -160,7 +160,7 @@ class ArrayIterator {
}
return false;
}
-bool seek(uint32_t n) const {
+bool seek(int64_t n) const {
Review Comment:
If using `int64_t` here, the function `seek` should check whether `n` is
negative.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] caomingjie-code commented on issue #9992: How to delete data.
caomingjie-code commented on issue #9992: URL: https://github.com/apache/incubator-doris/issues/9992#issuecomment-1154650997 麻烦问一下：这个rollup index 是指：PARTITION BY RANGE( 字段） 这个语句中的字段吗？ 还是说 DISTRIBUTED BY HASH(字段)。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
