[GitHub] [doris] github-actions[bot] commented on pull request #14814: [refactor](date function) simplify function template arguments
github-actions[bot] commented on PR #14814: URL: https://github.com/apache/doris/pull/14814#issuecomment-1336912875 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] ByteYue opened a new pull request, #14815: [chore](regression) add S3 config in conf file and change sf1DataPath
ByteYue opened a new pull request, #14815: URL: https://github.com/apache/doris/pull/14815 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [x] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [x] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname opened a new pull request, #14816: [Fix](Nereids)fix event channel loop until queue is not empty.
sohardforaname opened a new pull request, #14816: URL: https://github.com/apache/doris/pull/14816 # Proposed changes Issue Number: close #xxx ## Problem summary fix event channel loop until queue is not empty by queue.take(). ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #14814: [refactor](date function) simplify function template arguments
hello-stephen commented on PR #14814: URL: https://github.com/apache/doris/pull/14814#issuecomment-1336942448 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.9 seconds load time: 430 seconds storage size: 17123356287 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221205082224_clickbench_pr_57804.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14815: [chore](regression) add S3 config in conf file and change sf1DataPath
github-actions[bot] commented on PR #14815: URL: https://github.com/apache/doris/pull/14815#issuecomment-1336945201 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14815: [chore](regression) add S3 config in conf file and change sf1DataPath
github-actions[bot] commented on PR #14815: URL: https://github.com/apache/doris/pull/14815#issuecomment-1336945238 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #14816: [Fix](Nereids)fix event channel loop until queue is not empty.
hello-stephen commented on PR #14816: URL: https://github.com/apache/doris/pull/14816#issuecomment-1336955409 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.86 seconds load time: 469 seconds storage size: 17123356316 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221205083441_clickbench_pr_57818.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener opened a new issue, #14817: [Enhancement] Cleanup the session variable.
jackwener opened a new issue, #14817: URL: https://github.com/apache/doris/issues/14817 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description Some session variable is outdated. ### Solution _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener opened a new pull request, #14818: [Minor](Planner): remove redundant SessionVariable
jackwener opened a new pull request, #14818: URL: https://github.com/apache/doris/pull/14818 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yixiutt opened a new pull request, #14819: [branch_1.1](compaction) add some log
yixiutt opened a new pull request, #14819: URL: https://github.com/apache/doris/pull/14819 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] liaoxin01 commented on a diff in pull request #14722: [fix](vec) VMergeIterator add key same label for agg table
liaoxin01 commented on code in PR #14722:
URL: https://github.com/apache/doris/pull/14722#discussion_r1039311762
##
be/src/vec/olap/vgeneric_iterators.cpp:
##
@@ -123,6 +123,7 @@ bool VMergeIteratorContext::compare(const
VMergeIteratorContext& rhs) const {
if (_is_unique) {
Review Comment:
Can we use `_is_unique` to skip the same key in the agg model?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14796: [enhancement](compaction) add some trigger and delete useless log
github-actions[bot] commented on PR #14796: URL: https://github.com/apache/doris/pull/14796#issuecomment-1336976443 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] liaoxin01 commented on a diff in pull request #14722: [fix](vec) VMergeIterator add key same label for agg table
liaoxin01 commented on code in PR #14722:
URL: https://github.com/apache/doris/pull/14722#discussion_r1039311762
##
be/src/vec/olap/vgeneric_iterators.cpp:
##
@@ -123,6 +123,7 @@ bool VMergeIteratorContext::compare(const
VMergeIteratorContext& rhs) const {
if (_is_unique) {
Review Comment:
Can we use `_is_unique` to skip the same key in the agg model?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangbo opened a new issue, #14820: [Bug] Fix regression test failed in 1.1 version right outer join
wangbo opened a new issue, #14820:
URL: https://github.com/apache/doris/issues/14820
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no
similar issues.
### Version
branch-1.1-lts
### What's Wrong?
run regression test which suite named right_outer_join_wih_other_pred could
core
```
*** Aborted at 1669292783 (unix time) try "date -d @1669292783" if you are
using GNU date ***
*** SIGSEGV address not mapped to object (@0x0) received by PID 178700 (TID
0x7fd19cb68700) from PID 0; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int,
siginfo_t*, void*) at /root/palo/be/src/common/signal_handler.h:420
1# 0x7FD1E9A49400 in /lib64/libc.so.6
2#
doris::vectorized::ProcessHashTableProbe, std::integral_constant,
true>::do_process_with_other_join_conjunts(doris::vectorized::FixedKeyHashTableContext&, doris::vectorized::PODArray, 15ul, 16ul> const*, doris::vectorized::MutableBlock&,
doris::vectorized::Block*) at
/root/palo/be/src/vec/exec/join/vhash_join_node.cpp:572
3# auto doris::vectorized::HashJoinNode::get_next(doris::RuntimeState*,
doris::vectorized::Block*, bool*)::{lambda(auto:1&&, auto:2&&, auto:3,
auto:4)#2}::operator()&, std::integral_constant&, std::integral_constant,
std::integral_constant
>(doris::vectorized::FixedKeyHashTableContext&,
std::integral_constant&,
std::integral_constant, std::integral_constant) const
[clone .constprop.0] at /root/palo/be/src/vec/exec/join/vhash_join_node.cpp:988
4# doris::vectorized::HashJoinNode::get_next(doris::RuntimeState*,
doris::vectorized::Block*, bool*) at
/root/palo/be/src/vec/exec/join/vhash_join_node.cpp:1033
5# doris::vectorized::VSortNode::sort_input(doris::RuntimeState*) at
/root/palo/be/src/vec/exec/vsort_node.cpp:125
6# doris::vectorized::VSortNode::open(doris::RuntimeState*) at
/root/palo/be/src/vec/exec/vsort_node.cpp:61
7# doris::PlanFragmentExecutor::open_vectorized_internal() at
/root/palo/be/src/runtime/plan_fragment_executor.cpp:281
8# doris::PlanFragmentExecutor::open() at
/root/palo/be/src/runtime/plan_fragment_executor.cpp:256
9# doris::FragmentExecState::execute() at
/root/palo/be/src/runtime/fragment_mgr.cpp:250
10#
doris::FragmentMgr::_exec_actual(std::shared_ptr,
std::function) at
/root/palo/be/src/runtime/fragment_mgr.cpp:484
11# std::_Function_handler, std::function))(std::shared_ptr,
std::function)>
>::_M_invoke(std::_Any_data const&) at
/var/local/ldb-toolchain/include/c++/11/bits/std_function.h:291
12# doris::ThreadPool::dispatch_thread() at
/root/palo/be/src/util/threadpool.cpp:578
13# doris::Thread::supervise_thread(void*) at
/root/palo/be/src/util/thread.cpp:407
14# start_thread in /lib64/libpthread.so.0
15# clone in /lib64/libc.so.6
```
### What You Expected?
return correct result
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangbo opened a new pull request, #14821: [fix](vectorization) Fix right outer join core with conjunct
wangbo opened a new pull request, #14821: URL: https://github.com/apache/doris/pull/14821 # Proposed changes Issue Number: close #14820 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] nextdreamblue opened a new pull request, #14822: [fix][aggregate] agg table and unique table get different result when…
nextdreamblue opened a new pull request, #14822: URL: https://github.com/apache/doris/pull/14822 … non-vectorized when enable_vectorized_engine=false, agg table and unique table get different result, agg table will get replace column value with lower segment id from a rowset, but unique table get the one which has lower segment id. Signed-off-by: nextdreamblue # Proposed changes Issue Number: close #14811 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [x] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14822: [fix](aggregate) agg table and unique table get different result when…
github-actions[bot] commented on PR #14822: URL: https://github.com/apache/doris/pull/14822#issuecomment-1337001319 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14771: [Bug](expr) return error status for illegal expr
github-actions[bot] commented on PR #14771: URL: https://github.com/apache/doris/pull/14771#issuecomment-1337024024 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #14744: [Enhancement](status) Unify error code and enable customed err msg for BE internal errors
github-actions[bot] commented on code in PR #14744:
URL: https://github.com/apache/doris/pull/14744#discussion_r1039351687
##
be/test/runtime/disk_io_mgr_test.cpp:
##
@@ -52,8 +52,8 @@ class DiskIoMgrTest : public testing::Test {
void write_validate_callback(int num_writes, DiskIoMgr::WriteRange**
written_range,
DiskIoMgr* io_mgr, DiskIoMgr::RequestContext*
reader,
int32_t* data, Status expected_status, const
Status& status) {
-if (expected_status.code() == TStatusCode::CANCELLED) {
-EXPECT_TRUE(status.ok() || status.is_cancelled());
+if (expected_status.code() == E_CANCELLED) {
Review Comment:
warning: use of undeclared identifier 'E_CANCELLED' [clang-diagnostic-error]
```cpp
if (expected_status.code() == E_CANCELLED) {
^
```
##
be/test/olap/tablet_meta_manager_test.cpp:
##
@@ -92,7 +93,7 @@ TEST_F(TabletMetaManagerTest, TestSaveAndGetAndRemove) {
EXPECT_EQ(Status::OK(), s);
TabletMetaSharedPtr meta_read(new TabletMeta());
s = TabletMetaManager::get_meta(_data_dir, tablet_id, schema_hash,
meta_read);
Review Comment:
warning: '_data_dir' is a private member of 'doris::TabletMetaManagerTest'
[clang-diagnostic-error]
```cpp
s = TabletMetaManager::get_meta(_data_dir, tablet_id, schema_hash,
meta_read);
^
```
**be/test/olap/tablet_meta_manager_test.cpp:68:** declared private here
```cpp
DataDir* _data_dir;
^
```
##
be/test/runtime/load_channel_mgr_test.cpp:
##
@@ -421,7 +421,7 @@
}
row_batch.serialize(request.mutable_row_batch(), &uncompressed_size,
&compressed_size);
// DeltaWriter's write will return -215
-add_status = Status::OLAPInternalError(OLAP_ERR_TABLE_NOT_FOUND);
+add_status = Status::Error();
Review Comment:
warning: use of undeclared identifier 'TABLE_NOT_FOUND'; did you mean
'ErrorCode::TABLE_NOT_FOUND'? [clang-diagnostic-error]
```suggestion
add_status = Status::Error();
```
**be/src/common/status.h:85:** 'ErrorCode::TABLE_NOT_FOUND' declared here
```cpp
E(TABLE_NOT_FOUND, -215);
^
```
##
be/test/runtime/load_channel_mgr_test.cpp:
##
@@ -512,7 +512,7 @@
row_batch.commit_last_row();
}
row_batch.serialize(request.mutable_row_batch(), &uncompressed_size,
&compressed_size);
-close_status = Status::OLAPInternalError(OLAP_ERR_TABLE_NOT_FOUND);
+close_status = Status::Error();
Review Comment:
warning: use of undeclared identifier 'TABLE_NOT_FOUND'; did you mean
'ErrorCode::TABLE_NOT_FOUND'? [clang-diagnostic-error]
```suggestion
close_status = Status::Error();
```
**be/src/common/status.h:85:** 'ErrorCode::TABLE_NOT_FOUND' declared here
```cpp
E(TABLE_NOT_FOUND, -215);
^
```
##
be/test/runtime/load_channel_mgr_test.cpp:
##
@@ -512,7 +512,7 @@
row_batch.commit_last_row();
}
row_batch.serialize(request.mutable_row_batch(), &uncompressed_size,
&compressed_size);
Review Comment:
warning: 'compressed_size' is a private member of
'doris::LoadChannelMgrTest' [clang-diagnostic-error]
```cpp
row_batch.serialize(request.mutable_row_batch(), &uncompressed_size,
&compressed_size);
^
```
**be/test/runtime/load_channel_mgr_test.cpp:121:** declared private here
```cpp
size_t compressed_size = 0;
^
```
##
be/test/runtime/load_channel_mgr_test.cpp:
##
@@ -335,7 +335,7 @@ TEST_F(LoadChannelMgrTest, open_failed) {
}
request.set_num_senders(1);
request.set_need_gen_rollup(false);
-open_status = Status::OLAPInternalError(OLAP_ERR_TABLE_NOT_FOUND);
+open_status = Status::Error();
Review Comment:
warning: use of undeclared identifier 'TABLE_NOT_FOUND'; did you mean
'ErrorCode::TABLE_NOT_FOUND'? [clang-diagnostic-error]
```suggestion
open_status = Status::Error();
```
**be/src/common/status.h:85:** 'ErrorCode::TABLE_NOT_FOUND' declared here
```cpp
E(TABLE_NOT_FOUND, -215);
^
```
##
be/test/runtime/load_channel_mgr_test.cpp:
##
@@ -421,7 +421,7 @@
}
row_batch.serialize(request.mutable_row_batch(), &uncompressed_size,
&compressed_size);
Review Comment:
warning: 'uncompressed_size' is a private member of
'doris::LoadChannelMgrTest' [clang-diagnostic-error]
```cpp
row_batch.serialize(request.mutable_row_batch(), &uncompressed_size,
&compressed_size);
^
```
**be/test/runtime/load_cha
[GitHub] [doris] jacktengg opened a new issue, #14823: [Bug] coredump of cast expr
jacktengg opened a new issue, #14823:
URL: https://github.com/apache/doris/issues/14823
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no
similar issues.
### Version
master 1190fd4cd62132d3704feb49b16c0fb6cd451a1e
### What's Wrong?
The following sql will cause be coredump:
```
select
ref_0.`game_code` as c1
from
regression_test_usercases_37Wan.dwd_online_detail as ref_0
where
cast(
case
when BITMAP_EMPTY() is NULL then null
else null
end as bitmap
) is NULL;
```
coredump callstack:
```
*** Query id: 5b5d2f3de41c40d9-bba9f60dd4ac5ecc ***
*** Aborted at 1670232381 (unix time) try "date -d @1670232381" if you are
using GNU date ***
*** Current BE git commitID: 0650915bd ***
*** SIGSEGV address not mapped to object (@0x0) received by PID 3102731 (TID
0x7db14f5ff700) from PID 0; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int,
siginfo_t*, void*) at
/mnt/disk1/tengjianping/doris-test/be/src/common/signal_handler.h:420
1# os::Linux::chained_handler(int, siginfo*, void*) in
/mnt/disk1/tengjianping/local/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so
2# JVM_handle_linux_signal in
/mnt/disk1/tengjianping/local/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so
3# signalHandler(int, siginfo*, void*) in
/mnt/disk1/tengjianping/local/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so
4# 0x7F0010EEB400 in /lib64/libc.so.6
5#
doris::vectorized::VScanNode::_eval_const_conjuncts(doris::vectorized::VExpr*,
doris::vectorized::VExprContext*, doris::vectorized::VScanNode::PushDownType*)
at /mnt/disk1/tengjianping/doris-test/be/src/vec/exec/scan/vscan_node.cpp:575
6#
doris::vectorized::VScanNode::_normalize_predicate(doris::vectorized::VExpr*)
at /mnt/disk1/tengjianping/doris-test/be/src/vec/exec/scan/vscan_node.cpp:431
7# doris::vectorized::VScanNode::_normalize_conjuncts() at
/mnt/disk1/tengjianping/doris-test/be/src/vec/exec/scan/vscan_node.cpp:373
8# doris::vectorized::NewOlapScanNode::_process_conjuncts() at
/mnt/disk1/tengjianping/doris-test/be/src/vec/exec/scan/new_olap_scan_node.cpp:171
9# doris::vectorized::VScanNode::open(doris::RuntimeState*) at
/mnt/disk1/tengjianping/doris-test/be/src/vec/exec/scan/vscan_node.cpp:95
10# doris::PlanFragmentExecutor::open_vectorized_internal() at
/mnt/disk1/tengjianping/doris-test/be/src/runtime/plan_fragment_executor.cpp:278
11# doris::PlanFragmentExecutor::open() at
/mnt/disk1/tengjianping/doris-test/be/src/runtime/plan_fragment_executor.cpp:250
12# doris::FragmentExecState::execute() at
/mnt/disk1/tengjianping/doris-test/be/src/runtime/fragment_mgr.cpp:252
13#
doris::FragmentMgr::_exec_actual(std::shared_ptr,
std::function) at
/mnt/disk1/tengjianping/doris-test/be/src/runtime/fragment_mgr.cpp:499
14# std::_Function_handler)::{lambda()#1}>::_M_invoke(std::_Any_data
const&) at
/mnt/disk1/tengjianping/local/ldb_toolchain_clang13/include/c++/11/bits/std_function.h:291
15# doris::ThreadPool::dispatch_thread() at
/mnt/disk1/tengjianping/doris-test/be/src/util/threadpool.cpp:542
16# doris::Thread::supervise_thread(void*) at
/mnt/disk1/tengjianping/doris-test/be/src/util/thread.cpp:455
17# start_thread in /lib64/libpthread.so.0
18# __GI___clone in /lib64/libc.so.6
```
### What You Expected?
no coredump
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new issue, #14824: [Bug] Wrong type returned by function `days_sub`
Gabriel39 opened a new issue, #14824:
URL: https://github.com/apache/doris/issues/14824
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no
similar issues.
### Version
master
### What's Wrong?
Wrong result if run:
mysql [(none)]>select days_sub(cast('2022-01-01' as date), interval cast(2
as int) minute);
+--+
| days_sub(CAST('2022-01-01' AS DATE), INTERVAL CAST(2 AS INT) minute) |
+--+
| 2021-12-31 |
+--+
1 row in set (0.01 sec)
But if I change my sql below, it works well
mysql [(none)]>select days_sub('2022-01-01', interval cast(2 as int) minute);
+-+
| days_sub('2022-01-01 00:00:00', INTERVAL CAST(2 AS INT) minute) |
+-+
| 2021-12-31 23:58:00 |
+-+
1 row in set (0.00 sec)
### What You Expected?
Both of them should return 2021-12-31 23:58:00
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #14822: [fix](aggregate) agg table and unique table get different result when…
hello-stephen commented on PR #14822: URL: https://github.com/apache/doris/pull/14822#issuecomment-1337050612 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.08 seconds load time: 437 seconds storage size: 17123356212 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221205095330_clickbench_pr_57874.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jacktengg opened a new pull request, #14825: [fix](const_expr) fix coredump caused by unsupported cast const expr
jacktengg opened a new pull request, #14825: URL: https://github.com/apache/doris/pull/14825 # Proposed changes Issue Number: close #14823 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #14826: [Bug](date function) Fix date_add function
Gabriel39 opened a new pull request, #14826: URL: https://github.com/apache/doris/pull/14826 # Proposed changes Issue Number: close #14824 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #14814: [refactor](date function) simplify function template arguments
yiguolei merged PR #14814: URL: https://github.com/apache/doris/pull/14814 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [refactor](date function) simplify function template arguments (#14814)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 016e47b857 [refactor](date function) simplify function template
arguments (#14814)
016e47b857 is described below
commit 016e47b857a21d6d0b77fd3ad0d3364a59a67d0d
Author: Gabriel
AuthorDate: Mon Dec 5 17:59:47 2022 +0800
[refactor](date function) simplify function template arguments (#14814)
---
.../function_date_or_datetime_computation.cpp | 99 -
.../function_date_or_datetime_computation.h| 224 +++--
.../function_date_or_datetime_computation_v2.cpp | 174 +++-
3 files changed, 231 insertions(+), 266 deletions(-)
diff --git a/be/src/vec/functions/function_date_or_datetime_computation.cpp
b/be/src/vec/functions/function_date_or_datetime_computation.cpp
index 02dd86b58e..abaef68c4c 100644
--- a/be/src/vec/functions/function_date_or_datetime_computation.cpp
+++ b/be/src/vec/functions/function_date_or_datetime_computation.cpp
@@ -21,63 +21,48 @@
namespace doris::vectorized {
-using FunctionAddSeconds = FunctionDateOrDateTimeComputation<
-AddSecondsImpl>;
-using FunctionAddMinutes = FunctionDateOrDateTimeComputation<
-AddMinutesImpl>;
-using FunctionAddHours =
-FunctionDateOrDateTimeComputation>;
-using FunctionAddDays =
-FunctionDateOrDateTimeComputation>;
-using FunctionAddWeeks =
-FunctionDateOrDateTimeComputation>;
-using FunctionAddMonths =
-FunctionDateOrDateTimeComputation>;
-using FunctionAddQuarters = FunctionDateOrDateTimeComputation<
-AddQuartersImpl>;
-using FunctionAddYears =
-FunctionDateOrDateTimeComputation>;
-
-using FunctionSubSeconds = FunctionDateOrDateTimeComputation<
-SubtractSecondsImpl>;
-using FunctionSubMinutes = FunctionDateOrDateTimeComputation<
-SubtractMinutesImpl>;
-using FunctionSubHours = FunctionDateOrDateTimeComputation<
-SubtractHoursImpl>;
-using FunctionSubDays = FunctionDateOrDateTimeComputation<
-SubtractDaysImpl>;
-using FunctionSubWeeks = FunctionDateOrDateTimeComputation<
-SubtractWeeksImpl>;
-using FunctionSubMonths = FunctionDateOrDateTimeComputation<
-SubtractMonthsImpl>;
-using FunctionSubQuarters = FunctionDateOrDateTimeComputation<
-SubtractQuartersImpl>;
-using FunctionSubYears = FunctionDateOrDateTimeComputation<
-SubtractYearsImpl>;
-
-using FunctionDateDiff = FunctionDateOrDateTimeComputation>;
-using FunctionTimeDiff = FunctionDateOrDateTimeComputation>;
-using FunctionYearsDiff = FunctionDateOrDateTimeComputation>;
-using FunctionMonthsDiff = FunctionDateOrDateTimeComputation>;
-using FunctionDaysDiff = FunctionDateOrDateTimeComputation>;
-using FunctionWeeksDiff = FunctionDateOrDateTimeComputation>;
-using FunctionHoursDiff = FunctionDateOrDateTimeComputation>;
-using FunctionMinutesDiff = FunctionDateOrDateTimeComputation>;
-using FunctionSecondsDiff = FunctionDateOrDateTimeComputation>;
-
-using FunctionToYearWeekTwoArgs = FunctionDateOrDateTimeComputation<
-ToYearWeekTwoArgsImpl>;
-using FunctionToWeekTwoArgs = FunctionDateOrDateTimeComputation<
-ToWeekTwoArgsImpl>;
+using FunctionAddSeconds =
FunctionDateOrDateTimeComputation>;
+using FunctionAddMinutes =
FunctionDateOrDateTimeComputation>;
+using FunctionAddHours =
FunctionDateOrDateTimeComputation>;
+using FunctionAddDays =
FunctionDateOrDateTimeComputation>;
+using FunctionAddWeeks =
FunctionDateOrDateTimeComputation>;
+using FunctionAddMonths =
FunctionDateOrDateTimeComputation>;
+using FunctionAddQuarters =
FunctionDateOrDateTimeComputation>;
+using FunctionAddYears =
FunctionDateOrDateTimeComputation>;
+
+using FunctionSubSeconds =
FunctionDateOrDateTimeComputation>;
+using FunctionSubMinutes =
FunctionDateOrDateTimeComputation>;
+using FunctionSubHours =
FunctionDateOrDateTimeComputation>;
+using FunctionSubDays =
FunctionDateOrDateTimeComputation>;
+using FunctionSubWeeks =
FunctionDateOrDateTimeComputation>;
+using FunctionSubMonths =
FunctionDateOrDateTimeComputation>;
+using FunctionSubQuarters =
+
FunctionDateOrDateTimeComputation>;
+using FunctionSubYears =
FunctionDateOrDateTimeComputation>;
+
+using FunctionDateDiff =
+FunctionDateOrDateTimeComputation>;
+using FunctionTimeDiff =
+FunctionDateOrDateTimeComputation>;
+using FunctionYearsDiff =
+FunctionDateOrDateTimeComputation>;
+using FunctionMonthsDiff =
+FunctionDateOrDateTimeComputation>;
+using FunctionDaysDiff =
+FunctionDateOrDateTimeComputation>;
+using FunctionWeeksDiff =
+FunctionDateOrDateTimeComputation>;
+using FunctionHoursDiff =
+FunctionDateOrDateTimeComputation>;
+using FunctionMinutesDiff =
+FunctionDateOrDateTimeComputation>;
+using FunctionSecondsDiff =
+FunctionDateOrD
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #14825: [fix](const_expr) fix coredump caused by unsupported cast const expr
github-actions[bot] commented on code in PR #14825:
URL: https://github.com/apache/doris/pull/14825#discussion_r1039382236
##
be/src/vec/exec/scan/vscan_node.h:
##
@@ -132,18 +132,21 @@ class VScanNode : public ExecNode {
// 2. in/not in predicate
// 3. function predicate
// TODO: these interfaces should be change to become more common.
-virtual PushDownType _should_push_down_binary_predicate(
+virtual Status _should_push_down_binary_predicate(
VectorizedFnCall* fn_call, VExprContext* expr_ctx, StringRef*
constant_val,
-int* slot_ref_child, const std::function& fn_checker);
+int* slot_ref_child, const std::function& fn_checker,
+PushDownType& pdt);
virtual PushDownType _should_push_down_in_predicate(VInPredicate* in_pred,
VExprContext*
expr_ctx, bool is_not_in);
-virtual PushDownType _should_push_down_function_filter(VectorizedFnCall*
fn_call,
- VExprContext*
expr_ctx,
- StringVal*
constant_str,
-
doris_udf::FunctionContext** fn_ctx) {
-return PushDownType::UNACCEPTABLE;
+virtual Status _should_push_down_function_filter(VectorizedFnCall* fn_call,
Review Comment:
warning: parameter 'fn_call' is unused [misc-unused-parameters]
```suggestion
virtual Status _should_push_down_function_filter(VectorizedFnCall*
/*fn_call*/,
```
##
be/src/vec/exec/scan/vscan_node.h:
##
@@ -132,18 +132,21 @@
// 2. in/not in predicate
// 3. function predicate
// TODO: these interfaces should be change to become more common.
-virtual PushDownType _should_push_down_binary_predicate(
+virtual Status _should_push_down_binary_predicate(
VectorizedFnCall* fn_call, VExprContext* expr_ctx, StringRef*
constant_val,
-int* slot_ref_child, const std::function& fn_checker);
+int* slot_ref_child, const std::function& fn_checker,
+PushDownType& pdt);
virtual PushDownType _should_push_down_in_predicate(VInPredicate* in_pred,
VExprContext*
expr_ctx, bool is_not_in);
-virtual PushDownType _should_push_down_function_filter(VectorizedFnCall*
fn_call,
- VExprContext*
expr_ctx,
- StringVal*
constant_str,
-
doris_udf::FunctionContext** fn_ctx) {
-return PushDownType::UNACCEPTABLE;
+virtual Status _should_push_down_function_filter(VectorizedFnCall* fn_call,
+ VExprContext* expr_ctx,
+ StringVal* constant_str,
Review Comment:
warning: parameter 'constant_str' is unused [misc-unused-parameters]
```suggestion
StringVal*
/*constant_str*/,
```
##
be/src/vec/exec/scan/vscan_node.h:
##
@@ -132,18 +132,21 @@
// 2. in/not in predicate
// 3. function predicate
// TODO: these interfaces should be change to become more common.
-virtual PushDownType _should_push_down_binary_predicate(
+virtual Status _should_push_down_binary_predicate(
VectorizedFnCall* fn_call, VExprContext* expr_ctx, StringRef*
constant_val,
-int* slot_ref_child, const std::function& fn_checker);
+int* slot_ref_child, const std::function& fn_checker,
+PushDownType& pdt);
virtual PushDownType _should_push_down_in_predicate(VInPredicate* in_pred,
VExprContext*
expr_ctx, bool is_not_in);
-virtual PushDownType _should_push_down_function_filter(VectorizedFnCall*
fn_call,
- VExprContext*
expr_ctx,
- StringVal*
constant_str,
-
doris_udf::FunctionContext** fn_ctx) {
-return PushDownType::UNACCEPTABLE;
+virtual Status _should_push_down_function_filter(VectorizedFnCall* fn_call,
+ VExprContext* expr_ctx,
+ StringVal* constant_str,
+
doris_udf::FunctionContext** fn_ctx,
Review Comment:
warning: parameter 'fn_ctx' is unused [misc-unused-parameters]
```suggestion
doris_udf::FunctionContext** /*fn_ctx*/,
```
[GitHub] [doris] github-actions[bot] commented on pull request #14818: [Minor](Planner): remove redundant SessionVariable
github-actions[bot] commented on PR #14818: URL: https://github.com/apache/doris/pull/14818#issuecomment-1337071019 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14818: [Minor](Planner): remove redundant SessionVariable
github-actions[bot] commented on PR #14818: URL: https://github.com/apache/doris/pull/14818#issuecomment-1337071086 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #14216: [Feature](fuction) Support split_by_char function
github-actions[bot] commented on code in PR #14216:
URL: https://github.com/apache/doris/pull/14216#discussion_r1039392700
##
be/src/vec/functions/function_string.h:
##
@@ -1351,6 +1352,137 @@ class FunctionSplitPart : public IFunction {
}
};
+class FunctionSplitByChar : public IFunction {
+public:
+static constexpr auto name = "split_by_char";
+
+static FunctionPtr create() { return
std::make_shared(); }
+using NullMapType = PaddedPODArray;
+
+String get_name() const override { return name; }
+
+bool is_variadic() const override { return false; }
+
+size_t get_number_of_arguments() const override { return 2; }
+
+DataTypePtr get_return_type_impl(const DataTypes& arguments) const
override {
+DCHECK(is_string(arguments[0]))
+<< "first argument for function: " << name << " should be
string"
+<< " and arguments[0] is " << arguments[0]->get_name();
+DCHECK(is_string(arguments[1]))
+<< "second argument for function: " << name << " should be
char"
+<< " and arguments[1] is " << arguments[1]->get_name();
+return std::make_shared(arguments[0]);
+}
+
+Status execute_impl(FunctionContext* context, Block& block, const
ColumnNumbers& arguments,
Review Comment:
warning: parameter 'context' is unused [misc-unused-parameters]
```suggestion
Status execute_impl(FunctionContext* /*context*/, Block& block, const
ColumnNumbers& arguments,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #14742: [enhancement](memory) Add tablet schema cache metrics
yiguolei merged PR #14742: URL: https://github.com/apache/doris/pull/14742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [enhancement](memory) Add tablet schema cache metrics (#14742)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 8726bfa121 [enhancement](memory) Add tablet schema cache metrics
(#14742)
8726bfa121 is described below
commit 8726bfa121bd138453d4ff7285c338e182390f4f
Author: Xinyi Zou
AuthorDate: Mon Dec 5 18:19:13 2022 +0800
[enhancement](memory) Add tablet schema cache metrics (#14742)
---
be/src/olap/tablet_schema.cpp | 26 +-
be/src/olap/tablet_schema.h | 4 ++--
be/src/olap/tablet_schema_cache.h | 8
be/src/runtime/memory/mem_tracker_limiter.cpp | 1 +
be/src/util/doris_metrics.cpp | 6 ++
be/src/util/doris_metrics.h | 3 +++
6 files changed, 21 insertions(+), 27 deletions(-)
diff --git a/be/src/olap/tablet_schema.cpp b/be/src/olap/tablet_schema.cpp
index d9c46888c1..d684806454 100644
--- a/be/src/olap/tablet_schema.cpp
+++ b/be/src/olap/tablet_schema.cpp
@@ -441,18 +441,6 @@ void TabletColumn::to_schema_pb(ColumnPB* column) const {
}
}
-uint32_t TabletColumn::mem_size() const {
-auto size = sizeof(TabletColumn);
-size += _col_name.size();
-if (_has_default_value) {
-size += _default_value.size();
-}
-for (auto& sub_column : _sub_columns) {
-size += sub_column.mem_size();
-}
-return size;
-}
-
void TabletColumn::add_sub_column(TabletColumn& sub_column) {
_sub_columns.push_back(sub_column);
sub_column._parent = this;
@@ -557,6 +545,7 @@ void TabletSchema::clear_columns() {
}
void TabletSchema::init_from_pb(const TabletSchemaPB& schema) {
+SCOPED_MEM_COUNT(&_mem_size);
_keys_type = schema.keys_type();
_num_columns = 0;
_num_key_columns = 0;
@@ -729,19 +718,6 @@ void TabletSchema::to_schema_pb(TabletSchemaPB*
tablet_schema_pb) const {
tablet_schema_pb->set_compression_type(_compression_type);
}
-uint32_t TabletSchema::mem_size() const {
-auto size = sizeof(TabletSchema);
-for (auto& col : _cols) {
-size += col.mem_size();
-}
-
-for (auto& pair : _field_name_to_index) {
-size += pair.first.size();
-size += sizeof(pair.second);
-}
-return size;
-}
-
size_t TabletSchema::row_size() const {
size_t size = 0;
for (auto& column : _cols) {
diff --git a/be/src/olap/tablet_schema.h b/be/src/olap/tablet_schema.h
index 6702901e83..20b05d1801 100644
--- a/be/src/olap/tablet_schema.h
+++ b/be/src/olap/tablet_schema.h
@@ -45,7 +45,6 @@ public:
void init_from_pb(const ColumnPB& column);
void init_from_thrift(const TColumn& column);
void to_schema_pb(ColumnPB* column) const;
-uint32_t mem_size() const;
int32_t unique_id() const { return _unique_id; }
std::string name() const { return _col_name; }
@@ -150,7 +149,7 @@ public:
void append_column(TabletColumn column, bool is_dropped_column = false);
void copy_from(const TabletSchema& tablet_schema);
std::string to_key() const;
-uint32_t mem_size() const;
+int64_t mem_size() const { return _mem_size; };
size_t row_size() const;
int32_t field_index(const std::string& field_name) const;
@@ -243,6 +242,7 @@ private:
int32_t _sequence_col_idx = -1;
int32_t _schema_version = -1;
bool _disable_auto_compaction = false;
+int64_t _mem_size = 0;
};
bool operator==(const TabletSchema& a, const TabletSchema& b);
diff --git a/be/src/olap/tablet_schema_cache.h
b/be/src/olap/tablet_schema_cache.h
index 4dbdd93c6a..47bfcb16f8 100644
--- a/be/src/olap/tablet_schema_cache.h
+++ b/be/src/olap/tablet_schema_cache.h
@@ -24,6 +24,7 @@
#include
#include "olap/tablet_schema.h"
+#include "util/doris_metrics.h"
namespace doris {
@@ -49,6 +50,9 @@ public:
pb.ParseFromString(key);
tablet_schema_ptr->init_from_pb(pb);
_cache[key] = tablet_schema_ptr;
+DorisMetrics::instance()->tablet_schema_cache_count->increment(1);
+
DorisMetrics::instance()->tablet_schema_cache_memory_bytes->increment(
+tablet_schema_ptr->mem_size());
return tablet_schema_ptr;
}
return iter->second;
@@ -63,8 +67,12 @@ private:
for (;;) {
std::this_thread::sleep_for(std::chrono::seconds(tablet_schema_cache_recycle_interval));
std::lock_guard guard(_mtx);
+LOG(INFO) << "Tablet Schema Cache Capacity " << _cache.size();
for (auto iter = _cache.begin(), last = _cache.end(); iter !=
last;) {
if (iter->second.unique()) {
+
DorisMetrics::instance()->tablet_schema_cache_memory_bytes->increment(
+-iter->second->mem_size());
+
DorisMetrics::instance()->tablet_schema_cache_count
[GitHub] [doris] yiguolei closed pull request #14769: [fix](runtime profile) Fix HighWaterMarkCounter update max Bug
yiguolei closed pull request #14769: [fix](runtime profile) Fix HighWaterMarkCounter update max Bug URL: https://github.com/apache/doris/pull/14769 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](runtime profile) Fix HighWaterMarkCounter update max Bug #14769
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 382d35c7e1 [fix](runtime profile) Fix HighWaterMarkCounter update max
Bug #14769
382d35c7e1 is described below
commit 382d35c7e13edd838e6eecf7fc9e9f17bfe4
Author: Xinyi Zou
AuthorDate: Mon Dec 5 18:22:48 2022 +0800
[fix](runtime profile) Fix HighWaterMarkCounter update max Bug #14769
---
be/src/util/runtime_profile.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/be/src/util/runtime_profile.h b/be/src/util/runtime_profile.h
index 606e127274..c80507110e 100644
--- a/be/src/util/runtime_profile.h
+++ b/be/src/util/runtime_profile.h
@@ -120,9 +120,9 @@ public:
HighWaterMarkCounter(TUnit::type unit) : Counter(unit),
current_value_(0) {}
virtual void add(int64_t delta) {
-int64_t new_val = current_value_.fetch_add(delta,
std::memory_order_relaxed);
+current_value_.fetch_add(delta, std::memory_order_relaxed);
if (delta > 0) {
-UpdateMax(new_val);
+UpdateMax(current_value_);
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #14825: [fix](const_expr) fix coredump caused by unsupported cast const expr
hello-stephen commented on PR #14825: URL: https://github.com/apache/doris/pull/14825#issuecomment-1337096495 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.66 seconds load time: 423 seconds storage size: 17123356240 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221205102541_clickbench_pr_57890.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] 924060929 opened a new pull request, #14827: (Refactor) (Nerieds) Refactor aggregate function/plan/rules and support related cbo rules
924060929 opened a new pull request, #14827: URL: https://github.com/apache/doris/pull/14827 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HappenLee merged pull request #14787: [Refactor](pipeline) Refactor operator and builder code of pipeline
HappenLee merged PR #14787: URL: https://github.com/apache/doris/pull/14787 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #14801: [Chore](clang-tidy) remove misc-unused-parameters check
BiteThet commented on PR #14801: URL: https://github.com/apache/doris/pull/14801#issuecomment-1337108444 > Yes, but I prefer clang-tidy do not produce a warning on here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (382d35c7e1 -> b30cd86e9e)
This is an automated email from the ASF dual-hosted git repository.
lihaopeng pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
from 382d35c7e1 [fix](runtime profile) Fix HighWaterMarkCounter update max
Bug #14769
add b30cd86e9e [Refactor](pipeline) Refactor operator and builder code of
pipeline (#14787)
No new revisions were added by this update.
Summary of changes:
be/src/exec/exec_node.cpp | 9 -
be/src/exec/exec_node.h| 16 +-
be/src/pipeline/CMakeLists.txt | 5 +-
be/src/pipeline/exec/aggregation_sink_operator.cpp | 53 +---
be/src/pipeline/exec/aggregation_sink_operator.h | 34 +--
.../pipeline/exec/aggregation_source_operator.cpp | 41 +--
be/src/pipeline/exec/aggregation_source_operator.h | 22 +-
be/src/pipeline/exec/empty_set_operator.cpp| 20 +-
be/src/pipeline/exec/empty_set_operator.h | 29 +-
be/src/pipeline/exec/exchange_sink_operator.cpp| 58 ++--
be/src/pipeline/exec/exchange_sink_operator.h | 40 +--
be/src/pipeline/exec/exchange_source_operator.cpp | 31 +--
be/src/pipeline/exec/exchange_source_operator.h| 29 +-
be/src/pipeline/exec/olap_scan_operator.cpp| 33 ---
be/src/pipeline/exec/olap_scan_operator.h | 46
be/src/pipeline/exec/operator.cpp | 39 +--
be/src/pipeline/exec/operator.h| 294 +
be/src/pipeline/exec/repeat_operator.cpp | 56 +---
be/src/pipeline/exec/repeat_operator.h | 27 +-
be/src/pipeline/exec/result_sink_operator.cpp | 43 +--
be/src/pipeline/exec/result_sink_operator.h| 37 +--
be/src/pipeline/exec/scan_operator.cpp | 30 +--
be/src/pipeline/exec/scan_operator.h | 25 +-
be/src/pipeline/exec/sort_sink_operator.cpp| 27 +-
be/src/pipeline/exec/sort_sink_operator.h | 35 +--
be/src/pipeline/exec/sort_source_operator.cpp | 30 +--
be/src/pipeline/exec/sort_source_operator.h| 33 +--
.../exec/streaming_aggregation_sink_operator.cpp | 37 +--
.../exec/streaming_aggregation_sink_operator.h | 41 ++-
.../exec/streaming_aggregation_source_operator.cpp | 38 +--
.../exec/streaming_aggregation_source_operator.h | 26 +-
.../exec/table_function_operator.cpp} | 10 +-
be/src/pipeline/exec/table_function_operator.h | 68 +
be/src/pipeline/pipeline.cpp | 1 -
be/src/pipeline/pipeline.h | 2 +-
be/src/pipeline/pipeline_fragment_context.cpp | 59 ++---
be/src/pipeline/task_scheduler.h | 1 -
be/src/vec/exec/scan/new_olap_scan_node.cpp| 1 -
be/src/vec/exec/scan/vscan_node.h | 2 +
be/src/vec/exec/vaggregation_node.h| 4 +-
be/src/vec/sink/vdata_stream_sender.h | 2 +
be/src/vec/sink/vresult_sink.h | 2 +
42 files changed, 469 insertions(+), 967 deletions(-)
delete mode 100644 be/src/pipeline/exec/olap_scan_operator.cpp
delete mode 100644 be/src/pipeline/exec/olap_scan_operator.h
copy be/src/{runtime/result_writer.cpp =>
pipeline/exec/table_function_operator.cpp} (83%)
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #14826: [Bug](date function) Fix date_add function
hello-stephen commented on PR #14826: URL: https://github.com/apache/doris/pull/14826#issuecomment-1337113612 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 36.66 seconds load time: 434 seconds storage size: 17123356244 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221205104018_clickbench_pr_57896.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dutyu opened a new pull request, #14828: [fix](statistics) fix missing scanBytes and scanRows in query statist…
dutyu opened a new pull request, #14828: URL: https://github.com/apache/doris/pull/14828 …ics when enable_vectorized_engine=true. # Proposed changes A patch for [ISSUE-14750](https://github.com/apache/doris/pull/14750). There's one modification missing in ISSUE-14750. ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [x] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #14828: [fix](statistics) fix missing scanBytes and scanRows in query statist…
github-actions[bot] commented on code in PR #14828:
URL: https://github.com/apache/doris/pull/14828#discussion_r1039431796
##
be/src/vec/exec/vexchange_node.cpp:
##
@@ -116,6 +116,12 @@ void VExchangeNode::release_resource(RuntimeState* state) {
}
}
+Status ExchangeNode::collect_query_statistics(QueryStatistics* statistics) {
Review Comment:
warning: use of undeclared identifier 'ExchangeNode' [clang-diagnostic-error]
```cpp
Status ExchangeNode::collect_query_statistics(QueryStatistics* statistics) {
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #14396: [Feature](function) Support width_bucket function
github-actions[bot] commented on code in PR #14396:
URL: https://github.com/apache/doris/pull/14396#discussion_r1039444924
##
be/src/vec/functions/function_width_bucket.cpp:
##
@@ -0,0 +1,154 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "vec/columns/columns_number.h"
+#include "vec/data_types/data_type_nullable.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+#include "vec/functions/simple_function_factory.h"
+
+namespace doris::vectorized {
+class FunctionWidthBucket : public IFunction {
+public:
+static constexpr auto name = "width_bucket";
+static FunctionPtr create() { return
std::make_shared(); }
+
+/// Get function name.
+String get_name() const override { return name; }
+
+bool is_variadic() const override { return false; }
+
+size_t get_number_of_arguments() const override { return 4; }
+
+DataTypePtr get_return_type_impl(const DataTypes& arguments) const
override {
+if (arguments[0]->is_nullable()) {
+return make_nullable(std::make_shared());
+} else {
+return std::make_shared();
+}
+}
+
+Status execute_impl(FunctionContext* context, Block& block, const
ColumnNumbers& arguments,
Review Comment:
warning: parameter 'context' is unused [misc-unused-parameters]
```suggestion
Status execute_impl(FunctionContext* /*context*/, Block& block, const
ColumnNumbers& arguments,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener opened a new pull request, #14829: [enhance](Nereids): add DpHyperJob into Planner.
jackwener opened a new pull request, #14829: URL: https://github.com/apache/doris/pull/14829 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #14828: [fix](statistics) fix missing scanBytes and scanRows in query statist…
github-actions[bot] commented on code in PR #14828:
URL: https://github.com/apache/doris/pull/14828#discussion_r1039457905
##
be/src/vec/exec/vexchange_node.cpp:
##
@@ -116,6 +116,12 @@ void VExchangeNode::release_resource(RuntimeState* state) {
}
}
+Status VExchangeNode::collect_query_statistics(QueryStatistics* statistics) {
Review Comment:
warning: use of undeclared identifier 'ExchangeNode' [clang-diagnostic-error]
```cpp
Status ExchangeNode::collect_query_statistics(QueryStatistics* statistics) {
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yuxuan-luo opened a new pull request, #14830: [fix](build)fix get params display exception
yuxuan-luo opened a new pull request, #14830: URL: https://github.com/apache/doris/pull/14830 ## Problem summary fix get params display exception ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [√ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ √] No Need 3. Has document been added or modified: - [ ] Yes - [ √] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [√ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [√ ] No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] starocean999 opened a new pull request, #14831: [fix](agg)having clause should use alias if there is no group by clause
starocean999 opened a new pull request, #14831: URL: https://github.com/apache/doris/pull/14831 # Proposed changes Issue Number: close https://github.com/apache/doris/issues/14790 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14830: [fix](build)fix get params display exception
github-actions[bot] commented on PR #14830: URL: https://github.com/apache/doris/pull/14830#issuecomment-1337181339 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14830: [fix](build)fix get params display exception
github-actions[bot] commented on PR #14830: URL: https://github.com/apache/doris/pull/14830#issuecomment-1337181387 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14826: [Bug](date function) Fix date_add function
github-actions[bot] commented on PR #14826: URL: https://github.com/apache/doris/pull/14826#issuecomment-1337203188 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14826: [Bug](date function) Fix date_add function
github-actions[bot] commented on PR #14826: URL: https://github.com/apache/doris/pull/14826#issuecomment-1337203229 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring merged pull request #14819: [branch_1.1](compaction) add some log and fix promotion size bug
dataroaring merged PR #14819: URL: https://github.com/apache/doris/pull/14819 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.1-lts updated: [branch_1.1](compaction) add some log (#14819)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-1.1-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.1-lts by this push:
new bd9606d317 [branch_1.1](compaction) add some log (#14819)
bd9606d317 is described below
commit bd9606d317e83f8c4c0ddb4a3c8f5c2cf62284ca
Author: yixiutt <102007456+yixi...@users.noreply.github.com>
AuthorDate: Mon Dec 5 19:53:52 2022 +0800
[branch_1.1](compaction) add some log (#14819)
---
be/src/common/config.h | 2 +-
be/src/olap/compaction.cpp | 1 +
be/src/olap/cumulative_compaction_policy.cpp | 20 ++--
be/src/olap/cumulative_compaction_policy.h | 12 +---
be/src/olap/tablet.cpp | 2 +-
be/src/olap/tablet.h | 10 ++
be/test/olap/cumulative_compaction_policy_test.cpp | 11 ++-
7 files changed, 30 insertions(+), 28 deletions(-)
diff --git a/be/src/common/config.h b/be/src/common/config.h
index d3b45ba109..99092c0ff8 100644
--- a/be/src/common/config.h
+++ b/be/src/common/config.h
@@ -262,7 +262,7 @@
CONF_mInt64(cumulative_size_based_compaction_lower_size_mbytes, "64");
// cumulative compaction policy: min and max delta file's number
CONF_mInt64(min_cumulative_compaction_num_singleton_deltas, "5");
-CONF_mInt64(max_cumulative_compaction_num_singleton_deltas, "1000");
+CONF_mInt64(max_cumulative_compaction_num_singleton_deltas, "100");
// if compaction of a tablet failed, this tablet should not be chosen to
// compaction until this interval passes.
diff --git a/be/src/olap/compaction.cpp b/be/src/olap/compaction.cpp
index f1240a3648..104dc59cef 100644
--- a/be/src/olap/compaction.cpp
+++ b/be/src/olap/compaction.cpp
@@ -208,6 +208,7 @@ OLAPStatus Compaction::do_compaction_impl(int64_t permits) {
LOG(INFO) << "succeed to do " << compaction_name() << ". tablet=" <<
_tablet->full_name()
<< ", output_version=" << _output_version
<< ", current_max_version=" << current_max_version
+ << ", cumulative_point=" << _tablet->cumulative_layer_point()
<< ", disk=" << _tablet->data_dir()->path() << ", segments=" <<
segments_num
<< ". elapsed time=" << watch.get_elapse_second() << "s.
cumulative_compaction_policy="
<< (cumu_policy == nullptr ? "quick" : cumu_policy->name());
diff --git a/be/src/olap/cumulative_compaction_policy.cpp
b/be/src/olap/cumulative_compaction_policy.cpp
index 67e4d4aa08..943ee2a037 100644
--- a/be/src/olap/cumulative_compaction_policy.cpp
+++ b/be/src/olap/cumulative_compaction_policy.cpp
@@ -76,7 +76,7 @@ void
SizeBasedCumulativeCompactionPolicy::calculate_cumulative_point(
CHECK((*base_rowset_meta)->start_version() == 0);
int64_t promotion_size = 0;
-_calc_promotion_size(*base_rowset_meta, &promotion_size);
+_calc_promotion_size(tablet, *base_rowset_meta, &promotion_size);
int64_t prev_version = -1;
for (const RowsetMetaSharedPtr& rs : existing_rss) {
@@ -118,7 +118,7 @@ void
SizeBasedCumulativeCompactionPolicy::calculate_cumulative_point(
}
}
-void
SizeBasedCumulativeCompactionPolicy::_calc_promotion_size(RowsetMetaSharedPtr
base_rowset_meta,
+void SizeBasedCumulativeCompactionPolicy::_calc_promotion_size(Tablet* tablet,
RowsetMetaSharedPtr base_rowset_meta,
int64_t*
promotion_size) {
int64_t base_size = base_rowset_meta->total_disk_size();
*promotion_size = base_size * _size_based_promotion_ratio;
@@ -129,12 +129,12 @@ void
SizeBasedCumulativeCompactionPolicy::_calc_promotion_size(RowsetMetaSharedP
} else if (*promotion_size <= _size_based_promotion_min_size) {
*promotion_size = _size_based_promotion_min_size;
}
-_refresh_tablet_size_based_promotion_size(*promotion_size);
+_refresh_tablet_size_based_promotion_size(tablet, *promotion_size);
}
void
SizeBasedCumulativeCompactionPolicy::_refresh_tablet_size_based_promotion_size(
-int64_t promotion_size) {
-_tablet_size_based_promotion_size = promotion_size;
+Tablet* tablet, int64_t promotion_size) {
+tablet->set_cumulative_promotion_size(promotion_size);
}
void SizeBasedCumulativeCompactionPolicy::update_cumulative_point(
@@ -151,13 +151,13 @@ void
SizeBasedCumulativeCompactionPolicy::update_cumulative_point(
// if rowsets have no delete version, check output_rowset total disk
size
// satisfies promotion size.
size_t total_size = output_rowset->rowset_meta()->total_disk_size();
-if (total_size >= _tablet_size_based_promotion_size) {
+if (total_size >= tablet->cumulative_promotion_size()) {
tablet->set_cumulative_layer_point(output_rowset->end_versi
[GitHub] [doris] HappenLee opened a new pull request, #14832: [Exec](Profile) Register to Fetch Result time and Write Result time in FE to debug
HappenLee opened a new pull request, #14832: URL: https://github.com/apache/doris/pull/14832 # Proposed changes Before： ``` Execution Summary: - Analysis Time: 339.915us - Plan Time: 649.976us - Schedule Time: 14.412ms - Wait and Fetch Result Time: 4s548ms ``` After ``` Execution Summary: - Analysis Time: 339.915us - Plan Time: 649.976us - Schedule Time: 14.412ms - Wait and Fetch Result Time: 4s548ms ``` ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14832: [Exec](Profile) Register to Fetch Result time and Write Result time in FE to debug
github-actions[bot] commented on PR #14832: URL: https://github.com/apache/doris/pull/14832#issuecomment-1337219420 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14832: [Exec](Profile) Register to Fetch Result time and Write Result time in FE to debug
github-actions[bot] commented on PR #14832: URL: https://github.com/apache/doris/pull/14832#issuecomment-1337219495 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14796: [enhancement](compaction) add some trigger and delete useless log
github-actions[bot] commented on PR #14796: URL: https://github.com/apache/doris/pull/14796#issuecomment-1337242375 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] lide-reed merged pull request #14830: [fix](build)fix get params display exception
lide-reed merged PR #14830: URL: https://github.com/apache/doris/pull/14830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.1-lts updated: [fix](build)fix a typo of the build param string in build script (#14830)
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch branch-1.1-lts in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-1.1-lts by this push: new 1aec9318b6 [fix](build)fix a typo of the build param string in build script (#14830) 1aec9318b6 is described below commit 1aec9318b680b152596843c6dad580ba3fff3f00 Author: yuxuan-luo <119841515+yuxuan-...@users.noreply.github.com> AuthorDate: Mon Dec 5 20:18:14 2022 +0800 [fix](build)fix a typo of the build param string in build script (#14830) --- build.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build.sh b/build.sh index faa55feea1..7109fbe1b4 100755 --- a/build.sh +++ b/build.sh @@ -237,7 +237,7 @@ echo "Get params: BUILD_BE-- $BUILD_BE BUILD_FE-- $BUILD_FE BUILD_BROKER-- $BUILD_BROKER -BUILD_BROKER-- $BUILD_AUDIT +BUILD_AUDIT -- $BUILD_AUDIT BUILD_UI-- $BUILD_UI BUILD_SPARK_DPP -- $BUILD_SPARK_DPP PARALLEL-- $PARALLEL - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #14818: [Minor](Planner): remove redundant SessionVariable
hello-stephen commented on PR #14818: URL: https://github.com/apache/doris/pull/14818#issuecomment-1337252156 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.88 seconds load time: 453 seconds storage size: 17123356295 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221205122103_clickbench_pr_57972.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris-website] luzhijing closed pull request #169: Update JD_OLAP.md
luzhijing closed pull request #169: Update JD_OLAP.md URL: https://github.com/apache/doris-website/pull/169 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris-website] luzhijing closed pull request #168: [doc](blogs)typo in JD_OLAP.md
luzhijing closed pull request #168: [doc](blogs)typo in JD_OLAP.md URL: https://github.com/apache/doris-website/pull/168 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 closed issue #14824: [Bug] Wrong type returned by function `days_sub`
Gabriel39 closed issue #14824: [Bug] Wrong type returned by function `days_sub` URL: https://github.com/apache/doris/issues/14824 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 merged pull request #14826: [Bug](date function) Fix date_add function
Gabriel39 merged PR #14826: URL: https://github.com/apache/doris/pull/14826 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Bug](date function) Fix date_add function (#14826)
This is an automated email from the ASF dual-hosted git repository.
gabriellee pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new db4524c10e [Bug](date function) Fix date_add function (#14826)
db4524c10e is described below
commit db4524c10e79c09480f1bda63b7ff8abf38f8cd9
Author: Gabriel
AuthorDate: Mon Dec 5 20:34:20 2022 +0800
[Bug](date function) Fix date_add function (#14826)
---
.../java/org/apache/doris/analysis/DateLiteral.java| 18 ++
1 file changed, 18 insertions(+)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java
index 83f4473114..8c2fc20303 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java
@@ -909,14 +909,32 @@ public class DateLiteral extends LiteralExpr {
}
public DateLiteral plusHours(int hour) throws AnalysisException {
+if (type.isDate()) {
+return new DateLiteral(getTimeFormatter().plusHours(hour),
Type.DATETIME);
+}
+if (type.isDateV2()) {
+return new DateLiteral(getTimeFormatter().plusHours(hour),
Type.DATETIMEV2);
+}
return new DateLiteral(getTimeFormatter().plusHours(hour), type);
}
public DateLiteral plusMinutes(int minute) {
+if (type.isDate()) {
+return new DateLiteral(getTimeFormatter().plusMinutes(minute),
Type.DATETIME);
+}
+if (type.isDateV2()) {
+return new DateLiteral(getTimeFormatter().plusMinutes(minute),
Type.DATETIMEV2);
+}
return new DateLiteral(getTimeFormatter().plusMinutes(minute), type);
}
public DateLiteral plusSeconds(int second) {
+if (type.isDate()) {
+return new DateLiteral(getTimeFormatter().plusSeconds(second),
Type.DATETIME);
+}
+if (type.isDateV2()) {
+return new DateLiteral(getTimeFormatter().plusSeconds(second),
Type.DATETIMEV2);
+}
return new DateLiteral(getTimeFormatter().plusSeconds(second), type);
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #14828: [fix](statistics) fix missing scanBytes and scanRows in query statist…
hello-stephen commented on PR #14828: URL: https://github.com/apache/doris/pull/14828#issuecomment-1337274073 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.77 seconds load time: 438 seconds storage size: 17123356275 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221205124025_clickbench_pr_57986.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #14829: [enhance](Nereids): add DpHyperJob into Planner.
hello-stephen commented on PR #14829: URL: https://github.com/apache/doris/pull/14829#issuecomment-1337276572 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.91 seconds load time: 428 seconds storage size: 17123356267 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221205124243_clickbench_pr_57993.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #14831: [fix](agg)having clause should use alias if there is no group by clause
hello-stephen commented on PR #14831: URL: https://github.com/apache/doris/pull/14831#issuecomment-1337297603 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 36.66 seconds load time: 436 seconds storage size: 17123356270 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221205125954_clickbench_pr_58006.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #14832: [Exec](Profile) Register to Fetch Result time and Write Result time in FE to debug
hello-stephen commented on PR #14832: URL: https://github.com/apache/doris/pull/14832#issuecomment-1337300938 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.92 seconds load time: 424 seconds storage size: 17123356316 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221205130225_clickbench_pr_58021.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14775: [improvement](config) update high_priority_flush_thread_num_per_store default value to 6
github-actions[bot] commented on PR #14775: URL: https://github.com/apache/doris/pull/14775#issuecomment-1337300974 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring opened a new pull request, #14833: [fix](test) resolve view exists error
dataroaring opened a new pull request, #14833: URL: https://github.com/apache/doris/pull/14833 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xinyiZzz merged pull request #14776: [fix](tcmalloc_gc) optimize policy of tcmalloc gc
xinyiZzz merged PR #14776: URL: https://github.com/apache/doris/pull/14776 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](tcmalloc_gc) optimize policy of tcmalloc gc (#14776)
This is an automated email from the ASF dual-hosted git repository.
zouxinyi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 07472f7318 [fix](tcmalloc_gc) optimize policy of tcmalloc gc (#14776)
07472f7318 is described below
commit 07472f731819745e86da3ba6d4b81aa3f70a83df
Author: Yongqiang YANG <98214048+dataroar...@users.noreply.github.com>
AuthorDate: Mon Dec 5 21:16:35 2022 +0800
[fix](tcmalloc_gc) optimize policy of tcmalloc gc (#14776)
Release memory when memory pressure is above pressure limit and keep at
lease 2% memory as tcmalloc cache.
---
be/src/common/daemon.cpp | 21 +
1 file changed, 13 insertions(+), 8 deletions(-)
diff --git a/be/src/common/daemon.cpp b/be/src/common/daemon.cpp
index 0be20cd377..3687f99fc2 100644
--- a/be/src/common/daemon.cpp
+++ b/be/src/common/daemon.cpp
@@ -80,7 +80,7 @@ void Daemon::tcmalloc_gc_thread() {
double release_rates[10] = {1.0, 1.0, 1.0, 5.0, 5.0, 20.0, 50.0, 100.0,
500.0, 2000.0};
int64_t pressure_limit = 90;
bool is_performance_mode = false;
-size_t physical_limit_bytes =
+int64_t physical_limit_bytes =
std::min(MemInfo::physical_mem() -
MemInfo::sys_mem_available_low_water_mark(),
MemInfo::mem_limit());
@@ -115,13 +115,16 @@ void Daemon::tcmalloc_gc_thread() {
&tc_alloc_bytes);
MallocExtension::instance()->GetNumericProperty("generic.current_allocated_bytes",
&tc_used_bytes);
-int64_t tc_cached_bytes = tc_alloc_bytes - tc_used_bytes;
+int64_t tc_cached_bytes = (int64_t)tc_alloc_bytes -
(int64_t)tc_used_bytes;
int64_t to_free_bytes =
-(int64_t)tc_cached_bytes - (tc_used_bytes * max_cache_percent
/ 100);
+(int64_t)tc_cached_bytes - ((int64_t)tc_used_bytes *
max_cache_percent / 100);
+to_free_bytes = std::max(to_free_bytes, (int64_t)0);
int64_t memory_pressure = 0;
+int64_t rss_pressure = 0;
int64_t alloc_bytes = std::max(rss, tc_alloc_bytes);
memory_pressure = alloc_bytes * 100 / physical_limit_bytes;
+rss_pressure = rss * 100 / physical_limit_bytes;
expected_aggressive_decommit = init_aggressive_decommit;
if (memory_pressure > pressure_limit) {
@@ -129,11 +132,13 @@ void Daemon::tcmalloc_gc_thread() {
// Ideally, we should reuse cache and not allocate from system any
more,
// however, it is hard to set limit on cache of tcmalloc and doris
// use mmap in vectorized mode.
-if (last_memory_pressure <= pressure_limit) {
+// Limit cache capactiy is enough.
+if (rss_pressure > pressure_limit) {
int64_t min_free_bytes = alloc_bytes - physical_limit_bytes *
9 / 10;
to_free_bytes = std::max(to_free_bytes, min_free_bytes);
to_free_bytes = std::max(to_free_bytes, tc_cached_bytes * 30 /
100);
-to_free_bytes = std::min(to_free_bytes, tc_cached_bytes);
+// We assure that we have at least 500M bytes in cache.
+to_free_bytes = std::min(to_free_bytes, tc_cached_bytes - 500
* 1024 * 1024);
expected_aggressive_decommit = 1;
}
last_ms = kMaxLastMs;
@@ -147,7 +152,7 @@ void Daemon::tcmalloc_gc_thread() {
int release_rate_index = memory_pressure / 10;
double release_rate = 1.0;
-if (release_rate_index >= sizeof(release_rates)) {
+if (release_rate_index >= sizeof(release_rates) /
sizeof(release_rates[0])) {
release_rate = 2000.0;
} else {
release_rate = release_rates[release_rate_index];
@@ -161,7 +166,8 @@ void Daemon::tcmalloc_gc_thread() {
}
last_memory_pressure = memory_pressure;
-if (to_free_bytes > 0) {
+// We release at least 2% bytes once, frequent releasing hurts
performance.
+if (to_free_bytes > (physical_limit_bytes * 2 / 100)) {
last_ms += kIntervalMs;
if (last_ms >= kMaxLastMs) {
LOG(INFO) << "generic.current_allocated_bytes " <<
tc_used_bytes
@@ -175,7 +181,6 @@ void Daemon::tcmalloc_gc_thread() {
last_ms = 0;
}
} else {
-DCHECK(tc_cached_bytes <= tc_used_bytes * max_cache_percent / 100);
last_ms = 0;
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris-flink-connector] DongLiang-0 opened a new pull request, #90: [fix]Fix the NullPointerException caused by empty label when serializing
DongLiang-0 opened a new pull request, #90: URL: https://github.com/apache/doris-flink-connector/pull/90 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] lsy3993 opened a new pull request, #14834: [test](catalog)add emr catalog hive case
lsy3993 opened a new pull request, #14834: URL: https://github.com/apache/doris/pull/14834 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [x] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [x] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #14835: [Pipeline](dategen) Support datagen node
Gabriel39 opened a new pull request, #14835: URL: https://github.com/apache/doris/pull/14835 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14835: [Pipeline](dategen) Support datagen node
github-actions[bot] commented on PR #14835: URL: https://github.com/apache/doris/pull/14835#issuecomment-1337380359 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #14835: [Pipeline](dategen) Support datagen node
hello-stephen commented on PR #14835: URL: https://github.com/apache/doris/pull/14835#issuecomment-1337453087 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.03 seconds load time: 433 seconds storage size: 17123356321 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221205142118_clickbench_pr_58086.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] lsy3993 closed pull request #14834: [test](catalog)add emr catalog hive case
lsy3993 closed pull request #14834: [test](catalog)add emr catalog hive case URL: https://github.com/apache/doris/pull/14834 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yixiutt opened a new pull request, #14836: [bugfix](compaction) fix promotion size bug
yixiutt opened a new pull request, #14836: URL: https://github.com/apache/doris/pull/14836 # Proposed changes tablet_promotion_size can modify by all tablet cause cumulative_compaction_policy is global. fix: add a member for tablet to record promotion size ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #14836: [bugfix](compaction) fix promotion size bug
github-actions[bot] commented on code in PR #14836:
URL: https://github.com/apache/doris/pull/14836#discussion_r1039681888
##
be/src/olap/page_cache.cpp:
##
@@ -64,7 +64,7 @@ bool StoragePageCache::lookup(const CacheKey& key,
PageCacheHandle* handle,
void StoragePageCache::insert(const CacheKey& key, const Slice& data,
PageCacheHandle* handle,
segment_v2::PageTypePB page_type, bool
in_memory) {
-auto deleter = [](const doris::CacheKey& key, void* value) { delete[]
(uint8_t*)value; };
+auto deleter = [](const doris::CacheKey& key, void* value) {
delete[](uint8_t*) value; };
Review Comment:
warning: parameter 'key' is unused [misc-unused-parameters]
```suggestion
auto deleter = [](const doris::CacheKey& /*key*/, void* value) {
delete[](uint8_t*) value; };
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #14827: (Refactor) (Nerieds) Refactor aggregate function/plan/rules and support related cbo rules
hello-stephen commented on PR #14827: URL: https://github.com/apache/doris/pull/14827#issuecomment-1337491743 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.46 seconds load time: 451 seconds storage size: 17123356271 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221205143926_clickbench_pr_58109.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14836: [bugfix](compaction) fix promotion size bug
github-actions[bot] commented on PR #14836: URL: https://github.com/apache/doris/pull/14836#issuecomment-1337503107 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14796: [enhancement](compaction) add some trigger and delete useless log
github-actions[bot] commented on PR #14796: URL: https://github.com/apache/doris/pull/14796#issuecomment-1337505283 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Yukang-Lian closed pull request #14396: [Feature](function) Support width_bucket function
Yukang-Lian closed pull request #14396: [Feature](function) Support width_bucket function URL: https://github.com/apache/doris/pull/14396 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] luozenglin opened a new pull request, #14837: [enhancement](function) remove useless functions in fe
luozenglin opened a new pull request, #14837: URL: https://github.com/apache/doris/pull/14837 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] cambyzju opened a new issue, #14838: [Bug] insert_date_column of large data make BE crashed
cambyzju opened a new issue, #14838: URL: https://github.com/apache/doris/issues/14838 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version latest-master ### What's Wrong? *** Query id: 0-0 *** *** Aborted at 1670250093 (unix time) try "date -d @1670250093" if you are using GNU date *** *** Current BE git commitID: 1190fd4cd *** *** SIGSEGV address not mapped to object (@0x7f9af269e000) received by PID 82219 (TID 0x7f9a9f9ff640) from PID 18446744073481609216; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/common/signal_handler.h:420 1# 0x7F9AF3A8E080 in /opt/compiler/gcc-12/lib/libc.so.6 2# void doris::vectorized::PODArray, 15ul, 16ul>::push_back_without_reserve(long&&) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/vec/common/pod_array.h:424 3# doris::vectorized::ColumnVector::insert_date_column(char const*, unsigned long) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/vec/columns/column_vector.h:171 4# doris::vectorized::ColumnVector::insert_many_fix_len_data(char const*, unsigned long) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/vec/columns/column_vector.h:206 5# doris::vectorized::ColumnNullable::insert_many_fix_len_data(char const*, unsigned long) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/vec/columns/column_nullable.h:113 6# doris::segment_v2::BitShufflePageDecoder<(doris::FieldType)14>::next_batch(unsigned long*, COW::mutable_ptr&) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/olap/rowset/segment_v2/bitshuffle_page.h:397 7# doris::segment_v2::FileColumnIterator::next_batch(unsigned long*, COW::mutable_ptr&, bool*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/olap/rowset/segment_v2/column_reader.cpp:805 8# doris::segment_v2::ArrayFileColumnIterator::next_batch(unsigned long*, COW::mutable_ptr&, bool*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/olap/rowset/segment_v2/column_reader.cpp:585 9# doris::segment_v2::ArrayFileColumnIterator::read_by_rowids(unsigned int const*, unsigned long, COW::mutable_ptr&) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/olap/rowset/segment_v2/column_reader.cpp:607 10# doris::segment_v2::SegmentIterator::_read_columns_by_rowids(std::vector >&, std::vector >&, unsigned short*, unsigned long) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1092 11# doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1204 12# doris::BetaRowsetReader::next_block(doris::vectorized::Block*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/olap/rowset/beta_rowset_reader.cpp:299 13# doris::vectorized::VCollectIterator::Level0Iterator::next(doris::vectorized::Block*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/vec/olap/vcollect_iterator.cpp:268 14# doris::vectorized::VCollectIterator::Level1Iterator::_normal_next(doris::vectorized::Block*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/vec/olap/vcollect_iterator.cpp:534 15# doris::vectorized::VCollectIterator::Level1Iterator::_normal_next(doris::vectorized::Block*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/vec/olap/vcollect_iterator.cpp:542 16# doris::vectorized::VCollectIterator::Level1Iterator::next(doris::vectorized::Block*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/vec/olap/vcollect_iterator.cpp:361 17# doris::vectorized::VCollectIterator::next(doris::vectorized::Block*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/vec/olap/vcollect_iterator.cpp:187 18# doris::vectorized::BlockReader::_direct_next_block(doris::vectorized::Block*, doris::MemPool*, doris::ObjectPool*, bool*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/vec/olap/block_reader.cpp:168 19# doris::vectorized::BlockReader::next_block_with_aggregation(doris::vectorized::Block*, doris::MemPool*, doris::ObjectPool*, bool*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/vec/olap/block_reader.h:46 20# doris::vectorized::NewOlapScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/vec/exec/scan/new_olap_scanner.cpp:318 21# doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/vec/exec/scan/vscanner.cpp:56 22# doris::vectorized::ScannerScheduler::_scanner_scan(doris::vecto
[GitHub] [doris] hello-stephen commented on pull request #14836: [bugfix](compaction) fix promotion size bug
hello-stephen commented on PR #14836: URL: https://github.com/apache/doris/pull/14836#issuecomment-1337539527 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.28 seconds load time: 482 seconds storage size: 17123356312 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221205150703_clickbench_pr_58127.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] cambyzju opened a new pull request, #14839: [fix](ColumnVector) ColumnVector::insert_date_column crashed
cambyzju opened a new pull request, #14839: URL: https://github.com/apache/doris/pull/14839 # Proposed changes Issue Number: close https://github.com/apache/doris/issues/14838 ## Problem summary ColumnVector::insert_date_column make BE crashed with large data(>512 rows). ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14839: [fix](ColumnVector) ColumnVector::insert_date_column crashed
github-actions[bot] commented on PR #14839: URL: https://github.com/apache/doris/pull/14839#issuecomment-1337557391 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14808: [config](load) enable new load scan node by default
github-actions[bot] commented on PR #14808: URL: https://github.com/apache/doris/pull/14808#issuecomment-1337564734 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14808: [config](load) enable new load scan node by default
github-actions[bot] commented on PR #14808: URL: https://github.com/apache/doris/pull/14808#issuecomment-1337568322 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14837: [enhancement](function) remove useless functions in fe
github-actions[bot] commented on PR #14837: URL: https://github.com/apache/doris/pull/14837#issuecomment-1337573978 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14837: [enhancement](function) remove useless functions in fe
github-actions[bot] commented on PR #14837: URL: https://github.com/apache/doris/pull/14837#issuecomment-1337574050 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #14837: [enhancement](function) remove useless functions in fe
hello-stephen commented on PR #14837: URL: https://github.com/apache/doris/pull/14837#issuecomment-1337576270 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.9 seconds load time: 501 seconds storage size: 17123356305 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221205152803_clickbench_pr_58142.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman commented on a diff in pull request #14405: [improvement] (fix) Add check validity when create storage policy.
morningman commented on code in PR #14405:
URL: https://github.com/apache/doris/pull/14405#discussion_r1039759359
##
fe/fe-core/src/main/java/org/apache/doris/catalog/S3Resource.java:
##
@@ -121,12 +126,51 @@ protected void setProperties(Map
properties) throws DdlException
checkRequiredProperty(S3_ACCESS_KEY);
checkRequiredProperty(S3_SECRET_KEY);
checkRequiredProperty(S3_BUCKET);
+
+// default need check resource conf valid, so need fix ut and
regression case
+boolean needCheck = !properties.containsKey(S3_VALIDITY_CHECK)
+|| Boolean.parseBoolean(properties.get(S3_VALIDITY_CHECK));
+LOG.debug("s3 info need check validity : {}", needCheck);
+if (needCheck) {
+boolean available = pingS3();
+if (!available) {
+throw new DdlException("S3 can't use, please check your
properties");
+}
+}
+
// optional
checkOptionalProperty(S3_MAX_CONNECTIONS, DEFAULT_S3_MAX_CONNECTIONS);
checkOptionalProperty(S3_REQUEST_TIMEOUT_MS,
DEFAULT_S3_REQUEST_TIMEOUT_MS);
checkOptionalProperty(S3_CONNECTION_TIMEOUT_MS,
DEFAULT_S3_CONNECTION_TIMEOUT_MS);
}
+private boolean pingS3() {
+String bucket = "s3://" + properties.getOrDefault(S3_BUCKET, "") + "/";
+Map propertiesPing = new HashMap<>();
+propertiesPing.put("AWS_ACCESS_KEY",
properties.getOrDefault(S3_ACCESS_KEY, ""));
+propertiesPing.put("AWS_SECRET_KEY",
properties.getOrDefault(S3_SECRET_KEY, ""));
+propertiesPing.put("AWS_ENDPOINT", "http://"; +
properties.getOrDefault(S3_ENDPOINT, ""));
+propertiesPing.put("AWS_REGION", properties.getOrDefault(S3_REGION,
""));
+propertiesPing.put(S3Storage.USE_PATH_STYLE, "false");
+S3Storage storage = new S3Storage(propertiesPing);
+
+String testFile = bucket + properties.getOrDefault(S3_ROOT_PATH, "") +
"/test-object-valid.txt";
+String content = "doris will be better";
+Status status = storage.directUpload(content, testFile);
+if (status != Status.OK) {
+LOG.warn("ping update file status: {}, properties: {}", status,
propertiesPing);
+return false;
+}
+
+Status delete = storage.delete(testFile);
Review Comment:
wrap with `try .. finally`? To make sure the file will be deleted finally.
##
fe/fe-core/src/main/java/org/apache/doris/backup/S3Storage.java:
##
@@ -245,6 +245,9 @@ public Status directUpload(String content, String
remoteFile) {
} catch (UserException ue) {
LOG.error("connect to s3 failed: ", ue);
return new Status(Status.ErrCode.COMMON_ERROR, "connect to s3
failed: " + ue.getMessage());
+} catch (Exception e) {
+LOG.error("connect to s3 failed: ", e);
Review Comment:
Use `warn`, need to modify above all LOG.error to LOG.warn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14246: [fix](audit) fix duplicate audit log.
github-actions[bot] commented on PR #14246: URL: https://github.com/apache/doris/pull/14246#issuecomment-1337605662 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14821: [fix](vectorization) Fix right outer join core with conjunct
github-actions[bot] commented on PR #14821: URL: https://github.com/apache/doris/pull/14821#issuecomment-1337640720 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14821: [fix](vectorization) Fix right outer join core with conjunct
github-actions[bot] commented on PR #14821: URL: https://github.com/apache/doris/pull/14821#issuecomment-1337640820 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14809: [fix](fe)fix select from temporary partition bug
github-actions[bot] commented on PR #14809: URL: https://github.com/apache/doris/pull/14809#issuecomment-1337642674 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
