Re: [PR] [fix](insert) make group commit wal_manager exit elegantly [doris]
hust-hhb commented on PR #25250: URL: https://github.com/apache/doris/pull/25250#issuecomment-1762541826 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](insert) make group commit wal_manager exit elegantly [doris]
github-actions[bot] commented on code in PR #25250:
URL: https://github.com/apache/doris/pull/25250#discussion_r1359102785
##
be/test/vec/exec/vwal_scanner_test.cpp:
##
@@ -0,0 +1,267 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+#include
+
+#include
+#include
+
+#include "common/object_pool.h"
+#include "gen_cpp/Descriptors_types.h"
+#include "gen_cpp/PlanNodes_types.h"
+#include "olap/wal_manager.h"
+#include "runtime/descriptors.h"
+#include "runtime/memory/mem_tracker.h"
+#include "runtime/runtime_state.h"
+#include "runtime/user_function_cache.h"
+#include "vec/exec/scan/new_file_scan_node.h"
+#include "vec/exec/scan/vfile_scanner.h"
+
+namespace doris {
+
+namespace vectorized {
+class VWalScannerTest : public testing::Test {
+public:
+VWalScannerTest() : _runtime_state(TQueryGlobals()) {
+init();
+_profile = _runtime_state.runtime_profile();
+_runtime_state.init_mem_trackers();
+static_cast(_runtime_state.init(unique_id, query_options,
query_globals, _env));
+_runtime_state.set_query_ctx(query_ctx);
+}
+void init();
+
+void TearDown() override {
+
static_cast(io::global_local_filesystem()->delete_directory(wal_dir));
+SAFE_STOP(_env->_wal_manager);
+}
+
+protected:
+virtual void SetUp() override {}
+
+private:
+void init_desc_table();
+
+ExecEnv* _env = nullptr;
+std::string wal_dir = "./wal_test";
+int64_t db_id = 1;
+int64_t tb_id = 2;
+int64_t txn_id = 789;
Review Comment:
warning: 789 is a magic number; consider replacing it with a named constant
[readability-magic-numbers]
```cpp
int64_t txn_id = 789;
^
```
##
be/test/vec/exec/vwal_scanner_test.cpp:
##
@@ -0,0 +1,267 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+#include
+
+#include
+#include
+
+#include "common/object_pool.h"
+#include "gen_cpp/Descriptors_types.h"
+#include "gen_cpp/PlanNodes_types.h"
+#include "olap/wal_manager.h"
+#include "runtime/descriptors.h"
+#include "runtime/memory/mem_tracker.h"
+#include "runtime/runtime_state.h"
+#include "runtime/user_function_cache.h"
+#include "vec/exec/scan/new_file_scan_node.h"
+#include "vec/exec/scan/vfile_scanner.h"
+
+namespace doris {
+
+namespace vectorized {
Review Comment:
warning: nested namespaces can be concatenated
[modernize-concat-nested-namespaces]
```suggestion
namespace doris::vectorized {
```
be/test/vec/exec/vwal_scanner_test.cpp:265:
```diff
- } // namespace vectorized
- } // namespace doris
+ } // namespace doris
```
##
be/test/vec/exec/vwal_scanner_test.cpp:
##
@@ -0,0 +1,267 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for th
Re: [PR] [fix](insert) make group commit wal_manager exit elegantly [doris]
doris-robot commented on PR #25250: URL: https://github.com/apache/doris/pull/25250#issuecomment-1762547861 TeamCity be ut coverage result: Function Coverage: 36.64% (8227/22454) Line Coverage: 28.77% (65923/229118) Region Coverage: 27.48% (34191/124410) Branch Coverage: 24.19% (17403/71944) Coverage Report: http://coverage.selectdb-in.cc/coverage/d03d85cdd13340913d883fe7d18c65a0295cb976_d03d85cdd13340913d883fe7d18c65a0295cb976/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](docker)add docker-iceberg init tables [doris]
wuwenchi commented on PR #25424: URL: https://github.com/apache/doris/pull/25424#issuecomment-1762548103 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Improvement](hash-map) Separate fixed key hash map context creator [doris]
BiteThet opened a new pull request, #25438: URL: https://github.com/apache/doris/pull/25438 ## Proposed changes Separate fixed key hash map context creator ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](hash-map) Separate fixed key hash map context creator [doris]
BiteThet commented on PR #25438: URL: https://github.com/apache/doris/pull/25438#issuecomment-1762549182 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](insert) make group commit wal_manager exit elegantly [doris]
doris-robot commented on PR #25250: URL: https://github.com/apache/doris/pull/25250#issuecomment-1762549867 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.08 seconds stream load tsv: 552 seconds loaded 74807831229 Bytes, about 129 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 28.6 seconds inserted 1000 Rows, about 349K ops/s storage size: 17162220077 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](hash-map) Separate fixed key hash map context creator [doris]
github-actions[bot] commented on PR #25438: URL: https://github.com/apache/doris/pull/25438#issuecomment-1762551235 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] Fix [doris]
adonis0147 opened a new pull request, #25439: URL: https://github.com/apache/doris/pull/25439 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Fix [doris]
github-actions[bot] commented on PR #25439: URL: https://github.com/apache/doris/pull/25439#issuecomment-1762593281 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](hash-map) Separate fixed key hash map context creator [doris]
doris-robot commented on PR #25438: URL: https://github.com/apache/doris/pull/25438#issuecomment-1762597903 TeamCity be ut coverage result: Function Coverage: 36.34% (8144/22412) Line Coverage: 28.57% (65298/228557) Region Coverage: 27.24% (33813/124119) Branch Coverage: 24.05% (17257/71758) Coverage Report: http://coverage.selectdb-in.cc/coverage/fb63aa08879ca8a777b7d57a520aa04ad3e31959_fb63aa08879ca8a777b7d57a520aa04ad3e31959/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Chore](clang-tidy) remove unused code and adjust clang-tidy checks [doris]
doris-robot commented on PR #25405: URL: https://github.com/apache/doris/pull/25405#issuecomment-1761008194 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.04 seconds stream load tsv: 553 seconds loaded 74807831229 Bytes, about 129 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.1 seconds inserted 1000 Rows, about 343K ops/s storage size: 17162160129 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](statistics) use replication_num as replica num to calculate num delta rows [doris]
TangSiyang2001 commented on PR #25325: URL: https://github.com/apache/doris/pull/25325#issuecomment-1761008079 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](format) fix tablet_meta.cpp [doris]
github-actions[bot] commented on PR #25410: URL: https://github.com/apache/doris/pull/25410#issuecomment-1761008858 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](format) fix tablet_meta.cpp [doris]
github-actions[bot] commented on PR #25410: URL: https://github.com/apache/doris/pull/25410#issuecomment-1761009644 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](format) fix tablet_meta.cpp [doris]
github-actions[bot] commented on PR #25410: URL: https://github.com/apache/doris/pull/25410#issuecomment-1761009701 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](compaction) record base compaction schedule time and status [doris]
doris-robot commented on PR #25283: URL: https://github.com/apache/doris/pull/25283#issuecomment-1761017890 TeamCity be ut coverage result: Function Coverage: 36.29% (8144/22440) Line Coverage: 28.50% (65274/229015) Region Coverage: 27.20% (33814/124319) Branch Coverage: 24.00% (17252/71898) Coverage Report: http://coverage.selectdb-in.cc/coverage/1acfb114a1ca61914f0a2e758e4bb93f81a80d8b_1acfb114a1ca61914f0a2e758e4bb93f81a80d8b/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Bug](join) corner case cause the mark join + null aware left join core dump in regression test in pipeline query engine (#25087) [doris]
HappenLee opened a new pull request, #25411: URL: https://github.com/apache/doris/pull/25411 ## Proposed changes cherr-pick: #25087 ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](httpserver) update secure flag to false [doris]
vinlee19 opened a new pull request, #25412: URL: https://github.com/apache/doris/pull/25412 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](httpserver) update secure flag to false [doris]
vinlee19 commented on PR #25412: URL: https://github.com/apache/doris/pull/25412#issuecomment-1761019267 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](httpserver) update secure flag to false [doris]
github-actions[bot] commented on PR #25412: URL: https://github.com/apache/doris/pull/25412#issuecomment-1761022183 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](httpserver) update secure flag to false [doris]
github-actions[bot] commented on PR #25412: URL: https://github.com/apache/doris/pull/25412#issuecomment-1761022255 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pipelineX](refactor) Use class template to simplify join [doris]
github-actions[bot] commented on PR #25369: URL: https://github.com/apache/doris/pull/25369#issuecomment-1761023278 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pipelineX](refactor) Use class template to simplify join [doris]
github-actions[bot] commented on PR #25369: URL: https://github.com/apache/doris/pull/25369#issuecomment-1761023225 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] Using compute type in predicate column [doris]
yiguolei opened a new pull request, #25413: URL: https://github.com/apache/doris/pull/25413 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](join) corner case cause the mark join + null aware left join core dump in regression test in pipeline query engine (#25087) [doris]
github-actions[bot] commented on PR #25411: URL: https://github.com/apache/doris/pull/25411#issuecomment-1761026339 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) fix some case error in nereids_syntax_p0 [doris]
xiaokang merged PR #25341: URL: https://github.com/apache/doris/pull/25341 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-2.0 updated: [fix](Nereids) fix some case error in nereids_syntax_p0 (#25341)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new a9dce3ee681 [fix](Nereids) fix some case error in nereids_syntax_p0
(#25341)
a9dce3ee681 is described below
commit a9dce3ee681884d7f11d4b3236e45bcf9d6b2dd1
Author: 谢健
AuthorDate: Fri Oct 13 15:17:45 2023 +0800
[fix](Nereids) fix some case error in nereids_syntax_p0 (#25341)
---
.../data/nereids_syntax_p0/push_filter_through_ptopn.out| 6 +++---
regression-test/suites/nereids_syntax_p0/advance_mv.groovy | 4 ++--
regression-test/suites/nereids_syntax_p0/function.groovy| 2 +-
3 files changed, 6 insertions(+), 6 deletions(-)
diff --git
a/regression-test/data/nereids_syntax_p0/push_filter_through_ptopn.out
b/regression-test/data/nereids_syntax_p0/push_filter_through_ptopn.out
index 05343ae859e..7f6d7959a6f 100644
--- a/regression-test/data/nereids_syntax_p0/push_filter_through_ptopn.out
+++ b/regression-test/data/nereids_syntax_p0/push_filter_through_ptopn.out
@@ -32,7 +32,7 @@ PhysicalResultSink
-- !3 --
PhysicalResultSink
--PhysicalDistribute
-filter((T.b = 2) and (rn <= 2))
+filter((rn <= 2)(T.b = 2))
--PhysicalWindow
PhysicalQuickSort
--PhysicalPartitionTopN
@@ -42,9 +42,9 @@ PhysicalResultSink
PhysicalResultSink
--PhysicalDistribute
PhysicalProject
---filter((T.b = 2) and (rn <= 2))
+--filter((rn <= 2)(T.b = 2))
PhysicalWindow
---PhysicalPartitionTopN
+--PhysicalQuickSort
PhysicalDistribute
--PhysicalPartitionTopN
PhysicalProject
diff --git a/regression-test/suites/nereids_syntax_p0/advance_mv.groovy
b/regression-test/suites/nereids_syntax_p0/advance_mv.groovy
index b3aa733c38c..17ee0e0d041 100644
--- a/regression-test/suites/nereids_syntax_p0/advance_mv.groovy
+++ b/regression-test/suites/nereids_syntax_p0/advance_mv.groovy
@@ -99,10 +99,10 @@ suite("advance_mv") {
createMV("CREATE materialized VIEW mv2 AS SELECT abs(k1)+k2+1 tmp,
sum(abs(k2+2)+k3+3) FROM ${tbName2} GROUP BY tmp;")
explain {
-sql("SELECT abs(k1)+k2+1 tmp, sum(abs(k2+2)+k3+3) from FROM ${tbName2}
GROUP BY tmp;")
+sql("SELECT abs(k1)+k2+1 tmp, sum(abs(k2+2)+k3+3) FROM ${tbName2}
GROUP BY tmp;")
contains "(mv2)"
}
-order_qt_select_star "SELECT abs(k1)+k2+1 tmp, sum(abs(k2+2)+k3+3) from
FROM ${tbName2} GROUP BY tmp;"
+order_qt_select_star "SELECT abs(k1)+k2+1 tmp, sum(abs(k2+2)+k3+3) FROM
${tbName2} GROUP BY tmp;"
sql "CREATE materialized VIEW mv3 AS SELECT abs(k1)+k2+1 tmp,
abs(k2+2)+k3+3 FROM ${tbName2};"
int max_try_secs2 = 60
diff --git a/regression-test/suites/nereids_syntax_p0/function.groovy
b/regression-test/suites/nereids_syntax_p0/function.groovy
index 498273bad91..1efa58160ab 100644
--- a/regression-test/suites/nereids_syntax_p0/function.groovy
+++ b/regression-test/suites/nereids_syntax_p0/function.groovy
@@ -121,7 +121,7 @@ suite("nereids_function") {
}
test {
-sql """select "1" == "123", "%%" == "%%" """
+sql """select "1" = "123", "%%" = "%%" """
result([[false, true]])
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](planner) Fix `select table tablet` not effective [doris]
xinyiZzz commented on PR #25378: URL: https://github.com/apache/doris/pull/25378#issuecomment-1761028050 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](fix) case bug [doris]
morningman merged PR #25363: URL: https://github.com/apache/doris/pull/25363 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [test](fix) case bug (#25363)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new cee7a6889f5 [test](fix) case bug (#25363)
cee7a6889f5 is described below
commit cee7a6889f58845463663ff4018b432a319a8cc6
Author: shuke <37901441+shuke...@users.noreply.github.com>
AuthorDate: Fri Oct 13 02:25:15 2023 -0500
[test](fix) case bug (#25363)
---
regression-test/suites/load_p0/insert/test_insert.groovy | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/regression-test/suites/load_p0/insert/test_insert.groovy
b/regression-test/suites/load_p0/insert/test_insert.groovy
index 08e02d5f4b8..2f4a2989775 100644
--- a/regression-test/suites/load_p0/insert/test_insert.groovy
+++ b/regression-test/suites/load_p0/insert/test_insert.groovy
@@ -89,7 +89,7 @@ suite("test_insert") {
qt_select """ select k1, k2, k3, k4, k5, k6, k8, k10, k11, k12 from
${insert_tbl_dft} """
-sql 'drop table if exists f3'
+sql 'drop table if exists t3'
sql '''
create table t3 (
id int default "10"
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Using compute type in predicate column [doris]
github-actions[bot] commented on PR #25413: URL: https://github.com/apache/doris/pull/25413#issuecomment-1761035379 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](scanner) Remove the predicate that is always true for the segment [doris]
yiguolei merged PR #25366: URL: https://github.com/apache/doris/pull/25366 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [improvement](scanner) Remove the predicate that is always true for the segment (#25366)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 283bd59ebaa [improvement](scanner) Remove the predicate that is always
true for the segment (#25366)
283bd59ebaa is described below
commit 283bd59ebaa58284204d877e7b2e7486acc5c0f8
Author: Jerry Hu
AuthorDate: Fri Oct 13 02:25:38 2023 -0500
[improvement](scanner) Remove the predicate that is always true for the
segment (#25366)
By utilizing the zonemap index of the segment, we can ascertain if a
predicate is always true. For example, if the segment’s maximum value is 100
and the predicate is col < 101, then this predicate is always true for this
segment.
---
be/src/common/config.cpp | 2 +
be/src/common/config.h | 3 +
be/src/olap/column_predicate.h | 4 ++
be/src/olap/comparison_predicate.h | 23
be/src/olap/rowset/segment_v2/column_reader.cpp| 26 +
be/src/olap/rowset/segment_v2/column_reader.h | 3 +
be/src/olap/rowset/segment_v2/segment.cpp | 20 ++-
.../query_p0/test_select_with_predicate_prune.out | 25
.../test_select_with_predicate_prune.groovy| 67 ++
9 files changed, 172 insertions(+), 1 deletion(-)
diff --git a/be/src/common/config.cpp b/be/src/common/config.cpp
index 93708312705..b53ed53c3e0 100644
--- a/be/src/common/config.cpp
+++ b/be/src/common/config.cpp
@@ -1109,6 +1109,8 @@ DEFINE_Bool(exit_on_exception, "false");
DEFINE_String(doris_cgroup_cpu_path, "");
DEFINE_Bool(enable_cpu_hard_limit, "false");
+DEFINE_Bool(ignore_always_true_predicate_for_segment, "true");
+
// clang-format off
#ifdef BE_TEST
// test s3
diff --git a/be/src/common/config.h b/be/src/common/config.h
index 466f9919f09..4207b354410 100644
--- a/be/src/common/config.h
+++ b/be/src/common/config.h
@@ -1179,6 +1179,9 @@ DECLARE_mBool(exit_on_exception);
DECLARE_String(doris_cgroup_cpu_path);
DECLARE_Bool(enable_cpu_hard_limit);
+// Remove predicate that is always true for a segment.
+DECLARE_Bool(ignore_always_true_predicate_for_segment);
+
#ifdef BE_TEST
// test s3
DECLARE_String(test_s3_resource);
diff --git a/be/src/olap/column_predicate.h b/be/src/olap/column_predicate.h
index b98156f5fb8..05e84999a83 100644
--- a/be/src/olap/column_predicate.h
+++ b/be/src/olap/column_predicate.h
@@ -173,6 +173,10 @@ public:
return true;
}
+virtual bool is_always_true(const std::pair&
statistic) const {
+return false;
+}
+
virtual bool evaluate_del(const std::pair&
statistic) const {
return false;
}
diff --git a/be/src/olap/comparison_predicate.h
b/be/src/olap/comparison_predicate.h
index b6ac7217fcd..de966833f9d 100644
--- a/be/src/olap/comparison_predicate.h
+++ b/be/src/olap/comparison_predicate.h
@@ -174,6 +174,29 @@ public:
}
}
+bool is_always_true(const std::pair&
statistic) const override {
+if (statistic.first->is_null() || statistic.second->is_null()) {
+return false;
+}
+
+T tmp_min_value {};
+T tmp_max_value {};
+memcpy((char*)(&tmp_min_value), statistic.first->cell_ptr(),
sizeof(WarpperFieldType));
+memcpy((char*)(&tmp_max_value), statistic.second->cell_ptr(),
sizeof(WarpperFieldType));
+
+if constexpr (PT == PredicateType::LT) {
+return _value > tmp_max_value;
+} else if constexpr (PT == PredicateType::LE) {
+return _value >= tmp_max_value;
+} else if constexpr (PT == PredicateType::GT) {
+return _value < tmp_min_value;
+} else if constexpr (PT == PredicateType::GE) {
+return _value <= tmp_min_value;
+}
+
+return false;
+}
+
bool evaluate_del(const std::pair&
statistic) const override {
if (statistic.first->is_null() || statistic.second->is_null()) {
return false;
diff --git a/be/src/olap/rowset/segment_v2/column_reader.cpp
b/be/src/olap/rowset/segment_v2/column_reader.cpp
index 82e1e145393..73cd7730fd7 100644
--- a/be/src/olap/rowset/segment_v2/column_reader.cpp
+++ b/be/src/olap/rowset/segment_v2/column_reader.cpp
@@ -31,6 +31,7 @@
#include "io/fs/file_reader.h"
#include "olap/block_column_predicate.h"
#include "olap/column_predicate.h"
+#include "olap/comparison_predicate.h"
#include "olap/decimal12.h"
#include "olap/inverted_index_parser.h"
#include "olap/iterators.h"
@@ -338,6 +339,31 @@ bool ColumnReader::match_condition(const
AndBlockColumnPredicate* col_predicates
col_predicates);
}
+bool ColumnReader::prune_predicates_by_zone_map(std::vector&
predicates,
+const int column_id) const {
+if (_zone_map_inde
[PR] [date](parser) Support DateV1 keyword [doris]
Gabriel39 opened a new pull request, #25414: URL: https://github.com/apache/doris/pull/25414 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](compaction) record base compaction schedule time and status [doris]
doris-robot commented on PR #25283: URL: https://github.com/apache/doris/pull/25283#issuecomment-1761036883 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.46 seconds stream load tsv: 572 seconds loaded 74807831229 Bytes, about 124 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.3 seconds inserted 1000 Rows, about 341K ops/s storage size: 17162711996 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression-test](test) enable some cases [doris]
shuke987 commented on PR #25183: URL: https://github.com/apache/doris/pull/25183#issuecomment-1761044014 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [draft] disable pipeline tmp [doris]
shuke987 closed pull request #25221: [draft] disable pipeline tmp URL: https://github.com/apache/doris/pull/25221 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [draft] add change [doris]
shuke987 commented on PR #25223: URL: https://github.com/apache/doris/pull/25223#issuecomment-1761044377 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [date](parser) Support DateV1 keyword [doris]
Gabriel39 commented on PR #25414: URL: https://github.com/apache/doris/pull/25414#issuecomment-1761045517 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Opt](performance) refactor and opt time round floor function (#25026) [doris]
HappenLee opened a new pull request, #25415: URL: https://github.com/apache/doris/pull/25415 ## Proposed changes cherry pick #25026 ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [feature](Nereids) graphSimplifier should compare `edge1BeforeEdge2` and `edge2BeforeEdge1` [doris]
XieJiann opened a new pull request, #25416: URL: https://github.com/apache/doris/pull/25416 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Opt](performance) refactor and opt time round floor function (#25026) [doris]
github-actions[bot] commented on PR #25415: URL: https://github.com/apache/doris/pull/25415#issuecomment-1761056187 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](mark join) mark join column should be nullable [doris]
mrhhsg opened a new pull request, #25417: URL: https://github.com/apache/doris/pull/25417 ## Proposed changes pick #24910 into branch-2.0 ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Tvf) return empty set when tvf queries an empty file or an error uri [doris]
github-actions[bot] commented on PR #25280: URL: https://github.com/apache/doris/pull/25280#issuecomment-1761058735 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Tvf) return empty set when tvf queries an empty file or an error uri [doris]
github-actions[bot] commented on PR #25280: URL: https://github.com/apache/doris/pull/25280#issuecomment-1761058799 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Nereids) graphSimplifier should compare `edge1BeforeEdge2` and `edge2BeforeEdge1` [doris]
XieJiann commented on PR #25416: URL: https://github.com/apache/doris/pull/25416#issuecomment-1761059424 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (283bd59ebaa -> 2ec53ff60e4)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 283bd59ebaa [improvement](scanner) Remove the predicate that is always true for the segment (#25366) add 2ec53ff60e4 [fix](multi-table) fix single stream multi table load can not finish (#25379) No new revisions were added by this update. Summary of changes: be/src/runtime/routine_load/data_consumer_group.cpp| 7 ++- be/src/runtime/routine_load/data_consumer_group.h | 9 +++-- be/src/runtime/routine_load/routine_load_task_executor.cpp | 6 +- 3 files changed, 14 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](planner) Fix `select table tablet` not effective [doris]
xinyiZzz commented on PR #25378: URL: https://github.com/apache/doris/pull/25378#issuecomment-1761061876 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](multi-table) fix single stream multi table load can not finish [doris]
dataroaring merged PR #25379: URL: https://github.com/apache/doris/pull/25379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mark join) mark join column should be nullable [doris]
github-actions[bot] commented on PR #25417: URL: https://github.com/apache/doris/pull/25417#issuecomment-1761064655 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Tvf) return empty set when tvf queries an empty file or an error uri [doris]
morningman commented on PR #25280: URL: https://github.com/apache/doris/pull/25280#issuecomment-1761063871 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](tablet schedule) colocate balance between all groups [doris]
yujun777 commented on PR #23543: URL: https://github.com/apache/doris/pull/23543#issuecomment-1761064902 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pipelineX](fix) Fix use-after-free in shuffling [doris]
doris-robot commented on PR #25409: URL: https://github.com/apache/doris/pull/25409#issuecomment-1761069780 TeamCity be ut coverage result: Function Coverage: 36.31% (8146/22436) Line Coverage: 28.51% (65297/228997) Region Coverage: 27.21% (33823/124318) Branch Coverage: 24.00% (17256/71900) Coverage Report: http://coverage.selectdb-in.cc/coverage/ddf5306b2cbc49dfbdf631e9e7acac4568bfb8b0_ddf5306b2cbc49dfbdf631e9e7acac4568bfb8b0/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](format) fix tablet_meta.cpp [doris]
doris-robot commented on PR #25410: URL: https://github.com/apache/doris/pull/25410#issuecomment-1761069502 TeamCity be ut coverage result: Function Coverage: 36.30% (8144/22436) Line Coverage: 28.51% (65283/228997) Region Coverage: 27.20% (33810/124318) Branch Coverage: 24.00% (17255/71900) Coverage Report: http://coverage.selectdb-in.cc/coverage/27637d52feb03f107b6d67387d21f65f4f095ea4_27637d52feb03f107b6d67387d21f65f4f095ea4/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Tvf) return empty set when tvf queries an empty file or an error uri [doris]
BePPPower commented on code in PR #25280:
URL: https://github.com/apache/doris/pull/25280#discussion_r1357899958
##
fe/fe-core/src/main/java/org/apache/doris/tablefunction/LocalTableValuedFunction.java:
##
@@ -86,9 +86,9 @@ private void getFileListFromBackend() throws
AnalysisException {
Future response = proxy.glob(address,
requestBuilder.build());
PGlobResponse globResponse = response.get(5, TimeUnit.SECONDS);
if (globResponse.getStatus().getStatusCode() != 0) {
-throw new AnalysisException(
-"error code: " +
globResponse.getStatus().getStatusCode()
-+ ", error msg: " +
globResponse.getStatus().getErrorMsgsList());
+LOG.warn("get file from backend failed. error code: {}, error
msg: {}",
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0](pick) Delete unused function to improve test coverage (#2… [doris]
github-actions[bot] commented on PR #25406: URL: https://github.com/apache/doris/pull/25406#issuecomment-1761073387 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] Pick [bugfix](thirdparty) Upgrade aws s3 sdk to prevent mem leak (#25106) [doris]
ByteYue opened a new pull request, #25418: URL: https://github.com/apache/doris/pull/25418 During the use of the AWS S3 SDK, we found that there is a memory leak. According to the official issue, upgrading the SDK should resolve the issue. ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](case) change dynamic_partition.time_unit from day to month to a… [doris]
zhangstar333 merged PR #25361: URL: https://github.com/apache/doris/pull/25361 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](case) change dynamic_partition.time_unit from day to month to avoid the error that the intert data not in partition (#25361)
This is an automated email from the ASF dual-hosted git repository.
zhangstar333 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 1a25bb65b02 [fix](case) change dynamic_partition.time_unit from day to
month to avoid the error that the intert data not in partition (#25361)
1a25bb65b02 is described below
commit 1a25bb65b021baae55e6941b9fb15914238cba38
Author: Dongyang Li
AuthorDate: Fri Oct 13 16:02:17 2023 +0800
[fix](case) change dynamic_partition.time_unit from day to month to avoid
the error that the intert data not in partition (#25361)
[fix](case) change dynamic_partition.time_unit from day to month to avoid
the error that the intert data not in partition
Co-authored-by: stephen
---
regression-test/suites/index_p0/test_bitmap_index.groovy | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/regression-test/suites/index_p0/test_bitmap_index.groovy
b/regression-test/suites/index_p0/test_bitmap_index.groovy
index cf1da844562..321c152a11d 100644
--- a/regression-test/suites/index_p0/test_bitmap_index.groovy
+++ b/regression-test/suites/index_p0/test_bitmap_index.groovy
@@ -300,7 +300,7 @@ suite("test_bitmap_index") {
"replication_allocation" = "tag.location.default: 1",
"is_being_synced" = "false",
"dynamic_partition.enable" = "true",
-"dynamic_partition.time_unit" = "DAY",
+"dynamic_partition.time_unit" = "MONTH",
"dynamic_partition.time_zone" = "Asia/Shanghai",
"dynamic_partition.start" = "-30",
"dynamic_partition.end" = "1",
@@ -398,7 +398,7 @@ suite("test_bitmap_index") {
"replication_allocation" = "tag.location.default: 1",
"is_being_synced" = "false",
"dynamic_partition.enable" = "true",
-"dynamic_partition.time_unit" = "DAY",
+"dynamic_partition.time_unit" = "MONTH",
"dynamic_partition.time_zone" = "Asia/Shanghai",
"dynamic_partition.start" = "-30",
"dynamic_partition.end" = "1",
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pipelineX](fix) Fix use-after-free in shuffling [doris]
github-actions[bot] commented on PR #25409: URL: https://github.com/apache/doris/pull/25409#issuecomment-1761083163 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pipelineX](fix) Fix use-after-free in shuffling [doris]
github-actions[bot] commented on PR #25409: URL: https://github.com/apache/doris/pull/25409#issuecomment-1761083228 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Session](pipeline) Add force_shared_scan session variables [doris]
HappenLee opened a new pull request, #25419: URL: https://github.com/apache/doris/pull/25419 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Pick [bugfix](thirdparty) Upgrade aws s3 sdk to prevent mem leak (#25106) [doris]
github-actions[bot] commented on PR #25418: URL: https://github.com/apache/doris/pull/25418#issuecomment-1761088939 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](statistics) use replication_num as replica num to calculate num delta rows [doris]
doris-robot commented on PR #25325: URL: https://github.com/apache/doris/pull/25325#issuecomment-1761093900 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.56 seconds stream load tsv: 579 seconds loaded 74807831229 Bytes, about 123 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 28.7 seconds inserted 1000 Rows, about 348K ops/s storage size: 17162223614 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pipelineX](fix) Fix use-after-free in shuffling [doris]
doris-robot commented on PR #25409: URL: https://github.com/apache/doris/pull/25409#issuecomment-1761093946 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.68 seconds stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17162189617 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](format) fix tablet_meta.cpp [doris]
doris-robot commented on PR #25410: URL: https://github.com/apache/doris/pull/25410#issuecomment-1761093915 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.25 seconds stream load tsv: 573 seconds loaded 74807831229 Bytes, about 124 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17162512817 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](httpserver) update secure flag to false [doris]
doris-robot commented on PR #25412: URL: https://github.com/apache/doris/pull/25412#issuecomment-1761094190 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.17 seconds stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.0 seconds inserted 1000 Rows, about 344K ops/s storage size: 17162238090 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](hive-udf)(doc) minimize hive-udf and add some docs. [doris]
doris-robot commented on PR #24786: URL: https://github.com/apache/doris/pull/24786#issuecomment-1761094152 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.38 seconds stream load tsv: 575 seconds loaded 74807831229 Bytes, about 124 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 28.6 seconds inserted 1000 Rows, about 349K ops/s storage size: 17162172233 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](hive-udf) delete Logger to avoid Kryo serialize problem. [doris]
doris-robot commented on PR #25312: URL: https://github.com/apache/doris/pull/25312#issuecomment-1761094035 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.24 seconds stream load tsv: 579 seconds loaded 74807831229 Bytes, about 123 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 28.7 seconds inserted 1000 Rows, about 348K ops/s storage size: 17162367734 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](functions) add quantile_state_empty function signature [doris]
zhangstar333 commented on PR #25306: URL: https://github.com/apache/doris/pull/25306#issuecomment-1761096381 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](gensrc) Builtin function specification error for Branch-2.0 [doris]
xiaokang merged PR #25329: URL: https://github.com/apache/doris/pull/25329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-2.0 updated: [fix](gensrc) Builtin function specification error for Branch-2.0 (#25329)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 55a0cdd99eb [fix](gensrc) Builtin function specification error for
Branch-2.0 (#25329)
55a0cdd99eb is described below
commit 55a0cdd99eb8cbdce8ade7067449e61d8237d220
Author: minghong
AuthorDate: Fri Oct 13 16:15:50 2023 +0800
[fix](gensrc) Builtin function specification error for Branch-2.0 (#25329)
---
gensrc/script/doris_builtins_functions.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/gensrc/script/doris_builtins_functions.py
b/gensrc/script/doris_builtins_functions.py
index 304e7f773a4..1c989732653 100644
--- a/gensrc/script/doris_builtins_functions.py
+++ b/gensrc/script/doris_builtins_functions.py
@@ -461,7 +461,7 @@ visible_functions = {
[['array_apply'], 'ARRAY_SMALLINT', ['ARRAY_SMALLINT', 'VARCHAR',
'SMALLINT'], ''],
[['array_apply'], 'ARRAY_INT', ['ARRAY_INT', 'VARCHAR', 'INT'],
''],
[['array_apply'], 'ARRAY_BIGINT', ['ARRAY_BIGINT', 'VARCHAR',
'BIGINT'], ''],
-[['array_apply'], 'ARRAY_LARGEINT', ['ARRAY_LARGEINT', 'VARCHAR',
'ARRAY_LARGEINT'], ''],
+[['array_apply'], 'ARRAY_LARGEINT', ['ARRAY_LARGEINT', 'VARCHAR',
'LARGEINT'], ''],
[['array_apply'], 'ARRAY_FLOAT', ['ARRAY_FLOAT', 'VARCHAR',
'FLOAT'], ''],
[['array_apply'], 'ARRAY_DOUBLE',['ARRAY_DOUBLE', 'VARCHAR',
'DOUBLE'], ''],
[['array_apply'], 'ARRAY_DECIMALV2', ['ARRAY_DECIMALV2', 'VARCHAR',
'DECIMALV2'], ''],
@@ -469,7 +469,7 @@ visible_functions = {
[['array_apply'], 'ARRAY_DECIMAL64', ['ARRAY_DECIMAL64', 'VARCHAR',
'DECIMAL64'], ''],
[['array_apply'], 'ARRAY_DECIMAL128', ['ARRAY_DECIMAL128', 'VARCHAR',
'DECIMAL128'], ''],
[['array_apply'], 'ARRAY_DATETIME', ['ARRAY_DATETIME', 'VARCHAR',
'DATETIME'], ''],
-[['array_apply'], 'ARRAY_DATE', ['ARRAY_DATE', 'DATE', 'DATE'],
''],
+[['array_apply'], 'ARRAY_DATE', ['ARRAY_DATE', 'VARCHAR',
'DATE'], ''],
[['array_apply'], 'ARRAY_DATETIMEV2', ['ARRAY_DATETIMEV2', 'VARCHAR',
'DATETIMEV2'], ''],
[['array_apply'], 'ARRAY_DATEV2', ['ARRAY_DATEV2', 'VARCHAR',
'DATEV2'], ''],
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Session](pipeline) Add force_shared_scan session variables [doris]
github-actions[bot] commented on PR #25419: URL: https://github.com/apache/doris/pull/25419#issuecomment-1761100278 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Session](pipeline) Add force_shared_scan session variables [doris]
github-actions[bot] commented on PR #25419: URL: https://github.com/apache/doris/pull/25419#issuecomment-1761100052 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-2.0-var updated: [Bug](variant) make schema update in tablet atomic
This is an automated email from the ASF dual-hosted git repository.
eldenmoon pushed a commit to branch branch-2.0-var
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0-var by this push:
new f91ba2f0e4b [Bug](variant) make schema update in tablet atomic
f91ba2f0e4b is described below
commit f91ba2f0e4b1982e4ce0c21e41450219aa718b2f
Author: eldenmoon <15605149...@163.com>
AuthorDate: Tue Oct 10 14:27:10 2023 +0800
[Bug](variant) make schema update in tablet atomic
---
be/src/olap/delta_writer.cpp | 6 +-
be/src/olap/tablet.cpp | 10 +++---
2 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/be/src/olap/delta_writer.cpp b/be/src/olap/delta_writer.cpp
index 89707510043..23f44aa60e6 100644
--- a/be/src/olap/delta_writer.cpp
+++ b/be/src/olap/delta_writer.cpp
@@ -538,11 +538,7 @@ Status DeltaWriter::commit_txn(const PSlaveTabletNodes&
slave_tablet_nodes,
// _tabelt->tablet_schema: A(bigint), B(double)
// => update_schema: A(bigint), B(double), C(int), D(int)
RowsetWriterContext& rw_ctx = _rowset_writer->mutable_context();
-TabletSchemaSPtr update_schema = std::make_shared();
-vectorized::schema_util::get_least_common_schema(
-{_tablet->tablet_schema(), rw_ctx.tablet_schema},
update_schema);
-_tablet->update_by_least_common_schema(update_schema);
-VLOG_DEBUG << "dump updated tablet schema: " <<
update_schema->dump_structure();
+_tablet->update_by_least_common_schema(rw_ctx.tablet_schema);
}
Status res = _storage_engine->txn_manager()->commit_txn(_req.partition_id,
_tablet, _req.txn_id,
diff --git a/be/src/olap/tablet.cpp b/be/src/olap/tablet.cpp
index 6f6d6886a35..25148f662f0 100644
--- a/be/src/olap/tablet.cpp
+++ b/be/src/olap/tablet.cpp
@@ -2650,10 +2650,14 @@ void Tablet::update_max_version_schema(const
TabletSchemaSPtr& tablet_schema) {
}
}
-void Tablet::update_by_least_common_schema(const TabletSchemaSPtr&
least_common_schema) {
+void Tablet::update_by_least_common_schema(const TabletSchemaSPtr&
update_schema) {
std::lock_guard wrlock(_meta_lock);
-CHECK(_max_version_schema->schema_version() >=
least_common_schema->schema_version());
-_max_version_schema = least_common_schema;
+auto final_schema = std::make_shared();
+CHECK(_max_version_schema->schema_version() >=
update_schema->schema_version());
+vectorized::schema_util::get_least_common_schema(
+{_max_version_schema, update_schema}, final_schema);
+_max_version_schema = final_schema;
+VLOG_DEBUG << "dump updated tablet schema: " <<
final_schema->dump_structure();
}
// fetch value by row column
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](statistics)Collect stats for hive partition column using metadata. [doris]
Jibing-Li commented on code in PR #24853:
URL: https://github.com/apache/doris/pull/24853#discussion_r1357925254
##
fe/fe-core/src/main/java/org/apache/doris/statistics/HMSAnalysisTask.java:
##
@@ -144,27 +174,101 @@ private void getTableColumnStats() throws Exception {
// MAX(`r_regionkey`) AS max,
// 0 AS data_size,
// NOW() FROM `hive`.`tpch100`.`region`
-if (isPartitionOnly) {
-getPartitionNames();
-List partitionAnalysisSQLs = new ArrayList<>();
-for (String partId : this.partitionNames) {
-partitionAnalysisSQLs.add(generateSqlForPartition(partId));
+StringBuilder sb = new StringBuilder();
+sb.append(ANALYZE_TABLE_TEMPLATE);
+Map params = buildStatsParams("NULL");
+params.put("dataSizeFunction", getDataSizeFunction(col));
+params.put("nullCountExpr", getNullCountExpression());
+StringSubstitutor stringSubstitutor = new StringSubstitutor(params);
+String sql = stringSubstitutor.replace(sb.toString());
+executeInsertSql(sql);
+}
+
+private void getPartitionColumnStats() throws Exception {
+Set partitionNames = table.getPartitionNames();
+Set ndvPartValues = Sets.newHashSet();
+long numNulls = 0;
+long dataSize = 0;
+String min = null;
+String max = null;
+for (String names : partitionNames) {
+// names is like "date=20230101" for one level partition
+// and like "date=20230101/hour=12" for two level partition
+String[] parts = names.split("/");
+for (String part : parts) {
+if (part.startsWith(col.getName())) {
+String value = HiveUtil.getHivePartitionValue(part);
+// HIVE_DEFAULT_PARTITION hive partition value when the
partition name is not specified.
+if (value == null || value.isEmpty() ||
value.equals(HiveMetaStoreCache.HIVE_DEFAULT_PARTITION)) {
+numNulls += 1;
+continue;
+}
+ndvPartValues.add(value);
+dataSize += col.getType().isStringType() ? value.length()
: col.getType().getColumnSize();
+min = updateMinValue(min, value);
+max = updateMaxValue(max, value);
+}
+}
+}
+// Estimate the row count. This value is inaccurate if the table stats
is empty.
+TableStats tableStatsStatus =
Env.getCurrentEnv().getAnalysisManager().findTableStatsStatus(table.getId());
+long count = tableStatsStatus == null ? table.estimatedRowCount() :
tableStatsStatus.rowCount;
+dataSize = dataSize * count / partitionNames.size();
+numNulls = numNulls * count / partitionNames.size();
+int ndv = ndvPartValues.size();
+
+Map params = buildStatsParams("NULL");
+params.put("row_count", String.valueOf(count));
+params.put("ndv", String.valueOf(ndv));
+params.put("null_count", String.valueOf(numNulls));
+params.put("min", min);
+params.put("max", max);
+params.put("data_size", String.valueOf(dataSize));
+StringSubstitutor stringSubstitutor = new StringSubstitutor(params);
+String sql =
stringSubstitutor.replace(ANALYZE_PARTITION_COLUMN_TEMPLATE);
+executeInsertSql(sql);
+}
+
+private String updateMinValue(String currentMin, String value) {
Review Comment:
Added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](statistics)Collect stats for hive partition column using metadata. [doris]
Jibing-Li commented on PR #24853: URL: https://github.com/apache/doris/pull/24853#issuecomment-1761102872 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](sql-cache) Add partition update time for hms table and use it at sql-cache. (#24491) [doris]
xiaokang commented on PR #25382: URL: https://github.com/apache/doris/pull/25382#issuecomment-1761104984 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature-wip] (planner) Support transforming trino dialect SQL to logical plan [doris]
JingDas commented on PR #21855: URL: https://github.com/apache/doris/pull/21855#issuecomment-1761108963 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](column-id) fix null conn ctx in column id flusher and parser for `database` field in corresponding show stmt [doris]
TangSiyang2001 commented on PR #25393: URL: https://github.com/apache/doris/pull/25393#issuecomment-1761113565 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](path gc) fix data dir path gc [doris]
yujun777 opened a new pull request, #25420: URL: https://github.com/apache/doris/pull/25420 ## Proposed changes Data dir path gc include gc by tablet and gc by rowset, both use the same one thread. If notify to gc by rowset, but the gc thread is just waiting for gc by tablet, then this thread will not try to perform gc by rowset, it's just always waiting for gc by thread. Vice versa. Fix this. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](alter colocate group) Support alter colocate group replica allocation [doris]
xiaokang merged PR #25387: URL: https://github.com/apache/doris/pull/25387 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-2.0 updated: [feature](alter colocate group) Support alter colocate group replica allocation #23320 (#25387)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new abdeaec91dd [feature](alter colocate group) Support alter colocate
group replica allocation #23320 (#25387)
abdeaec91dd is described below
commit abdeaec91ddc1250014198b7691a64bff4965ecb
Author: yujun
AuthorDate: Fri Oct 13 16:22:43 2023 +0800
[feature](alter colocate group) Support alter colocate group replica
allocation #23320 (#25387)
---
.../Alter/ALTER-COLOCATE-GROUP.md | 84 +
docs/sidebars.json | 1 +
.../Alter/ALTER-COLOCATE-GROUP.md | 88 ++
.../Alter/ALTER-WORKLOAD-GROUP.md | 4 +-
fe/fe-core/src/main/cup/sql_parser.cup | 17 ++
.../doris/analysis/AlterColocateGroupStmt.java | 81 +
.../apache/doris/analysis/ColocateGroupName.java | 70
.../apache/doris/catalog/ColocateGroupSchema.java | 4 +
.../apache/doris/catalog/ColocateTableIndex.java | 191 -
.../org/apache/doris/catalog/PartitionInfo.java| 4 +
.../clone/ColocateTableCheckerAndBalancer.java | 133 +-
.../org/apache/doris/clone/TabletScheduler.java| 18 +-
.../doris/common/proc/TabletHealthProcDir.java | 5 +
.../doris/httpv2/meta/ColocateMetaService.java | 2 +-
.../org/apache/doris/journal/JournalEntity.java| 1 +
.../org/apache/doris/master/ReportHandler.java | 5 +
.../apache/doris/persist/ColocatePersistInfo.java | 27 ++-
.../java/org/apache/doris/persist/EditLog.java | 9 +
.../org/apache/doris/persist/OperationType.java| 4 +
.../main/java/org/apache/doris/qe/DdlExecutor.java | 3 +
fe/fe-core/src/main/jflex/sql_scanner.flex | 1 +
.../java/org/apache/doris/alter/AlterTest.java | 181 +--
.../org/apache/doris/utframe/UtFrameUtils.java | 7 +-
.../alter_p2/test_alter_colocate_group.groovy | 170 ++
24 files changed, 1076 insertions(+), 34 deletions(-)
diff --git
a/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-COLOCATE-GROUP.md
b/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-COLOCATE-GROUP.md
new file mode 100644
index 000..54c87c05e67
--- /dev/null
+++
b/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-COLOCATE-GROUP.md
@@ -0,0 +1,84 @@
+---
+{
+"title": "ALTER-COLOCATE-GROUP",
+"language": "en"
+}
+---
+
+
+
+## ALTER-COLOCATE-GROUP
+
+### Name
+
+ALTER COLOCATE GROUP
+
+
+
+### Description
+
+This statement is used to modify the colocation group.
+
+Syntax:
+
+```sql
+ALTER COLOCATE GROUP [database.]group
+SET (
+property_list
+);
+```
+
+NOTE:
+
+1. If the colocate group is global, that is, its name starts with
`__global__`, then it does not belong to any database;
+
+2. property_list is a colocation group attribute, currently only supports
modifying `replication_num` and `replication_allocation`. After modifying these
two attributes of the colocation group, at the same time, change the attribute
`default.replication_allocation`, the attribute
`dynamic.replication_allocation` of the table of the group, and the
`replication_allocation` of the existing partition to be the same as it.
+
+### Example
+
+1. Modify the number of copies of a global group
+
+ ```sql
+ # Set "colocate_with" = "__global__foo" when creating the table
+
+ ALTER COLOCATE GROUP __global__foo
+ SET (
+ "replication_num"="1"
+ );
+ ```
+
+2. Modify the number of copies of a non-global group
+
+ ```sql
+ # Set "colocate_with" = "bar" when creating the table, and the Database
is "example_db"
+
+ ALTER COLOCATE GROUP example_db.bar
+ SET (
+ "replication_num"="1"
+ );
+ ```
+
+### Keywords
+
+```sql
+ALTER, COLOCATE, GROUP
+```
+
+### Best Practice
diff --git a/docs/sidebars.json b/docs/sidebars.json
index 744260a9a31..3b58f867dc7 100644
--- a/docs/sidebars.json
+++ b/docs/sidebars.json
@@ -795,6 +795,7 @@
"sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-PARTITION",
"sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN",
"sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-RESOURCE",
+
"sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-COLOCATE-GROUP",
"sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-WORKLOAD-GROUP",
"sql-manual/sql-reference/Data-Definition-Statements/Alter/CANCEL-ALTER-TABLE"
Re: [PR] [fix](path gc) fix data dir path gc [doris]
yujun777 commented on PR #25420: URL: https://github.com/apache/doris/pull/25420#issuecomment-1761115661 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](functions) add quantile_state_empty function signature [doris]
github-actions[bot] commented on PR #25306: URL: https://github.com/apache/doris/pull/25306#issuecomment-1761115740 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-2.0 updated: [improvement](checkpoint) checkpoint thread update tablet invert index #25098 (#25388)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new fd860825c06 [improvement](checkpoint) checkpoint thread update tablet
invert index #25098 (#25388)
fd860825c06 is described below
commit fd860825c06049f8883b361a35858f382899b6b8
Author: yujun
AuthorDate: Fri Oct 13 16:23:59 2023 +0800
[improvement](checkpoint) checkpoint thread update tablet invert index
#25098 (#25388)
Checkpoint thread doesn't update tablet invert index, then in checkpoint
thread TabletInvertedIndex.getTablet/getReplica will return null. It may cause
some problems. Fix this, let checkpoint thread also update tablet invert index.
---
.../doris/alter/MaterializedViewHandler.java | 8 +-
.../apache/doris/catalog/CatalogRecycleBin.java| 6 +-
.../main/java/org/apache/doris/catalog/Env.java| 9 +-
.../apache/doris/catalog/TabletInvertedIndex.java | 12 ---
.../org/apache/doris/catalog/TempPartitions.java | 2 +-
.../apache/doris/datasource/InternalCatalog.java | 110 ++---
6 files changed, 61 insertions(+), 86 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/alter/MaterializedViewHandler.java
b/fe/fe-core/src/main/java/org/apache/doris/alter/MaterializedViewHandler.java
index b9f1ba3585f..bf79c8a7359 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/alter/MaterializedViewHandler.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/alter/MaterializedViewHandler.java
@@ -1010,11 +1010,9 @@ public class MaterializedViewHandler extends
AlterHandler {
for (Partition partition : olapTable.getPartitions()) {
MaterializedIndex rollupIndex =
partition.deleteRollupIndex(rollupIndexId);
-if (!Env.isCheckpointThread()) {
-// remove from inverted index
-for (Tablet tablet : rollupIndex.getTablets()) {
-invertedIndex.deleteTablet(tablet.getId());
-}
+// remove from inverted index
+for (Tablet tablet : rollupIndex.getTablets()) {
+invertedIndex.deleteTablet(tablet.getId());
}
}
String rollupIndexName = olapTable.getIndexNameById(rollupIndexId);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java
index 759a34ad445..890a76ee24b 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java
@@ -409,7 +409,7 @@ public class CatalogRecycleBin extends MasterDaemon
implements Writable {
RecycleTableInfo tableInfo = idToTable.remove(tableId);
idToRecycleTime.remove(tableId);
Table table = tableInfo.getTable();
-if (table.getType() == TableType.OLAP && !Env.isCheckpointThread()) {
+if (table.getType() == TableType.OLAP) {
Env.getCurrentEnv().onEraseOlapTable((OlapTable) table, true);
}
LOG.info("replay erase table[{}]", tableId);
@@ -519,9 +519,7 @@ public class CatalogRecycleBin extends MasterDaemon
implements Writable {
}
Partition partition = partitionInfo.getPartition();
-if (!Env.isCheckpointThread()) {
-Env.getCurrentEnv().onErasePartition(partition);
-}
+Env.getCurrentEnv().onErasePartition(partition);
LOG.info("replay erase partition[{}]", partitionId);
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
index ea4410f60c6..6748a8b4afc 100755
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
@@ -1888,10 +1888,8 @@ public class Env {
public long loadRecycleBin(DataInputStream dis, long checksum) throws
IOException {
recycleBin.readFields(dis);
-if (!isCheckpointThread()) {
-// add tablet in Recycle bin to TabletInvertedIndex
-recycleBin.addTabletToInvertedIndex();
-}
+// add tablet in Recycle bin to TabletInvertedIndex
+recycleBin.addTabletToInvertedIndex();
// create DatabaseTransactionMgr for db in recycle bin.
// these dbs do not exist in `idToDb` of the catalog.
for (Long dbId : recycleBin.getAllDbIds()) {
@@ -5281,7 +5279,7 @@ public class Env {
}
}
-if (!isReplay) {
+if (!isReplay && !Env.isCheckpointThread()) {
// drop all replicas
AgentBatchTask batchTask = new AgentBatchTask();
for (Partition partition : olapTable.getAllPartitions()) {
@@ -53
Re: [PR] [improvement](checkpoint) checkpoint thread update tablet invert index [doris]
xiaokang merged PR #25388: URL: https://github.com/apache/doris/pull/25388 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](compaction) record base compaction schedule time and status [doris]
zhannngchen commented on PR #25283: URL: https://github.com/apache/doris/pull/25283#issuecomment-1761117771 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) bind sink should use full base schema [doris]
xiaokang commented on PR #25398: URL: https://github.com/apache/doris/pull/25398#issuecomment-1761118648 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Pick](branch2.0)fix tosql of date literal [doris]
xiaokang merged PR #25399: URL: https://github.com/apache/doris/pull/25399 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Chore](clang-tidy) remove unused code and adjust clang-tidy checks [doris]
BiteThet merged PR #25405: URL: https://github.com/apache/doris/pull/25405 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: remove unused code and adjust clang-tidy checks (#25405)
This is an automated email from the ASF dual-hosted git repository.
panxiaolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new f4e2eb65646 remove unused code and adjust clang-tidy checks (#25405)
f4e2eb65646 is described below
commit f4e2eb656465c393f0f3c39a67669dee486671ce
Author: Pxl
AuthorDate: Fri Oct 13 16:27:37 2023 +0800
remove unused code and adjust clang-tidy checks (#25405)
remove unused code and adjust clang-tidy checks
---
.clang-tidy| 18 +++
be/src/vec/columns/column.h| 4 --
be/src/vec/columns/column_array.cpp| 5 --
be/src/vec/columns/column_array.h | 1 -
be/src/vec/columns/column_complex.h| 2 -
be/src/vec/columns/column_decimal.h| 1 -
be/src/vec/columns/column_dictionary.h | 2 -
be/src/vec/columns/column_map.cpp | 6 ---
be/src/vec/columns/column_map.h| 1 -
be/src/vec/columns/column_nullable.cpp | 91 +++---
be/src/vec/columns/column_nullable.h | 20 ++--
be/src/vec/columns/column_string.cpp | 5 --
be/src/vec/columns/column_string.h | 2 -
be/src/vec/columns/column_struct.cpp | 6 ---
be/src/vec/columns/column_struct.h | 1 -
be/src/vec/columns/column_vector.h | 2 -
be/src/vec/columns/predicate_column.h | 2 -
17 files changed, 61 insertions(+), 108 deletions(-)
diff --git a/.clang-tidy b/.clang-tidy
index f571e162545..aad12455701 100644
--- a/.clang-tidy
+++ b/.clang-tidy
@@ -4,21 +4,15 @@ Checks: |
clang-analyzer-*,
-*,
bugprone-redundant-branch-condition,
- modernize-use-override,
- modernize-use-equals-default,
- modernize-use-equals-delete,
- modernize-use-nodiscard,
- modernize-use-nullptr,
- modernize-use-bool-literals,
- modernize-use-using,
+ modernize-*,
+ -modernize-use-trailing-return-type,
+ -modernize-use-nodiscard,
misc-redundant-expression,
misc-unused-*,
-misc-unused-parameters,
- readability-make-member-function-const,
- readability-non-const-parameter,
- readability-static-accessed-through-instance,
- readability-redundant-*,
- readability-braces-around-statements,
+ readability-*,
+ -readability-identifier-length,
+ -readability-implicit-bool-conversion,
portability-simd-intrinsics,
performance-type-promotion-in-math-fn,
performance-faster-string-find,
diff --git a/be/src/vec/columns/column.h b/be/src/vec/columns/column.h
index 45ca44ee057..b21c5d83862 100644
--- a/be/src/vec/columns/column.h
+++ b/be/src/vec/columns/column.h
@@ -545,10 +545,6 @@ public:
/// Zero, if could not be determined.
virtual size_t allocated_bytes() const = 0;
-/// Make memory region readonly with mprotect if it is large enough.
-/// The operation is slow and performed only for debug builds.
-virtual void protect() {}
-
/// If the column contains subcolumns (such as Array, Nullable, etc), do
callback on them.
/// Shallow: doesn't do recursive calls; don't do call for itself.
using ColumnCallback = std::function;
diff --git a/be/src/vec/columns/column_array.cpp
b/be/src/vec/columns/column_array.cpp
index 06e3e35ce0b..53144d883f8 100644
--- a/be/src/vec/columns/column_array.cpp
+++ b/be/src/vec/columns/column_array.cpp
@@ -434,11 +434,6 @@ size_t ColumnArray::allocated_bytes() const {
return get_data().allocated_bytes() + get_offsets().allocated_bytes();
}
-void ColumnArray::protect() {
-get_data().protect();
-get_offsets().protect();
-}
-
ColumnPtr ColumnArray::convert_to_full_column_if_const() const {
/// It is possible to have an array with constant data and non-constant
offsets.
/// Example is the result of expression: replicate('hello', [1])
diff --git a/be/src/vec/columns/column_array.h
b/be/src/vec/columns/column_array.h
index 42d263dea31..668abd1ef62 100644
--- a/be/src/vec/columns/column_array.h
+++ b/be/src/vec/columns/column_array.h
@@ -174,7 +174,6 @@ public:
void reserve(size_t n) override;
size_t byte_size() const override;
size_t allocated_bytes() const override;
-void protect() override;
ColumnPtr replicate(const IColumn::Offsets& replicate_offsets) const
override;
void replicate(const uint32_t* counts, size_t target_size, IColumn&
column) const override;
ColumnPtr convert_to_full_column_if_const() const override;
diff --git a/be/src/vec/columns/column_complex.h
b/be/src/vec/columns/column_complex.h
index 9e959d4a64f..b25ae9f1577 100644
--- a/be/src/vec/columns/column_complex.h
+++ b/be/src/vec/columns/column_complex.h
@@ -118,8 +118,6 @@ public:
size_t allocated_bytes() const override { return byte_size(); }
-void protect() override {}
-
void insert_value(T value) { data.emplace_back(std::move(value)); }
[[noreturn]] void get_permutation(bool reverse, size_t limit, int
nan_direction_hint,
diff --git a/be/src/vec/columns/
[doris] branch branch-2.0 updated: [fix](test) Fix analyze test p0 cases (#25401)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 5f7d3b97581 [fix](test) Fix analyze test p0 cases (#25401)
5f7d3b97581 is described below
commit 5f7d3b975815cc7125aa9b648e9d6ff137efc6a2
Author: AKIRA <33112463+kikyou1...@users.noreply.github.com>
AuthorDate: Fri Oct 13 16:28:08 2023 +0800
[fix](test) Fix analyze test p0 cases (#25401)
---
regression-test/suites/statistics/analyze_stats.groovy | 14 +-
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/regression-test/suites/statistics/analyze_stats.groovy
b/regression-test/suites/statistics/analyze_stats.groovy
index 1c374495034..3fa926ab06d 100644
--- a/regression-test/suites/statistics/analyze_stats.groovy
+++ b/regression-test/suites/statistics/analyze_stats.groovy
@@ -112,7 +112,7 @@ suite("test_analyze") {
Thread.sleep(1000 * 60)
sql """
-SELECT COUNT(*) FROM ${tbl};
+SELECT * FROM ${tbl};
"""
sql """
@@ -123,7 +123,7 @@ suite("test_analyze") {
try {
sql """
-SELECT COUNT(*) FROM ${tbl};
+SELECT * FROM ${tbl};
"""
} catch (Exception e) {
exception = e
@@ -138,7 +138,7 @@ suite("test_analyze") {
"""
sql """
-SELECT COUNT(*) FROM ${tbl};
+SELECT * FROM ${tbl};
"""
sql """
@@ -147,7 +147,7 @@ suite("test_analyze") {
try {
sql """
-SELECT COUNT(*) FROM ${tbl};
+SELECT * FROM ${tbl};
"""
} catch (Exception e) {
exception = e
@@ -874,7 +874,11 @@ PARTITION `p599` VALUES IN (599)
"""
sql """
-INSERT INTO test_600_partition_table_analyze VALUES
(0),(1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13),(14),(15),(16),(17),(18),(19),(20),(21),(22),(23),(24),(25),(26),(27),(28),(29),(30),(31),(32),(33),(34),(35),(36),(37),(38),(39),(40),(41),(42),(43),(44),(45),(46),(47),(48),(49),(50),(51),(52),(53),(54),(55),(56),(57),(58),(59),(60),(61),(62),(63),(64),(65),(66),(67),(68),(69),(70),(71),(72),(73),(74),(75),(76),(77),(78),(79),(80),(81),(82),(83),(84),(85),(86),(87),(8
[...]
+INSERT INTO test_600_partition_table_analyze VALUES
(0),(1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13),(14),(15),(16),(17),(18),(19),(20),(21),(22),(23),(24),(25),(26),(27),(28),(29),(30),(31),(32),(33),(34),(35),(36),(37),(38),(39),(40),(41),(42),(43),(44),(45),(46),(47),(48),(49),(50),(51),(52),(53),(54),(55),(56),(57),(58),(59),(60),(61),(62),(63),(64),(65),(66),(67),(68),(69),(70),(71),(72),(73),(74),(75),(76),(77),(78),(79),(80),(81),(82),(83),(84),(85),(86),(87),(8
[...]
+"""
+
+sql """
+INSERT INTO test_600_partition_table_analyze VALUES
(301),(302),(303),(304),(305),(306),(307),(308),(309),(310),(311),(312),(313),(314),(315),(316),(317),(318),(319),(320),(321),(322),(323),(324),(325),(326),(327),(328),(329),(330),(331),(332),(333),(334),(335),(336),(337),(338),(339),(340),(341),(342),(343),(344),(345),(346),(347),(348),(349),(350),(351),(352),(353),(354),(355),(356),(357),(358),(359),(360),(361),(362),(363),(364),(365),(366),(367),(368),(369),(370),(371),(372),
[...]
"""
sql """ANALYZE TABLE test_600_partition_table_analyze WITH SYNC"""
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test) Fix analyze test p0 cases [doris]
xiaokang merged PR #25401: URL: https://github.com/apache/doris/pull/25401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Pick "[Bug](ScanNode) Fix potential incorrect query result caused by concurrent NewOlapScanNode initialization and Compaction (#24638)" [doris]
github-actions[bot] commented on PR #25127: URL: https://github.com/apache/doris/pull/25127#issuecomment-1761125553 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [draft] add change [doris]
hello-stephen commented on PR #25223: URL: https://github.com/apache/doris/pull/25223#issuecomment-1761124457 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 47.49 seconds stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s insert into select: 29.5 seconds inserted 1000 Rows, about 338K ops/s storage size: 17164171251 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Opt](performance) Optimize timeround with minute / second [doris]
xiaokang commented on PR #25404: URL: https://github.com/apache/doris/pull/25404#issuecomment-1761128268 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) init Date/DateV2Literal should check non-zero time fields [doris]
xiaokang commented on PR #25402: URL: https://github.com/apache/doris/pull/25402#issuecomment-1761127431 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](path gc) fix data dir path gc [doris]
github-actions[bot] commented on PR #25420: URL: https://github.com/apache/doris/pull/25420#issuecomment-1761128184 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
