[GitHub] [doris] luozenglin commented on a diff in pull request #13193: [feature](function) support `initcap` string function.
luozenglin commented on code in PR #13193:
URL: https://github.com/apache/doris/pull/13193#discussion_r990745209
##
be/src/exprs/string_functions.cpp:
##
@@ -350,6 +350,29 @@ StringVal StringFunctions::upper(FunctionContext* context,
const StringVal& str)
return result;
}
+StringVal StringFunctions::initcap(FunctionContext* context, const StringVal&
str) {
+if (str.is_null) {
+return StringVal::null();
+}
+StringVal result(context, str.len);
+if (UNLIKELY(result.is_null)) {
Review Comment:
I have removed it. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] luozenglin commented on a diff in pull request #13193: [feature](function) support `initcap` string function.
luozenglin commented on code in PR #13193:
URL: https://github.com/apache/doris/pull/13193#discussion_r990745229
##
be/src/vec/functions/function_string.cpp:
##
@@ -265,6 +265,45 @@ struct TransferImpl {
}
};
+// Capitalize first letter
+struct NameToInitcap {
+static constexpr auto name = "initcap";
+};
+
+struct InitcapImpl {
+static Status vector(const ColumnString::Chars& data, const
ColumnString::Offsets& offsets,
+ ColumnString::Chars& res_data, ColumnString::Offsets&
res_offsets) {
+size_t offset_size = offsets.size();
+res_offsets.resize(offsets.size());
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #13209: [Regression](outfile) Fix concurrency test failure caused by outfile
Gabriel39 opened a new pull request, #13209: URL: https://github.com/apache/doris/pull/13209 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] liaoxin01 opened a new pull request, #13210: [enhancement](regression-test) add sync for unique table debug test
liaoxin01 opened a new pull request, #13210: URL: https://github.com/apache/doris/pull/13210 # Proposed changes Issue Number: close #xxx ## Problem summary To avoid accidental result errors, add sync for for unique table debug test. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] starocean999 opened a new pull request, #13211: [fix](sort)should not change resolvedTupleExprs in toThrift method
starocean999 opened a new pull request, #13211: URL: https://github.com/apache/doris/pull/13211 # Proposed changes Issue Number: close #xxx ## Problem summary The toThrift method will be called mutilple times for sending data to different be but the changes of resolvedTupleExprs should be done only once. This pr move the changing of resolvedTupleExprs member to init method instead. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataalive commented on issue #13202: [Bug] doris fe 不支持keepalived 部署场景
dataalive commented on issue #13202: URL: https://github.com/apache/doris/issues/13202#issuecomment-1272478521 conf 中 priority_networks是如何配置的，麻烦贴下？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] AshinGau opened a new pull request, #13212: [feature-wip](multi-catalog) Optimize the performance of boolean & dictionary decoding
AshinGau opened a new pull request, #13212: URL: https://github.com/apache/doris/pull/13212 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [x] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [x] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13193: [feature](function) support `initcap` string function.
github-actions[bot] commented on PR #13193: URL: https://github.com/apache/doris/pull/13193#issuecomment-1272478642 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13193: [feature](function) support `initcap` string function.
github-actions[bot] commented on PR #13193: URL: https://github.com/apache/doris/pull/13193#issuecomment-1272478649 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] gongpibin commented on issue #13202: [Bug] doris fe 不支持keepalived 部署场景
gongpibin commented on issue #13202: URL: https://github.com/apache/doris/issues/13202#issuecomment-1272482317 > conf中priority_networks是如何配置的，贴下？  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hf200012 merged pull request #13203: [typo](docs) fix the bad link of delete-recover.md
hf200012 merged PR #13203: URL: https://github.com/apache/doris/pull/13203 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (9c64fde8f5 -> 207e913b55)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 9c64fde8f5 [tools](banchmark) upgrade date type (#13197) add 207e913b55 fix the bad link fo delete-recover.md (#13203) No new revisions were added by this update. Summary of changes: docs/zh-CN/docs/admin-manual/data-admin/delete-recover.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] stalary opened a new issue, #13213: [Enhancement] Support `SHOW ALL BACKUP` return all database backup task
stalary opened a new issue, #13213: URL: https://github.com/apache/doris/issues/13213 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description Support `SHOW ALL BACKUP` return all database backup task ### Solution _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13175: [typo](docs)Fix Docs 404 Url
github-actions[bot] commented on PR #13175: URL: https://github.com/apache/doris/pull/13175#issuecomment-1272484559 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hf200012 merged pull request #13156: [typo](docs)Fix the jump link 404 in delete recover.md
hf200012 merged PR #13156: URL: https://github.com/apache/doris/pull/13156 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [typo](docs)Fix the jump link 404 in delete recover.md (#13156)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new e5fbecc621 [typo](docs)Fix the jump link 404 in delete recover.md (#13156) e5fbecc621 is described below commit e5fbecc621bef7ce786724240e279d7a050dfef5 Author: yuanyuan8983 <99315889+yuanyuan8...@users.noreply.github.com> AuthorDate: Sun Oct 9 16:12:34 2022 +0800 [typo](docs)Fix the jump link 404 in delete recover.md (#13156) * [typo](docs)Fix the jump link 404 in delete-recover.md --- docs/en/docs/admin-manual/data-admin/delete-recover.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/en/docs/admin-manual/data-admin/delete-recover.md b/docs/en/docs/admin-manual/data-admin/delete-recover.md index f02c4ada15..e9944ae9cd 100644 --- a/docs/en/docs/admin-manual/data-admin/delete-recover.md +++ b/docs/en/docs/admin-manual/data-admin/delete-recover.md @@ -50,4 +50,4 @@ RECOVER PARTITION p1 FROM example_tbl; ## More Help -For more detailed syntax and best practices used by RECOVER, please refer to the [RECOVER](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RECOVER.md) command manual, You can also type `HELP RECOVER` on the MySql client command line for more help. +For more detailed syntax and best practices used by RECOVER, please refer to the [RECOVER](../../sql-manual/sql-reference/Database-Administration-Statements/RECOVER) command manual, You can also type `HELP RECOVER` on the MySql client command line for more help. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13196: [chore](be config) remove config use_mmap_allocate_chunk
github-actions[bot] commented on PR #13196: URL: https://github.com/apache/doris/pull/13196#issuecomment-1272485172 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13196: [chore](be config) remove config use_mmap_allocate_chunk
github-actions[bot] commented on PR #13196: URL: https://github.com/apache/doris/pull/13196#issuecomment-1272485179 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13157: [typo](docs)fix docs 404 url
github-actions[bot] commented on PR #13157: URL: https://github.com/apache/doris/pull/13157#issuecomment-1272485214 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #13214: [Improvement](like function) avoid to convert const column to full column
Gabriel39 opened a new pull request, #13214: URL: https://github.com/apache/doris/pull/13214 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #13196: [chore](be config) remove config use_mmap_allocate_chunk
yiguolei merged PR #13196: URL: https://github.com/apache/doris/pull/13196 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [chore](be config) remove config use_mmap_allocate_chunk #13196
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new dc2d33298b [chore](be config) remove config use_mmap_allocate_chunk
#13196
dc2d33298b is described below
commit dc2d33298b225489033f547d1bb6d15c6daf0e1b
Author: yiguolei <676222...@qq.com>
AuthorDate: Sun Oct 9 16:19:59 2022 +0800
[chore](be config) remove config use_mmap_allocate_chunk #13196
This config is never used online and there exist bugs if enable this
config. So that I remove this config and related tests.
Co-authored-by: yiguolei
---
be/src/common/config.h | 8 --
be/src/runtime/memory/system_allocator.cpp | 33 ++--
be/test/runtime/memory/chunk_allocator_test.cpp | 1 -
be/test/runtime/memory/system_allocator_test.cpp | 5 +---
docs/en/docs/admin-manual/config/be-config.md| 6 -
docs/zh-CN/docs/admin-manual/config/be-config.md | 6 -
6 files changed, 3 insertions(+), 56 deletions(-)
diff --git a/be/src/common/config.h b/be/src/common/config.h
index 80e821c536..e0baed3da4 100644
--- a/be/src/common/config.h
+++ b/be/src/common/config.h
@@ -437,14 +437,6 @@ CONF_Int32(max_free_io_buffers, "128");
// including MemPool, ChunkAllocator, BufferPool, DiskIO free buffer.
CONF_Bool(disable_mem_pools, "false");
-// Whether to allocate chunk using mmap. If you enable this, you'd better to
-// increase vm.max_map_count's value whose default value is 65530.
-// you can do it as root via "sysctl -w vm.max_map_count=262144" or
-// "echo 262144 > /proc/sys/vm/max_map_count"
-// NOTE: When this is set to true, you must set chunk_reserved_bytes_limit
-// to a relative large number or the performance is very very bad.
-CONF_Bool(use_mmap_allocate_chunk, "false");
-
// The reserved bytes limit of Chunk Allocator, usually set as a percentage of
mem_limit.
// defaults to bytes if no unit is given, the number of bytes must be a
multiple of 2.
// must larger than 0. and if larger than physical memory size, it will be set
to physical memory size.
diff --git a/be/src/runtime/memory/system_allocator.cpp
b/be/src/runtime/memory/system_allocator.cpp
index c05c28861b..fe268ec8d3 100644
--- a/be/src/runtime/memory/system_allocator.cpp
+++ b/be/src/runtime/memory/system_allocator.cpp
@@ -30,26 +30,11 @@ namespace doris {
#define PAGE_SIZE (4 * 1024) // 4K
uint8_t* SystemAllocator::allocate(size_t length) {
-if (config::use_mmap_allocate_chunk) {
-return allocate_via_mmap(length);
-} else {
-return allocate_via_malloc(length);
-}
+return allocate_via_malloc(length);
}
void SystemAllocator::free(uint8_t* ptr, size_t length) {
-if (config::use_mmap_allocate_chunk) {
-auto res = munmap(ptr, length);
-if (res != 0) {
-char buf[64];
-LOG(ERROR) << "fail to free memory via munmap, errno=" << errno
- << ", errmsg=" << strerror_r(errno, buf, 64);
-} else {
-RELEASE_THREAD_MEM_TRACKER(length);
-}
-} else {
-::free(ptr);
-}
+::free(ptr);
}
uint8_t* SystemAllocator::allocate_via_malloc(size_t length) {
@@ -65,18 +50,4 @@ uint8_t* SystemAllocator::allocate_via_malloc(size_t length)
{
return (uint8_t*)ptr;
}
-uint8_t* SystemAllocator::allocate_via_mmap(size_t length) {
-CONSUME_THREAD_MEM_TRACKER(length);
-auto ptr = (uint8_t*)mmap(nullptr, length, PROT_READ | PROT_WRITE,
MAP_ANONYMOUS | MAP_PRIVATE,
- -1, 0);
-if (ptr == MAP_FAILED) {
-char buf[64];
-LOG(ERROR) << "fail to allocate memory via mmap, errno=" << errno
- << ", errmsg=" << strerror_r(errno, buf, 64);
-RELEASE_THREAD_MEM_TRACKER(length);
-return nullptr;
-}
-return ptr;
-}
-
} // namespace doris
diff --git a/be/test/runtime/memory/chunk_allocator_test.cpp
b/be/test/runtime/memory/chunk_allocator_test.cpp
index aaf3d91b13..b3854724eb 100644
--- a/be/test/runtime/memory/chunk_allocator_test.cpp
+++ b/be/test/runtime/memory/chunk_allocator_test.cpp
@@ -28,7 +28,6 @@
namespace doris {
TEST(ChunkAllocatorTest, Normal) {
-config::use_mmap_allocate_chunk = true;
for (size_t size = 4096; size <= 1024 * 1024; size <<= 1) {
Chunk chunk;
EXPECT_TRUE(ChunkAllocator::instance()->allocate(size, &chunk).ok());
diff --git a/be/test/runtime/memory/system_allocator_test.cpp
b/be/test/runtime/memory/system_allocator_test.cpp
index 3d7a471aa5..6a155ce052 100644
--- a/be/test/runtime/memory/system_allocator_test.cpp
+++ b/be/test/runtime/memory/system_allocator_test.cpp
@@ -23,9 +23,7 @@
namespace doris {
-template
void test_normal() {
-config::use_mmap_allocate_chunk = use_mmap;
{
auto ptr = SystemAllocator::alloc
[GitHub] [doris] yiguolei merged pull request #13175: [typo](docs)Fix Docs 404 Url
yiguolei merged PR #13175: URL: https://github.com/apache/doris/pull/13175 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (dc2d33298b -> cfade2dfe0)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from dc2d33298b [chore](be config) remove config use_mmap_allocate_chunk #13196 add cfade2dfe0 [typo](docs)Fix Docs 404 Url #13175 No new revisions were added by this update. Summary of changes: .../Data-Definition-Statements/Alter/ALTER-TABLE-PARTITION.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13136: [Enhancement](statistics) optimize the default configuration related to statistics, etc.
github-actions[bot] commented on PR #13136: URL: https://github.com/apache/doris/pull/13136#issuecomment-1272487554 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13136: [Enhancement](statistics) optimize the default configuration related to statistics, etc.
github-actions[bot] commented on PR #13136: URL: https://github.com/apache/doris/pull/13136#issuecomment-1272487561 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #10170: [Enhancement](optimizer) Support select table sample
weizhengte commented on code in PR #10170:
URL: https://github.com/apache/doris/pull/10170#discussion_r990726382
##
fe/fe-core/src/main/java/org/apache/doris/analysis/TableRef.java:
##
@@ -160,6 +161,12 @@ public TableRef(TableName name, String alias,
PartitionNames partitionNames, Arr
hasExplicitAlias = false;
}
this.partitionNames = partitionNames;
+if (sampleTabletIds != null) {
+this.sampleTabletIds = sampleTabletIds;
+}
+if (tableSample != null) {
Review Comment:
It seems that its initial value is null, `protected TableSample tableSample
= null`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13206: [fix](Nereids) plan broadcast join for right semi join by mistake
github-actions[bot] commented on PR #13206: URL: https://github.com/apache/doris/pull/13206#issuecomment-1272488417 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13206: [fix](Nereids) plan broadcast join for right semi join by mistake
github-actions[bot] commented on PR #13206: URL: https://github.com/apache/doris/pull/13206#issuecomment-1272488425 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] caoliang-web opened a new pull request, #13215: [typo](docs)fix 404 error url
caoliang-web opened a new pull request, #13215: URL: https://github.com/apache/doris/pull/13215 # Proposed changes Issue Number: close #xxx ## Problem summary fix 404 error url ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener merged pull request #13206: [fix](Nereids) plan broadcast join for right semi join by mistake
jackwener merged PR #13206: URL: https://github.com/apache/doris/pull/13206 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](Nereids) plan broadcast on right semi join by mistake (#13206)
This is an automated email from the ASF dual-hosted git repository.
jakevin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new da933ecd21 [fix](Nereids) plan broadcast on right semi join by mistake
(#13206)
da933ecd21 is described below
commit da933ecd2109fdd1fe3c8faf64ba0bf807d21e2a
Author: morrySnow <101034200+morrys...@users.noreply.github.com>
AuthorDate: Sun Oct 9 16:32:12 2022 +0800
[fix](Nereids) plan broadcast on right semi join by mistake (#13206)
---
.../src/main/java/org/apache/doris/nereids/trees/plans/JoinType.java | 4 ++--
fe/fe-core/src/main/java/org/apache/doris/nereids/util/JoinUtils.java | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/JoinType.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/JoinType.java
index 369636573d..764eecd154 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/JoinType.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/JoinType.java
@@ -92,8 +92,8 @@ public enum JoinType {
return this == INNER_JOIN;
}
-public final boolean isReturnUnmatchedRightJoin() {
-return this == RIGHT_OUTER_JOIN || this == RIGHT_ANTI_JOIN || this ==
FULL_OUTER_JOIN;
+public final boolean isRightJoin() {
+return this == RIGHT_OUTER_JOIN || this == RIGHT_ANTI_JOIN || this ==
RIGHT_SEMI_JOIN;
}
public final boolean isFullOuterJoin() {
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/util/JoinUtils.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/util/JoinUtils.java
index 23fdcdbca1..b65843d457 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/nereids/util/JoinUtils.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/util/JoinUtils.java
@@ -57,7 +57,7 @@ public class JoinUtils {
}
public static boolean couldBroadcast(Join join) {
-return !(join.getJoinType().isReturnUnmatchedRightJoin());
+return !(join.getJoinType().isRightJoin() ||
join.getJoinType().isFullOuterJoin());
}
private static final class JoinSlotCoverageChecker {
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow merged pull request #13136: [Enhancement](statistics) optimize the default configuration related to statistics, etc.
morrySnow merged PR #13136: URL: https://github.com/apache/doris/pull/13136 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [enhancement](statistics) optimize the default configuration related to statistics, etc. (#13136)
This is an automated email from the ASF dual-hosted git repository.
morrysnow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 15fc3c2c89 [enhancement](statistics) optimize the default
configuration related to statistics, etc. (#13136)
15fc3c2c89 is described below
commit 15fc3c2c896e5a2949be7f47cc4802e1077c93d8
Author: ElvinWei
AuthorDate: Sun Oct 9 16:34:20 2022 +0800
[enhancement](statistics) optimize the default configuration related to
statistics, etc. (#13136)
This pr is mainly to optimize statistical tasks. Includes the following:
1. No longer generate statistics tasks for empty tables, and move the logic
of skipping empty partitions to the process of task generation.
2. Adjusted the default configuration related to statistics to improve the
efficiency of statistics collection, parameters include
`cbo_concurrency_statistics_task_num`,`statistic_job_scheduler_execution_interval_ms`
and `statistic_task_scheduler_execution_interval_ms`.
3. Optimize the display of statistical tasks.
4. In addition, some `org.apache.parquet.Strings` packages are changed to
`com.google.common.base.Strings` to avoid the exception that Strings cannot be
found in local debug.
etc.
---
.../org/apache/doris/analysis/AnalyzeStmt.java | 20
.../clone/ColocateTableCheckerAndBalancer.java | 2 +-
.../main/java/org/apache/doris/common/Config.java | 6 ++---
.../java/org/apache/doris/policy/PolicyMgr.java| 2 +-
.../org/apache/doris/statistics/StatisticsJob.java | 8 ++-
.../doris/statistics/StatisticsJobManager.java | 6 +
.../doris/statistics/StatisticsJobScheduler.java | 28 +++---
.../apache/doris/statistics/StatisticsTask.java| 2 +-
.../statistics/StatisticsJobSchedulerTest.java | 17 +
9 files changed, 56 insertions(+), 35 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/AnalyzeStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/AnalyzeStmt.java
index 2da3cf70c0..3b6004aeb3 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/AnalyzeStmt.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/AnalyzeStmt.java
@@ -43,6 +43,7 @@ import com.google.common.collect.Maps;
import com.google.common.collect.Sets;
import org.apache.commons.lang.StringUtils;
+import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Optional;
@@ -150,11 +151,11 @@ public class AnalyzeStmt extends DdlStmt {
try {
OlapTable olapTable = (OlapTable) table;
List partitionNames = getPartitionNames();
-if (partitionNames.isEmpty() && olapTable.isPartitioned()) {
-partitionNames.addAll(olapTable.getPartitionNames());
+List newPartitionNames = new
ArrayList<>(partitionNames);
+if (newPartitionNames.isEmpty() && olapTable.isPartitioned()) {
+newPartitionNames.addAll(olapTable.getPartitionNames());
}
-List notEmptyPartition =
getNotEmptyPartition(olapTable, partitionNames);
-tableIdToPartitionName.put(table.getId(), notEmptyPartition);
+tableIdToPartitionName.put(table.getId(), newPartitionNames);
} finally {
table.readUnlock();
}
@@ -332,17 +333,6 @@ public class AnalyzeStmt extends DdlStmt {
optProperties.put(CBO_STATISTICS_TASK_TIMEOUT_SEC,
String.valueOf(taskTimeout));
}
-private List getNotEmptyPartition(OlapTable olapTable,
List partitionNames) {
-List notEmptyPartition = Lists.newArrayList();
-for (String partitionName : partitionNames) {
-Partition partition = olapTable.getPartition(partitionName);
-if (partition != null && partition.getDataSize() > 0) {
-notEmptyPartition.add(partitionName);
-}
-}
-return notEmptyPartition;
-}
-
@Override
public String toSql() {
StringBuilder sb = new StringBuilder();
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/clone/ColocateTableCheckerAndBalancer.java
b/fe/fe-core/src/main/java/org/apache/doris/clone/ColocateTableCheckerAndBalancer.java
index c7848b3937..490743a67f 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/clone/ColocateTableCheckerAndBalancer.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/clone/ColocateTableCheckerAndBalancer.java
@@ -41,13 +41,13 @@ import org.apache.doris.system.Backend;
import org.apache.doris.system.SystemInfoService;
import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
import com.google.common.collect.Lists;
import com.google.common.collect.Maps;
import com.google.common.collect.Sets;
import com.google.c
[GitHub] [doris] github-actions[bot] commented on pull request #13162: [fix](be) Open the project expressions properly.
github-actions[bot] commented on PR #13162: URL: https://github.com/apache/doris/pull/13162#issuecomment-1272489197 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #13091: [feature](nereids) dump physical tree and memo
morrySnow commented on PR #13091: URL: https://github.com/apache/doris/pull/13091#issuecomment-1272491390 please add a debug string example in commit msg~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataalive commented on issue #13174: [Bug] Doris创建物化视图导致be集群全部宕机
dataalive commented on issue #13174: URL: https://github.com/apache/doris/issues/13174#issuecomment-1272491419 这个我们在最新的版本中验证了下，应该不会导致崩溃了，到时候升级后可以关注下。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener opened a new pull request, #13216: [improve](Nereids): split otherJoinCondition with List.
jackwener opened a new pull request, #13216: URL: https://github.com/apache/doris/pull/13216 # Proposed changes Issue Number: close #xxx ## Problem summary split otherJoinCondition with List. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hf200012 commented on issue #13202: [Bug] doris fe 不支持keepalived 部署场景
hf200012 commented on issue #13202: URL: https://github.com/apache/doris/issues/13202#issuecomment-1272491886 这个是使用了同一个IP段的IP造成的，VIP换成一个和本机IP段不一样的就可以了 比如你这个里面VIP换成一个192.168.67.x -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] caiconghui opened a new pull request, #13217: [fix](be_ut) fix endless loop in parquet scan when do parquet_scanner_test ut
caiconghui opened a new pull request, #13217: URL: https://github.com/apache/doris/pull/13217 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 opened a new pull request, #13218: [fix](planner) Check by partition instead of bucket
Kikyou1997 opened a new pull request, #13218: URL: https://github.com/apache/doris/pull/13218 # Proposed changes Issue Number: noissue ## Problem summary Use partition type instead of bucket number to determine if a `AlterColumnStats` is valid ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on pull request #13218: [fix](planner) Check by partition instead of bucket
Kikyou1997 commented on PR #13218: URL: https://github.com/apache/doris/pull/13218#issuecomment-1272492142 @morrySnow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] gongpibin commented on issue #13202: [Bug] doris fe 不支持keepalived 部署场景
gongpibin commented on issue #13202: URL: https://github.com/apache/doris/issues/13202#issuecomment-1272492621 > 这个是使用了同一个IP段的IP造成的，VIP换成一个和本机IP段不一样的就可以了 比如你这个里面VIP换成一个192.168.67.x 不是的，VIP地址需要和网卡一个网段的 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yixiutt opened a new pull request, #13219: [compaction](http_action) enable be run manual compaction concurrently
yixiutt opened a new pull request, #13219: URL: https://github.com/apache/doris/pull/13219 In some case, we need to run manual compaction via http interface concurrently, we remove the mutex and tablet's compaction lock is enough to prevent concurrent compaction in tablet. # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13219: [compaction](http_action) enable be run manual compaction concurrently
github-actions[bot] commented on PR #13219: URL: https://github.com/apache/doris/pull/13219#issuecomment-1272494436 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13219: [compaction](http_action) enable be run manual compaction concurrently
github-actions[bot] commented on PR #13219: URL: https://github.com/apache/doris/pull/13219#issuecomment-1272494429 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring merged pull request #13189: [fix](rowset) fix that rowset writer doesn't process the return value , which may result in data loss
dataroaring merged PR #13189: URL: https://github.com/apache/doris/pull/13189 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring closed issue #13188: [Bug] failed to initialize storage reader. tablet=17764.1993551234.7745d334f645b261-bacfae93e6bebc83, res=Internal error(error -3109)
dataroaring closed issue #13188: [Bug] failed to initialize storage reader. tablet=17764.1993551234.7745d334f645b261-bacfae93e6bebc83, res=Internal error(error -3109) URL: https://github.com/apache/doris/issues/13188 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (15fc3c2c89 -> 89514fc964)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 15fc3c2c89 [enhancement](statistics) optimize the default configuration related to statistics, etc. (#13136) add 89514fc964 [fix](rowset) fix that rowset writer doesn't process the return value, which may result in data loss (#13189) No new revisions were added by this update. Summary of changes: be/src/olap/rowset/beta_rowset_writer.cpp | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] gavinchou commented on a diff in pull request #13219: [compaction](http_action) enable be run manual compaction concurrently
gavinchou commented on code in PR #13219:
URL: https://github.com/apache/doris/pull/13219#discussion_r990761883
##
be/src/http/action/compaction_action.cpp:
##
@@ -94,18 +91,7 @@ Status CompactionAction::_handle_run_compaction(HttpRequest*
req, std::string* j
return _execute_compaction_callback(tablet, compaction_type);
});
std::future future_obj = task.get_future();
-
-{
-// 3.1 check is there compaction running
-std::lock_guard lock(_compaction_running_mutex);
-if (_is_compaction_running) {
-return Status::TooManyTasks("Manual compaction task is running");
-} else {
-// 3.2 execute the compaction task and set compaction task running
-_is_compaction_running = true;
-std::thread(std::move(task)).detach();
-}
-}
+std::thread(std::move(task)).detach();
Review Comment:
Should we set a limit to it? e.g. count the number of running compaction,
and limit with it.
```
std::packaged_task task([this, tablet, compaction_type]() {
++static_cnt;
std::shared_ptr defer_cnt(nullptr, [](...) { --static_cnt; });
return _execute_compaction_callback(tablet, compaction_type);
});
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yixiutt commented on a diff in pull request #13219: [compaction](http_action) enable be run manual compaction concurrently
yixiutt commented on code in PR #13219:
URL: https://github.com/apache/doris/pull/13219#discussion_r990763061
##
be/src/http/action/compaction_action.cpp:
##
@@ -94,18 +91,7 @@ Status CompactionAction::_handle_run_compaction(HttpRequest*
req, std::string* j
return _execute_compaction_callback(tablet, compaction_type);
});
std::future future_obj = task.get_future();
-
-{
-// 3.1 check is there compaction running
-std::lock_guard lock(_compaction_running_mutex);
-if (_is_compaction_running) {
-return Status::TooManyTasks("Manual compaction task is running");
-} else {
-// 3.2 execute the compaction task and set compaction task running
-_is_compaction_running = true;
-std::thread(std::move(task)).detach();
-}
-}
+std::thread(std::move(task)).detach();
Review Comment:
Parallel run case may fail if we add a limit here。 And this interface should
not expose to users but we use it internal, so, it's controllable。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] ByteYue opened a new pull request, #13220: [Bug](replace function) fix be oom when use replace with an empty old str
ByteYue opened a new pull request, #13220: URL: https://github.com/apache/doris/pull/13220 # Proposed changes Issue Number: close #xxx ## Problem summary The former logic in replace function didn't check whether the input old str is empty or not. The `string.find` function of cpp stl would return pos 0, which would result in an endless string replace logic loop, and end up with OOM. ## Checklist(Required) 1. Does it affect the original behavior: - [x] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [x] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [x] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhannngchen commented on pull request #13219: [compaction](http_action) enable be run manual compaction concurrently
zhannngchen commented on PR #13219: URL: https://github.com/apache/doris/pull/13219#issuecomment-1272503022 use `_push_tablet_into_submitted_compaction` and `_pop_tablet_into_submitted_compaction` instead? Is there any problem if we allow many compaction task of same type for same tablet running at the same time? Note that `_cumulative_compaction_lock` is hold separately during prepare phase and execute phase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13209: [Regression](outfile) Fix concurrency test failure caused by outfile
github-actions[bot] commented on PR #13209: URL: https://github.com/apache/doris/pull/13209#issuecomment-1272503874 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13209: [Regression](outfile) Fix concurrency test failure caused by outfile
github-actions[bot] commented on PR #13209: URL: https://github.com/apache/doris/pull/13209#issuecomment-1272503882 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] FreeOnePlus opened a new issue, #13221: [Bug] SM4 encryption function encryption result error
FreeOnePlus opened a new issue, #13221: URL: https://github.com/apache/doris/issues/13221 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version 1.1.1 ### What's Wrong? In the sample results of Doris, the SM4 encryption function returns the encryption content as' aDjwRflBrDjhBZIOFNw3Tg==', but the encryption results of the encryption program written in Java and the test website are both' LjZFimmUHZO4TghQCZ8z4w==' The test website is: http://lzltool.com/SM4 The test is as shown in the figure   ### What You Expected?    ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei opened a new pull request, #13222: [chore](release build) copy license and notice file to output folder and strip debug info from meta tool
yiguolei opened a new pull request, #13222: URL: https://github.com/apache/doris/pull/13222 # Proposed changes When build a doris release, the release manager need do a lot of works and sometimes he or she may forget something. And sometimes the release maybe not fully tested because the release is build manually by the release manager and the env maybe wrong. In this PR do follow things: 1. add licenses and notices file to our binary folder as default so that the release manager do not need copy them manually. 2. strip debug info from meta tool by default since debug info is useless for meta tool. After this we could get the binary from github pipeline in the future and the binary is fully tested and it is more stable. ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 4. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 5. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 6. Does it need to update dependencies: - [ ] Yes - [ ] No 7. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13222: [chore](release build) copy license and notice file to output folder and strip debug info from meta tool
github-actions[bot] commented on PR #13222:
URL: https://github.com/apache/doris/pull/13222#issuecomment-1272509372
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/3213792704";) output.
shellcheck errors
```
'shellcheck ' returned error 1 finding the following syntactical issues:
--
In build.sh line 102:
cp -r -p ${DORIS_HOME}/NOTICE.txt "$1/"
^---^ SC2086 (info): Double quote to prevent globbing
and word splitting.
Did you mean:
cp -r -p "${DORIS_HOME}"/NOTICE.txt "$1/"
In build.sh line 103:
cp -r -p ${DORIS_HOME}/dist/LICENSE-dist.txt "$1/"
^---^ SC2086 (info): Double quote to prevent globbing
and word splitting.
Did you mean:
cp -r -p "${DORIS_HOME}"/dist/LICENSE-dist.txt "$1/"
In build.sh line 104:
cp -r -p ${DORIS_HOME}/dist/licenses "$1/"
^---^ SC2086 (info): Double quote to prevent globbing
and word splitting.
Did you mean:
cp -r -p "${DORIS_HOME}"/dist/licenses "$1/"
In build.sh line 456:
copy_common_files ${DORIS_OUTPUT}/fe/
^-^ SC2086 (info): Double quote to prevent
globbing and word splitting.
Did you mean:
copy_common_files "${DORIS_OUTPUT}"/fe/
In build.sh line 502:
copy_common_files ${DORIS_OUTPUT}/be/
^-^ SC2086 (info): Double quote to prevent
globbing and word splitting.
Did you mean:
copy_common_files "${DORIS_OUTPUT}"/be/
In build.sh line 514:
copy_common_files ${DORIS_OUTPUT}/apache_hdfs_broker/
^-^ SC2086 (info): Double quote to prevent
globbing and word splitting.
Did you mean:
copy_common_files "${DORIS_OUTPUT}"/apache_hdfs_broker/
For more information:
https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing
...
--
You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
# shellcheck disable=
above the line that contains the issue, where is the error code;
3. Add '-e ' to the SHELLCHECK_OPTS setting in your .yml action file.
```
shfmt errors
```
'shfmt ' found no issues.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow merged pull request #13162: [fix](be) Open the project expressions properly.
morrySnow merged PR #13162: URL: https://github.com/apache/doris/pull/13162 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (89514fc964 -> fc711d89c8)
This is an automated email from the ASF dual-hosted git repository. morrysnow pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 89514fc964 [fix](rowset) fix that rowset writer doesn't process the return value, which may result in data loss (#13189) add fc711d89c8 [fix](projections) Open the project expressions properly. (#13162) No new revisions were added by this update. Summary of changes: be/src/exec/exec_node.cpp | 3 +-- .../suites/correctness/test_view_varchar_length.groovy| 8 2 files changed, 5 insertions(+), 6 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow closed issue #13150: [Bug](ExecNode) unreachable projections open cause projections uninitialized
morrySnow closed issue #13150: [Bug](ExecNode) unreachable projections open cause projections uninitialized URL: https://github.com/apache/doris/issues/13150 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13220: [Bug](replace function) fix be infinite loop and oom when use replace with an empty old str
github-actions[bot] commented on PR #13220: URL: https://github.com/apache/doris/pull/13220#issuecomment-1272516600 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13220: [Bug](replace function) fix be infinite loop and oom when use replace with an empty old str
github-actions[bot] commented on PR #13220: URL: https://github.com/apache/doris/pull/13220#issuecomment-1272516607 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 merged pull request #13209: [Regression](outfile) Fix concurrency test failure caused by outfile
Gabriel39 merged PR #13209: URL: https://github.com/apache/doris/pull/13209 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Regression](outfile) Fix concurrency test failure caused by outfile (#13209)
This is an automated email from the ASF dual-hosted git repository.
gabriellee pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new f2159709a8 [Regression](outfile) Fix concurrency test failure caused
by outfile (#13209)
f2159709a8 is described below
commit f2159709a81d5e2ecf31a81369c8fa84955f9369
Author: Gabriel
AuthorDate: Sun Oct 9 19:09:44 2022 +0800
[Regression](outfile) Fix concurrency test failure caused by outfile
(#13209)
---
regression-test/suites/export_p0/test_outfile.groovy | 2 +-
regression-test/suites/export_p0/test_outfile_expr.groovy | 4 ++--
regression-test/suites/export_p0/test_outfile_parquet.groovy | 2 +-
regression-test/suites/export_p0/test_outfile_separator.groovy | 6 +++---
4 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/regression-test/suites/export_p0/test_outfile.groovy
b/regression-test/suites/export_p0/test_outfile.groovy
index 91ab0ef0a9..0dfd86d036 100644
--- a/regression-test/suites/export_p0/test_outfile.groovy
+++ b/regression-test/suites/export_p0/test_outfile.groovy
@@ -49,7 +49,7 @@ suite("test_outfile") {
return
}
def tableName = "outfile_test"
-def outFilePath = """${context.file.parent}/tmp"""
+def outFilePath = """${context.file.parent}/test_outfile"""
try {
sql """ DROP TABLE IF EXISTS ${tableName} """
sql """
diff --git a/regression-test/suites/export_p0/test_outfile_expr.groovy
b/regression-test/suites/export_p0/test_outfile_expr.groovy
index a549e56bde..0e3f5adbb9 100644
--- a/regression-test/suites/export_p0/test_outfile_expr.groovy
+++ b/regression-test/suites/export_p0/test_outfile_expr.groovy
@@ -21,7 +21,7 @@ import java.nio.charset.StandardCharsets
import java.nio.file.Files
import java.nio.file.Paths
-suite("test_outfile") {
+suite("test_outfile_expr") {
StringBuilder strBuilder = new StringBuilder()
strBuilder.append("curl --location-trusted -u " + context.config.jdbcUser
+ ":" + context.config.jdbcPassword)
strBuilder.append(" http://"; + context.config.feHttpAddress +
"/rest/v1/config/fe")
@@ -45,7 +45,7 @@ suite("test_outfile") {
}
}
if (!enableOutfileToLocal) {
-logger.warn("Please set enable_outfile_to_local to true to run
test_outfile")
+logger.warn("Please set enable_outfile_to_local to true to run
test_outfile_expr")
return
}
def tableName = "outfile_test_expr"
diff --git a/regression-test/suites/export_p0/test_outfile_parquet.groovy
b/regression-test/suites/export_p0/test_outfile_parquet.groovy
index 2804b1e0e5..ac710410e9 100644
--- a/regression-test/suites/export_p0/test_outfile_parquet.groovy
+++ b/regression-test/suites/export_p0/test_outfile_parquet.groovy
@@ -53,7 +53,7 @@ suite("test_outfile_parquet") {
}
def tableName = "outfile_parquet_test"
def tableName2 = "outfile_parquet_test2"
-def outFilePath = """${context.file.parent}/tmp"""
+def outFilePath = """${context.file.parent}/test_outfile_parquet"""
try {
sql """ DROP TABLE IF EXISTS ${tableName} """
sql """
diff --git a/regression-test/suites/export_p0/test_outfile_separator.groovy
b/regression-test/suites/export_p0/test_outfile_separator.groovy
index 009508c8ca..07766db680 100644
--- a/regression-test/suites/export_p0/test_outfile_separator.groovy
+++ b/regression-test/suites/export_p0/test_outfile_separator.groovy
@@ -21,7 +21,7 @@ import java.nio.charset.StandardCharsets
import java.nio.file.Files
import java.nio.file.Paths
-suite("test_outfile") {
+suite("test_outfile_separator") {
StringBuilder strBuilder = new StringBuilder()
strBuilder.append("curl --location-trusted -u " + context.config.jdbcUser
+ ":" + context.config.jdbcPassword)
strBuilder.append(" http://"; + context.config.feHttpAddress +
"/rest/v1/config/fe")
@@ -45,12 +45,12 @@ suite("test_outfile") {
}
}
if (!enableOutfileToLocal) {
-logger.warn("Please set enable_outfile_to_local to true to run
test_outfile")
+logger.warn("Please set enable_outfile_to_local to true to run
test_outfile_separator")
return
}
def dbName = context.config.getDbNameByFile(context.file)
def tableName = "outfile_test_separator"
-def outFilePath = """${context.file.parent}/tmp_separator"""
+def outFilePath = """${context.file.parent}/test_outfile_separator"""
try {
sql """ DROP TABLE IF EXISTS ${tableName} """
sql """
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #10170: [Enhancement](optimizer) Support select table sample
morrySnow commented on code in PR #10170:
URL: https://github.com/apache/doris/pull/10170#discussion_r990772731
##
fe/fe-core/src/main/java/org/apache/doris/analysis/TableRef.java:
##
@@ -197,17 +205,49 @@ protected TableRef(TableRef other) {
lateralViewRefs.add((LateralViewRef) viewRef.clone());
}
}
+if (other.sampleTabletIds.size() != 0) {
Review Comment:
why not copy if size is 0?
##
fe/fe-core/src/main/java/org/apache/doris/analysis/TableSample.java:
##
@@ -0,0 +1,101 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.common.AnalysisException;
+
+/*
+ * To represent following stmt:
+ * TABLESAMPLE (10 PERCENT)
+ * TABLESAMPLE (100 ROWS)
+ * TABLESAMPLE (10 PERCENT) REPEATABLE (123)
+ * TABLESAMPLE (100 ROWS) REPEATABLE (123)R
+ *
+ * references:
+ *
https://simplebiinsights.com/sql-server-tablesample-retrieving-random-data-from-sql-server/
+ * https://sqlrambling.net/2018/01/24/tablesample-basic-examples/
+ */
+public class TableSample implements ParseNode {
+
+private final Long sampleValue;
+private final boolean isPercent;
+private final Long seek;
+
+public TableSample(boolean isPercent, Long sampleValue) {
+this.sampleValue = sampleValue;
+this.isPercent = isPercent;
+this.seek = -1L;
+}
+
+public TableSample(boolean isPercent, Long sampleValue, Long seek) {
+this.sampleValue = sampleValue;
+this.isPercent = isPercent;
+this.seek = seek;
+}
+
+public TableSample(TableSample other) {
+this.sampleValue = other.sampleValue;
+this.isPercent = other.isPercent;
+this.seek = other.seek;
+}
+
+public Long getSampleValue() {
+return sampleValue;
+}
+
+public boolean isPercent() {
+return isPercent;
+}
+
+public Long getSeek() {
+return seek;
+}
+
+@Override
+public void analyze(Analyzer analyzer) throws AnalysisException {
+if (sampleValue <= 0 || (isPercent && sampleValue > 100)) {
+throw new AnalysisException("table sample value must be greater
than 0, percent need less than 100.");
+}
+}
+
+@Override
+public String toSql() {
+if (sampleValue == null) {
+return "";
+}
+StringBuilder sb = new StringBuilder();
+sb.append("TABLESAMPLE ( ");
+sb.append(sampleValue);
+if (isPercent) {
+sb.append(" PERCENT ");
+} else {
+sb.append(" ROWS ");
+}
+sb.append(")");
+if (seek != 0) {
+sb.append(" REPEATABLE ");
+sb.append(seek);
Review Comment:
miss parentheses
##
fe/fe-core/src/main/java/org/apache/doris/analysis/TableRef.java:
##
@@ -160,6 +161,12 @@ public TableRef(TableName name, String alias,
PartitionNames partitionNames, Arr
hasExplicitAlias = false;
}
this.partitionNames = partitionNames;
+if (sampleTabletIds != null) {
+this.sampleTabletIds = sampleTabletIds;
+}
+if (tableSample != null) {
Review Comment:
i think zhengte is right, even if tableSample is null, it has no side effect.
##
fe/fe-core/src/main/java/org/apache/doris/analysis/TupleDescriptor.java:
##
@@ -159,6 +164,80 @@ public void setTable(TableIf tbl) {
table = tbl;
}
+public Set getSampleTabletIds() {
+return sampleTabletIds;
+}
+
+/**
+ * First, determine how many rows to sample from each partition according
to the number of partitions.
+ * Then determine the number of Tablets to be selected for each partition
according to the average number
+ * of rows of Tablet,
+ * If seek is not specified, the specified number of Tablets are
pseudo-randomly selected from each partition.
+ * If seek is specified, it will be selected sequentially from the seek
tablet of the partition.
+ * And add the manually specified Tablet id to the selected Tablet.
+ * simpleTabletNums = simpleRows / partitionNums /
[GitHub] [doris] jackwener opened a new issue, #13223: [Enhancement] ShowViewStmtTest UT often fail
jackwener opened a new issue, #13223: URL: https://github.com/apache/doris/issues/13223 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description ShowViewStmtTest UT often fail ``` Caused by: io.grpc.netty.shaded.io.netty.channel.unix.Errors$NativeIoException: bind(..) failed: Address already in use ``` ### Solution _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] liaoxin01 opened a new pull request, #13224: [fix](rowset) fix that rowset writer doesn't process the return value which may result in data loss
liaoxin01 opened a new pull request, #13224: URL: https://github.com/apache/doris/pull/13224 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13224: [fix](rowset) fix that rowset writer doesn't process the return value which may result in data loss
github-actions[bot] commented on PR #13224: URL: https://github.com/apache/doris/pull/13224#issuecomment-1272521307 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] liaoxin01 closed pull request #9453: [improvement](regression-test) add regression test for decimal datatype
liaoxin01 closed pull request #9453: [improvement](regression-test) add regression test for decimal datatype URL: https://github.com/apache/doris/pull/9453 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13224: [fix](rowset) fix that rowset writer doesn't process the return value which may result in data loss
github-actions[bot] commented on PR #13224: URL: https://github.com/apache/doris/pull/13224#issuecomment-1272521300 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] liaoxin01 closed issue #9596: [Bug] [Load-Vectorized] core dump on KeyCoder::encode_ascending
liaoxin01 closed issue #9596: [Bug] [Load-Vectorized] core dump on KeyCoder::encode_ascending URL: https://github.com/apache/doris/issues/9596 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring merged pull request #13210: [enhancement](regression-test) add sync for unique table debug test
dataroaring merged PR #13210: URL: https://github.com/apache/doris/pull/13210 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [enhancement](regression-test) add sync for unique table debug test (#13210)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 3302e0b57e [enhancement](regression-test) add sync for unique table
debug test (#13210)
3302e0b57e is described below
commit 3302e0b57e91339591f230d533ae122e2baa7c41
Author: Xin Liao
AuthorDate: Sun Oct 9 19:32:28 2022 +0800
[enhancement](regression-test) add sync for unique table debug test (#13210)
---
.../suites/data_model_p0/unique/test_unique_table_debug_data.groovy| 3 +++
1 file changed, 3 insertions(+)
diff --git
a/regression-test/suites/data_model_p0/unique/test_unique_table_debug_data.groovy
b/regression-test/suites/data_model_p0/unique/test_unique_table_debug_data.groovy
index 4be1174692..db0c4e4ac7 100644
---
a/regression-test/suites/data_model_p0/unique/test_unique_table_debug_data.groovy
+++
b/regression-test/suites/data_model_p0/unique/test_unique_table_debug_data.groovy
@@ -46,6 +46,7 @@ suite("test_unique_table_debug_data") {
sql "insert into ${tbName} values(1,1),(2,1);"
sql "insert into ${tbName} values(1,11),(2,11);"
sql "insert into ${tbName} values(3,1);"
+sql "sync"
qt_select_init "select * from ${tbName} order by a, b"
@@ -70,10 +71,12 @@ suite("test_unique_table_debug_data") {
time 1 // limit inflight 10s
}
+sql "sync"
qt_select_batch_delete "select * from ${tbName} order by a, b"
// delete rows with a = 2:
sql "delete from ${tbName} where a = 2;"
+sql "sync"
qt_select_sql_delete "select * from ${tbName} order by a, b"
// enable skip_delete_predicate, rows deleted with delete statement is
returned:
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #13218: [fix](planner) Check by partition instead of bucket
morrySnow commented on code in PR #13218:
URL: https://github.com/apache/doris/pull/13218#discussion_r990777302
##
fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsManager.java:
##
@@ -86,12 +87,12 @@ public void alterTableStatistics(AlterTableStatsStmt stmt)
throws AnalysisExcept
* @throws AnalysisException if table, column or partition not exist
*/
public void alterColumnStatistics(AlterColumnStatsStmt stmt) throws
AnalysisException {
-Table table = validateTableName(stmt.getTableName());
+OlapTable table = (OlapTable) validateTableName(stmt.getTableName());
Review Comment:
we should do instanceof and cast when we do partition check, since not only
OlapTable has statistics
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener merged pull request #13183: [schema change](fix) fix coredump of schema change
jackwener merged PR #13183: URL: https://github.com/apache/doris/pull/13183 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [schema change](fix) fix coredump of schema change (#13183)
This is an automated email from the ASF dual-hosted git repository.
jakevin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 7b2fdd26a1 [schema change](fix) fix coredump of schema change (#13183)
7b2fdd26a1 is described below
commit 7b2fdd26a19e90fb29cd02c56679dcf5a73ed2ed
Author: TengJianPing <18241664+jackte...@users.noreply.github.com>
AuthorDate: Sun Oct 9 19:44:00 2022 +0800
[schema change](fix) fix coredump of schema change (#13183)
When schema change and compaction is executing simutaneously, both
nullable and not nullable data can be read for the same column, need to
reset _nullmap for each Block when converting Block data, or else Column
case will be wrong.
---
be/src/vec/olap/olap_data_convertor.cpp | 1 +
1 file changed, 1 insertion(+)
diff --git a/be/src/vec/olap/olap_data_convertor.cpp
b/be/src/vec/olap/olap_data_convertor.cpp
index 58ab5d6579..33598c4e98 100644
--- a/be/src/vec/olap/olap_data_convertor.cpp
+++ b/be/src/vec/olap/olap_data_convertor.cpp
@@ -160,6 +160,7 @@ void
OlapBlockDataConvertor::OlapColumnDataConvertorBase::set_source_column(
void
OlapBlockDataConvertor::OlapColumnDataConvertorBase::clear_source_column() {
// just to reduce the source column's ref count to 1
_typed_column.column = nullptr;
+_nullmap = nullptr;
}
// This should be called only in SegmentWriter. If you want to access nullmap
in Convertor,
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hf200012 merged pull request #13157: [typo](docs)fix docs 404 url
hf200012 merged PR #13157: URL: https://github.com/apache/doris/pull/13157 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [typo](docs)fix docs 404 url (#13157)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new b9516b50c1 [typo](docs)fix docs 404 url (#13157) b9516b50c1 is described below commit b9516b50c11d11ba934140cb20b75e555a0468d6 Author: zy-kkk AuthorDate: Sun Oct 9 20:02:48 2022 +0800 [typo](docs)fix docs 404 url (#13157) * fix docs 404 url --- docs/en/docs/ecosystem/logstash.md | 4 +- docs/en/docs/ecosystem/udf/contribute-udf.md | 4 +- docs/en/docs/faq/install-faq.md| 4 - .../Alter/ALTER-TABLE-ROLLUP.md| 1 - .../Drop/DROP-DATABASE.md | 2 +- .../SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 120 + .../sql-reference/Show-Statements/SHOW-STATUS.md | 17 ++- docs/zh-CN/docs/ecosystem/logstash.md | 2 +- .../Alter/ALTER-TABLE-PARTITION.md | 2 +- .../Alter/ALTER-TABLE-ROLLUP.md| 2 +- .../Drop/DROP-DATABASE.md | 2 +- .../SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 2 +- 12 files changed, 137 insertions(+), 25 deletions(-) diff --git a/docs/en/docs/ecosystem/logstash.md b/docs/en/docs/ecosystem/logstash.md index d5dfe17a41..c6ab151d1f 100644 --- a/docs/en/docs/ecosystem/logstash.md +++ b/docs/en/docs/ecosystem/logstash.md @@ -28,9 +28,9 @@ under the License. This plugin is used to output data to Doris for logstash, use the HTTP protocol to interact with the Doris FE Http interface, and import data through Doris's stream load. -[Learn more about Doris Stream Load ](../data-operate/import/import-way/stream-load-manual.html) +[Learn more about Doris Stream Load ](../data-operate/import/import-way/stream-load-manual) -[Learn more about Doris](../) +[Learn more about Doris](/) ## Install and compile diff --git a/docs/en/docs/ecosystem/udf/contribute-udf.md b/docs/en/docs/ecosystem/udf/contribute-udf.md index 075bd41a5f..c7c01035ae 100644 --- a/docs/en/docs/ecosystem/udf/contribute-udf.md +++ b/docs/en/docs/ecosystem/udf/contribute-udf.md @@ -14,7 +14,7 @@ to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 + http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an @@ -121,4 +121,4 @@ The user manual needs to include: UDF function definition description, applicabl When you meet the conditions and prepare the code, you can contribute UDF to the Doris community after the document. Simply submit the request (PR) on [Github](https://github.com/apache/incubator-doris). See the specific submission method: [Pull Request (PR)](https://help.github.com/articles/about-pull-requests/). -Finally, when the PR assessment is passed and merged. Congratulations, your UDF becomes a third-party UDF supported by Doris. You can check it out in the ecological expansion section of [Doris official website](/en)~. +Finally, when the PR assessment is passed and merged. Congratulations, your UDF becomes a third-party UDF supported by Doris. You can check it out in the ecological expansion section of [Doris official website](/)~. diff --git a/docs/en/docs/faq/install-faq.md b/docs/en/docs/faq/install-faq.md index a8f805e9c2..b8f5f88537 100644 --- a/docs/en/docs/faq/install-faq.md +++ b/docs/en/docs/faq/install-faq.md @@ -155,10 +155,6 @@ In many cases, we need to troubleshoot problems through logs. The format and vie Logs starting with F are Fatal logs. For example, F0916 , indicating the Fatal log on September 16th. Fatal logs usually indicate a program assertion error, and an assertion error will directly cause the process to exit (indicating a bug in the program). Welcome to the WeChat group, github discussion or dev mail group for help. - 4. Minidump(removed) - - Mindump is a function added after Doris version 0.15. For details, please refer to [document](https://doris.apache.org/zh-CN/developer-guide/minidump.html). - 2. FE FE is a java process, and the robustness is better than the C/C++ program. Usually the reason for FE to hang up may be OOM (Out-of-Memory) or metadata write failure. These errors usually have an error stack in fe.log or fe.out. Further investigation is required based on the error stack information. diff --git a/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-ROLLUP.md b/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-ROLLUP.md index 2991aa24b4..596e7cfcc5 100644 --- a/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statement
[GitHub] [doris] zhannngchen commented on a diff in pull request #12866: [enhancement](compaction) introduce segment compaction (#12609)
zhannngchen commented on code in PR #12866:
URL: https://github.com/apache/doris/pull/12866#discussion_r990780225
##
be/src/olap/delta_writer.cpp:
##
@@ -295,9 +307,13 @@ Status DeltaWriter::close() {
return Status::OLAPInternalError(OLAP_ERR_ALREADY_CANCELLED);
}
-RETURN_NOT_OK(_flush_memtable_async());
+auto s = _flush_memtable_async();
_mem_table.reset();
-return Status::OK();
+if (OLAP_UNLIKELY(!s.ok())) {
Review Comment:
You can simply use `return s;`?
##
be/src/olap/delta_writer.cpp:
##
@@ -56,8 +56,6 @@ DeltaWriter::~DeltaWriter() {
_garbage_collection();
}
-_mem_table.reset();
Review Comment:
Why remove it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13211: [fix](sort)should not change resolvedTupleExprs in toThrift method
github-actions[bot] commented on PR #13211: URL: https://github.com/apache/doris/pull/13211#issuecomment-1272531589 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13211: [fix](sort)should not change resolvedTupleExprs in toThrift method
github-actions[bot] commented on PR #13211: URL: https://github.com/apache/doris/pull/13211#issuecomment-1272531596 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring merged pull request #13208: [fix](test) resolve load in tpch_sf100_unique_p2 and tpch_sf10_unique_p2
dataroaring merged PR #13208: URL: https://github.com/apache/doris/pull/13208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] cambyzju opened a new pull request, #13225: [fix](array-type) fix get_data_at for zero element array
cambyzju opened a new pull request, #13225: URL: https://github.com/apache/doris/pull/13225 # Proposed changes Issue Number: close #13119 ## Problem summary Describe your changes. We should get data ptr after check element size. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (b9516b50c1 -> 581494dea8)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from b9516b50c1 [typo](docs)fix docs 404 url (#13157) add 581494dea8 [fix](test) resolve load in tpch_sf100_unique_p2 and tpch_sf10_unique_p2 (#13208) No new revisions were added by this update. Summary of changes: .../suites/tpch_sf100_unique_p2/load_four_step/load.groovy | 10 +- .../suites/tpch_sf100_unique_p2/load_one_step/load.groovy | 6 +++--- .../suites/tpch_sf100_unique_p2/load_three_step/load.groovy| 8 .../suites/tpch_sf100_unique_p2/load_two_step/load.groovy | 6 +++--- .../suites/tpch_sf10_unique_p2/load_four_step/load.groovy | 10 +- .../suites/tpch_sf10_unique_p2/load_one_step/load.groovy | 6 +++--- .../suites/tpch_sf10_unique_p2/load_three_step/load.groovy | 8 .../suites/tpch_sf10_unique_p2/load_two_step/load.groovy | 6 +++--- 8 files changed, 30 insertions(+), 30 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hf200012 opened a new pull request, #13226: [refactor](datax)Refactoring doris writer code
hf200012 opened a new pull request, #13226: URL: https://github.com/apache/doris/pull/13226 Refactoring doris writer code # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [x] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [x] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hf200012 closed pull request #13226: [refactor](datax)Refactoring doris writer code
hf200012 closed pull request #13226: [refactor](datax)Refactoring doris writer code URL: https://github.com/apache/doris/pull/13226 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] starocean999 opened a new pull request, #13227: [fix](join) should call getOutputTblRefIds to get child's tuple info
starocean999 opened a new pull request, #13227: URL: https://github.com/apache/doris/pull/13227 # Proposed changes Issue Number: close #xxx ## Problem summary in hash join node, should call getOutputTblRefIds to get child's tuple info, this function works for both inlineViewRef and baseTblRef ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13217: [fix](be_ut) fix endless loop in parquet scan when do parquet_scanner_test ut and es_http_scan_node_test failure
github-actions[bot] commented on PR #13217: URL: https://github.com/apache/doris/pull/13217#issuecomment-1272553122 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13217: [fix](be_ut) fix endless loop in parquet scan when do parquet_scanner_test ut and es_http_scan_node_test failure
github-actions[bot] commented on PR #13217: URL: https://github.com/apache/doris/pull/13217#issuecomment-1272553367 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xinyiZzz merged pull request #13217: [fix](be_ut) fix endless loop in parquet scan when do parquet_scanner_test ut and es_http_scan_node_test failure
xinyiZzz merged PR #13217: URL: https://github.com/apache/doris/pull/13217 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.1-lts updated: [fix](be_ut) fix endless loop in parquet scan when do parquet_scanner_test ut and es_http_scan_node_test failure (#13217)
This is an automated email from the ASF dual-hosted git repository.
zouxinyi pushed a commit to branch branch-1.1-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.1-lts by this push:
new 1539b661cb [fix](be_ut) fix endless loop in parquet scan when do
parquet_scanner_test ut and es_http_scan_node_test failure (#13217)
1539b661cb is described below
commit 1539b661cbcb5321f6a6843141212d54e7d01e31
Author: caiconghui <55968745+caicong...@users.noreply.github.com>
AuthorDate: Sun Oct 9 22:26:25 2022 +0800
[fix](be_ut) fix endless loop in parquet scan when do parquet_scanner_test
ut and es_http_scan_node_test failure (#13217)
Co-authored-by: caiconghui1
---
be/src/runtime/memory/mem_tracker_limiter.cpp | 2 ++
be/test/exec/parquet_scanner_test.cpp | 1 +
2 files changed, 3 insertions(+)
diff --git a/be/src/runtime/memory/mem_tracker_limiter.cpp
b/be/src/runtime/memory/mem_tracker_limiter.cpp
index a8e6315f74..9b4eedea65 100644
--- a/be/src/runtime/memory/mem_tracker_limiter.cpp
+++ b/be/src/runtime/memory/mem_tracker_limiter.cpp
@@ -73,7 +73,9 @@ MemTrackerLimiter::MemTrackerLimiter(int64_t byte_limit,
const std::string& labe
MemTrackerLimiter::~MemTrackerLimiter() {
// TCMalloc hook will be triggered during destructor memtracker, may cause
crash.
+#ifndef BE_TEST
if (_label == "Process") doris::thread_context_ptr._init = false;
+#endif
DCHECK(remain_child_count() == 0 || _label == "Process");
// In order to ensure `consumption of all limiter trackers` + `orphan
tracker consumption` = `process tracker consumption`
// in real time. Merge its consumption into orphan when parent is process,
to avoid repetition.
diff --git a/be/test/exec/parquet_scanner_test.cpp
b/be/test/exec/parquet_scanner_test.cpp
index e299f71c59..2e5eeded8f 100644
--- a/be/test/exec/parquet_scanner_test.cpp
+++ b/be/test/exec/parquet_scanner_test.cpp
@@ -421,6 +421,7 @@ void ParquetScannerTest::init() {
}
TEST_F(ParquetScannerTest, normal) {
+config::parquet_reader_max_buffer_size = 50;
BrokerScanNode scan_node(&_obj_pool, _tnode, *_desc_tbl);
scan_node.init(_tnode);
auto status = scan_node.prepare(&_runtime_state);
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] luozenglin opened a new pull request, #13228: [feature](agg) support `any`,`any_value` agg functions.
luozenglin opened a new pull request, #13228: URL: https://github.com/apache/doris/pull/13228 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [x] No - [ ] No Need 3. Has document been added or modified: - [x] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13143: [test](jdbc) add jdbc and hive regression test
github-actions[bot] commented on PR #13143:
URL: https://github.com/apache/doris/pull/13143#issuecomment-1272565640
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/3214629210";) output.
shellcheck errors
```
'shellcheck ' returned error 1 finding the following syntactical issues:
--
In docker/thirdparties/docker-compose/hive/gen_env.sh line 35:
echo "FS_PORT=${FS_PORT}" >>"${ROOT}"/hadoop-hive.env
^-- SC2129 (style): Consider using { cmd1; cmd2; } >> file instead of
individual redirects.
For more information:
https://www.shellcheck.net/wiki/SC2129 -- Consider using { cmd1; cmd2; }
>>...
--
You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
# shellcheck disable=
above the line that contains the issue, where is the error code;
3. Add '-e ' to the SHELLCHECK_OPTS setting in your .yml action file.
```
shfmt errors
```
'shfmt ' found no issues.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Toms1999 opened a new pull request, #13229: Fully resolved Mysql external table issues
Toms1999 opened a new pull request, #13229: URL: https://github.com/apache/doris/pull/13229 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman commented on a diff in pull request #13224: [fix](rowset) fix that rowset writer doesn't process the return value which may result in data loss
morningman commented on code in PR #13224:
URL: https://github.com/apache/doris/pull/13224#discussion_r990808904
##
be/src/olap/rowset/beta_rowset_writer.cpp:
##
@@ -173,7 +173,11 @@ OLAPStatus
BetaRowsetWriter::flush_single_memtable(MemTable* memtable, int64_t*
RowsetSharedPtr BetaRowsetWriter::build() {
// TODO(lingbin): move to more better place, or in a CreateBlockBatch?
for (auto& wblock : _wblocks) {
Review Comment:
I think we should fix it in a more complete way, to return Status instead of
nullptr, which is very error-prone.
And use `RowsetSharedPtr` as an out parameter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13222: [chore](release build) copy license and notice file to output folder and strip debug info from meta tool
github-actions[bot] commented on PR #13222: URL: https://github.com/apache/doris/pull/13222#issuecomment-1272578114 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13222: [chore](release build) copy license and notice file to output folder and strip debug info from meta tool
github-actions[bot] commented on PR #13222: URL: https://github.com/apache/doris/pull/13222#issuecomment-1272578104 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13212: [feature-wip](multi-catalog) Optimize the performance of boolean & dictionary decoding
github-actions[bot] commented on PR #13212: URL: https://github.com/apache/doris/pull/13212#issuecomment-1272578940 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13212: [feature-wip](multi-catalog) Optimize the performance of boolean & dictionary decoding
github-actions[bot] commented on PR #13212: URL: https://github.com/apache/doris/pull/13212#issuecomment-1272578937 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
