Re: [I] [Bug] postgresql catalog insert into 指定列名顺序失败 [doris]
Yulei-Yang commented on issue #28120: URL: https://github.com/apache/doris/issues/28120#issuecomment-1844844980 fixed in master: https://github.com/apache/doris/pull/27855 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] postgresql catalog insert into 指定列名顺序失败 [doris]
wz11wz commented on issue #28120: URL: https://github.com/apache/doris/issues/28120#issuecomment-1844845531 这是来自QQ邮箱的假期自动回复邮件。您好，我最近正在休假中，无法亲自回复您的邮件。我将在假期结束后，尽快给您回复。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) tag 2.0.3-rc05 created (now 97d6021c03d)
This is an automated email from the ASF dual-hosted git repository. kxiao pushed a change to tag 2.0.3-rc05 in repository https://gitbox.apache.org/repos/asf/doris.git at 97d6021c03d (commit) No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] Revert "Add blog" [doris-website]
httpshirley opened a new pull request, #350: URL: https://github.com/apache/doris-website/pull/350 Reverts apache/doris-website#349 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) rewrite memtable memory limiter rules [doris]
gavinchou commented on PR #27759: URL: https://github.com/apache/doris/pull/27759#issuecomment-1844850992 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) rewrite memtable memory limiter rules [doris]
github-actions[bot] commented on PR #27759: URL: https://github.com/apache/doris/pull/27759#issuecomment-1844852189 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [DRAFT] Release Note 2.0.3 [doris]
morningman commented on issue #27909: URL: https://github.com/apache/doris/issues/27909#issuecomment-1844852499 > 对 Automatic Collection、统计信息有点兴趣，最近有什么开发计划吗？ Welcome, I will contact you later -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Revert "Add blog" [doris-website]
luzhijing merged PR #350: URL: https://github.com/apache/doris-website/pull/350 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated: Revert "Add blog (#349)" (#350)
This is an automated email from the ASF dual-hosted git repository.
luzhijing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new c7ed859cdb0 Revert "Add blog (#349)" (#350)
c7ed859cdb0 is described below
commit c7ed859cdb07340766f89375e74650e1a106f03a
Author: Hu Yanjun <100749531+httpshir...@users.noreply.github.com>
AuthorDate: Thu Dec 7 16:05:38 2023 +0800
Revert "Add blog (#349)" (#350)
This reverts commit b36ea61bf82d8fa9140f2c4374a0203d43c4b1c2.
---
...-by-enabling-seven-times-faster-log-analysis.md | 142 -
...apache-doris-vs-starrocks-query-performance.png | Bin 760906 -> 0 bytes
...pache-doris-vs-starrocks-writing-throughput.png | Bin 102274 -> 0 bytes
static/images/cyber-security-inverted-index.png| Bin 457762 -> 0 bytes
...-security-log-storage-and-analysis-platform.png | Bin 119593 -> 0 bytes
...er-for-visualized-operation-and-maintenance.png | Bin 341077 -> 0 bytes
static/images/doris-manager-webui-showcase.png | Bin 329312 -> 0 bytes
7 files changed, 142 deletions(-)
diff --git
a/blog/empowering-cyber-security-by-enabling-seven-times-faster-log-analysis.md
b/blog/empowering-cyber-security-by-enabling-seven-times-faster-log-analysis.md
deleted file mode 100644
index f6195922646..000
---
a/blog/empowering-cyber-security-by-enabling-seven-times-faster-log-analysis.md
+++ /dev/null
@@ -1,142 +0,0 @@
-{
-'title': 'Empowering cyber security by enabling 7 times faster log
analysis',
-'summary': "This is about how a cyber security service provider built its
log storage and analysis system (LSAS) and realized 3X data writing speed, 7X
query execution speed, and visualized management.",
-'date': '2023--12-07',
-'author': 'Apache Doris',
-'tags': ['Best Practice'],
-}
-
-
-
-
-
-This is about how a cyber security service provider built its log storage and
analysis system (LSAS) and realized 3X data writing speed, 7X query execution
speed, and visualized management.
-
-## Log storage & analysis platform
-
-In this use case, the LSAS collects system logs from its enterprise users,
scans them, and detects viruses. It also provides data management and file
tracking services.
-
-Within the LSAS, it scans local files and uploads the file information as MD5
values to its cloud engine and identifies suspicious viruses. The cloud engine
returns a log entry to tell the risk level of the files. The log entry includes
messages like `file_name`, `file_size`, `file_level`, and `event_time`. Such
information goes into a Topic in Apache Kafka, and then the real-time data
warehouse normalizes the log messages. After that, all log data will be backed
up to the offline data w [...]
-
-
-
-The above process comes down to log writing and analysis, and the company
faced some issues in both processes with their old system, which used StarRocks
as the analytic engine.
-
-### Slow data writing
-
-The cloud engine interacts with tens of millions of terminal software and
digests over 100 billion logs every day. The enormous data size poses a big
challenge. The LSAS used to rely on StarRocks for log storage. With the
ever-increasing daily log influx, data writing gradually slows down. The severe
backlogs during peak times undermines system stability. They tried scaling the
cluster from 3 nodes to 13 nodes, but the writing speed wasn't substantially
improved.
-
-### Slow query execution
-
-From an execution standpoint, extracting security information from logs
involves a lot of keyword matching in the text fields (URL, payload, etc.). The
StarRocks-based system does that by the SQL LIKE operator, which implements
full scanning and brutal-force matching. In that way, queries on a
100-billion-row table often take one or several minutes. After screening out
irrelevant data based on time range, the query response time still ranges from
seconds to dozens of seconds, and it gets [...]
-
-## Architectural upgrade
-
-In the search for a new database tool, the cyber security company set their
eye on [Apache Doris](https://doris.apache.org/zh-CN/), which happened to have
sharpened itself up in [version
2.0](https://doris.apache.org/zh-CN/blog/release-note-2.0.0) for log analysis.
It supports [inverted
index](https://doris.apache.org/docs/dev/data-table/index/inverted-index/) to
empower text search, and [NGram
BloomFilter](https://doris.apache.org/docs/dev/data-table/index/ngram-bloomfilter-index?_highl
[...]
-
-Although StarRocks was a fork of Apache Doris, it has rewritten part of the
code and is now very different from Apache Doris in terms of features. The
foregoing inverted index and NGram BloomFilter are a fragment of the current
advancements that Apache Dori
[PR] add blog (fixed) [doris-website]
httpshirley opened a new pull request, #351: URL: https://github.com/apache/doris-website/pull/351 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](docs) Fix partition cache design principles [doris]
xinyiZzz commented on PR #28110: URL: https://github.com/apache/doris/pull/28110#issuecomment-1844858885 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] add blog (fixed) [doris-website]
luzhijing merged PR #351: URL: https://github.com/apache/doris-website/pull/351 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated: add blog (fixed) (#351)
This is an automated email from the ASF dual-hosted git repository.
luzhijing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 2c375e8ab0a add blog (fixed) (#351)
2c375e8ab0a is described below
commit 2c375e8ab0a6381547035d8f1ab876310d956489
Author: Hu Yanjun <100749531+httpshir...@users.noreply.github.com>
AuthorDate: Thu Dec 7 16:09:59 2023 +0800
add blog (fixed) (#351)
---
...-by-enabling-seven-times-faster-log-analysis.md | 142 +
...apache-doris-vs-starrocks-query-performance.png | Bin 0 -> 760906 bytes
...pache-doris-vs-starrocks-writing-throughput.png | Bin 0 -> 102274 bytes
static/images/cyber-security-inverted-index.png| Bin 0 -> 457762 bytes
...-security-log-storage-and-analysis-platform.png | Bin 0 -> 119593 bytes
...er-for-visualized-operation-and-maintenance.png | Bin 0 -> 341077 bytes
static/images/doris-manager-webui-showcase.png | Bin 0 -> 329312 bytes
7 files changed, 142 insertions(+)
diff --git
a/blog/empowering-cyber-security-by-enabling-seven-times-faster-log-analysis.md
b/blog/empowering-cyber-security-by-enabling-seven-times-faster-log-analysis.md
new file mode 100644
index 000..7ea61bf2e0e
--- /dev/null
+++
b/blog/empowering-cyber-security-by-enabling-seven-times-faster-log-analysis.md
@@ -0,0 +1,142 @@
+---
+{
+'title': 'Empowering cyber security by enabling 7 times faster log
analysis',
+'summary': "This is about how a cyber security service provider built its
log storage and analysis system (LSAS) and realized 3X data writing speed, 7X
query execution speed, and visualized management.",
+'date': '2023-12-07',
+'author': 'Apache Doris',
+'tags': ['Best Practice'],
+}
+
+
+---
+
+
+
+This is about how a cyber security service provider built its log storage and
analysis system (LSAS) and realized 3X data writing speed, 7X query execution
speed, and visualized management.
+
+## Log storage & analysis platform
+
+In this use case, the LSAS collects system logs from its enterprise users,
scans them, and detects viruses. It also provides data management and file
tracking services.
+
+Within the LSAS, it scans local files and uploads the file information as MD5
values to its cloud engine and identifies suspicious viruses. The cloud engine
returns a log entry to tell the risk level of the files. The log entry includes
messages like `file_name`, `file_size`, `file_level`, and `event_time`. Such
information goes into a Topic in Apache Kafka, and then the real-time data
warehouse normalizes the log messages. After that, all log data will be backed
up to the offline data w [...]
+
+
+
+The above process comes down to log writing and analysis, and the company
faced some issues in both processes with their old system, which used StarRocks
as the analytic engine.
+
+### Slow data writing
+
+The cloud engine interacts with tens of millions of terminal software and
digests over 100 billion logs every day. The enormous data size poses a big
challenge. The LSAS used to rely on StarRocks for log storage. With the
ever-increasing daily log influx, data writing gradually slows down. The severe
backlogs during peak times undermines system stability. They tried scaling the
cluster from 3 nodes to 13 nodes, but the writing speed wasn't substantially
improved.
+
+### Slow query execution
+
+From an execution standpoint, extracting security information from logs
involves a lot of keyword matching in the text fields (URL, payload, etc.). The
StarRocks-based system does that by the SQL LIKE operator, which implements
full scanning and brutal-force matching. In that way, queries on a
100-billion-row table often take one or several minutes. After screening out
irrelevant data based on time range, the query response time still ranges from
seconds to dozens of seconds, and it gets [...]
+
+## Architectural upgrade
+
+In the search for a new database tool, the cyber security company set their
eye on [Apache Doris](https://doris.apache.org/zh-CN/), which happened to have
sharpened itself up in [version
2.0](https://doris.apache.org/zh-CN/blog/release-note-2.0.0) for log analysis.
It supports [inverted
index](https://doris.apache.org/docs/dev/data-table/index/inverted-index/) to
empower text search, and [NGram
BloomFilter](https://doris.apache.org/docs/dev/data-table/index/ngram-bloomfilter-index?_highl
[...]
+
+Although StarRocks was a fork of Apache Doris, it has rewritten part of the
code and is now very different from Apache Doris in terms of features. The
foregoing inverted index and NGram BloomFilter are a fragment of the current
advancements that Apache Doris has made.
+
+They tried Apache Doris out to evaluate its writing speed, query performance
Re: [PR] [improvement](insert) make all case can run with replaying wal [doris]
doris-robot commented on PR #27688: URL: https://github.com/apache/doris/pull/27688#issuecomment-1844861717 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Tpch sf100 test result on commit a824db0821c0da665bc93db932cae3ce29ebb49f, data reload: false run tpch-sf100 query with default conf and session variables q1 4723442744674427 q2 361 153 157 153 q3 1453126211801180 q4 927 891 891 q5 3158314331493143 q6 248 133 125 125 q7 996 495 484 484 q8 2189219021682168 q9 6658669366486648 q10 3219325132663251 q11 323 206 194 194 q12 352 205 202 202 q13 4541386137813781 q14 245 211 222 211 q15 556 521 533 521 q16 445 400 382 382 q17 999 573 590 573 q18 7374720669466946 q19 1523135314391353 q20 546 339 370 339 q21 3083266627142666 q22 357 294 301 294 Total cold run time: 44460 ms Total hot run time: 39932 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 4405440843584358 q2 266 167 174 167 q3 3533353035043504 q4 2367236323522352 q5 5743574157345734 q6 240 119 120 119 q7 2371190118761876 q8 3518350735143507 q9 9020894689898946 q10 3894398240103982 q11 490 393 396 393 q12 757 590 590 590 q13 4320358535503550 q14 278 268 246 246 q15 570 517 516 516 q16 511 459 466 459 q17 1863188618541854 q18 8670830983328309 q19 1741172917231723 q20 2267194819391939 q21 6509616761266126 q22 511 415 431 415 Total cold run time: 63844 ms Total hot run time: 60665 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [chore](memory) Warning in log when turning on THP [doris]
xinyiZzz opened a new pull request, #28122: URL: https://github.com/apache/doris/pull/28122 ## Proposed changes Doris not recommend turning on THP, which may cause the BE process to use more memory and cannot be freed in time. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](memory) Warning in log when turning on THP [doris]
xinyiZzz commented on PR #28122: URL: https://github.com/apache/doris/pull/28122#issuecomment-1844862544 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](Nereids) let create table compatible with legacy planner [doris]
starocean999 commented on PR #28078: URL: https://github.com/apache/doris/pull/28078#issuecomment-1844866706 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](docs) Fix partition cache design principles [doris]
github-actions[bot] commented on PR #28110: URL: https://github.com/apache/doris/pull/28110#issuecomment-1844866686 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](stream load)add test case and doc for arrow type of stream load [doris]
wuwenchi commented on PR #28098: URL: https://github.com/apache/doris/pull/28098#issuecomment-1844870269 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](stream load)add test case and doc for arrow type of stream load [doris]
wuwenchi commented on PR #28098: URL: https://github.com/apache/doris/pull/28098#issuecomment-1844870611 run piplinex_p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](insert) make all case can run with replaying wal [doris]
hust-hhb commented on PR #27688: URL: https://github.com/apache/doris/pull/27688#issuecomment-1844871919 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](memory) Warning in log when turning on THP [doris]
github-actions[bot] commented on code in PR #28122:
URL: https://github.com/apache/doris/pull/28122#discussion_r1418549186
##
be/src/util/mem_info.cpp:
##
@@ -460,6 +460,19 @@ void MemInfo::init() {
_s_sys_mem_available_warning_water_mark =
_s_sys_mem_available_low_water_mark * 2;
}
+std::ifstream
sys_transparent_hugepage("/sys/kernel/mm/transparent_hugepage/enabled",
+ std::ios::in);
+std::string hugepage_enable;
+getline(sys_transparent_hugepage, hugepage_enable);
+if (sys_transparent_hugepage.is_open()) sys_transparent_hugepage.close();
Review Comment:
warning: statement should be inside braces
[readability-braces-around-statements]
```suggestion
if (sys_transparent_hugepage.is_open()) {
sys_transparent_hugepage.close();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [doc](statistics)Update external catalog statistics doc. [doris]
Jibing-Li opened a new pull request, #28123: URL: https://github.com/apache/doris/pull/28123 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default again [doris]
doris-robot commented on PR #28105: URL: https://github.com/apache/doris/pull/28105#issuecomment-1844875314 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Tpch sf100 test result on commit 4a186190464b191384b6ae7db4c20cfe7c5599fa, data reload: false run tpch-sf100 query with default conf and session variables q1 4719449444484448 q2 372 142 158 142 q3 1471124712561247 q4 1115928 903 903 q5 3189316531973165 q6 255 128 133 128 q7 989 494 492 492 q8 220621782178 q9 6655666966546654 q10 3239325132693251 q11 328 204 208 204 q12 354 214 216 214 q13 4553381438393814 q14 246 215 219 215 q15 574 526 533 526 q16 444 394 397 394 q17 1009572 599 572 q18 7427746169896989 q19 1532132614351326 q20 1135298 312 298 q21 3096263926542639 q22 350 288 296 288 Total cold run time: 45258 ms Total hot run time: 40087 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 4385437244204372 q2 266 167 169 167 q3 3532351835313518 q4 2370236123612361 q5 5777573657135713 q6 238 119 123 119 q7 2386188118481848 q8 3520351735123512 q9 9037899289618961 q10 3914398740083987 q11 509 387 373 373 q12 766 605 592 592 q13 4301360835453545 q14 293 246 250 246 q15 577 528 515 515 q16 521 490 457 457 q17 1902186118681861 q18 8606829583508295 q19 1718172117211721 q20 2269195519331933 q21 6544619761876187 q22 506 418 422 418 Total cold run time: 63937 ms Total hot run time: 60701 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) forbid two TVF in one fragment since the limit of coordinator [doris]
doris-robot commented on PR #28114: URL: https://github.com/apache/doris/pull/28114#issuecomment-1844886821 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Tpch sf100 test result on commit 2ce7957b7967809870580368232fe01da45f9f44, data reload: false run tpch-sf100 query with default conf and session variables q1 4685442044764420 q2 370 145 158 145 q3 1454123212361232 q4 1110930 893 893 q5 3150316431653164 q6 248 130 127 127 q7 1002499 485 485 q8 2199221521912191 q9 6642668966406640 q10 3245326632683266 q11 331 210 203 203 q12 362 213 211 211 q13 4542385538233823 q14 240 208 220 208 q15 563 528 533 528 q16 435 386 400 386 q17 1026614 589 589 q18 7705748486857484 q19 1529140214331402 q20 510 306 407 306 q21 3093262426822624 q22 360 290 293 290 Total cold run time: 44801 ms Total hot run time: 40617 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 4404437843884378 q2 272 162 174 162 q3 3552352935143514 q4 2382236923772369 q5 5743572657285726 q6 242 122 121 121 q7 2378188218551855 q8 3516351935223519 q9 9069895789808957 q10 3909398939813981 q11 511 379 401 379 q12 774 595 594 594 q13 4290353135693531 q14 280 251 260 251 q15 572 520 520 520 q16 508 457 488 457 q17 1858185918621859 q18 8680911282498249 q19 1722173917831739 q20 2231194719401940 q21 6503615161476147 q22 497 415 405 405 Total cold run time: 63893 ms Total hot run time: 60653 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](partial update) Fix missing rowsets during doing alignment when flushing memtable due to compaction [doris]
github-actions[bot] commented on code in PR #28062:
URL: https://github.com/apache/doris/pull/28062#discussion_r1418568157
##
be/src/olap/olap_common.h:
##
@@ -493,13 +493,18 @@ inline RowsetId extract_rowset_id(std::string_view
filename) {
class DeleteBitmap;
// merge on write context
struct MowContext {
-MowContext(int64_t version, int64_t txnid, const RowsetIdUnorderedSet& ids,
+MowContext(int64_t version, int64_t txnid, RowsetIdUnorderedSet& ids,
std::shared_ptr db)
: max_version(version), txn_id(txnid), rowset_ids(ids),
delete_bitmap(db) {}
+void update_rowset_ids_with_lock(std::function callback) {
Review Comment:
warning: method 'update_rowset_ids_with_lock' can be made static
[readability-convert-member-functions-to-static]
```suggestion
static void update_rowset_ids_with_lock(std::function callback) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](stream load)add test case and doc for arrow type of stream load [doris]
hello-stephen commented on PR #28098: URL: https://github.com/apache/doris/pull/28098#issuecomment-1844890034 run piplinex_p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](Nereids) improve Shape check cases [doris]
englefly commented on PR #28124: URL: https://github.com/apache/doris/pull/28124#issuecomment-1844892343 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
svn commit: r65874 - in /dev/doris/2.0.3: ./ apache-doris-2.0.3-src.tar.gz apache-doris-2.0.3-src.tar.gz.asc apache-doris-2.0.3-src.tar.gz.sha512
Author: kxiao Date: Thu Dec 7 08:33:26 2023 New Revision: 65874 Log: doris-2.0.3 release Added: dev/doris/2.0.3/ dev/doris/2.0.3/apache-doris-2.0.3-src.tar.gz (with props) dev/doris/2.0.3/apache-doris-2.0.3-src.tar.gz.asc (with props) dev/doris/2.0.3/apache-doris-2.0.3-src.tar.gz.sha512 Added: dev/doris/2.0.3/apache-doris-2.0.3-src.tar.gz == Binary file - no diff available. Propchange: dev/doris/2.0.3/apache-doris-2.0.3-src.tar.gz -- svn:mime-type = application/x-gzip Added: dev/doris/2.0.3/apache-doris-2.0.3-src.tar.gz.asc == Binary file - no diff available. Propchange: dev/doris/2.0.3/apache-doris-2.0.3-src.tar.gz.asc -- svn:mime-type = application/pgp-signature Added: dev/doris/2.0.3/apache-doris-2.0.3-src.tar.gz.sha512 == --- dev/doris/2.0.3/apache-doris-2.0.3-src.tar.gz.sha512 (added) +++ dev/doris/2.0.3/apache-doris-2.0.3-src.tar.gz.sha512 Thu Dec 7 08:33:26 2023 @@ -0,0 +1 @@ +5441112e97a773ff7cb37ed925beb6159372370a6a39984ba225c720bc1aede51dd833583543fab1f34c462829f093f96ab8a4358e733c299f7041d2a3da420c apache-doris-2.0.3-src.tar.gz - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [opt](Nereids) improve Shape check cases [doris]
englefly opened a new pull request, #28124: URL: https://github.com/apache/doris/pull/28124 ## Proposed changes 1. fix parallel instance num when running shape check 2. stop auto analyze when running shape check 3. verify column stats during shape check Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](move-memtable) support variant in move-memtable [doris]
github-actions[bot] commented on PR #28084: URL: https://github.com/apache/doris/pull/28084#issuecomment-1844893725 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [DRAFT] Release Note 2.0.3 [doris]
xiaokang commented on issue #27909: URL: https://github.com/apache/doris/issues/27909#issuecomment-1844893803 > 撒时候正式发布啊 This week. Thanks for your attention. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default again [doris]
doris-robot commented on PR #28105: URL: https://github.com/apache/doris/pull/28105#issuecomment-1844894917 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.44 seconds stream load tsv: 589 seconds loaded 74807831229 Bytes, about 121 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 34 seconds loaded 861443392 Bytes, about 24 MB/s insert into select: 28.6 seconds inserted 1000 Rows, about 349K ops/s storage size: 17194683749 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression-test](Variant) Add more cases related to schema changes [doris]
doris-robot commented on PR #27958: URL: https://github.com/apache/doris/pull/27958#issuecomment-1844894972 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 43.89 seconds stream load tsv: 582 seconds loaded 74807831229 Bytes, about 122 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17194399697 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) forbid two TVF in one fragment since the limit of coordinator [doris]
doris-robot commented on PR #28114: URL: https://github.com/apache/doris/pull/28114#issuecomment-1844895045 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 43.93 seconds stream load tsv: 588 seconds loaded 74807831229 Bytes, about 121 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 35 seconds loaded 861443392 Bytes, about 23 MB/s insert into select: 28.8 seconds inserted 1000 Rows, about 347K ops/s storage size: 17194455344 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](nereids) Support inner join query rewrite by materialized view [doris]
doris-robot commented on PR #27922: URL: https://github.com/apache/doris/pull/27922#issuecomment-1844895205 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 43.96 seconds stream load tsv: 586 seconds loaded 74807831229 Bytes, about 121 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17193694728 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](split_by_string) Fix split by string core on column string [doris]
yiguolei merged PR #28030: URL: https://github.com/apache/doris/pull/28030 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression-test](Variant) Add more cases related to schema changes [doris]
doris-robot commented on PR #27958: URL: https://github.com/apache/doris/pull/27958#issuecomment-1844895592 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Tpch sf100 test result on commit fb1cd539020779f32d0ff0aeade3834316889f3e, data reload: false run tpch-sf100 query with default conf and session variables q1 4655444644504446 q2 360 155 159 155 q3 1446127612591259 q4 1118918 919 918 q5 3117312531503125 q6 246 131 131 131 q7 993 505 482 482 q8 2187221521682168 q9 67056692 q10 3192324532443244 q11 314 210 213 210 q12 353 210 208 208 q13 4679410037883788 q14 246 210 220 210 q15 558 537 519 519 q16 443 392 387 387 q17 996 605 577 577 q18 7438787768856885 q19 1497134114131341 q20 551 332 309 309 q21 3101263926752639 q22 354 290 290 290 Total cold run time: 44549 ms Total hot run time: 39957 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 4417438743914387 q2 268 164 169 164 q3 3532359536543595 q4 2394236323632363 q5 5714572557425725 q6 240 122 120 120 q7 2394191218711871 q8 3503351435153514 q9 9082902590319025 q10 3922397340023973 q11 491 377 395 377 q12 756 607 583 583 q13 4297352435823524 q14 282 264 254 254 q15 567 520 518 518 q16 515 452 443 443 q17 1866185218581852 q18 8709820581178117 q19 1712172217041704 q20 2253194919391939 q21 6470614461596144 q22 504 428 407 407 Total cold run time: 63888 ms Total hot run time: 60599 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [DRAFT] Release Note 2.0.3 [doris]
xiaokang commented on issue #27909: URL: https://github.com/apache/doris/issues/27909#issuecomment-1844895702 中文版本的Release Note 感谢Doris社区参与2.0.3版本的100多位开发者和用户，这版本包含了统计信息、倒排索引、复杂数据类型、数据湖、分布式副本管理等近1000项改进与修复。 ## 1 行为变更 - 复杂数据类型array/map/struct的输出格式改成跟输入格式以及JSON规范保持一致，跟之前版本的主要变化是日期和字符串用双引号括起来，ARRAY/MAP内部的空值显示为null而不是NULL。 - https://github.com/apache/doris/pull/25946 - 默认情况下，当用户属性 `resource_tags.location` 没有设置时，只能使用 default 资源组的节点，而之前版本中可以访问任意节点。 - https://github.com/apache/doris/pull/25331 - 支持 SHOW_VIEW 权限，拥有 SELECT 或 LOAD 权限的用户将不再能够执行 `SHOW CREATE VIEW` 语句，必须单独授予 SHOW_VIEW 权限。 - https://github.com/apache/doris/pull/25370 ## 2 新功能 ### 2.1 支持自动统计信息收集 通过收集统计信息有助于优化器了解数据分布特性，选择更优的计划以大幅提升查询效率。从2.0.3版本开始正式支持，默认为全天开启状态。 更多信息请参考：https://doris.apache.org/docs/query-acceleration/statistics/ ### 2.2 数据湖更多系统支持复杂数据类型 - JAVA UDF、JDBC、Hudi MOR 表等功能支持复杂数据类型 - https://github.com/apache/doris/pull/24810 - https://github.com/apache/doris/pull/26236 - Paimon catalog 支持 复杂数据类型 - https://github.com/apache/doris/pull/25364 - Paimon catalog 支持 paimon 0.5 版本 - https://github.com/apache/doris/pull/24985 ### 2.3 增加更多内置函数 - 新优化器支持BitmapAgg函数 - https://github.com/apache/doris/pull/25508 - 支持SHA系列摘要函数 - https://github.com/apache/doris/pull/24342 - 聚合函数min_by和max_by支持bitmap数据类型 - https://github.com/apache/doris/pull/25430 - 增加milliseconds/microseconds_add/sub/diff函数 - https://github.com/apache/doris/pull/24114 - 增加json_insert, json_replace, json_set JSON函数 - https://github.com/apache/doris/pull/24384 ## 3 改进和优化 ### 3.1 性能优化 - 在过滤率高的倒排索引MATCH WHERE条件和过滤率低的普通WHERE条件组合时，大幅降低索引列的IO - 优化经过where条件过滤后随机读数据的效率 - 优化在JSON数据类型上使用老的get_json_xx 函数的性能，提升2-4倍 - 支持配置降低读数据线程的优先级，保证写入的CPU资源和实时性 - 增加返回largeint的uuid-numeric函数，性能比返回string的uuid函数快20倍 - 优化了case when 的性能提升3倍 - 在存储引擎执行中裁剪不必要的谓词计算 - 支持 count 算子下推到存储层 - 优化支持 and or 表达式中包含nullable 类型的计算性能 - 支持更多场景下limit算子提前到join前执行的改写，以提升执行效率 - 增加消除inline view中的无用的order by算子，以提升执行效率 - 优化了部分情况下的基数估计和代价模型的准确性，以提升执行效率 - 优化了jdbc catalog 的谓词下推逻辑和大小写逻辑 - 优化了file cache的第一次开启后的读取效率 - 优化hive 表sql cache 策略，使用 hms 中存储的分区更新时间作为 cache 是否失效的判断，提高cache命中率。 - 优化mow compaction 效率 - 优化了外表查询的线程分配逻辑，降低内存使用 - 优化column reader 的内存使用 ### 3.2 分布式副本管理改进 包括跳过删除分区，colocate group ，持续写时均衡失败，冷热分层表不能均衡等 ### 3.3 安全性提升 - 审计日志插件的配置使用token代替明文密码增强安全性 - https://github.com/apache/doris/pull/26278 - log4j配置安全性增强 - https://github.com/apache/doris/pull/24861 - 日志中不显示用户敏感信息 - https://github.com/apache/doris/pull/26912 ## 4 bugfix和稳定性提升 ### 4.1 复杂数据类型 - 修复了map/struct对定长CHAR(n)没有正确截断的问题 - https://github.com/apache/doris/pull/25725 - 修复了struct嵌套map/array写入失败的问题 - https://github.com/apache/doris/pull/26973 - 修复了count distinct不支持array/map/struct的问题 - https://github.com/apache/doris/pull/25483 - 解决query中出现delete复杂类型之后，升级到2.0.3 过程中出现be crash的问题 - https://github.com/apache/doris/pull/26006 - 修复了jsonb在where条件中be crash的问题 - https://github.com/apache/doris/pull/27325 - 修复了outer join中有array类型时be crash的问题 - https://github.com/apache/doris/pull/25669 - 修复 orc 格式 decimal 类型读取错误的问题 - https://github.com/apache/doris/pull/26548 - https://github.com/apache/doris/pull/25977 - https://github.com/apache/doris/pull/26633 ### 4.2 倒排索引 - 修复了关闭倒排索引查询时OR NOT组合where条件结果错误的问题 - https://github.com/apache/doris/pull/26327 - 修复了空数组的倒排索引写入时be crash的问题 - https://github.com/apache/doris/pull/25984 - 修复输出为空的情况下index compaction be crash的问题 - https://github.com/apache/doris/pull/25486 - 修复新增列没有写入数据时，增加倒排索引be crash的问题 - https://github.com/apache/doris/pull/27276 - 修复1.2误建倒排索引后升级2.0等情况下倒排索引硬链缺失和泄露的问题 - https://github.com/apache/doris/pull/26903 ### 4.3 物化视图 - 修复group by语句中包括重复表达式导致BE crash 的问题 - https://github.com/apache/doris/pull/27523 - 禁止视图创建时group by 子句中使用float/doubld 类型 - https://github.com/apache/doris/pull/25823 - 增强支持了select 查询命中物化视图的功能 - https://github.com/apache/doris/pull/24691 - 修复当使用了表的alias时，物化视图不能命中的问题 - https://github.com/apache/doris/pull/25321 - 修复了创建物化视图中使用percentile_approx的问题 - https://github.com/apache/doris/pull/26528 ### 4.4 采样查询 - 修复table sample功能在partition table 上无法正常工作的问题 - https://github.com/apache/doris/pull/25912 - 修复table sample指定tablet 无法工作的问题 - https://github.com/apache/doris/pull/25378 ### 4.5 主键表 - 修复基于主键条件更新的空指针异常 - https://github.com/apache/doris/pull/26881 - 修复部分列更新字段名大小写问题 - https://github.com/apache/doris/pull/27223 - 修复 schema change 时 mow 会出现重复 key 的问题 - https://github.com/apache/doris/pull/25705 ### 4.6 导入和compaction - 修复 routineload 一流多表时 unkown slot descriptor 错误 - https://github.com/
[PR] [improve](env) Add disk usage in not ready msg [doris]
w41ter opened a new pull request, #28125: URL: https://github.com/apache/doris/pull/28125 ## Proposed changes Issue Number: close #xxx When the disk space is insufficient, BDBJE will stop writing. This PR adds the disk usage log to locate the problem faster when not ready. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (495c01bdfd3 -> 1a46cf6fb50)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 495c01bdfd3 [fix](session-variable) enable_unicode_name_support need forward to master (#28112) add 1a46cf6fb50 [fix](split_by_string) Fix split by string core on column string (#28030) No new revisions were added by this update. Summary of changes: be/src/vec/functions/function_string.h | 100 + .../string_functions/test_split_by_string.out | 20 + .../string_functions/test_split_by_string.groovy | 60 + 3 files changed, 164 insertions(+), 16 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](env) Add disk usage in not ready msg [doris]
w41ter commented on PR #28125: URL: https://github.com/apache/doris/pull/28125#issuecomment-1844895920 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Error while running github feature from .asf.yaml in doris!
An error occurred while running github feature in .asf.yaml!: You can only have a maximum of 10 external triage collaborators, please contact vp-in...@apache.org to request an exception. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](statistics)Escape load stats sql. [doris]
doris-robot commented on PR #28117: URL: https://github.com/apache/doris/pull/28117#issuecomment-1844897969 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 43.86 seconds stream load tsv: 585 seconds loaded 74807831229 Bytes, about 121 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.0 seconds inserted 1000 Rows, about 344K ops/s storage size: 17194371833 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](log) Log to trace before wait rpc timeout [doris]
github-actions[bot] commented on PR #28024: URL: https://github.com/apache/doris/pull/28024#issuecomment-1844898154 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](variant) fallback to none partial update for mow table [doris]
doris-robot commented on PR #28116: URL: https://github.com/apache/doris/pull/28116#issuecomment-1844898238 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 43.47 seconds stream load tsv: 584 seconds loaded 74807831229 Bytes, about 122 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 34 seconds loaded 861443392 Bytes, about 24 MB/s insert into select: 29.2 seconds inserted 1000 Rows, about 342K ops/s storage size: 17194696702 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](log) Log to trace before wait rpc timeout [doris]
github-actions[bot] commented on PR #28024: URL: https://github.com/apache/doris/pull/28024#issuecomment-1844898220 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Variant) ColumnObject need to be finalized when doing ColumnObj… [doris]
doris-robot commented on PR #28119: URL: https://github.com/apache/doris/pull/28119#issuecomment-1844898380 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 44.6 seconds stream load tsv: 586 seconds loaded 74807831229 Bytes, about 121 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.1 seconds inserted 1000 Rows, about 343K ops/s storage size: 17193908969 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](move-memtable) support variant in move-memtable [doris]
eldenmoon merged PR #28084: URL: https://github.com/apache/doris/pull/28084 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (1a46cf6fb50 -> 9c63dfd692a)
This is an automated email from the ASF dual-hosted git repository. eldenmoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 1a46cf6fb50 [fix](split_by_string) Fix split by string core on column string (#28030) add 9c63dfd692a [improve](move-memtable) support variant in move-memtable (#28084) No new revisions were added by this update. Summary of changes: be/src/olap/rowset/beta_rowset_writer.cpp | 7 ++- be/src/olap/rowset/beta_rowset_writer.h | 3 ++- be/src/olap/rowset/beta_rowset_writer_v2.cpp | 5 +++-- be/src/olap/rowset/beta_rowset_writer_v2.h| 3 ++- be/src/olap/rowset/rowset_writer.h| 5 - be/src/olap/rowset/segment_creator.cpp| 15 --- be/src/olap/rowset/segment_creator.h | 12 be/src/runtime/load_stream.cpp| 13 +++-- be/src/runtime/load_stream_writer.cpp | 5 +++-- be/src/runtime/load_stream_writer.h | 2 +- be/src/vec/sink/load_stream_stub.cpp | 6 +- be/src/vec/sink/load_stream_stub.h| 3 ++- gensrc/proto/internal_service.proto | 1 + regression-test/data/variant_p0/load.out | 2 +- .../suites/variant_github_events_p0/load.groovy | 1 + regression-test/suites/variant_p0/complexjson.groovy | 1 + 16 files changed, 59 insertions(+), 25 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Error while running github feature from .asf.yaml in doris!
An error occurred while running github feature in .asf.yaml!: You can only have a maximum of 10 external triage collaborators, please contact vp-in...@apache.org to request an exception. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](nereids) Support inner join query rewrite by materialized view [doris]
doris-robot commented on PR #27922: URL: https://github.com/apache/doris/pull/27922#issuecomment-1844908238 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Tpch sf100 test result on commit be85ee2b1c5b5a5c7aa5c1c96c0f960fd54747c6, data reload: false run tpch-sf100 query with default conf and session variables q1 47084468 q2 379 177 158 158 q3 1467127711981198 q4 1104916 881 881 q5 3162315431393139 q6 250 132 129 129 q7 1001504 494 494 q8 2228219221852185 q9 6735668166376637 q10 3240326732783267 q11 326 209 207 207 q12 358 211 215 211 q13 4556385838003800 q14 254 212 220 212 q15 566 522 522 522 q16 443 388 392 388 q17 1010603 575 575 q18 7541812279857985 q19 1516138413861384 q20 556 304 341 304 q21 3071265926582658 q22 358 286 294 286 Total cold run time: 44829 ms Total hot run time: 41064 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 4411441443794379 q2 266 163 173 163 q3 3537351435203514 q4 2393237623642364 q5 5722573957285728 q6 238 121 121 121 q7 2383184718691847 q8 3507351935153515 q9 9049904290179017 q10 3905399840203998 q11 510 394 399 394 q12 763 604 588 588 q13 4325356235703562 q14 283 265 249 249 q15 560 519 524 519 q16 496 478 487 478 q17 1888183318501833 q18 8780856784028402 q19 1691172417561724 q20 2290194319651943 q21 6531617561606160 q22 497 418 432 418 Total cold run time: 64025 ms Total hot run time: 60916 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](stream load)add test case and doc for arrow type of stream load [doris]
morningman commented on code in PR #28098:
URL: https://github.com/apache/doris/pull/28098#discussion_r1418581374
##
docs/zh-CN/docs/ecosystem/spark-doris-connector.md:
##
@@ -266,7 +266,8 @@ kafkaSource.selectExpr("CAST(value as STRING)")
| doris.write.fields | --|
指定写入Doris表的字段或者字段顺序，多列之间使用逗号分隔。默认写入时要按照Doris表字段顺序写入全部字段。
|
| sink.batch.size | 1 | 单次写BE的最大行数
|
| sink.max-retries | 1 | 写BE失败之后的重试次数
|
-| sink.properties.*| --| Stream Load
的导入参数。例如:指定列分隔符:`'sink.properties.column_separator' = ','`、
指定写入数据格式:`'sink.properties.format' = 'json'`
[更多参数详情](https://doris.apache.org/zh-CN/docs/dev/data-operate/import/import-way/stream-load-manual#%E5%88%9B%E5%BB%BA%E5%AF%BC%E5%85%A5)
|
+| sink.properties.format | csv | Stream Load
的数据格式。共支持3种格式：csv，json，arrow（1.4.0版本开始支持）
[更多参数详情](https://doris.apache.org/zh-CN/docs/dev/data-operate/import/import-way/stream-load-manual#%E5%88%9B%E5%BB%BA%E5%AF%BC%E5%85%A5)
|
Review Comment:
```suggestion
| sink.properties.format | csv | Stream Load
的数据格式。共支持3种格式：csv，json，arrow（2.1.0版本开始支持）
[更多参数详情](https://doris.apache.org/zh-CN/docs/dev/data-operate/import/import-way/stream-load-manual#%E5%88%9B%E5%BB%BA%E5%AF%BC%E5%85%A5)
|
```
##
docs/zh-CN/docs/ecosystem/spark-doris-connector.md:
##
@@ -266,7 +266,8 @@ kafkaSource.selectExpr("CAST(value as STRING)")
| doris.write.fields | --|
指定写入Doris表的字段或者字段顺序，多列之间使用逗号分隔。默认写入时要按照Doris表字段顺序写入全部字段。
|
| sink.batch.size | 1 | 单次写BE的最大行数
|
| sink.max-retries | 1 | 写BE失败之后的重试次数
|
-| sink.properties.*| --| Stream Load
的导入参数。例如:指定列分隔符:`'sink.properties.column_separator' = ','`、
指定写入数据格式:`'sink.properties.format' = 'json'`
[更多参数详情](https://doris.apache.org/zh-CN/docs/dev/data-operate/import/import-way/stream-load-manual#%E5%88%9B%E5%BB%BA%E5%AF%BC%E5%85%A5)
|
+| sink.properties.format | csv | Stream Load
的数据格式。共支持3种格式：csv，json，arrow（1.4.0版本开始支持）
[更多参数详情](https://doris.apache.org/zh-CN/docs/dev/data-operate/import/import-way/stream-load-manual#%E5%88%9B%E5%BB%BA%E5%AF%BC%E5%85%A5)
|
Review Comment:
```suggestion
| sink.properties.format | csv | Stream Load
的数据格式。共支持3种格式：csv，json，arrow（2.1.0版本开始支持）
[更多参数详情](https://doris.apache.org/zh-CN/docs/dev/data-operate/import/import-way/stream-load-manual#%E5%88%9B%E5%BB%BA%E5%AF%BC%E5%85%A5)
|
```
##
docs/zh-CN/docs/data-operate/import/import-way/stream-load-manual.md:
##
@@ -154,8 +154,11 @@ Stream Load 由于使用的是 HTTP 协议，所以所有导入任务有关的
- format
- 指定导入数据格式，支持 `csv` 和 `json`，默认是 `csv`
- 支持 `csv_with_names`
(csv文件行首过滤)、`csv_with_names_and_types`(csv文件前两行过滤)、`parquet`、`orc`
+ 指定导入数据格式，支持 `csv`、 `json` 和 `arrow` ，默认是 `csv`。
+
+ 支持 `csv_with_names`
(csv文件行首过滤)、`csv_with_names_and_types`(csv文件前两行过滤)、`parquet`、`orc`。
+
+ 支持 `arrow`格式。
Review Comment:
```suggestion
支持 `arrow`格式。
```
##
docs/zh-CN/docs/data-operate/import/import-way/stream-load-manual.md:
##
@@ -154,8 +154,11 @@ Stream Load 由于使用的是 HTTP 协议，所以所有导入任务有关的
- format
- 指定导入数据格式，支持 `csv` 和 `json`，默认是 `csv`
- 支持 `csv_with_names`
(csv文件行首过滤)、`csv_with_names_and_types`(csv文件前两行过滤)、`parquet`、`orc`
+ 指定导入数据格式，支持 `csv`、 `json` 和 `arrow` ，默认是 `csv`。
+
+ 支持 `csv_with_n
Re: [PR] [improve](env) Ensure next majority is met before drop an alive follower [doris]
w41ter commented on PR #28101: URL: https://github.com/apache/doris/pull/28101#issuecomment-1844917659 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default again [doris]
github-actions[bot] commented on PR #28105: URL: https://github.com/apache/doris/pull/28105#issuecomment-1844918250 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default again [doris]
github-actions[bot] commented on PR #28105: URL: https://github.com/apache/doris/pull/28105#issuecomment-1844918369 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](memory) Warning in log when turning on THP [doris]
doris-robot commented on PR #28122: URL: https://github.com/apache/doris/pull/28122#issuecomment-1844921736 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 44.66 seconds stream load tsv: 585 seconds loaded 74807831229 Bytes, about 121 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.0 seconds inserted 1000 Rows, about 344K ops/s storage size: 17194929759 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](statistics)Escape load stats sql. [doris]
doris-robot commented on PR #28117: URL: https://github.com/apache/doris/pull/28117#issuecomment-1844925983 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Tpch sf100 test result on commit f47898970011604b051f12a87e2628d6969e8e24, data reload: false run tpch-sf100 query with default conf and session variables q1 4698447344404440 q2 370 157 159 157 q3 1469126611971197 q4 1122899 855 855 q5 3132314231793142 q6 244 128 130 128 q7 999 504 489 489 q8 2198222421952195 q9 6687667467056674 q10 3217380132643264 q11 319 204 198 198 q12 355 212 216 212 q13 4615379138073791 q14 239 213 219 213 q15 565 516 520 516 q16 446 390 386 386 q17 1006630 629 629 q18 7490744569646964 q19 1522139914471399 q20 515 310 345 310 q21 3118262026802620 q22 357 296 296 296 Total cold run time: 44683 ms Total hot run time: 40075 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 4404440043934393 q2 266 164 172 164 q3 3531351135153511 q4 2381237923732373 q5 5737573557335733 q6 240 123 123 123 q7 2398188318501850 q8 3518351935263519 q9 9047897189558955 q10 3915399439853985 q11 500 400 377 377 q12 769 583 605 583 q13 4316357135623562 q14 284 256 263 256 q15 569 522 513 513 q16 504 456 459 456 q17 1883185218671852 q18 8576824782718247 q19 1745178017501750 q20 2249193819301930 q21 6543615461546154 q22 500 417 432 417 Total cold run time: 63875 ms Total hot run time: 60703 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Feature] (join) support sort merge join [doris]
hqx871 commented on issue #20126: URL: https://github.com/apache/doris/issues/20126#issuecomment-1844936837 Hi team, any update on this issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Variant) ColumnObject need to be finalized when doing ColumnObj… [doris]
doris-robot commented on PR #28119: URL: https://github.com/apache/doris/pull/28119#issuecomment-1844937027 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Tpch sf100 test result on commit 1a7e54bc0375608ff33d557a56b6e9ad4952130c, data reload: false run tpch-sf100 query with default conf and session variables q1 4685455344304430 q2 367 148 156 148 q3 1469124812371237 q4 1117935 910 910 q5 3155319531763176 q6 249 133 126 126 q7 1019503 490 490 q8 2233222321962196 q9 67086713 q10 3241325932353235 q11 321 215 206 206 q12 361 215 213 213 q13 4555381938143814 q14 248 214 221 214 q15 576 526 532 526 q16 451 390 393 390 q17 1018591 611 591 q18 8835731369966996 q19 1518188413821382 q20 548 851 376 376 q21 3150269026762676 q22 369 296 303 296 Total cold run time: 46193 ms Total hot run time: 40294 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 4425439443854385 q2 274 171 179 171 q3 3552355035263526 q4 2381236423632363 q5 5773575157605751 q6 240 122 123 122 q7 2375188618511851 q8 3521353535393535 q9 9107908090559055 q10 3896400839843984 q11 521 403 418 403 q12 766 593 588 588 q13 4294359035623562 q14 290 264 262 262 q15 579 526 521 521 q16 505 458 469 458 q17 1908184618641846 q18 8641826781248124 q19 1748178217711771 q20 2282196819321932 q21 6544618161696169 q22 505 420 438 420 Total cold run time: 64127 ms Total hot run time: 60799 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] Doris Roadmap 2023 [doris]
hqx871 commented on issue #16392: URL: https://github.com/apache/doris/issues/16392#issuecomment-1844940672 Hi team, any update about MergeSortJoin? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](variant) fallback to none partial update for mow table [doris]
doris-robot commented on PR #28116: URL: https://github.com/apache/doris/pull/28116#issuecomment-1844950055 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Tpch sf100 test result on commit 8d4ecb27778c0df472fc9d600092648416bec186, data reload: false run tpch-sf100 query with default conf and session variables q1 4747442644744426 q2 369 148 158 148 q3 1473123812191219 q4 1128961 928 928 q5 3161320031923192 q6 250 131 130 130 q7 1002502 491 491 q8 2219224021932193 q9 6680667666616661 q10 3219324532723245 q11 325 211 203 203 q12 356 221 217 217 q13 4536386738403840 q14 242 212 217 212 q15 562 527 519 519 q16 445 385 385 385 q17 1003594 580 580 q18 8275755173657365 q19 1514142014341420 q20 566 323 308 308 q21 3129271028062710 q22 362 287 298 287 Total cold run time: 45563 ms Total hot run time: 40679 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 4413436643884366 q2 267 164 169 164 q3 3533354035333533 q4 2374236923602360 q5 5732571957285719 q6 241 120 124 120 q7 2370186618891866 q8 3521351235193512 q9 9036902389788978 q10 3924399540093995 q11 511 381 376 376 q12 776 596 585 585 q13 4326357535433543 q14 281 261 252 252 q15 567 525 528 525 q16 502 484 457 457 q17 1865186618681866 q18 8627847782888288 q19 1716177217461746 q20 2272194719461946 q21 6522616361796163 q22 493 434 420 420 Total cold run time: 63869 ms Total hot run time: 60780 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
DarvenDuan commented on PR #27828: URL: https://github.com/apache/doris/pull/27828#issuecomment-1844952316 run pipelinex_p0 10 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](Nereids) let create table compatible with legacy planner [doris]
doris-robot commented on PR #28078: URL: https://github.com/apache/doris/pull/28078#issuecomment-1844953848 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 44.07 seconds stream load tsv: 584 seconds loaded 74807831229 Bytes, about 122 MB/s stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17194568543 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](jdbc) support call function to pass sql directly to jdbc catalog [doris]
github-actions[bot] commented on PR #26492: URL: https://github.com/apache/doris/pull/26492#issuecomment-1844956377 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](memory) Warning in log when turning on THP [doris]
doris-robot commented on PR #28122: URL: https://github.com/apache/doris/pull/28122#issuecomment-1844961393 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Tpch sf100 test result on commit 9f6861b7c0da81a36c0c3a16e310bc3322521b77, data reload: false run tpch-sf100 query with default conf and session variables q1 4694446444784464 q2 384 171 158 158 q3 1469120512411205 q4 1121928 923 923 q5 3195321032213210 q6 246 128 125 125 q7 1033500 487 487 q8 2217220822032203 q9 6684667966546654 q10 3228326932823269 q11 319 212 199 199 q12 360 226 215 215 q13 4581501243774377 q14 240 211 219 211 q15 571 519 522 519 q16 443 396 389 389 q17 1020582 599 582 q18 7392765670857085 q19 1528137814261378 q20 529 353 302 302 q21 3126269026432643 q22 361 285 298 285 Total cold run time: 44741 ms Total hot run time: 40883 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 4402437244024372 q2 267 164 172 164 q3 3551353035383530 q4 2373236523752365 q5 5737575757515751 q6 240 117 121 117 q7 2376186019181860 q8 3531353435283528 q9 9066901389808980 q10 3914400740174007 q11 498 383 390 383 q12 770 599 600 599 q13 4293356735593559 q14 281 265 240 240 q15 575 517 516 516 q16 527 464 477 464 q17 1876186918851869 q18 8685810283548102 q19 1729175317301730 q20 2259196219231923 q21 6530617961386138 q22 514 441 422 422 Total cold run time: 63994 ms Total hot run time: 60619 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [only test] support some operator local_shuffle [doris]
Mryange opened a new pull request, #28126: URL: https://github.com/apache/doris/pull/28126 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [only test]Pipeline x fragment test [doris]
Mryange closed pull request #28095: [only test]Pipeline x fragment test URL: https://github.com/apache/doris/pull/28095 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [only test] support some operator local_shuffle [doris]
Mryange commented on PR #28126: URL: https://github.com/apache/doris/pull/28126#issuecomment-1844965098 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [doc](statistics)Update external catalog statistics doc. [doris]
Jibing-Li commented on PR #28123: URL: https://github.com/apache/doris/pull/28123#issuecomment-1844968294 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](nereids) Support inner join query rewrite by materialized view [doris]
seawinde commented on PR #27922: URL: https://github.com/apache/doris/pull/27922#issuecomment-1844968605 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](env) Add disk usage in not ready msg [doris]
doris-robot commented on PR #28125: URL: https://github.com/apache/doris/pull/28125#issuecomment-1844969482 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 44.64 seconds stream load tsv: 588 seconds loaded 74807831229 Bytes, about 121 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.1 seconds inserted 1000 Rows, about 343K ops/s storage size: 17195025001 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](Nereids) let create table compatible with legacy planner [doris]
doris-robot commented on PR #28078: URL: https://github.com/apache/doris/pull/28078#issuecomment-1844970873 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Tpch sf100 test result on commit 3b7c7943af8ef8f5b1ba07520e37b7b434a06ff0, data reload: false run tpch-sf100 query with default conf and session variables q1 4719446444724464 q2 372 186 157 157 q3 1470122412701224 q4 1116916 933 916 q5 3183319231853185 q6 246 130 133 130 q7 1005487 489 487 q8 2225223821932193 q9 6692670966536653 q10 3213327032633263 q11 331 210 214 210 q12 354 216 211 211 q13 4533383237703770 q14 244 216 218 216 q15 568 516 518 516 q16 455 390 397 390 q17 1016637 561 561 q18 7364748473487348 q19 1540139414271394 q20 548 334 299 299 q21 3086268726852685 q22 354 292 301 292 Total cold run time: 44634 ms Total hot run time: 40564 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 4460444244294429 q2 266 165 170 165 q3 3537352135333521 q4 2395237323722372 q5 5723572557475725 q6 244 120 122 120 q7 2390188318661866 q8 3528353335293529 q9 9049900889648964 q10 3910399439853985 q11 502 372 399 372 q12 771 604 600 600 q13 4300359035623562 q14 290 246 252 246 q15 569 527 521 521 q16 501 444 491 444 q17 1890186518791865 q18 8681824193548241 q19 1738174917841749 q20 2263194919251925 q21 6524620461526152 q22 491 418 424 418 Total cold run time: 64022 ms Total hot run time: 60771 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression-test](Variant) add compaction case for variant and fix bugs [doris]
xiaokang commented on code in PR #28066:
URL: https://github.com/apache/doris/pull/28066#discussion_r1418620690
##
be/src/vec/functions/function_cast.h:
##
@@ -2013,8 +2013,16 @@ class FunctionCast final : public IFunctionBase {
if (!variant.is_finalized()) {
variant.assume_mutable()->finalize();
}
-
-if (variant.is_scalar_variant()) {
+// It's important to convert as many elements as possible in this
context. For instance,
+// if the root of this variant column is a number column,
converting it to a number column
+// is acceptable. However, if the destination type is a string and
root is none scalar root, then
+// we should convert the entire tree to a string.
+bool is_root_valuable =
+variant.is_scalar_variant() ||
+(!variant.is_null_root() &&
+
!WhichDataType(remove_nullable(variant.get_root_type())).is_nothing() &&
+ !WhichDataType(data_type_to).is_string());
Review Comment:
data_type_to is not string?
##
be/src/vec/json/path_in_data.cpp:
##
@@ -151,10 +151,17 @@ size_t PathInData::Hash::operator()(const PathInData&
value) const {
}
PathInData PathInData::pop_front() const {
+return pop_nfront(1);
+}
+
+PathInData PathInData::pop_nfront(size_t n) const {
+if (n >= parts.size()) {
+return {};
+}
PathInData new_path;
Parts new_parts;
if (!parts.empty()) {
-std::copy(parts.begin() + 1, parts.end(),
std::back_inserter(new_parts));
+std::copy(parts.begin() + n, parts.end(),
std::back_inserter(new_parts));
Review Comment:
Does 'pop_nfront' remove n front element using std::copy?
##
be/src/olap/rowset/segment_v2/hierarchical_data_reader.cpp:
##
@@ -32,11 +32,12 @@ namespace doris {
namespace segment_v2 {
Status HierarchicalDataReader::create(std::unique_ptr* reader,
+ vectorized::PathInData path,
const SubcolumnColumnReaders::Node* node,
const SubcolumnColumnReaders::Node* root,
bool output_as_raw_json) {
// None leave node need merge with root
-auto* stream_iter = new HierarchicalDataReader(node->path,
output_as_raw_json);
+auto* stream_iter = new HierarchicalDataReader(path, output_as_raw_json);
Review Comment:
what's the difference?
##
be/src/vec/json/path_in_data.cpp:
##
@@ -151,10 +151,17 @@ size_t PathInData::Hash::operator()(const PathInData&
value) const {
}
PathInData PathInData::pop_front() const {
+return pop_nfront(1);
+}
+
+PathInData PathInData::pop_nfront(size_t n) const {
Review Comment:
pop_nfront is only used once, why change it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [only test] support some operator local_shuffle [doris]
github-actions[bot] commented on code in PR #28126:
URL: https://github.com/apache/doris/pull/28126#discussion_r1418648304
##
be/src/pipeline/exec/sort_source_operator.h:
##
@@ -72,6 +72,7 @@ class SortSourceOperatorX final : public
OperatorX {
SourceState& source_state) override;
bool is_source() const override { return true; }
+ExchangeType get_local_exchange_type() const override { return
ExchangeType::PASSTHROUGH; }
Review Comment:
warning: method 'get_local_exchange_type' can be made static
[readability-convert-member-functions-to-static]
```suggestion
static ExchangeType get_local_exchange_type() override { return
ExchangeType::PASSTHROUGH; }
```
##
be/src/pipeline/exec/sort_sink_operator.h:
##
@@ -94,6 +94,8 @@ class SortSinkOperatorX final : public
DataSinkOperatorX {
Status sink(RuntimeState* state, vectorized::Block* in_block,
SourceState source_state) override;
+ExchangeType get_local_exchange_type() const override { return
ExchangeType::PASSTHROUGH; }
Review Comment:
warning: method 'get_local_exchange_type' can be made static
[readability-convert-member-functions-to-static]
```suggestion
static ExchangeType get_local_exchange_type() override { return
ExchangeType::PASSTHROUGH; }
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](env) Add disk usage in not ready msg [doris]
github-actions[bot] commented on PR #28125: URL: https://github.com/apache/doris/pull/28125#issuecomment-1844977021 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](env) Add disk usage in not ready msg [doris]
github-actions[bot] commented on PR #28125: URL: https://github.com/apache/doris/pull/28125#issuecomment-1844977095 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Chore](docs)Fix job error docs [doris]
CalvinKirs opened a new pull request, #28127: URL: https://github.com/apache/doris/pull/28127 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Chore](docs)Fix job error docs [doris]
CalvinKirs commented on PR #28127: URL: https://github.com/apache/doris/pull/28127#issuecomment-1844979245 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids)runtime filter translator failed on set operator [doris]
englefly commented on PR #28102: URL: https://github.com/apache/doris/pull/28102#issuecomment-1844979791 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhance](fe) Iceberg table in HMS catalog supports broker scan [doris]
WinkerDu commented on PR #28107: URL: https://github.com/apache/doris/pull/28107#issuecomment-1844980822 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](regression-test) load_to_single_tablet assertEquals usage [doris]
qidaye opened a new pull request, #28128: URL: https://github.com/apache/doris/pull/28128 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](regression-test) load_to_single_tablet assertEquals usage [doris]
qidaye commented on PR #28128: URL: https://github.com/apache/doris/pull/28128#issuecomment-1844981707 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](env) Ensure next majority is met before drop an alive follower [doris]
github-actions[bot] commented on PR #28101: URL: https://github.com/apache/doris/pull/28101#issuecomment-1844982972 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](env) Add disk usage in not ready msg [doris]
doris-robot commented on PR #28125: URL: https://github.com/apache/doris/pull/28125#issuecomment-1844982272 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Tpch sf100 test result on commit 970c561b1b70b3c1e50c263e0515275beb612370, data reload: false run tpch-sf100 query with default conf and session variables q1 4711446344344434 q2 362 144 142 142 q3 1450121612361216 q4 1099944 894 894 q5 3170322031503150 q6 251 131 127 127 q7 965 503 485 485 q8 2217223221952195 q9 6685671966966696 q10 3224326832913268 q11 326 210 195 195 q12 356 221 213 213 q13 4781398737823782 q14 245 221 225 221 q15 579 520 529 520 q16 454 390 400 390 q17 1016583 594 583 q18 7766732891087328 q19 1556139614531396 q20 554 597 304 304 q21 3111267226872672 q22 357 294 304 294 Total cold run time: 45235 ms Total hot run time: 40505 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 4433438843764376 q2 270 172 170 170 q3 3524352135303521 q4 2374238223772377 q5 5736574057395739 q6 242 120 121 120 q7 2415187618621862 q8 3542352735343527 q9 9037901890169016 q10 3919398140123981 q11 510 400 409 400 q12 765 599 585 585 q13 4323357935813579 q14 277 252 249 249 q15 579 514 515 514 q16 498 461 478 461 q17 1869186018631860 q18 8795827481328132 q19 1731174417801744 q20 2272195319421942 q21 6526620561736173 q22 516 419 424 419 Total cold run time: 64153 ms Total hot run time: 60747 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](env) Ensure next majority is met before drop an alive follower [doris]
github-actions[bot] commented on PR #28101: URL: https://github.com/apache/doris/pull/28101#issuecomment-1844983036 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) explain graph insert-select NPE [doris]
github-actions[bot] commented on PR #28007: URL: https://github.com/apache/doris/pull/28007#issuecomment-1844983511 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Error while running github feature from .asf.yaml in doris!
An error occurred while running github feature in .asf.yaml!: You can only have a maximum of 10 external triage collaborators, please contact vp-in...@apache.org to request an exception. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (9c63dfd692a -> bc12a059157)
This is an automated email from the ASF dual-hosted git repository.
englefly pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
from 9c63dfd692a [improve](move-memtable) support variant in move-memtable
(#28084)
add bc12a059157 [fix](Nereids) explain graph insert-select NPE (#28007)
No new revisions were added by this update.
Summary of changes:
.../doris/common/profile/PlanTreeBuilder.java | 14 --
.../explain/explain_graph.groovy} | 56 ++
2 files changed, 34 insertions(+), 36 deletions(-)
copy regression-test/suites/{nereids_rules_p0/runtime_filter/rf_prune.groovy
=> nereids_p0/explain/explain_graph.groovy} (69%)
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) explain graph insert-select NPE [doris]
englefly merged PR #28007: URL: https://github.com/apache/doris/pull/28007 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [improve](stack trace) avoid print stack trace in csv and json reader [doris]
HHoflittlefish777 opened a new pull request, #28129: URL: https://github.com/apache/doris/pull/28129 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) rewrite memtable memory limiter rules [doris]
dataroaring merged PR #27759: URL: https://github.com/apache/doris/pull/27759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (bc12a059157 -> 84a651d9761)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from bc12a059157 [fix](Nereids) explain graph insert-select NPE (#28007) add 84a651d9761 [improve](load) rewrite memtable memory limiter rules (#27759) No new revisions were added by this update. Summary of changes: be/src/common/config.cpp| 8 +- be/src/common/config.h | 6 + be/src/olap/memtable_memory_limiter.cpp | 263 +++- be/src/olap/memtable_memory_limiter.h | 23 +-- be/src/util/mem_info.cpp| 7 + 5 files changed, 152 insertions(+), 155 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) rewrite memtable memory limiter rules [doris]
github-actions[bot] commented on PR #27759: URL: https://github.com/apache/doris/pull/27759#issuecomment-1844986873 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression-test](Variant) Add more cases related to schema changes [doris]
eldenmoon commented on PR #27958: URL: https://github.com/apache/doris/pull/27958#issuecomment-1844985979 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Error while running github feature from .asf.yaml in doris!
An error occurred while running github feature in .asf.yaml!: You can only have a maximum of 10 external triage collaborators, please contact vp-in...@apache.org to request an exception. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [typo](docs) add doris complex type for flink-doris-connector [doris]
vinlee19 opened a new pull request, #28130: URL: https://github.com/apache/doris/pull/28130 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](stack trace) avoid print stack trace in csv and json reader [doris]
HHoflittlefish777 commented on PR #28129: URL: https://github.com/apache/doris/pull/28129#issuecomment-1844987643 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [typo](docs) add doris complex type for flink-doris-connector [doris]
vinlee19 commented on PR #28130: URL: https://github.com/apache/doris/pull/28130#issuecomment-1844987801 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](move-memtable) make StreamWait time configurable [doris]
dataroaring merged PR #28086: URL: https://github.com/apache/doris/pull/28086 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
