[GitHub] [doris] kaijchen commented on a diff in pull request #21746: [feature](partial update) add failure tolerance for strict mode partial update stream load
kaijchen commented on code in PR #21746:
URL: https://github.com/apache/doris/pull/21746#discussion_r1266317972
##
be/src/olap/delta_writer.h:
##
@@ -124,6 +124,10 @@ class DeltaWriter {
int64_t total_received_rows() const { return _total_received_rows; }
+void update_filtered_rows(int64_t rows) { _num_rows_filtered += rows; }
Review Comment:
This is unused, please remove
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21915: [Bug](storage) add lock on base tablet when create_tablet
github-actions[bot] commented on PR #21915: URL: https://github.com/apache/doris/pull/21915#issuecomment-1639616500 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HappenLee commented on pull request #21845: [opt](join) Opt the performance of join probe
HappenLee commented on PR #21845: URL: https://github.com/apache/doris/pull/21845#issuecomment-1639617191 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zclllyybb commented on pull request #21808: [fix](profile) Fix wrong instance number in query profile
zclllyybb commented on PR #21808: URL: https://github.com/apache/doris/pull/21808#issuecomment-1639623570 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21845: [opt](join) Opt the performance of join probe
github-actions[bot] commented on PR #21845: URL: https://github.com/apache/doris/pull/21845#issuecomment-1639625851 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] qzsee commented on pull request #21906: [Bug][Colocate] when adding a table to the colocate group, we should check that the number of buckets per partition is the same
qzsee commented on PR #21906: URL: https://github.com/apache/doris/pull/21906#issuecomment-1639635433 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zy-kkk merged pull request #21804: [branch-1.2-lts](cherry-pick) enable strong consistency by syncing max journal…
zy-kkk merged PR #21804: URL: https://github.com/apache/doris/pull/21804 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-lts updated: [branch-1.2-lts](cherry-pick) enable strong consistency by syncing max journal… (#21804)
This is an automated email from the ASF dual-hosted git repository.
zykkk pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new c6c2dc20e4 [branch-1.2-lts](cherry-pick) enable strong consistency by
syncing max journal… (#21804)
c6c2dc20e4 is described below
commit c6c2dc20e418f95f4c0ab0ed40de061ba71ed5e4
Author: Siyang Tang <82279870+tangsiyang2...@users.noreply.github.com>
AuthorDate: Tue Jul 18 15:13:52 2023 +0800
[branch-1.2-lts](cherry-pick) enable strong consistency by syncing max
journal… (#21804)
Add a session var & config enable_strong_consistency_read to solve the
problem that loading result may be shortly invisible to follwers, to meet users
requirements in strong consistency read scenario.
Will sync max journal id from master and wait for replaying.
---
docs/en/docs/advanced/variables.md | 4 +
docs/zh-CN/docs/advanced/variables.md | 5 +
.../java/org/apache/doris/qe/MasterOpExecutor.java | 124 -
.../java/org/apache/doris/qe/SessionVariable.java | 5 +
.../java/org/apache/doris/qe/StmtExecutor.java | 9 ++
.../apache/doris/service/FrontendServiceImpl.java | 5 +
gensrc/thrift/FrontendService.thrift | 1 +
7 files changed, 125 insertions(+), 28 deletions(-)
diff --git a/docs/en/docs/advanced/variables.md
b/docs/en/docs/advanced/variables.md
index 27769a6fae..7152fa49bb 100644
--- a/docs/en/docs/advanced/variables.md
+++ b/docs/en/docs/advanced/variables.md
@@ -647,6 +647,10 @@ Translated with www.DeepL.com/Translator (free version)
+--+
```
+* `enable_strong_consistency_read`
+
+ Used to enable strong consistent reading. By default, Doris supports strong
consistency within the same session, that is, changes to data within the same
session are visible in real time. If you want strong consistent reads between
sessions, set this variable to true.
+
***
Supplementary instructions on statement execution timeout control
diff --git a/docs/zh-CN/docs/advanced/variables.md
b/docs/zh-CN/docs/advanced/variables.md
index 939f69b6c2..2309389367 100644
--- a/docs/zh-CN/docs/advanced/variables.md
+++ b/docs/zh-CN/docs/advanced/variables.md
@@ -632,6 +632,11 @@ try (Connection conn =
DriverManager.getConnection("jdbc:mysql://127.0.0.1:9030/
| 1000 |
+--+
```
+
+* `enable_strong_consistency_read`
+
+ 用以开启强一致读。Doris
默认支持同一个会话内的强一致性，即同一个会话内对数据的变更操作是实时可见的。如需要会话间的强一致读，则需将此变量设置为true。
+
***
关于语句执行超时控制的补充说明
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/MasterOpExecutor.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/MasterOpExecutor.java
index 57e4fe67aa..b3ccdf3bbe 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/MasterOpExecutor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/MasterOpExecutor.java
@@ -19,6 +19,7 @@ package org.apache.doris.qe;
import org.apache.doris.analysis.RedirectStatus;
import org.apache.doris.common.ClientPool;
+import org.apache.doris.common.DdlException;
import org.apache.doris.common.telemetry.Telemetry;
import org.apache.doris.thrift.FrontendService;
import org.apache.doris.thrift.TMasterOpRequest;
@@ -26,11 +27,13 @@ import org.apache.doris.thrift.TMasterOpResult;
import org.apache.doris.thrift.TNetworkAddress;
import org.apache.doris.thrift.TUniqueId;
+import com.google.common.collect.ImmutableMap;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.context.Context;
import io.opentelemetry.context.Scope;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
+import org.apache.thrift.TException;
import org.apache.thrift.transport.TTransportException;
import java.nio.ByteBuffer;
@@ -63,24 +66,40 @@ public class MasterOpExecutor {
this.shouldNotRetry = !isQuery;
}
+/**
+ * used for simply syncing journal with master under strong consistency
mode
+ */
+public MasterOpExecutor(ConnectContext ctx) {
+this(null, ctx, RedirectStatus.FORWARD_WITH_SYNC, true);
+}
+
public void execute() throws Exception {
Span forwardSpan =
ctx.getTracer().spanBuilder("forward").setParent(Context.current())
.startSpan();
-try (Scope scope = forwardSpan.makeCurrent()) {
-forward();
+try (Scope ignored = forwardSpan.makeCurrent()) {
+result = forward(buildStmtForwardParams());
} catch (Exception e) {
forwardSpan.recordException(e);
throw e;
} finally {
forwardSpan.end();
}
+waitOnReplaying();
+}
+
+public void syncJournal() throws Exception {
+result = forward(buildSyncJournalParmas());
+waitOnReplaying();
+}
+
+private void waitOnReplaying() throws
[GitHub] [doris] kaijchen commented on a diff in pull request #21746: [feature](partial update) add failure tolerance for strict mode partial update stream load
kaijchen commented on code in PR #21746:
URL: https://github.com/apache/doris/pull/21746#discussion_r1266335203
##
be/src/olap/delta_writer.cpp:
##
@@ -398,6 +398,7 @@ Status DeltaWriter::close_wait(const PSlaveTabletNodes&
slave_tablet_nodes,
_mem_table.reset();
+_num_rows_filtered += _rowset_writer->num_rows_filtered();
Review Comment:
I think we don't need to accumulate this value again in DeltaWriter, because
there is only one rowset writer in one delta writer.
```cpp
int64_t DeltaWriter::num_rows_filtered() {
return _rowset_writer == nullptr ? 0 :
_rowset_writer->num_rows_filtered();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21746: [feature](partial update) add failure tolerance for strict mode partial update stream load
github-actions[bot] commented on PR #21746: URL: https://github.com/apache/doris/pull/21746#issuecomment-1639639061 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] kaijchen commented on a diff in pull request #21746: [feature](partial update) add failure tolerance for strict mode partial update stream load
kaijchen commented on code in PR #21746: URL: https://github.com/apache/doris/pull/21746#discussion_r1266337350 ## be/src/olap/rowset/segment_v2/segment_writer.cpp: ## @@ -408,6 +412,7 @@ Status SegmentWriter::append_block_with_partial_content(const vectorized::Block* auto mutable_full_columns = full_block.mutate_columns(); RETURN_IF_ERROR(fill_missing_columns(mutable_full_columns, use_default_or_null_flag, has_default_or_nullable)); + Review Comment: Please do not change unrelated lines (to avoid merge conflict) ```suggestion ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] CalvinKirs opened a new pull request, #21916: [WIP][Feature](Job)Support Event Scheduler
CalvinKirs opened a new pull request, #21916: URL: https://github.com/apache/doris/pull/21916 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21746: [feature](partial update) add failure tolerance for strict mode partial update stream load
github-actions[bot] commented on PR #21746: URL: https://github.com/apache/doris/pull/21746#issuecomment-1639658780 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 opened a new pull request, #21917: [fix](pipeline) Fix pipeline that cause plenty timeout of p0 cases
Kikyou1997 opened a new pull request, #21917: URL: https://github.com/apache/doris/pull/21917 ## Proposed changes Disable full auto analyze ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on pull request #21917: [fix](pipeline) Fix pipeline that cause plenty timeout of p0 cases
Kikyou1997 commented on PR #21917: URL: https://github.com/apache/doris/pull/21917#issuecomment-1639659499 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zddr commented on pull request #21910: [improvement](multi catalog)paimon support filesystem metastore
zddr commented on PR #21910: URL: https://github.com/apache/doris/pull/21910#issuecomment-1639667609 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] kaijchen commented on a diff in pull request #21746: [feature](partial update) add failure tolerance for strict mode partial update stream load
kaijchen commented on code in PR #21746:
URL: https://github.com/apache/doris/pull/21746#discussion_r1266353671
##
be/src/olap/delta_writer.h:
##
@@ -182,6 +186,8 @@ class DeltaWriter {
// total rows num written by DeltaWriter
int64_t _total_received_rows = 0;
+// number of rows filtered in strict mode partial update
+int64_t _num_rows_filtered = 0;
Review Comment:
this is unused
##
be/src/olap/delta_writer.h:
##
@@ -124,6 +124,10 @@ class DeltaWriter {
int64_t total_received_rows() const { return _total_received_rows; }
+int64_t DeltaWriter::num_rows_filtered() {
Review Comment:
This does not compile, please move it to `.cpp` file.
In header, it should just be
```cpp
int64_t num_rows_filtered() const;
```
In cpp file (add the const)
```cpp
int64_t DeltaWriter::num_rows_filtered() const {
return _rowset_writer == nullptr ? 0 :
_rowset_writer->num_rows_filtered();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 opened a new pull request, #21918: [doc](stats) Add doc for analyze
Kikyou1997 opened a new pull request, #21918: URL: https://github.com/apache/doris/pull/21918 ## Proposed changes Add description for some fe config about analyze ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on pull request #21918: [doc](stats) Add doc for analyze
Kikyou1997 commented on PR #21918: URL: https://github.com/apache/doris/pull/21918#issuecomment-1639672461 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21910: [improvement](multi catalog)paimon support filesystem metastore
github-actions[bot] commented on PR #21910: URL: https://github.com/apache/doris/pull/21910#issuecomment-1639678661 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21917: [fix](pipeline) Fix pipeline that cause plenty timeout of p0 cases
github-actions[bot] commented on PR #21917: URL: https://github.com/apache/doris/pull/21917#issuecomment-1639680286 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 opened a new pull request, #21919: [fix](nereids) Disable auto analyze temporarily
Kikyou1997 opened a new pull request, #21919: URL: https://github.com/apache/doris/pull/21919 ## Proposed changes It may cause plenty of p0 cases timeout ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on pull request #21919: [fix](nereids) Disable auto analyze temporarily
Kikyou1997 commented on PR #21919: URL: https://github.com/apache/doris/pull/21919#issuecomment-1639687322 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #21845: [opt](join) Opt the performance of join probe
hello-stephen commented on PR #21845: URL: https://github.com/apache/doris/pull/21845#issuecomment-1639687291 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 51.04 seconds stream load tsv: 536 seconds loaded 74807831229 Bytes, about 133 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17162489593 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21746: [feature](partial update) add failure tolerance for strict mode partial update stream load
github-actions[bot] commented on PR #21746: URL: https://github.com/apache/doris/pull/21746#issuecomment-1639693564 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] kaijchen commented on pull request #21746: [feature](partial update) add failure tolerance for strict mode partial update stream load
kaijchen commented on PR #21746: URL: https://github.com/apache/doris/pull/21746#issuecomment-1639694541 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #21727: [refactor](Nereids) refactor cte analyze, rewrite and reuse code
morrySnow commented on PR #21727: URL: https://github.com/apache/doris/pull/21727#issuecomment-1639713984 run compile feut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yujun777 commented on pull request #21856: [Improvement](tablet clone) impr tablet sched speed and fix tablet sched fail too many times
yujun777 commented on PR #21856: URL: https://github.com/apache/doris/pull/21856#issuecomment-1639723604 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #21755: [fix](stacktrace) Speed up stack trace
xinyiZzz commented on PR #21755: URL: https://github.com/apache/doris/pull/21755#issuecomment-1639729192 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wslkxup opened a new pull request, #21920: [doc]Fix parameter name `storage_root_path`
wslkxup opened a new pull request, #21920: URL: https://github.com/apache/doris/pull/21920 Inconsistent parameter names in the documentation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #21919: [fix](nereids) Disable auto analyze temporarily
hello-stephen commented on PR #21919: URL: https://github.com/apache/doris/pull/21919#issuecomment-1639738221 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 52.61 seconds stream load tsv: 508 seconds loaded 74807831229 Bytes, about 140 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.3 seconds inserted 1000 Rows, about 341K ops/s storage size: 17161765155 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zclllyybb commented on pull request #21305: [fix](datetime) refactor insert behavior to make same with mysql
zclllyybb commented on PR #21305: URL: https://github.com/apache/doris/pull/21305#issuecomment-1639739237 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xavierzhao opened a new issue, #21921: [Bug] Be down
xavierzhao opened a new issue, #21921: URL: https://github.com/apache/doris/issues/21921 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version  ### What's Wrong?   ### What You Expected? fixed this issue ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21755: [fix](stacktrace) Speed up stack trace
github-actions[bot] commented on PR #21755: URL: https://github.com/apache/doris/pull/21755#issuecomment-1639741393 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] bobhan1 commented on pull request #20563: [Enhancement](window-funnel)add different modes for window_funnel() function
bobhan1 commented on PR #20563: URL: https://github.com/apache/doris/pull/20563#issuecomment-1639750492 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] adonis0147 commented on pull request #21881: [fix](spark load) Hive table properties do not take effect when creating spark session
adonis0147 commented on PR #21881: URL: https://github.com/apache/doris/pull/21881#issuecomment-1639752470 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] imx7 closed issue #21832: [Bug] flink通过flink-connector-doris写入Map类型的数据时候，当K为Timestamp的类型时会写入失败
imx7 closed issue #21832: [Bug] flink通过flink-connector-doris写入Map类型的数据时候，当K为Timestamp的类型时会写入失败 URL: https://github.com/apache/doris/issues/21832 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhangstar333 opened a new pull request, #21922: [bug](test) fix regression test case failed with curdate
zhangstar333 opened a new pull request, #21922: URL: https://github.com/apache/doris/pull/21922 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt opened a new pull request, #21923: [Bug](excution) fix ScannerContext is done make query failed
BiteThet opened a new pull request, #21923: URL: https://github.com/apache/doris/pull/21923 ## Proposed changes fix ScannerContext is done make query failed ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #21923: [Bug](excution) fix ScannerContext is done make query failed
BiteThet commented on PR #21923: URL: https://github.com/apache/doris/pull/21923#issuecomment-1639795632 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21922: [bug](test) fix regression test case failed with curdate
github-actions[bot] commented on PR #21922: URL: https://github.com/apache/doris/pull/21922#issuecomment-1639798536 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21922: [bug](test) fix regression test case failed with curdate
github-actions[bot] commented on PR #21922: URL: https://github.com/apache/doris/pull/21922#issuecomment-1639798604 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21923: [Bug](excution) fix ScannerContext is done make query failed
github-actions[bot] commented on PR #21923: URL: https://github.com/apache/doris/pull/21923#issuecomment-1639799228 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21923: [Bug](excution) fix ScannerContext is done make query failed
github-actions[bot] commented on PR #21923: URL: https://github.com/apache/doris/pull/21923#issuecomment-1639799308 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21923: [Bug](excution) fix ScannerContext is done make query failed
github-actions[bot] commented on PR #21923: URL: https://github.com/apache/doris/pull/21923#issuecomment-1639806862 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #21881: [fix](spark load) Hive table properties do not take effect when creating spark session
hello-stephen commented on PR #21881: URL: https://github.com/apache/doris/pull/21881#issuecomment-1639811676 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 51.43 seconds stream load tsv: 509 seconds loaded 74807831229 Bytes, about 140 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.1 seconds inserted 1000 Rows, about 343K ops/s storage size: 17169101010 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] czzmmc commented on issue #18904: [Feature] Support batch operation for Java UDAF to get better performance
czzmmc commented on issue #18904: URL: https://github.com/apache/doris/issues/18904#issuecomment-1639814085 > now, doris java udaf implement is batch call, like: for(int i=0;i
[GitHub] [doris] starocean999 commented on pull request #21827: [fix](nereids)ExtractAndNormalizeWindowExpression rule should push down correct exprs to child
starocean999 commented on PR #21827: URL: https://github.com/apache/doris/pull/21827#issuecomment-1639818874 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname opened a new pull request, #21924: [Fix](Nereids)fix insert into default value exception
sohardforaname opened a new pull request, #21924: URL: https://github.com/apache/doris/pull/21924 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #21924: [Fix](Nereids)fix insert into default value exception
sohardforaname commented on PR #21924: URL: https://github.com/apache/doris/pull/21924#issuecomment-1639824795 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #21922: [bug](test) fix regression test case failed with curdate
BiteThet commented on PR #21922: URL: https://github.com/apache/doris/pull/21922#issuecomment-1639829320 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #21925: test
Gabriel39 opened a new pull request, #21925: URL: https://github.com/apache/doris/pull/21925 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21925: test
github-actions[bot] commented on PR #21925: URL: https://github.com/apache/doris/pull/21925#issuecomment-1639842918 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #21727: [refactor](Nereids) refactor cte analyze, rewrite and reuse code
morrySnow commented on PR #21727: URL: https://github.com/apache/doris/pull/21727#issuecomment-1639849708 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21902: [stats](nereids) dump col stats for all physical plan node and cost details in memo
github-actions[bot] commented on PR #21902: URL: https://github.com/apache/doris/pull/21902#issuecomment-1639865432 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21902: [stats](nereids) dump col stats for all physical plan node and cost details in memo
github-actions[bot] commented on PR #21902: URL: https://github.com/apache/doris/pull/21902#issuecomment-1639865495 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen opened a new pull request, #21926: [ckb](debug) debug ckb, base be750e88b26153a578462385072536f9191dc753
hello-stephen opened a new pull request, #21926: URL: https://github.com/apache/doris/pull/21926 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #21926: [ckb](debug) debug ckb, base be750e88b26153a578462385072536f9191dc753
hello-stephen commented on PR #21926: URL: https://github.com/apache/doris/pull/21926#issuecomment-1639894856 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen opened a new pull request, #21927: [ckb](debug) debug ckb, base 05cf0955063852ccba24219e6ca6d1c360dff025
hello-stephen opened a new pull request, #21927: URL: https://github.com/apache/doris/pull/21927 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #21927: [ckb](debug) debug ckb, base 05cf0955063852ccba24219e6ca6d1c360dff025
hello-stephen commented on PR #21927: URL: https://github.com/apache/doris/pull/21927#issuecomment-1639898452 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21925: test
github-actions[bot] commented on PR #21925: URL: https://github.com/apache/doris/pull/21925#issuecomment-1639901065 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21926: [ckb](debug) debug ckb, base be750e88b26153a578462385072536f9191dc753
github-actions[bot] commented on PR #21926: URL: https://github.com/apache/doris/pull/21926#issuecomment-1639904468 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] pingchunzhang commented on pull request #21917: [fix](pipeline) Fix pipeline that cause plenty timeout of p0 cases
pingchunzhang commented on PR #21917: URL: https://github.com/apache/doris/pull/21917#issuecomment-1639906306 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Yukang-Lian commented on issue #20131: [Enhancement] support create or replace view
Yukang-Lian commented on issue #20131: URL: https://github.com/apache/doris/issues/20131#issuecomment-1639906895 cc @yiguolei -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21927: [ckb](debug) debug ckb, base 05cf0955063852ccba24219e6ca6d1c360dff025
github-actions[bot] commented on PR #21927: URL: https://github.com/apache/doris/pull/21927#issuecomment-1639908642 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21917: [fix](pipeline) Fix pipeline that cause plenty timeout of p0 cases
github-actions[bot] commented on PR #21917: URL: https://github.com/apache/doris/pull/21917#issuecomment-1639911717 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname opened a new pull request, #21929: [Fix](regression-test)fix nereids_p0/javaudf and nereids_p0/outfile cases.
sohardforaname opened a new pull request, #21929: URL: https://github.com/apache/doris/pull/21929 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt merged pull request #21923: [Bug](excution) fix ScannerContext is done make query failed
BiteThet merged PR #21923: URL: https://github.com/apache/doris/pull/21923 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Bug](excution) fix ScannerContext is done make query failed (#21923)
This is an automated email from the ASF dual-hosted git repository.
panxiaolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 3089e4b3b6 [Bug](excution) fix ScannerContext is done make query
failed (#21923)
3089e4b3b6 is described below
commit 3089e4b3b6bb236dbf0fb596865357189134daaa
Author: Pxl
AuthorDate: Tue Jul 18 17:58:00 2023 +0800
[Bug](excution) fix ScannerContext is done make query failed (#21923)
fix ScannerContext is done make query failed
---
be/src/vec/exec/scan/pip_scanner_context.h | 2 +-
be/src/vec/exec/scan/scanner_context.cpp | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/be/src/vec/exec/scan/pip_scanner_context.h
b/be/src/vec/exec/scan/pip_scanner_context.h
index c720c22d04..b71b72ab3c 100644
--- a/be/src/vec/exec/scan/pip_scanner_context.h
+++ b/be/src/vec/exec/scan/pip_scanner_context.h
@@ -48,7 +48,7 @@ public:
set_status_on_error(Status::Cancelled("cancelled"), false);
}
-if (!_process_status.ok()) {
+if (!status().ok()) {
return _process_status;
}
}
diff --git a/be/src/vec/exec/scan/scanner_context.cpp
b/be/src/vec/exec/scan/scanner_context.cpp
index 049283647c..2f535e4947 100644
--- a/be/src/vec/exec/scan/scanner_context.cpp
+++ b/be/src/vec/exec/scan/scanner_context.cpp
@@ -199,7 +199,7 @@ Status ScannerContext::get_block_from_queue(RuntimeState*
state, vectorized::Blo
// Wait for block from queue
if (wait) {
SCOPED_TIMER(_scanner_wait_batch_timer);
-while (!(!_blocks_queue.empty() || _is_finished ||
!_process_status.ok() ||
+while (!(!_blocks_queue.empty() || _is_finished || !status().ok() ||
state->is_cancelled())) {
_blocks_queue_added_cv.wait(l);
}
@@ -231,7 +231,7 @@ bool ScannerContext::set_status_on_error(const Status&
status, bool need_lock) {
if (need_lock) {
l.lock();
}
-if (_process_status.ok()) {
+if (this->status().ok()) {
_process_status = status;
_status_error = true;
_blocks_queue_added_cv.notify_one();
@@ -327,7 +327,7 @@ std::string ScannerContext::debug_string() {
" status: {}, _should_stop: {}, _is_finished: {}, free blocks: {},"
" limit: {}, _num_running_scanners: {}, _num_scheduling_ctx: {},
_max_thread_num: {},"
" _block_per_scanner: {}, _cur_bytes_in_queue: {},
MAX_BYTE_OF_QUEUE: {}",
-ctx_id, _scanners.size(), _blocks_queue.size(),
_process_status.ok(), _should_stop,
+ctx_id, _scanners.size(), _blocks_queue.size(), status().ok(),
_should_stop,
_is_finished, _free_blocks.size_approx(), limit,
_num_running_scanners,
_num_scheduling_ctx, _max_thread_num, _block_per_scanner,
_cur_bytes_in_queue,
_max_bytes_in_queue);
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] ixzc opened a new pull request, #21930: [typo](docs) Replace invalid mysql-connector-java download package.
ixzc opened a new pull request, #21930: URL: https://github.com/apache/doris/pull/21930 ## Proposed changes Issue Number: close #xxx Replace invalid mysql-connector-java download package. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #21915: [Bug](storage) add lock on base tablet when create_tablet
BiteThet commented on PR #21915: URL: https://github.com/apache/doris/pull/21915#issuecomment-1639917528 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yujun777 commented on pull request #21856: [Improvement](tablet clone) impr tablet sched speed and fix tablet sched fail too many times
yujun777 commented on PR #21856: URL: https://github.com/apache/doris/pull/21856#issuecomment-1639920926 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei commented on pull request #21899: [fix](two level hash table) fix dead loop when converting to two level hash table for zero value
yiguolei commented on PR #21899: URL: https://github.com/apache/doris/pull/21899#issuecomment-1639926223 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xzj7019 commented on pull request #21883: [enhance](nereids) enhance runtime filter pushdown
xzj7019 commented on PR #21883: URL: https://github.com/apache/doris/pull/21883#issuecomment-1639926823 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HappenLee commented on pull request #21845: [opt](join) Opt the performance of join probe
HappenLee commented on PR #21845: URL: https://github.com/apache/doris/pull/21845#issuecomment-1639928089 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhannngchen commented on pull request #21488: [enhancement](merge-on-write) support concurrent delete bitmap calc while close_wait
zhannngchen commented on PR #21488: URL: https://github.com/apache/doris/pull/21488#issuecomment-1639928417 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21915: [Bug](storage) add lock on base tablet when create_tablet
github-actions[bot] commented on PR #21915: URL: https://github.com/apache/doris/pull/21915#issuecomment-1639929131 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt merged pull request #21878: [feature](profile) add process hashtable time in join node
BiteThet merged PR #21878: URL: https://github.com/apache/doris/pull/21878 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [feature](profile) add process hashtable time in join node (#21878)
This is an automated email from the ASF dual-hosted git repository.
panxiaolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new c36d225a27 [feature](profile) add process hashtable time in join node
(#21878)
c36d225a27 is described below
commit c36d225a27a0954fb0193381a77b8defa7bf4aa2
Author: Mryange <59914473+mrya...@users.noreply.github.com>
AuthorDate: Tue Jul 18 18:09:42 2023 +0800
[feature](profile) add process hashtable time in join node (#21878)
add process hashtable time in join node
---
be/src/vec/exec/join/process_hash_table_probe.h | 2 +-
be/src/vec/exec/join/process_hash_table_probe_impl.h | 4 +++-
be/src/vec/exec/join/vhash_join_node.cpp | 2 ++
be/src/vec/exec/join/vhash_join_node.h | 1 +
4 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/be/src/vec/exec/join/process_hash_table_probe.h
b/be/src/vec/exec/join/process_hash_table_probe.h
index 385e3c82cd..db43daf20d 100644
--- a/be/src/vec/exec/join/process_hash_table_probe.h
+++ b/be/src/vec/exec/join/process_hash_table_probe.h
@@ -105,7 +105,7 @@ struct ProcessHashTableProbe {
RuntimeProfile::Counter* _search_hashtable_timer;
RuntimeProfile::Counter* _build_side_output_timer;
RuntimeProfile::Counter* _probe_side_output_timer;
-
+RuntimeProfile::Counter* _probe_process_hashtable_timer;
static constexpr int PROBE_SIDE_EXPLODE_RATE = 3;
};
diff --git a/be/src/vec/exec/join/process_hash_table_probe_impl.h
b/be/src/vec/exec/join/process_hash_table_probe_impl.h
index 14181b160a..4e9180bed9 100644
--- a/be/src/vec/exec/join/process_hash_table_probe_impl.h
+++ b/be/src/vec/exec/join/process_hash_table_probe_impl.h
@@ -47,7 +47,8 @@
ProcessHashTableProbe::ProcessHashTableProbe(HashJoinNode* join_node
_rows_returned_counter(join_node->_rows_returned_counter),
_search_hashtable_timer(join_node->_search_hashtable_timer),
_build_side_output_timer(join_node->_build_side_output_timer),
- _probe_side_output_timer(join_node->_probe_side_output_timer) {}
+ _probe_side_output_timer(join_node->_probe_side_output_timer),
+
_probe_process_hashtable_timer(join_node->_probe_process_hashtable_timer) {}
template
template
@@ -1070,6 +1071,7 @@ Status
ProcessHashTableProbe::process_data_in_hashtable(HashTableTyp
Block*
output_block,
bool* eos)
{
using Mapped = typename HashTableType::Mapped;
+SCOPED_TIMER(_probe_process_hashtable_timer);
if constexpr (std::is_same_v ||
std::is_same_v) {
hash_table_ctx.init_once();
diff --git a/be/src/vec/exec/join/vhash_join_node.cpp
b/be/src/vec/exec/join/vhash_join_node.cpp
index 81e71881a2..c048c4cc2f 100644
--- a/be/src/vec/exec/join/vhash_join_node.cpp
+++ b/be/src/vec/exec/join/vhash_join_node.cpp
@@ -471,6 +471,8 @@ Status HashJoinNode::prepare(RuntimeState* state) {
ADD_CHILD_TIMER(probe_phase_profile,
"ProbeWhenBuildSideOutputTime", "ProbeTime");
_probe_side_output_timer =
ADD_CHILD_TIMER(probe_phase_profile,
"ProbeWhenProbeSideOutputTime", "ProbeTime");
+_probe_process_hashtable_timer =
+ADD_CHILD_TIMER(probe_phase_profile,
"ProbeWhenProcessHashTableTime", "ProbeTime");
_open_timer = ADD_TIMER(runtime_profile(), "OpenTime");
_allocate_resource_timer = ADD_TIMER(runtime_profile(),
"AllocateResourceTime");
_process_other_join_conjunct_timer = ADD_TIMER(runtime_profile(),
"OtherJoinConjunctTime");
diff --git a/be/src/vec/exec/join/vhash_join_node.h
b/be/src/vec/exec/join/vhash_join_node.h
index 3d85d6b227..096a9148cc 100644
--- a/be/src/vec/exec/join/vhash_join_node.h
+++ b/be/src/vec/exec/join/vhash_join_node.h
@@ -304,6 +304,7 @@ private:
RuntimeProfile::Counter* _search_hashtable_timer;
RuntimeProfile::Counter* _build_side_output_timer;
RuntimeProfile::Counter* _probe_side_output_timer;
+RuntimeProfile::Counter* _probe_process_hashtable_timer;
RuntimeProfile::Counter* _build_side_compute_hash_timer;
RuntimeProfile::Counter* _build_side_merge_block_timer;
RuntimeProfile::Counter* _build_runtime_filter_timer;
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21907: [bugfix](deserialize ) pack struct to avoid parse wrong content for file header
github-actions[bot] commented on PR #21907: URL: https://github.com/apache/doris/pull/21907#issuecomment-1639939704 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #21926: [ckb](debug) debug ckb, base be750e88b26153a578462385072536f9191dc753
hello-stephen commented on PR #21926: URL: https://github.com/apache/doris/pull/21926#issuecomment-1639939753 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 51.75 seconds stream load tsv: 509 seconds loaded 74807831229 Bytes, about 140 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17161119359 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21907: [bugfix](deserialize ) pack struct to avoid parse wrong content for file header
github-actions[bot] commented on PR #21907: URL: https://github.com/apache/doris/pull/21907#issuecomment-1639939785 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] catpineapple opened a new pull request, #21931: [feature-wip](dbt) exchange table temp to target table atomically
catpineapple opened a new pull request, #21931: URL: https://github.com/apache/doris/pull/21931 ## Proposed changes Issue Number: close #xxx exchange table temp to target table atomically ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] catpineapple commented on pull request #21931: [feature-wip](dbt) exchange table temp to target table atomically
catpineapple commented on PR #21931: URL: https://github.com/apache/doris/pull/21931#issuecomment-1639940266 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21845: [opt](join) Opt the performance of join probe
github-actions[bot] commented on PR #21845: URL: https://github.com/apache/doris/pull/21845#issuecomment-1639941164 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #21895: [Fix](multi catalog)Fix hive partition value contains special character bug.
yiguolei merged PR #21895: URL: https://github.com/apache/doris/pull/21895 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-lts updated: [Fix](multi catalog)Fix hive partition value contains special character bug. #21895
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new 89bc972d4c [Fix](multi catalog)Fix hive partition value contains
special character bug. #21895
89bc972d4c is described below
commit 89bc972d4c5f2880791b4cd50ef66ab3d3023231
Author: Jibing-Li <64681310+jibing...@users.noreply.github.com>
AuthorDate: Tue Jul 18 18:14:56 2023 +0800
[Fix](multi catalog)Fix hive partition value contains special character
bug. #21895
Hive partition value may contain special characters, such as '/'. This may
cause the partition value splitter to get wrong partition values because it use
'/' to split partition value. So we need to split the encoded partition name
and then decode it to the original value.
---
.../doris/datasource/hive/HiveMetaStoreCache.java | 14 --
.../hive/test_hive_special_char_partition.out | 51 ++
.../hive/test_hive_special_char_partition.groovy | 51 ++
3 files changed, 113 insertions(+), 3 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetaStoreCache.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetaStoreCache.java
index 169ed3727b..06a72c112b 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetaStoreCache.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetaStoreCache.java
@@ -179,8 +179,9 @@ public class HiveMetaStoreCache {
Map> idToUniqueIdsMap =
Maps.newHashMapWithExpectedSize(partitionNames.size());
long idx = 0;
for (String partitionName : partitionNames) {
+String decodedPartitionName;
try {
-partitionName = URLDecoder.decode(partitionName,
StandardCharsets.UTF_8.name());
+decodedPartitionName = URLDecoder.decode(partitionName,
StandardCharsets.UTF_8.name());
} catch (UnsupportedEncodingException e) {
// It should not be here
throw new RuntimeException(e);
@@ -188,7 +189,7 @@ public class HiveMetaStoreCache {
long partitionId = idx++;
ListPartitionItem listPartitionItem =
toListPartitionItem(partitionName, key.types);
idToPartitionItem.put(partitionId, listPartitionItem);
-partitionNameToIdMap.put(partitionName, partitionId);
+partitionNameToIdMap.put(decodedPartitionName, partitionId);
}
Map> uidToPartitionRange = null;
@@ -219,7 +220,14 @@ public class HiveMetaStoreCache {
for (String part : parts) {
String[] kv = part.split("=");
Preconditions.checkState(kv.length == 2, partitionName);
-values.add(new PartitionValue(kv[1],
HIVE_DEFAULT_PARTITION.equals(kv[1])));
+String decodedValue = null;
+try {
+decodedValue = URLDecoder.decode(kv[1],
StandardCharsets.UTF_8.name());
+} catch (UnsupportedEncodingException e) {
+// It should not be here
+throw new RuntimeException(e);
+}
+values.add(new PartitionValue(decodedValue,
HIVE_DEFAULT_PARTITION.equals(decodedValue)));
}
try {
PartitionKey key =
PartitionKey.createListPartitionKeyWithTypes(values, types);
diff --git
a/regression-test/data/external_table_emr_p2/hive/test_hive_special_char_partition.out
b/regression-test/data/external_table_emr_p2/hive/test_hive_special_char_partition.out
new file mode 100644
index 00..0bd26b1276
--- /dev/null
+++
b/regression-test/data/external_table_emr_p2/hive/test_hive_special_char_partition.out
@@ -0,0 +1,51 @@
+-- This file is automatically generated. You should know what you did if you
want to edit this
+-- !1 --
+name# 2023#01#01
+name1 2023/01/01
+name10 2023<01><01>
+name11 2023\\01\\01
+name12 2023.01.01
+name2 2023 01 01
+name3 2023:01:01
+name4 2023?01?01
+name5 2023=01=01
+name6 2023%01%01
+name8 2023"01"01
+name9 2023'01'01
+
+-- !2 --
+name2
+
+-- !3 --
+name1
+
+-- !4 --
+name4 2023?01?01
+
+-- !5 --
+name12 2023.01.01
+
+-- !6 --
+name10 2023<01><01>
+
+-- !7 --
+name3 2023:01:01
+
+-- !8 --
+name5 2023=01=01
+
+-- !9 --
+name8 2023"01"01
+
+-- !10 --
+name9 2023'01'01
+
+-- !11 --
+name11 2023\\01\\01
+
+-- !12 --
+name6 2023%01%01
+
+-- !13 --
+name# 2023#01#01
+
diff --git
a/regression-test/suites/external_table_emr_p2/hive/test_hive_special_char_partition.groovy
b/regression-test/suites/external_table_emr_p2/hive/test_hive_special_char_partition.groovy
new file mode 100644
index 00..cb862469f6
--- /dev/null
+++
b/regression-test/suites/external_table_emr_p2/hive/test_hive_special_char_partition.groovy
@@ -0,0 +1,51 @@
+// Licensed to the Apache Software Fou
[GitHub] [doris] caiconghui opened a new pull request, #21932: [fix](status) fix cmake warning with message 'Manually-specified variables were not used by the project:ENABLE_STACKTRACE'
caiconghui opened a new pull request, #21932: URL: https://github.com/apache/doris/pull/21932 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring merged pull request #21635: [refactor](load) cleanup segment flush logic in beta rowset writer
dataroaring merged PR #21635: URL: https://github.com/apache/doris/pull/21635 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] qzsee commented on pull request #21906: [Bug][Colocate] when adding a table to the colocate group, we should check that the number of buckets per partition is the same
qzsee commented on PR #21906: URL: https://github.com/apache/doris/pull/21906#issuecomment-1639946218 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [refactor](load) cleanup segment flush logic in beta rowset writer (#21635)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 2013dcd0e9 [refactor](load) cleanup segment flush logic in beta rowset
writer (#21635)
2013dcd0e9 is described below
commit 2013dcd0e96fb79493884dc933e9f2e6f941986d
Author: Kaijie Chen
AuthorDate: Tue Jul 18 18:17:57 2023 +0800
[refactor](load) cleanup segment flush logic in beta rowset writer (#21635)
---
be/src/olap/memtable_flush_executor.cpp | 7 +-
be/src/olap/rowset/beta_rowset_writer.cpp | 222 +-
be/src/olap/rowset/beta_rowset_writer.h | 47 ---
be/src/olap/rowset/rowset_writer.h| 18 +--
be/src/olap/rowset/segcompaction.cpp | 2 +-
be/src/olap/tablet.cpp| 3 +-
6 files changed, 132 insertions(+), 167 deletions(-)
diff --git a/be/src/olap/memtable_flush_executor.cpp
b/be/src/olap/memtable_flush_executor.cpp
index cf478393af..70cd74e214 100644
--- a/be/src/olap/memtable_flush_executor.cpp
+++ b/be/src/olap/memtable_flush_executor.cpp
@@ -99,8 +99,11 @@ Status FlushToken::_do_flush_memtable(MemTable* memtable,
int32_t segment_id, in
int64_t duration_ns;
SCOPED_RAW_TIMER(&duration_ns);
std::unique_ptr block = memtable->to_block();
-
SKIP_MEMORY_CHECK(RETURN_IF_ERROR(_rowset_writer->unfold_variant_column_and_flush_block(
-block.get(), segment_id, memtable->flush_mem_tracker(),
flush_size)));
+{
+SCOPED_CONSUME_MEM_TRACKER(memtable->flush_mem_tracker());
+SKIP_MEMORY_CHECK(RETURN_IF_ERROR(
+_rowset_writer->flush_memtable(block.get(), segment_id,
flush_size)));
+}
_memtable_stat += memtable->stat();
DorisMetrics::instance()->memtable_flush_total->increment(1);
DorisMetrics::instance()->memtable_flush_duration_us->increment(duration_ns /
1000);
diff --git a/be/src/olap/rowset/beta_rowset_writer.cpp
b/be/src/olap/rowset/beta_rowset_writer.cpp
index d99c8e9ee4..4aa7815423 100644
--- a/be/src/olap/rowset/beta_rowset_writer.cpp
+++ b/be/src/olap/rowset/beta_rowset_writer.cpp
@@ -89,10 +89,10 @@ BetaRowsetWriter::~BetaRowsetWriter() {
if (!fs) {
return;
}
-auto max_segment_id = std::max(_num_segment.load(),
_next_segment_id.load());
-for (int i = 0; i < max_segment_id; ++i) {
-std::string seg_path = BetaRowset::segment_file_path(
-_context.rowset_dir, _context.rowset_id, _segment_start_id
+ i);
+DCHECK_LE(_segment_start_id + _num_segment, _next_segment_id);
+for (int i = _segment_start_id; i < _next_segment_id; ++i) {
+std::string seg_path =
+BetaRowset::segment_file_path(_context.rowset_dir,
_context.rowset_id, i);
// Even if an error is encountered, these files that have not been
cleaned up
// will be cleaned up by the GC background. So here we only print
the error
// message when we encounter an error.
@@ -134,11 +134,9 @@ Status BetaRowsetWriter::add_block(const
vectorized::Block* block) {
return Status::OK();
}
if (UNLIKELY(_segment_writer == nullptr)) {
-FlushContext ctx;
-ctx.block = block;
-RETURN_IF_ERROR(_create_segment_writer(&_segment_writer, &ctx));
+RETURN_IF_ERROR(_create_segment_writer(_segment_writer,
allocate_segment_id()));
}
-return _add_block(block, &_segment_writer);
+return _add_block(block, _segment_writer);
}
Status BetaRowsetWriter::_generate_delete_bitmap(int32_t segment_id) {
@@ -446,46 +444,38 @@ Status
BetaRowsetWriter::_segcompaction_ramaining_if_necessary() {
return status;
}
-Status BetaRowsetWriter::_do_add_block(const vectorized::Block* block,
-
std::unique_ptr* segment_writer,
- size_t row_offset, size_t
input_row_num) {
-auto s = (*segment_writer)->append_block(block, row_offset, input_row_num);
+Status BetaRowsetWriter::_add_rows(const vectorized::Block* block,
+ std::unique_ptr&
segment_writer,
+ size_t row_offset, size_t input_row_num) {
+auto s = segment_writer->append_block(block, row_offset, input_row_num);
if (UNLIKELY(!s.ok())) {
return Status::Error("failed to append block:
{}", s.to_string());
}
+_raw_num_rows_written += input_row_num;
return Status::OK();
}
Status BetaRowsetWriter::_add_block(const vectorized::Block* block,
-
std::unique_ptr* segment_writer,
-const FlushContext* flush_ctx) {
+
std::unique_ptr& segment_writer) {
size_t block_size_in_bytes = block->bytes();
size_t block_row_nu
[doris] branch master updated (2013dcd0e9 -> d6d27ef428)
This is an automated email from the ASF dual-hosted git repository. morrysnow pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 2013dcd0e9 [refactor](load) cleanup segment flush logic in beta rowset writer (#21635) add d6d27ef428 [fix](Nereids) join other conjuncts should get slot from join output (#21840) No new revisions were added by this update. Summary of changes: .../nereids/rules/rewrite/AdjustNullable.java | 3 + .../join/test_other_conjuncts_nullable.groovy | 64 ++ 2 files changed, 67 insertions(+) create mode 100644 regression-test/suites/nereids_p0/join/test_other_conjuncts_nullable.groovy - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow merged pull request #21840: [fix](Nereids) join other conjuncts should get slot from join output
morrySnow merged PR #21840: URL: https://github.com/apache/doris/pull/21840 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] kaijchen commented on pull request #21725: [refactor](load) split segment flusher from rowset writer
kaijchen commented on PR #21725: URL: https://github.com/apache/doris/pull/21725#issuecomment-1639953587 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21932: [fix](status) fix cmake warning with message 'Manually-specified variables were not used by the project:ENABLE_STACKTRACE'
github-actions[bot] commented on PR #21932: URL: https://github.com/apache/doris/pull/21932#issuecomment-1639955214 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] bobhan1 opened a new pull request, #21933: [feature](delete) support delete on value column for unique table
bobhan1 opened a new pull request, #21933: URL: https://github.com/apache/doris/pull/21933 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21932: [fix](status) fix cmake warning with message 'Manually-specified variables were not used by the project:ENABLE_STACKTRACE'
github-actions[bot] commented on PR #21932: URL: https://github.com/apache/doris/pull/21932#issuecomment-1639955261 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #21932: [fix](status) fix cmake warning with message 'Manually-specified variables were not used by the project:ENABLE_STACKTRACE'
yiguolei merged PR #21932: URL: https://github.com/apache/doris/pull/21932 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-lts updated: [fix](status) fix cmake warning with message 'Manually-specified variables were not used by the project:ENABLE_STACKTRACE' (#21932)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new b06177977d [fix](status) fix cmake warning with message
'Manually-specified variables were not used by the project:ENABLE_STACKTRACE'
(#21932)
b06177977d is described below
commit b06177977d8735844c357903436682d9f8d6b54e
Author: caiconghui <55968745+caicong...@users.noreply.github.com>
AuthorDate: Tue Jul 18 18:25:50 2023 +0800
[fix](status) fix cmake warning with message 'Manually-specified variables
were not used by the project:ENABLE_STACKTRACE' (#21932)
Co-authored-by: caiconghui1
---
be/CMakeLists.txt | 4
be/src/common/status.h | 4 ++--
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/be/CMakeLists.txt b/be/CMakeLists.txt
index 9307e018f0..09f607b80a 100644
--- a/be/CMakeLists.txt
+++ b/be/CMakeLists.txt
@@ -531,6 +531,10 @@ if (USE_JEMALLOC)
set(CXX_COMMON_FLAGS "${CXX_COMMON_FLAGS} -DUSE_JEMALLOC")
endif()
+if (ENABLE_STACKTRACE)
+add_compile_options(-DENABLE_STACKTRACE)
+endif()
+
# STRICT_MEMORY_USE=ON` expects BE to use less memory, and gives priority to
ensuring stability
# when the cluster memory is limited.
# TODO In the future, expect a dynamic soft memory limit, combined with
real-time memory usage of the cluster,
diff --git a/be/src/common/status.h b/be/src/common/status.h
index cefc8d6402..6f49de3ec2 100644
--- a/be/src/common/status.h
+++ b/be/src/common/status.h
@@ -470,8 +470,8 @@ inline std::ostream& operator<<(std::ostream& ostr, const
Status& status) {
ostr << '[' << status.code_as_string() << ']';
ostr << (status._err_msg ? status._err_msg->_msg : "");
#ifdef ENABLE_STACKTRACE
-if (status->_err_msg && !status->_err_msg._stack.empty()) {
-ostr << '\n' << status->_err_msg._stack;
+if (status._err_msg && !status._err_msg->_stack.empty()) {
+ostr << '\n' << status._err_msg->_stack;
}
#endif
return ostr;
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] kaijchen commented on pull request #21725: [refactor](load) split segment flusher from rowset writer
kaijchen commented on PR #21725: URL: https://github.com/apache/doris/pull/21725#issuecomment-1639957130 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
