[GitHub] [doris] guanyiming opened a new issue, #11071: [Bug]
guanyiming opened a new issue, #11071: URL: https://github.com/apache/doris/issues/11071 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version com.sleepycat.je.EnvironmentFailureException: (JE 18.3.12) Environment must be closed, caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 18.3.12) 192.168.33.137_9010_1658384636665(2147483647):/usr/local/doris/fe/doris-meta/bdb Clock delta: 28667 ms. between Feeder: 192.168.33.135_9010_1658384237261 and this Replica exceeds max permissible delta: 5000 ms. HANDSHAKE_ERROR: Error during the handshake between two nodes. Some validity or compatibility check failed, preventing further communication between the nodes. Environment is invalid and must be closed. Originally thrown by HA thread: RepNode 192.168.33.137_9010_1658384636665(-1) Originally thrown by HA thread: RepNode 192.168.33.137_9010_1658384636665(-1) at com.sleepycat.je.EnvironmentFailureException.wrapSelf(EnvironmentFailureException.java:230) ~[je-18.3.12.jar:18.3.12] at com.sleepycat.je.dbi.EnvironmentImpl.checkIfInvalid(EnvironmentImpl.java:1835) ~[je-18.3.12.jar:18.3.12] at com.sleepycat.je.dbi.DbEnvPool.getEnvironment(DbEnvPool.java:151) ~[je-18.3.12.jar:18.3.12] at com.sleepycat.je.Environment.makeEnvironmentImpl(Environment.java:278) ~[je-18.3.12.jar:18.3.12] at com.sleepycat.je.Environment.(Environment.java:258) ~[je-18.3.12.jar:18.3.12] at com.sleepycat.je.rep.ReplicatedEnvironment.(ReplicatedEnvironment.java:605) ~[je-18.3.12.jar:18.3.12] at com.sleepycat.je.rep.ReplicatedEnvironment.(ReplicatedEnvironment.java:464) ~[je-18.3.12.jar:18.3.12] at com.sleepycat.je.rep.ReplicatedEnvironment.(ReplicatedEnvironment.java:538) ~[je-18.3.12.jar:18.3.12] at org.apache.doris.journal.bdbje.BDBEnvironment.setup(BDBEnvironment.java:151) ~[doris-fe.jar:1.0-SNAPSHOT] at org.apache.doris.journal.bdbje.BDBJEJournal.open(BDBJEJournal.java:303) ~[doris-fe.jar:1.0-SNAPSHOT] at org.apache.doris.persist.EditLog.open(EditLog.java:847) ~[doris-fe.jar:1.0-SNAPSHOT] at org.apache.doris.catalog.Catalog.initialize(Catalog.java:852) ~[doris-fe.jar:1.0-SNAPSHOT] at org.apache.doris.PaloFe.start(PaloFe.java:128) ~[doris-fe.jar:1.0-SNAPSHOT] at org.apache.doris.PaloFe.main(PaloFe.java:63) ~[doris-fe.jar:1.0-SNAPSHOT] Caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 18.3.12) 192.168.33.137_9010_1658384636665(2147483647):/usr/local/doris/fe/doris-meta/bdb Clock delta: 28667 ms. between Feeder: 192.168.33.135_9010_1658384237261 and this Replica exceeds max permissible delta: 5000 ms. HANDSHAKE_ERROR: Error during the handshake between two nodes. Some validity or compatibility check failed, preventing further communication between the nodes. Environment is invalid and must be closed. Originally thrown by HA thread: RepNode 192.168.33.137_9010_1658384636665(-1) Originally thrown by HA thread: RepNode 192.168.33.137_9010_1658384636665(-1) at com.sleepycat.je.rep.stream.ReplicaFeederHandshake.checkClockSkew(ReplicaFeederHandshake.java:432) ~[je-18.3.12.jar:18.3.12] at com.sleepycat.je.rep.stream.ReplicaFeederHandshake.execute(ReplicaFeederHandshake.java:269) ~[je-18.3.12.jar:18.3.12] at com.sleepycat.je.rep.impl.node.Replica.initReplicaLoop(Replica.java:709) ~[je-18.3.12.jar:18.3.12] at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoopInternal(Replica.java:485) ~[je-18.3.12.jar:18.3.12] at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoop(Replica.java:412) ~[je-18.3.12.jar:18.3.12] at com.sleepycat.je.rep.impl.node.RepNode.run(RepNode.java:1869) ~[je-18.3.12.jar:18.3.12] ### What's Wrong? add new fe ERROR ### What You Expected? version 1.1.0 ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h
[doris-website] branch master updated: url error
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 124001aab1 url error 124001aab1 is described below commit 124001aab16bb69d6dbe6698b4a1402160e3ef42 Author: jiafeng.zhang AuthorDate: Thu Jul 21 15:10:42 2022 +0800 url error url error --- docs/ecosystem/flink-doris-connector.md | 2 +- .../current/ecosystem/flink-doris-connector.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/ecosystem/flink-doris-connector.md b/docs/ecosystem/flink-doris-connector.md index 2c8b9dd5fd..ce6b6eb861 100644 --- a/docs/ecosystem/flink-doris-connector.md +++ b/docs/ecosystem/flink-doris-connector.md @@ -26,7 +26,7 @@ under the License. # Flink Doris Connector -> This document applies to flink-doris-connector versions after 1.1.0, for versions before 1.1.0 refer to [here](https://doris.apache.org/1.0/extending-doris/flink-doris-connector.html) +> This document applies to flink-doris-connector versions after 1.1.0, for versions before 1.1.0 refer to [here](https://doris.apache.org/docs/1.0/extending-doris/flink-doris-connector) diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/flink-doris-connector.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/flink-doris-connector.md index 6d8dd62a18..b0c7899d7b 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/flink-doris-connector.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/flink-doris-connector.md @@ -28,7 +28,7 @@ under the License. # Flink Doris Connector -> 本文档适用于flink-doris-connector 1.1.0之后的版本，1.1.0之前的版本参考[这里](https://doris.apache.org/zh-CN/1.0/extending-doris/flink-doris-connector.html) +> 本文档适用于flink-doris-connector 1.1.0之后的版本，1.1.0之前的版本参考[这里](https://doris.apache.org/zh-CN/docs/1.0/extending-doris/flink-doris-connector) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman opened a new pull request, #11072: [improvement](arrow) add arrow block convertion time profile
morningman opened a new pull request, #11072: URL: https://github.com/apache/doris/pull/11072 # Proposed changes Issue Number: close #xxx ## Problem Summary: Add a new timer `ConvertArrowBlockTimer` for `FileArrowScanner` to record the time of converting arrow batch to doris column block. ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No) 3. Has document been added or modified: (No) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] SaintBacchus commented on a diff in pull request #11046: [feature-wip][multi-catalog] Support orc format file split for file scan node
SaintBacchus commented on code in PR #11046:
URL: https://github.com/apache/doris/pull/11046#discussion_r926346121
##
be/src/exec/arrow/orc_reader.cpp:
##
@@ -52,6 +55,31 @@ Status ORCReaderWrap::init_reader(const TupleDescriptor*
tuple_desc,
return Status::EndOfFile("Empty Orc File");
}
+int64_t row_number = 0;
+int end_group = _total_groups;
+for (int i = 0; i < _total_groups; i++) {
+int64_t _offset =
_reader->GetRawORCReader()->getStripe(i)->getOffset();
+int64_t row =
_reader->GetRawORCReader()->getStripe(i)->getNumberOfRows();
+if (_offset < _range_start_offset) {
+row_number += row;
+} else if (_offset == _range_start_offset) {
+_current_group = i;
+}
+if (_range_start_offset + _range_size <= _offset) {
+end_group = i;
+break;
+}
+}
+LOG(INFO) << "This reader read orc file from offset: " <<
_range_start_offset
+ << " with size: " << _range_size << ". Also mean that read from
strip id from "
+ << _current_group << " to " << end_group;
+_total_groups = end_group;
Review Comment:
Yeah
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] SaintBacchus commented on a diff in pull request #11046: [feature-wip][multi-catalog] Support orc format file split for file scan node
SaintBacchus commented on code in PR #11046:
URL: https://github.com/apache/doris/pull/11046#discussion_r926345730
##
be/src/exec/arrow/orc_reader.cpp:
##
@@ -52,6 +55,31 @@ Status ORCReaderWrap::init_reader(const TupleDescriptor*
tuple_desc,
return Status::EndOfFile("Empty Orc File");
}
+int64_t row_number = 0;
+int end_group = _total_groups;
+for (int i = 0; i < _total_groups; i++) {
+int64_t _offset =
_reader->GetRawORCReader()->getStripe(i)->getOffset();
+int64_t row =
_reader->GetRawORCReader()->getStripe(i)->getNumberOfRows();
+if (_offset < _range_start_offset) {
+row_number += row;
+} else if (_offset == _range_start_offset) {
+_current_group = i;
Review Comment:
In my test for orc file , `_range_start_offset` is one of the `_offset` list
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] SaintBacchus commented on a diff in pull request #11046: [feature-wip][multi-catalog] Support orc format file split for file scan node
SaintBacchus commented on code in PR #11046:
URL: https://github.com/apache/doris/pull/11046#discussion_r926346121
##
be/src/exec/arrow/orc_reader.cpp:
##
@@ -52,6 +55,31 @@ Status ORCReaderWrap::init_reader(const TupleDescriptor*
tuple_desc,
return Status::EndOfFile("Empty Orc File");
}
+int64_t row_number = 0;
+int end_group = _total_groups;
+for (int i = 0; i < _total_groups; i++) {
+int64_t _offset =
_reader->GetRawORCReader()->getStripe(i)->getOffset();
+int64_t row =
_reader->GetRawORCReader()->getStripe(i)->getNumberOfRows();
+if (_offset < _range_start_offset) {
+row_number += row;
+} else if (_offset == _range_start_offset) {
+_current_group = i;
+}
+if (_range_start_offset + _range_size <= _offset) {
+end_group = i;
+break;
+}
+}
+LOG(INFO) << "This reader read orc file from offset: " <<
_range_start_offset
+ << " with size: " << _range_size << ". Also mean that read from
strip id from "
+ << _current_group << " to " << end_group;
+_total_groups = end_group;
Review Comment:
in the impl of arrow, `seek` function is just a value set.
```
Status Seek(int64_t row_number) {
ARROW_RETURN_IF(row_number >= NumberOfRows(),
Status::Invalid("Out of bounds row number: ",
row_number));
current_row_ = row_number;
return Status::OK();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener opened a new pull request, #11073: [enhance](nereids): use optional for combine()
jackwener opened a new pull request, #11073: URL: https://github.com/apache/doris/pull/11073 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. use optional for combine() If input list is empty, combine will return true. It's strange. Use optional to correct it. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zy-kkk opened a new pull request, #11074: [doc]Correct jump link error in flink-doris-connector
zy-kkk opened a new pull request, #11074: URL: https://github.com/apache/doris/pull/11074 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: update users wall and download style (#13)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new fed70282cc update users wall and download style (#13)
fed70282cc is described below
commit fed70282cca348b84919ef92a85b52e9decb83d1
Author: wangyongfeng <943155...@qq.com>
AuthorDate: Thu Jul 21 15:53:22 2022 +0800
update users wall and download style (#13)
Co-authored-by: wangyf0555
---
.gitignore| 23 +++
src/pages/download/index.scss | 6 +-
static/images/icon/u21.png| Bin 20333 -> 5981 bytes
3 files changed, 28 insertions(+), 1 deletion(-)
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 00..c1d977f9ed
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,23 @@
+# Dependencies
+/node_modules
+
+# Production
+/build
+
+# Generated files
+.docusaurus
+.cache-loader
+
+# Misc
+.DS_Store
+.env.local
+.env.development.local
+.env.test.local
+.env.production.local
+package-lock.json
+
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+
+*.zip
\ No newline at end of file
diff --git a/src/pages/download/index.scss b/src/pages/download/index.scss
index 2671f1a3cb..2fef49bb55 100644
--- a/src/pages/download/index.scss
+++ b/src/pages/download/index.scss
@@ -54,7 +54,7 @@
text-align: center;
font-size: var(--global-font-size-medium);
color: var(--global-colors-text-primary);
-height: 3.125rem;
+min-height: 3.125rem;
display: flex;
align-items: center;
justify-content: center;
@@ -111,6 +111,10 @@
flex-direction: column;
align-items: flex-start;
+label {
+width: 100%;
+}
+
.tabs-radio {
width: 100%;
}
diff --git a/static/images/icon/u21.png b/static/images/icon/u21.png
index 9748ba5b82..7fd60faaae 100644
Binary files a/static/images/icon/u21.png and b/static/images/icon/u21.png
differ
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #11074: [doc]Correct jump link error in flink-doris-connector
github-actions[bot] commented on PR #11074: URL: https://github.com/apache/doris/pull/11074#issuecomment-1191164893 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #11074: [doc]Correct jump link error in flink-doris-connector
github-actions[bot] commented on PR #11074: URL: https://github.com/apache/doris/pull/11074#issuecomment-1191164921 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] JNSimba opened a new pull request, #11075: [Doc] update flink connector faq
JNSimba opened a new pull request, #11075: URL: https://github.com/apache/doris/pull/11075 ## Problem Summary: update flink connector Add FAQ ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] 924060929 commented on a diff in pull request #11043: [feature] (Nereids) Merge memo group recursively
924060929 commented on code in PR #11043:
URL: https://github.com/apache/doris/pull/11043#discussion_r926376161
##
fe/fe-core/src/main/java/org/apache/doris/nereids/memo/Memo.java:
##
@@ -188,19 +189,24 @@ private void mergeGroup(Group source, Group destination) {
children.set(i, destination);
}
}
-if (groupExpressions.containsKey(groupExpression)) {
-// TODO: need to merge group recursively
-
groupExpression.getOwnerGroup().removeGroupExpression(groupExpression);
+GroupExpression that = groupExpressions.get(groupExpression);
+if (that != null) {
+mergeGroup(groupExpression.getOwnerGroup(),
that.getOwnerGroup());
} else {
groupExpressions.put(groupExpression, groupExpression);
}
}
-for (GroupExpression groupExpression : source.getLogicalExpressions())
{
-source.removeGroupExpression(groupExpression);
+
+Iterator iterator =
source.getLogicalExpressions().iterator();
+while (iterator.hasNext()) {
+GroupExpression groupExpression = iterator.next();
+iterator.remove();
destination.addGroupExpression(groupExpression);
}
-for (GroupExpression groupExpression :
source.getPhysicalExpressions()) {
-source.removeGroupExpression(groupExpression);
+iterator = source.getPhysicalExpressions().iterator();
Review Comment:
```suggestion
if (source != destination) {
// copy sourceGroup's physicalExpressions to destination group
destination.addPhysicalExpressions(source.getPhysicalExpressions());
source.clearPhysicalExpressions();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #11075: [Doc] update flink connector faq
github-actions[bot] commented on PR #11075: URL: https://github.com/apache/doris/pull/11075#issuecomment-1191182970 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #11075: [Doc] update flink connector faq
github-actions[bot] commented on PR #11075: URL: https://github.com/apache/doris/pull/11075#issuecomment-1191182995 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] 03/03: Merge branch 'master' of https://github.com/apache/doris-website
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git commit 6b8e0b3eda716b8a0897230d6c5fa2506398fdf4 Merge: a96e723bee fed70282cc Author: jiafeng.zhang AuthorDate: Thu Jul 21 16:17:53 2022 +0800 Merge branch 'master' of https://github.com/apache/doris-website # Conflicts: # .gitignore .gitignore| 29 +++-- src/pages/download/index.scss | 6 +- static/images/icon/u21.png| Bin 20333 -> 5981 bytes 3 files changed, 28 insertions(+), 7 deletions(-) diff --cc .gitignore index 11aa77384a,c1d977f9ed..ead04b614e --- a/.gitignore +++ b/.gitignore @@@ -1,7 -1,23 +1,24 @@@ - community/.DS_Store - docs/admin-manual/.DS_Store + # Dependencies + /node_modules + + # Production + /build + + # Generated files .docusaurus - build - docs/advanced/.DS_Store - .github/.DS_Store - node_modules + .cache-loader + + # Misc + .DS_Store + .env.local + .env.development.local + .env.test.local + .env.production.local + package-lock.json + + npm-debug.log* + yarn-debug.log* + yarn-error.log* + -*.zip ++*.zip ++*.zip - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] 02/03: flink connector fqa
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git commit a96e723bee6488e0082521c8a970891e2050e73e Author: jiafeng.zhang AuthorDate: Thu Jul 21 16:16:11 2022 +0800 flink connector fqa flink connector fqa --- .gitignore | 7 +++ 1 file changed, 7 insertions(+) diff --git a/.gitignore b/.gitignore new file mode 100644 index 00..11aa77384a --- /dev/null +++ b/.gitignore @@ -0,0 +1,7 @@ +community/.DS_Store +docs/admin-manual/.DS_Store +.docusaurus +build +docs/advanced/.DS_Store +.github/.DS_Store +node_modules - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] 01/03: flink connector fqa
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git commit 420fb7e0ad2dce1241428ba461cc8b6168b12ed8 Author: jiafeng.zhang AuthorDate: Thu Jul 21 16:14:59 2022 +0800 flink connector fqa flink connector fqa --- docs/ecosystem/flink-doris-connector.md | 11 +-- .../current/ecosystem/flink-doris-connector.md| 8 +++- 2 files changed, 16 insertions(+), 3 deletions(-) diff --git a/docs/ecosystem/flink-doris-connector.md b/docs/ecosystem/flink-doris-connector.md index ce6b6eb861..b63362c4df 100644 --- a/docs/ecosystem/flink-doris-connector.md +++ b/docs/ecosystem/flink-doris-connector.md @@ -356,9 +356,12 @@ source.sinkTo(builder.build()); | doris.read.field| --| N | List of column names in the Doris table, separated by commas | | doris.filter.query | --| N | Filter expression of the query, which is transparently transmitted to Doris. Doris uses this expression to complete source-side data filtering. | | sink.label-prefix | -- | Y | The label prefix used by stream load imports. In the 2pc scenario, global uniqueness is required to ensure the EOS semantics of Flink. | -| sink.properties.* | -- | N | The stream load parameters. eg: sink.properties.column_separator' = ',' Setting 'sink.properties.escape_delimiters' = 'true' if you want to use a control char as a separator, so that such as '\\x01' will translate to binary 0x01 Support JSON format import, you need to enable both 'sink.properties.format' ='json' and 'sink.properties.strip_outer_array' ='true'| +| sink.properties.* | -- | N | The stream load parameters. eg: sink.properties.column_separator' = ',' Setting 'sink.properties.escape_delimiters' = 'true' if you want to use a control char as a separator, so that such as '\\x01' will translate to binary 0x01Support JSON format import, you need to enable both 'sink.properties.format' ='json' and 'sink.properties.strip_outer_array' ='true' | | sink.enable-delete | true | N | Whether to enable deletion. This option requires Doris table to enable batch delete function (0.15+ version is enabled by default), and only supports Uniq model.| | sink.enable-2pc | true | N| Whether to enable two-phase commit (2pc), the default is true, to ensure Exactly-Once semantics. For two-phase commit, please refer to [here](../data-operate/import/import-way/stream-load-manual.md). | +| sink.max-retries | 1 | N| In the 2pc scenario, the number of retries after the commit phase fails. | +| sink.buffer-size | 1048576(1MB) | N| Write data cache buffer size, in bytes. It is not recommended to modify, the default configuration is sufficient. | +| sink.buffer-count| 3 | N| The number of write data cache buffers, it is not recommended to modify, the default configuration is sufficient. @@ -438,7 +441,7 @@ The most suitable scenario for using Flink Doris Connector is to synchronize sou ### common problem -1. Bitmap type write +1. **Bitmap type write** ```sql CREATE TABLE bitmap_sink ( @@ -456,3 +459,7 @@ WITH ( 'sink.properties.columns' = 'dt,page,user_id,user_id=to_bitmap(user_id)' ) +2. **errCode = 2, detailMessage = Label [label_0_1] has already been used, relate to txn [19650]** + +In the Exactly-Once scenario, the Flink Job must be restarted from the latest Checkpoint/Savepoint, otherwise the above error will be reported. +When Exactly-Once is not required, it can also be solved by turning off 2PC commits (sink.enable-2pc=false) or changing to a different sink.label-prefix. \ No newline at end of file diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/flink-doris-connector.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/flink-doris-connector.md index b0c7899d7b..77a1301ec8 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/flink-doris-connector.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/
[doris-website] branch master updated (fed70282cc -> 6b8e0b3eda)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git from fed70282cc update users wall and download style (#13) new 420fb7e0ad flink connector fqa new a96e723bee flink connector fqa new 6b8e0b3eda Merge branch 'master' of https://github.com/apache/doris-website The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .gitignore| 3 ++- docs/ecosystem/flink-doris-connector.md | 11 +-- .../current/ecosystem/flink-doris-connector.md| 8 +++- 3 files changed, 18 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xy720 merged pull request #10824: [Fix] (array-type) add the supported sub-type and regression
xy720 merged PR #10824: URL: https://github.com/apache/doris/pull/10824 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: Add the supported sub-type for array (#10824)
This is an automated email from the ASF dual-hosted git repository.
xuyang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 5f6f35e886 Add the supported sub-type for array (#10824)
5f6f35e886 is described below
commit 5f6f35e88664eae4033ee6ed5b17e91e111dac68
Author: carlvinhust2012
AuthorDate: Thu Jul 21 16:29:17 2022 +0800
Add the supported sub-type for array (#10824)
1.This pr is used for adding the supported sub-type for array which has
been modified in #9916
2.add regression test for the supported sub-type
Co-authored-by: hucheng01
---
.../java/org/apache/doris/analysis/TypeDef.java| 14 +-
.../data/load/broker_load/simple_array.json| 5 +
.../data/load/broker_load/simple_object_array.json | 5 +
.../data/load/broker_load/test_array_load.out | 33
.../suites/load/broker_load/test_array_load.groovy | 198 +
5 files changed, 252 insertions(+), 3 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/TypeDef.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/TypeDef.java
index 91ce563896..7af29a8fc2 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/TypeDef.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/TypeDef.java
@@ -117,10 +117,18 @@ public class TypeDef implements ParseNode {
if (type.isNull()) {
throw new AnalysisException("Unsupported data type: " +
type.toSql());
}
-if (!type.getPrimitiveType().isIntegerType()
-&& !type.getPrimitiveType().isCharFamily()) {
-throw new AnalysisException("Array column just support INT/VARCHAR
sub-type");
+// check whether the array sub-type is supported
+Boolean isSupportType = false;
+for (Type subType : Type.getArraySubTypes()) {
+if (type.getPrimitiveType() == subType.getPrimitiveType()) {
+isSupportType = true;
+break;
+}
}
+if (!isSupportType) {
+throw new AnalysisException("Array unsupported sub-type: " +
type.toSql());
+}
+
if (type.getPrimitiveType().isStringType()
&& !type.isAssignedStrLenInColDefinition()) {
type.setLength(1);
diff --git a/regression-test/data/load/broker_load/simple_array.json
b/regression-test/data/load/broker_load/simple_array.json
new file mode 100644
index 00..15fc2a3964
--- /dev/null
+++ b/regression-test/data/load/broker_load/simple_array.json
@@ -0,0 +1,5 @@
+[{"k1": 1, "k2": [1,2,3,4,5], "k3": [32767,32768,32769], "k4":
[65534,65535,65536], "k5": ["a","b","c","d","e"], "k6": ["hello","world"],
"k7": ["1991-01-01"], "k8": ["1991-01-01 00:00:00"], "k9": [0.33,0.67], "k10":
[3.1415926,0.878787878], "k11": [1,1.2,1.3]},
+{"k1": 2, "k2": [6,7,8,9,10], "k3": [32767,32768,32769], "k4":
[65534,65535,65536], "k5": ["a","b","c","d","e"], "k6": ["hello","world"],
"k7": ["1991-01-01"], "k8": ["1991-01-01 00:00:00"], "k9": [0.33,0.67], "k10":
[3.1415926,0.878787878], "k11": [1,1.2,1.3]},
+{"k1": 3, "k2": [], "k3": [32767,32768,32769], "k4": [null,null,65536], "k5":
["a","b","c","d","e"], "k6": ["happy","birthday"], "k7": ["1991-01-01"], "k8":
["1991-01-01 00:00:00"], "k9": [0.33,0.67], "k10": [3.1415926,0.878787878],
"k11": [1,1.2,1.3]},
+{"k1": 4, "k2": [null], "k3": [32767,32768,32769], "k4": [ null,null,65536],
"k5": ["a","b","c","d","e"], "k6": ["hello","world"], "k7": ["1991-01-01"],
"k8": ["1991-01-01 00:00:00"], "k9": [0.33,0.67], "k10":
[3.1415926,0.878787878], "k11": [1,1.2,1.3]},
+{"k1": 5, "k2": [null,null], "k3": [32767,32768,null], "k4":
[65534,null,65536], "k5": ["a","b","c","d","e"], "k6": ["hello","world"], "k7":
["1991-01-01"], "k8": ["1991-01-01 00:00:00"], "k9": [0.33,0.67], "k10":
[3.1415926,0.878787878], "k11": [1,1.2,1.3]}]
\ No newline at end of file
diff --git a/regression-test/data/load/broker_load/simple_object_array.json
b/regression-test/data/load/broker_load/simple_object_array.json
new file mode 100644
index 00..ca57e52676
--- /dev/null
+++ b/regression-test/data/load/broker_load/simple_object_array.json
@@ -0,0 +1,5 @@
+{"k1": 1, "k2": [1,2,3,4,5], "k3": [32767,32768,32769], "k4":
[65534,65535,65536], "k5": ["a","b","c","d","e"], "k6": ["hello","world"],
"k7": ["1991-01-01"], "k8": ["1991-01-01 00:00:00"], "k9": [0.33,0.67], "k10":
[3.1415926,0.878787878], "k11": [1,1.2,1.3]}
+{"k1": 2, "k2": [6,7,8,9,10], "k3": [32767,32768,32769], "k4":
[65534,65535,65536], "k5": ["a","b","c","d","e"], "k6": ["hello","world"],
"k7": ["1991-01-01"], "k8": ["1991-01-01 00:00:00"], "k9": [0.33,0.67], "k10":
[3.1415926,0.878787878], "k11": [1,1.2,1.3]}
+{"k1": 3, "k2": [], "k3": [32767,32768,32769], "k4": [null,null,65536], "k5":
["a","b","c","d","e"], "k6": ["happy","birthday"], "k7": ["1991-01-01"], "k8":
["1
[GitHub] [doris] yinzhijian commented on pull request #11055: [Bug] Total capacity metrics was not updated after extending filesystem partition online (#11053)
yinzhijian commented on PR #11055: URL: https://github.com/apache/doris/pull/11055#issuecomment-1191203084 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: update readme
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 01d36f1ebf update readme 01d36f1ebf is described below commit 01d36f1ebff2138bfd152082d186a0699c899bbd Author: jiafeng.zhang AuthorDate: Thu Jul 21 16:56:18 2022 +0800 update readme update readme --- README.md | 44 +++- 1 file changed, 43 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index aaba2fa1e1..0e6a227de3 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,46 @@ -# Website + + +# Doris document website + +This repo is for [Apache Doris Website](https://doris.apache.org) + +And it use Github Action to automatically sync content from [Apache Doris Code Repo](https://github.com/apache/doris) + +There are 2 Github Actions: + +1. cron-deploy-website.yml + + It will sync at 01:00 AM everyday from Doris's master branch. + +2. manual-deploy-website.yml + + It can only be triggered manually, and you can specify the branch name you want to sync. + +## View the website + +To view the website, navigate to +[https://doris.apache.org](https://doris.apache.org) +or +[https://doris.apache.org](https://doris.apache.org) + +## Run & Build Website This website is built using [Docusaurus 2](https://docusaurus.io/), a modern static website generator. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #11066: [Refactor](Nereids) Remove expression type.
morrySnow commented on code in PR #11066:
URL: https://github.com/apache/doris/pull/11066#discussion_r926439060
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/StringRegexPredicate.java:
##
@@ -29,15 +29,21 @@
* Such as: like, regexp
*/
public abstract class StringRegexPredicate extends Expression implements
BinaryExpression {
+
+/**
+ * like or regexp
+ */
+protected final String symbol;
+
/**
* Constructor of StringRegexPredicate.
*
- * @param nodeType node type of expression
* @param left left child of string regex
* @param rightright child of string regex
*/
-public StringRegexPredicate(ExpressionType nodeType, Expression left,
Expression right) {
-super(nodeType, left, right);
+public StringRegexPredicate(Expression left, Expression right, String
symbol) {
Review Comment:
nit: mod java doc parameter list
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei commented on pull request #10823: [enhancement] Refactor to improve the usability of MemTracker (step2)
yiguolei commented on PR #10823: URL: https://github.com/apache/doris/pull/10823#issuecomment-1191241703 This PR need fast merge because it modified memtracker and too many files. It will not affect logic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #10823: [enhancement] Refactor to improve the usability of MemTracker (step2)
yiguolei merged PR #10823: URL: https://github.com/apache/doris/pull/10823 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: add github star (#14)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 80729d06e1 add github star (#14)
80729d06e1 is described below
commit 80729d06e1f80a3b6eef307ff1e33f3fd5f0579c
Author: wangyongfeng <943155...@qq.com>
AuthorDate: Thu Jul 21 17:13:59 2022 +0800
add github star (#14)
* update users wall and download style
* add github star
Co-authored-by: wangyf0555
---
.gitignore | 3 +-
docusaurus.config.js | 424 ++---
package.json | 1 +
src/scss/components/navbar.scss| 115 +-
src/theme/Navbar/Content/index.d.ts| 2 +
src/theme/Navbar/Content/index.tsx | 83
src/theme/Navbar/Content/styles.module.css | 8 +
.../Navbar/MobileSidebar/PrimaryMenu/index.d.ts| 2 +
.../Navbar/MobileSidebar/PrimaryMenu/index.tsx | 41 ++
9 files changed, 459 insertions(+), 220 deletions(-)
diff --git a/.gitignore b/.gitignore
index ead04b614e..c1d977f9ed 100644
--- a/.gitignore
+++ b/.gitignore
@@ -20,5 +20,4 @@ npm-debug.log*
yarn-debug.log*
yarn-error.log*
-*.zip
-*.zip
+*.zip
\ No newline at end of file
diff --git a/docusaurus.config.js b/docusaurus.config.js
index 888d754fc7..fce2d93dd1 100644
--- a/docusaurus.config.js
+++ b/docusaurus.config.js
@@ -1,229 +1,229 @@
// @ts-check
// Note: type annotations allow type checking and IDEs autocompletion
// @ts-ignore
-const versions = require('./versions.json');
-const lightCodeTheme = require('prism-react-renderer/themes/github');
+const versions = require("./versions.json");
+const lightCodeTheme = require("prism-react-renderer/themes/github");
const showAllVersions = true;
/** @type {import('@docusaurus/types').Config} */
const config = {
-title: 'Apache Doris',
-titleDelimiter: '-',
-tagline: 'Apache Doris',
-url: 'https://doris.apache.org',
-baseUrl: '/',
-onBrokenLinks: 'ignore',
-onBrokenMarkdownLinks: 'ignore',
-favicon: 'images/favicon.ico',
-i18n: {
-defaultLocale: 'en',
-locales: ['en', 'zh-CN'],
-localeConfigs: {
-en: {
-label: 'EN',
+ title: "Apache Doris",
+ titleDelimiter: "-",
+ tagline: "Apache Doris",
+ url: "https://doris.apache.org";,
+ baseUrl: "/",
+ onBrokenLinks: "ignore",
+ onBrokenMarkdownLinks: "ignore",
+ favicon: "images/favicon.ico",
+ i18n: {
+defaultLocale: "en",
+locales: ["en", "zh-CN"],
+localeConfigs: {
+ en: {
+label: "EN",
+ },
+ "zh-CN": {
+label: "中文",
+ },
+},
+ },
+ // scripts: ['/js/redirect.js'],
+ stylesheets: [
+"https://fonts.googleapis.com/css?family=Montserrat:500";,
+"https://fonts.googleapis.com/css?family=Noto+Sans+SC:400";,
+ ],
+ organizationName: "doris", // Usually your GitHub org/user name.
+ projectName: "doris", // Usually your repo name.
+ plugins: [
+"docusaurus-plugin-sass",
+[
+ "content-docs",
+ /** @type {import('@docusaurus/plugin-content-docs').Options} */
+ ({
+id: "community",
+path: "community",
+routeBasePath: "/community",
+sidebarPath: require.resolve("./sidebarsCommunity.json"),
+ }),
+],
+ ],
+ presets: [
+[
+ "classic",
+ /** @type {import('@docusaurus/preset-classic').Options} */
+ ({
+docs: {
+ lastVersion: "current",
+ versions: {
+current: {
+ label: "1.1",
+ path: "",
+},
+"1.0": {
+ banner: "none",
},
-'zh-CN': {
-label: '中文',
+0.15: {
+ banner: "none",
},
+ },
+ sidebarPath: require.resolve("./sidebars.json"),
+ onlyIncludeVersions:
+process.env.NODE_ENV === "development" && !showAllVersions
+ ? ["current"]
+ : ["current", ...versions],
},
-},
-// scripts: ['/js/redirect.js'],
-stylesheets: [
-'https://fonts.googleapis.com/css?family=Montserrat:500',
-'https://fonts.googleapis.com/css?family=Noto+Sans+SC:400',
-],
-organizationName: 'doris', // Usually your GitHub org/user name.
-projectName: 'doris', // Usually your repo name.
-plugins: [
-'docusaurus-plugin-sass',
-[
-'content-docs',
-/** @type {import('@docusaurus/plugin-content-docs').Options} */
-({
-id: 'community',
-path: 'community',
-routeBasePath: '/community',
-sidebarPath: require.resolve('./sidebarsCommunity.json'),
-}),
-
[GitHub] [doris] github-actions[bot] commented on pull request #11066: [Refactor](Nereids) Remove expression type.
github-actions[bot] commented on PR #11066: URL: https://github.com/apache/doris/pull/11066#issuecomment-1191240484 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] 924060929 commented on a diff in pull request #11066: [Refactor](Nereids) Remove expression type.
924060929 commented on code in PR #11066:
URL: https://github.com/apache/doris/pull/11066#discussion_r926449371
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/Arithmetic.java:
##
@@ -189,6 +165,24 @@ public int hashCode() {
@Override
public String toString() {
-return toSql();
+return stringBuilder(Object::toString);
Review Comment:
good refactor
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow closed pull request #10510: [chore]add license header check in fe's checkstyle
morrySnow closed pull request #10510: [chore]add license header check in fe's checkstyle URL: https://github.com/apache/doris/pull/10510 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #10510: [chore]add license header check in fe's checkstyle
morrySnow commented on PR #10510: URL: https://github.com/apache/doris/pull/10510#issuecomment-1191265950 to many conflict, commit a new PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow opened a new pull request, #11076: [chore](FE)add license header check in fe's checkstyle
morrySnow opened a new pull request, #11076: URL: https://github.com/apache/doris/pull/11076 ## Problem Summary: Add license header check in fe's checkstyle -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman merged pull request #10977: [feature-wip][multi-catalog] Support s3 storage for file scan node
morningman merged PR #10977: URL: https://github.com/apache/doris/pull/10977 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [feature-wip](multi-catalog) Support s3 storage for file scan node (#10977)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 7147a7c290 [feature-wip](multi-catalog) Support s3 storage for file
scan node (#10977)
7147a7c290 is described below
commit 7147a7c2906bdc488732a8c742db59ff0adcfa25
Author: huangzhaowei
AuthorDate: Thu Jul 21 17:38:53 2022 +0800
[feature-wip](multi-catalog) Support s3 storage for file scan node (#10977)
This is an example of s3 hms_catalog:
```sql
CREATE CATALOG hms_catalog properties(
"type" = "hms",
"hive.metastore.uris"="thrift://localhost:9083",
"AWS_ACCESS_KEY" = "your access key",
"AWS_SECRET_KEY"="your secret key",
"AWS_ENDPOINT"="s3 endpoint",
"AWS_REGION"="s3-region",
"fs.s3a.paging.maximum"="1000");
```
All these params are necessary;
---
.gitignore | 1 +
be/src/io/file_factory.cpp | 52 ++
be/src/io/file_factory.h | 14 ++
be/src/vec/exec/file_arrow_scanner.cpp | 10 +++--
be/src/vec/exec/file_text_scanner.cpp | 8 ++--
bin/start_be.sh| 10 +
.../java/org/apache/doris/catalog/HiveTable.java | 5 +++
.../doris/datasource/DataSourceProperty.java | 24 ++
.../planner/external/ExternalFileScanNode.java | 33 +-
.../planner/external/ExternalFileScanProvider.java | 4 +-
.../planner/external/ExternalHiveScanProvider.java | 46 ++-
.../external/ExternalIcebergScanProvider.java | 14 ++
gensrc/thrift/PlanNodes.thrift | 2 +
13 files changed, 181 insertions(+), 42 deletions(-)
diff --git a/.gitignore b/.gitignore
index c8cf1a765b..e743211fac 100644
--- a/.gitignore
+++ b/.gitignore
@@ -33,6 +33,7 @@ package-lock.json
# ignore project file
.cproject
.project
+.cache
.settings/
**/.idea/
**/.vscode/
diff --git a/be/src/io/file_factory.cpp b/be/src/io/file_factory.cpp
index 9937eaa6d5..46d19669b9 100644
--- a/be/src/io/file_factory.cpp
+++ b/be/src/io/file_factory.cpp
@@ -130,3 +130,55 @@ doris::Status doris::FileFactory::create_file_reader(
}
return Status::OK();
}
+
+doris::Status doris::FileFactory::_new_file_reader(doris::ExecEnv* env,
RuntimeProfile* profile,
+ const TFileScanRangeParams&
params,
+ const
doris::TFileRangeDesc& range,
+ FileReader*&
file_reader_ptr) {
+doris::TFileType::type type = params.file_type;
+
+if (type == TFileType::FILE_STREAM) {
+return Status::InternalError("UnSupport UniquePtr For FileStream
type");
+}
+
+switch (type) {
+case TFileType::FILE_S3: {
+file_reader_ptr = new BufferedReader(
+profile, new S3Reader(params.properties, range.path,
range.start_offset));
+break;
+}
+case TFileType::FILE_HDFS: {
+FileReader* hdfs_reader = nullptr;
+RETURN_IF_ERROR(HdfsReaderWriter::create_reader(params.hdfs_params,
range.path,
+range.start_offset,
&hdfs_reader));
+file_reader_ptr = new BufferedReader(profile, hdfs_reader);
+break;
+}
+default:
+return Status::InternalError("UnSupport File Reader Type: " +
std::to_string(type));
+}
+
+return Status::OK();
+}
+
+doris::Status doris::FileFactory::create_file_reader(doris::ExecEnv* env,
RuntimeProfile* profile,
+ const
TFileScanRangeParams& params,
+ const
doris::TFileRangeDesc& range,
+
std::shared_ptr& file_reader) {
+FileReader* file_reader_ptr;
+RETURN_IF_ERROR(_new_file_reader(env, profile, params, range,
file_reader_ptr));
+file_reader.reset(file_reader_ptr);
+
+return Status::OK();
+}
+
+doris::Status doris::FileFactory::create_file_reader(doris::ExecEnv* env,
RuntimeProfile* profile,
+ const
TFileScanRangeParams& params,
+ const
doris::TFileRangeDesc& range,
+
std::unique_ptr& file_reader) {
+FileReader* file_reader_ptr;
+RETURN_IF_ERROR(_new_file_reader(env, profile, params, range,
file_reader_ptr));
+file_reader.reset(file_reader_ptr);
+
+return Status::OK();
+}
diff --git a/be/src/io/file_factory.h b/be/src/io/file_factory.h
index c1350d6682..c36b611f9e 100644
--- a/be/src/io/file_factory.h
+++ b/be/src/io/file_factory.h
@@ -48,6 +48,16 @@ public:
[GitHub] [doris] wangshuo128 opened a new pull request, #11077: [Feature](Nereids) Add an annotation @Developing
wangshuo128 opened a new pull request, #11077: URL: https://github.com/apache/doris/pull/11077 # Proposed changes Issue Number: close #xxx Currently, the new optimizer is under development. We want to merge not fully developed features into the code, but not be used in the main code path. Added an annotation `@Developing` to mark these features. ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #10781: [feature-wip](array-type) add function array_union/array_except/array_intersect
github-actions[bot] commented on PR #10781: URL: https://github.com/apache/doris/pull/10781#issuecomment-1191274182 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #10781: [feature-wip](array-type) add function array_union/array_except/array_intersect
github-actions[bot] commented on PR #10781: URL: https://github.com/apache/doris/pull/10781#issuecomment-1191274126 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix stream load doc
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new dd34a72fc0 fix stream load doc new aa8dff6a93 Merge branch 'master' of https://github.com/apache/doris-website dd34a72fc0 is described below commit dd34a72fc0b5eee3a262c396dc57b09989d52e61 Author: jiafeng.zhang AuthorDate: Thu Jul 21 18:12:22 2022 +0800 fix stream load doc fix stream load doc --- .../import/import-way/stream-load-manual.md| 8 +++- .../Show-Statements/SHOW-STREAM-LOAD.md| 1 + .../import/import-way/stream-load-manual.md| 8 +++- .../sql-reference/Show-Statements/SHOW-LOAD.md | 22 -- 4 files changed, 27 insertions(+), 12 deletions(-) diff --git a/docs/data-operate/import/import-way/stream-load-manual.md b/docs/data-operate/import/import-way/stream-load-manual.md index a1e8eb9e74..0b0a67913b 100644 --- a/docs/data-operate/import/import-way/stream-load-manual.md +++ b/docs/data-operate/import/import-way/stream-load-manual.md @@ -216,7 +216,7 @@ The number of rows in the original file = `dpp.abnorm.ALL + dpp.norm.ALL` "status": "Success", "msg": "transaction [18037] abort successfully." } - ``` + ``` ### Return results @@ -298,6 +298,12 @@ The following main explanations are given for the Stream load import result para Users can't cancel Stream load manually. Stream load will be cancelled automatically by the system after a timeout or import error. +### View Stream Load + +Users can view completed stream load tasks through `show stream load`. + +By default, BE does not record Stream Load records. If you want to view the records that need to be enabled on BE `enable_stream_load_record=true`, you need to restart BE here + ## Relevant System Configuration ### FE configuration diff --git a/docs/sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md b/docs/sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md index 46d1a8b015..3ac7ba14ee 100644 --- a/docs/sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md +++ b/docs/sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md @@ -50,6 +50,7 @@ SHOW STREAM LOAD illustrate: +1. The default is to not record Stream Load records. If you want to view the records that need to be enabled on BE, `enable_stream_load_record=true`, you need to restart BE here 1. If db_name is not specified, the current default db is used 2. If LABEL LIKE is used, it will match the tasks whose label of the Stream Load task contains label_matcher 3. If LABEL = is used, it will match the specified label exactly diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-way/stream-load-manual.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-way/stream-load-manual.md index efc21985d0..606ca5f2a2 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-way/stream-load-manual.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-way/stream-load-manual.md @@ -227,7 +227,7 @@ Stream Load 由于使用的是 HTTP 协议，所以所有导入任务有关的 "status": "Success", "msg": "transaction [18037] abort successfully." } - ``` + ``` ### 返回结果 @@ -310,6 +310,12 @@ Stream Load 由于使用的是 HTTP 协议，所以所有导入任务有关的 用户无法手动取消 Stream Load，Stream Load 在超时或者导入错误后会被系统自动取消。 +### 查看 Stream Load + +用户可以通过 `show stream load` 来查看已经完成的 stream load 任务。 + +默认 BE 是不记录 Stream Load 的记录，如果你要查看需要再 BE 上启用记录，`enable_stream_load_record=true`，这里需要重启BE + ## 相关系统配置 ### FE配置 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Show-Statements/SHOW-LOAD.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Show-Statements/SHOW-LOAD.md index 0cbbf581be..a58a99369e 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Show-Statements/SHOW-LOAD.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Show-Statements/SHOW-LOAD.md @@ -50,20 +50,22 @@ SHOW LOAD 说明： +1) 默认是不记录 Stream Load 的记录，如果你要查看需要再 BE 上启用记录，`enable_stream_load_record=true`，这里需要重启 BE + 1) 如果不指定 db_name，使用当前默认db - + 2) 如果使用 LABEL LIKE，则会匹配导入任务的 label 包含 label_matcher 的导入任务 3) 如果使用 LABEL = ，则精确匹配指定的 label 4) 如果指定了 STATE，则匹配 LOAD 状态 - + 5) 可以使用 ORDER BY 对任意列组合进行排序 - + 6) 如果指定了 LIMIT，则显示 limit 条匹配记录。否则全部显示 7) 如果指定了 OFFSET，则从偏移量offset开始显示查询结果。默认情况下偏移量为0。 - + 8) 如果是使用 broker/mini load，则 URL 列中的连接可以使用以下命令查看： ```sql @@ -73,38 +75,38 @@ SHOW LOAD ### Example 1. 展示默认 db 的所有导入任务 - + ```sql SHOW LOAD; ``` 2. 展示指定 db 的导入任务，label 中包含字符串 "2014_01_02"，展示最老的10个 - + ```sql SHOW LOAD FROM
[GitHub] [doris] github-actions[bot] commented on pull request #11077: [Feature](Nereids) Add an annotation @Developing
github-actions[bot] commented on PR #11077: URL: https://github.com/apache/doris/pull/11077#issuecomment-1191304620 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hf200012 opened a new pull request, #11078: [doc]Stream load doc fix
hf200012 opened a new pull request, #11078: URL: https://github.com/apache/doris/pull/11078 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yixiutt opened a new issue, #11079: [Bug] doris master ProcessHashTableProbe core
yixiutt opened a new issue, #11079:
URL: https://github.com/apache/doris/issues/11079
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and
found no similar issues.
### Version
master
### What's Wrong?
0x558fa1d341c8 in doris::signal::(anonymous
namespace)::FailureSignalHandler (signal_number=11, signal_info=0x7f66e429e370,
ucontext=0x7f66e429e240) at
/home/zcp/repo_center/doris_master/be/src/common/signal_handler.h:372 #1
#2 0x558fa1d341c8 in doris::signal::(anonymous
namespace)::FailureSignalHandler (signal_number=6, signal_info=0x7f66e429ea30,
ucontext=0x7f66e429e900) at
/home/zcp/repo_center/doris_master/be/src/common/signal_handler.h:372 #3
#4 _GI_raise (sig=sig@entry=6) at
../sysdeps/unix/sysv/linux/raise.c:50 #5 0x7f693536e859 in __GI_abort () at
abort.c:79 #6 0x558fa1d0a42e in __sanitizer::Abort() () #7
0x558fa1d15d18 in __sanitizer::Die() () #8 0x558fa1cf5ae4 in
__asan::ScopedInErrorReport::~ScopedInErrorReport() () #9 0x558fa1cf53a1 in
__asan::ReportGenericError(unsigned long, unsigned long, unsigned long,
unsigned long, bool, unsigned long, unsigned int, bool) () #10
0x558fa1cf6117 in __as
an_report_load8 () #11 0x558fa21082d1 in
doris::vectorized::PODArrayBase<1ul, 4096ul, Allocator, 15ul,
16ul>::capacity (this=0x6032139b9300) at
/home/zcp/repo_center/doris_master/be/src/vec/common/pod_array.h:203 #12
0x558fa2100aa2 in doris::vectorized::PODArrayBase<1ul, 4096ul,
Allocator, 15ul, 16ul>::reserve<>(unsigned long) (
this=0x6032139b9300, n=1) at
/home/zcp/repo_center/doris_master/be/src/vec/common/pod_array.h:212 #13
0x558fa4ea1701 in doris::vectorized::ColumnVector::replicate
(this=0x6040027cfe90, counts=0x602000cfa010, target_size=1, column=...) at
/home/zcp/repo_center/doris_master/be/src/vec/columns/column_vector.cpp:374 #14
0x558fa5b90431 in
doris::vectorized::ProcessHashTableProbe,
false>::probe_side_output_column (this=0x7f66e42a18d0, mcol=...,
output_slot_flags=..., size=1) at /home/zcp/rep
o_center/doris_master/be/src/vec/exec/join/vhash_join_node.cpp:258 #15
0x558fa55bca4c in
doris::vectorized::ProcessHashTableProbe,
false>::do_process_with_other_join_conjunts (this=0x7f66e42a18d0,
hash_table_ctx=..., null_map=0x604002f68960, mutable_block=...,
output_block=0x60700270b450) at
/home/zcp/repo_center/doris_master/be/src/vec/exec/join/vhash_join_node.cpp:488
#16 0x558fa541eaa0 in
operator()&,
std::integral_constant, std::integral_constant >
(closure=0x7f66e42a2650, arg=..., join_op_variants=...,
have_other_join_conjunct=..., probe_ignore_null=...) at
/home/zcp/repo_center/doris_master/be/src/vec/exec/join/vhash_join_node.cpp:928
#17 0x558fa557717b in std::invoke_impl,
doris::vectorized::SerializedHashTableContext&,
std::integral_constant&,
std::integral_constant, std::integral_constant
>(std::invoke_other, struct {...} &&) ( -Type for more, q to quit, c to
continue without paging- __f=...) at
/var/local/ldb_toolchain/include/c+/11/bits/invoke.h:61 #18 0x558fa555e7e1
in std::invoke, doris::vectorized::SerializedHashTableContext&,
std::integral_constant&,
std::integral_constant, std::integral_constant
>(struct {...} &&) (fn=...) at
/var/local/ldb_toolchain/include/c/11/bits/invoke.h:96 #19 0x558fa541ed24
in std::detail::variant::gen
_vtable_impl
(doris::vectorized::HashJoinNode::get_next(doris::RuntimeState*,
doris::vectorized::Block*, bool*)::&&, std::variant,
doris::vectorized::PrimaryTypeHashTableContext,
doris::vectorized::PrimaryTypeHashTableContext,
doris::vectorized::PrimaryTypeHashTableContext,
doris::vectorized::PrimaryTypeHashTableContext,
doris::vectorized::PrimaryTypeHashTableContext,
doris::vectorized::FixedKeyHashTableContext,
doris::vectorized::FixedKeyHashTableContext,
doris::vectorized::FixedKeyHashTableContext,
doris::vectorized::FixedKeyHashTableConte
xt,
doris::vectorized::FixedKeyHashTableContext,
doris::vectorized::FixedKeyHashTableContext
>&, std::variant, std::integral_constant, std::integral_constant, std::integral_constant, std::integral_constant, std::integral_constant, std::integral_constant, std::integral_constant, std::integral_constant, std::integral_constant >&, std::variant,
std::integral_const
ant >&&, std::variant,
std::integral_constant >&&)>, std::integer_sequence >::visit_invoke(struct {...} &&, std::variant,
doris::vectorized::PrimaryTypeHashTableContext,
doris::vectorized::PrimaryTypeHashTableContext,
doris::vectorized::PrimaryTypeHashTableContext,
doris::vectorized::PrimaryTypeHashTableContext,
doris::vectorized::PrimaryTypeHashTableContext,
doris::vectorized::FixedKeyHashTableContext,
doris::vectorized::FixedKeyHashTableContext,
doris::vectorized::FixedKeyHashTableContext,
doris::vectorized::FixedKeyHashTableContext,
doris::vectorized::FixedKe
yHashTableContext,
doris::vectoriz
[doris-website] branch master updated: Add Regression Testing Doc
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new e5b812025a Add Regression Testing Doc
e5b812025a is described below
commit e5b812025af030bc81570a4ae274c53e21fabd29
Author: jiafeng.zhang
AuthorDate: Thu Jul 21 18:33:32 2022 +0800
Add Regression Testing Doc
Add Regression Testing Doc
---
community/developer-guide/regression-testing.md | 30 +
sidebarsCommunity.json | 3 ++-
2 files changed, 32 insertions(+), 1 deletion(-)
diff --git a/community/developer-guide/regression-testing.md
b/community/developer-guide/regression-testing.md
new file mode 100644
index 00..c5981fd3ca
--- /dev/null
+++ b/community/developer-guide/regression-testing.md
@@ -0,0 +1,30 @@
+---
+{
+"title": "Regression Testing",
+"language": "en"
+}
+
+---
+
+
+
+# Regression Testing
+
+TODO
\ No newline at end of file
diff --git a/sidebarsCommunity.json b/sidebarsCommunity.json
index ce08f0d880..0305e881ad 100644
--- a/sidebarsCommunity.json
+++ b/sidebarsCommunity.json
@@ -53,7 +53,8 @@
"developer-guide/cpp-diagnostic-code",
"developer-guide/how-to-share-blogs",
"developer-guide/bitmap-hll-file-format",
-"developer-guide/github-checks"
+"developer-guide/github-checks",
+"developer-guide/regression-testing"
]
}
]
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] caiconghui commented on a diff in pull request #11078: [doc]Stream load doc fix
caiconghui commented on code in PR #11078: URL: https://github.com/apache/doris/pull/11078#discussion_r926523324 ## docs/zh-CN/docs/sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md: ## @@ -50,6 +50,7 @@ SHOW STREAM LOAD 说明： +1. 默认 BE 是不记录 Stream Load 的记录，如果你要查看需要再 BE 上启用记录，`enable_stream_load_record=true`，这里需要重启BE Review Comment: ```suggestion 1. 默认 BE 是不记录 Stream Load 的记录，如果你要查看需要在BE 上启用记录，`enable_stream_load_record=true`，这里需要重启BE ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] caiconghui commented on a diff in pull request #11078: [doc]Stream load doc fix
caiconghui commented on code in PR #11078: URL: https://github.com/apache/doris/pull/11078#discussion_r926523324 ## docs/zh-CN/docs/sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md: ## @@ -50,6 +50,7 @@ SHOW STREAM LOAD 说明： +1. 默认 BE 是不记录 Stream Load 的记录，如果你要查看需要再 BE 上启用记录，`enable_stream_load_record=true`，这里需要重启BE Review Comment: ```suggestion 1. 默认 BE 是不记录 Stream Load 的记录，如果你要查看需要在BE 上启用记录，`enable_stream_load_record=true`，这里不需要重启BE ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hf200012 opened a new issue, #11080: [improvement]Remove the website build framework from the docs directory
hf200012 opened a new issue, #11080: URL: https://github.com/apache/doris/issues/11080 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version master ### What's Wrong? 1. Remove the website build framework from the docs directory The current build framework is in the doris-website repo 2. Remove the build pipeline docs build ### What You Expected? no ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hf200012 opened a new pull request, #11081: [improvement]Remove the website build framework from the docs directory
hf200012 opened a new pull request, #11081: URL: https://github.com/apache/doris/pull/11081 # Proposed changes Issue Number: close #11080 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] carlvinhust2012 opened a new pull request, #11082: [regression] (array-type) add the cases for csv/orc/parquet file format
carlvinhust2012 opened a new pull request, #11082: URL: https://github.com/apache/doris/pull/11082 # Proposed changes 1. this pr is used to add the cases for csv/orc/parquet file format. Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No) 3. Has document been added or modified: (No) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #11076: [chore](FE)add license header check in fe's checkstyle
morrySnow commented on PR #11076: URL: https://github.com/apache/doris/pull/11076#issuecomment-1191339178 @924060929 PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: style fix
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 0e4c3e0f40 style fix
0e4c3e0f40 is described below
commit 0e4c3e0f40f4dbe20b559ff53b11d84dd04d6fa6
Author: jiafeng.zhang
AuthorDate: Thu Jul 21 18:53:06 2022 +0800
style fix
style fix
---
.../current/developer-guide/regression-testing.md| 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/regression-testing.md
b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/regression-testing.md
index d92b25c148..984115d7a3 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/regression-testing.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/developer-guide/regression-testing.md
@@ -136,7 +136,7 @@ sql action用于提交sql并获取结果，如果查询失败则会抛出异常
参数如下
- String sql: 输入的sql字符串
-- return List>: 查询结果，如果是DDL/DML，则返回一行一列，唯一的值是updateRowCount
+- `return List>`: 查询结果，如果是DDL/DML，则返回一行一列，唯一的值是updateRowCount
下面的样例代码存放于`${DORIS_HOME}/regression-test/suites/demo/sql_action.groovy`:
```groovy
@@ -269,13 +269,13 @@ test action可以使用更复杂的校验规则来测试，比如验证行数、
可用参数
- String sql: 输入的sql字符串
-- List> result: 提供一个List对象，用于比较真实查询结果与List对象是否相等
-- Iterator resultIterator: 提供一个Iterator对象，用于比较真实查询结果与Iterator是否相等
+- `List> result`: 提供一个List对象，用于比较真实查询结果与List对象是否相等
+- `Iterator resultIterator`: 提供一个Iterator对象，用于比较真实查询结果与Iterator是否相等
- String resultFile:
提供一个文件Uri(可以是本地文件相对路径，或http(s)路径)，用于比较真实查询结果与http响应流是否相等，格式与.out文件格式类似，但没有块头和注释
- String exception: 校验抛出的异常是否包含某些字符串
- long rowNum: 验证结果行数
- long time: 验证执行时间是否小于这个值，单位是毫秒
-- Closure>, Throwable, Long, Long> check:
自定义回调校验，可传入结果、异常、时间。存在回调函数时，其他校验方式会失效。
+- `Closure>, Throwable, Long, Long> check`:
自定义回调校验，可传入结果、异常、时间。存在回调函数时，其他校验方式会失效。
下面的样例代码存放于`${DORIS_HOME}/regression-test/suites/demo/test_action.groovy`:
```groovy
@@ -373,8 +373,8 @@ explain action用来校验explain返回的字符串是否包含某些字符串
- String sql: 查询的sql，需要去掉sql中的explain
- String contains: 校验explain是否包含某些字符串，可多次调用校验同时多个结果
- String notContains: 校验explain是否不含某些字符串，可多次调用校验同时多个结果
-- Closure check: 自定义校验回调函数，可以获取返回的字符串，存在校验函数时，其他校验方式会失效
-- Closure check: 自定义校验回调函数，可以额外获取异常和时间
+- `Closure check`: 自定义校验回调函数，可以获取返回的字符串，存在校验函数时，其他校验方式会失效
+- `Closure check`: 自定义校验回调函数，可以额外获取异常和时间
下面的样例代码存放于`${DORIS_HOME}/regression-test/suites/demo/explain_action.groovy`:
```groovy
@@ -419,12 +419,12 @@ streamLoad action用于导入数据
- String db: db，默认值为regression-conf.groovy中的defaultDb
- String table: 表名
- String file: 要导入的文件路径，可以写data目录下的相对路径，或者写http url来导入网络文件
-- Iterator> inputIterator: 要导入的迭代器
+- `Iterator> inputIterator`: 要导入的迭代器
- String inputText: 要导入的文本, 较为少用
- InputStream inputStream: 要导入的字节流，较为少用
- long time: 验证执行时间是否小于这个值，单位是毫秒
- void set(String key, String value): 设置stream
load的http请求的header，如label、columnSeparator
-- Closure check:
自定义校验回调函数，可以获取返回结果、异常和超时时间。当存在回调函数时，其他校验项会失效。
+- `Closure check`:
自定义校验回调函数，可以获取返回结果、异常和超时时间。当存在回调函数时，其他校验项会失效。
下面的样例代码存放于`${DORIS_HOME}/regression-test/suites/demo/streamLoad_action.groovy`:
```groovy
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] adonis0147 opened a new pull request, #11083: [enhancement](workflow) Add a workflow for Clang check
adonis0147 opened a new pull request, #11083: URL: https://github.com/apache/doris/pull/11083 # Proposed changes ~~Issue Number: close #xxx~~ ## Problem Summary: Currently, we should support building from source by Clang, but we lack related checks. It is __*inconvenient*__ to inspect and fix compilation errors reported by Clang manually. This pr adds a workflow for Clang check. ### REMARK For the sake of speed, the workflow copy the prebuilt third-party packages from [docker image](https://hub.docker.com/r/apache/doris) directly rather than build them from source. We must update the image if we modify the third-party packages. ## Checklist(Required) 1. Does it affect the original behavior: No 2. Has unit tests been added: No Need 3. Has document been added or modified: No Need 4. Does it need to update dependencies: No 5. Are there any changes that cannot be rolled back: No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #11078: [doc]Stream load doc fix
github-actions[bot] commented on PR #11078: URL: https://github.com/apache/doris/pull/11078#issuecomment-1191355467 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #11078: [doc]Stream load doc fix
github-actions[bot] commented on PR #11078: URL: https://github.com/apache/doris/pull/11078#issuecomment-1191355500 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataalive commented on issue #10561: [Bug] 根据doris安装文档镜像编译，一直都报这个问题，版本都是最新的
dataalive commented on issue #10561: URL: https://github.com/apache/doris/issues/10561#issuecomment-1191365545 > 可以尝试下1.1对应的镜像看看，我尝试了下没遇到这个问题。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] mrhhsg opened a new pull request, #11084: [improvement][agg]Process aggregated results in the vectorized way
mrhhsg opened a new pull request, #11084: URL: https://github.com/apache/doris/pull/11084 # Proposed changes Issue Number: close #xxx ## Problem Summary: Aggregated results are currently processed row by row, this is an inefficient way of accessing memory and will call virtual functions too many times. Here test on clickbench(https://github.com/ClickHouse/ClickBench) with SQL: ```sql SELECT UserID, COUNT(*) FROM hits GROUP BY UserID ORDER BY COUNT(*) DESC LIMIT 10; ``` master: https://user-images.githubusercontent.com/1179834/180204059-997f16f1-9b51-4351-9b33-bbd8409b6b7a.png";> agg_result_vec: https://user-images.githubusercontent.com/1179834/180204114-70655f00-5b38-4d6d-9069-fefaac541026.png";> ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hf200012 commented on issue #11071: [Bug]
hf200012 commented on issue #11071: URL: https://github.com/apache/doris/issues/11071#issuecomment-1191379460 你需要服务器之间做时钟同步 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] caiconghui commented on a diff in pull request #10711: [Feature] Add interface to check tablet segment lost
caiconghui commented on code in PR #10711:
URL: https://github.com/apache/doris/pull/10711#discussion_r926577728
##
be/src/olap/tablet_manager.cpp:
##
@@ -1333,4 +1333,30 @@ void
TabletManager::get_all_tablets_storage_format(TCheckStorageFormatResult* re
result->__isset.v2_tablets = true;
}
+std::set TabletManager::check_all_tablet_segment(bool repair) {
+std::set bad_tablets;
+for (const auto& tablets_shard : _tablets_shards) {
+std::shared_lock rdlock(tablets_shard.lock);
Review Comment:
read lock but modify tablet meta?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] caiconghui commented on a diff in pull request #10711: [Feature] Add interface to check tablet segment lost
caiconghui commented on code in PR #10711:
URL: https://github.com/apache/doris/pull/10711#discussion_r926578048
##
be/src/olap/rowset/beta_rowset.cpp:
##
@@ -228,4 +228,21 @@ bool BetaRowset::check_file_exist() {
return true;
}
+bool BetaRowset::check_current_rowset_segment() {
+auto fs = _rowset_meta->fs();
+if (!fs) {
+return false;
+}
+for (int seg_id = 0; seg_id < num_segments(); ++seg_id) {
+auto seg_path = segment_file_path(seg_id);
+std::shared_ptr segment;
+auto s = segment_v2::Segment::open(fs, seg_path, seg_id, _schema,
&segment);
+if (!s.ok()) {
+LOG(WARNING) << "Beta rowset segment Check. segment not exist.
file=" << seg_path;
Review Comment:
not ok always mean file not exist?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #10880: [regression] Add tpcds sf1 test under unique table with zstd.
hello-stephen commented on PR #10880: URL: https://github.com/apache/doris/pull/10880#issuecomment-1191388594 please ignore q78 q30, since they can't pass now(which later will be fix), and update q99 q62 q19's out file to correct answer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris-flink-connector] hf200012 merged pull request #49: [Feature] Support Flink 1.15
hf200012 merged PR #49: URL: https://github.com/apache/doris-flink-connector/pull/49 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-flink-connector] branch master updated: [Feature] Support Flink 1.15 (#49)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-flink-connector.git
The following commit(s) were added to refs/heads/master by this push:
new 7171929 [Feature] Support Flink 1.15 (#49)
7171929 is described below
commit 7171929fb4ab64d96e75ad3f58752e64a1a75f11
Author: wudi <676366...@qq.com>
AuthorDate: Thu Jul 21 19:54:36 2022 +0800
[Feature] Support Flink 1.15 (#49)
* flink 1.15 support
---
.github/workflows/build-extension.yml | 4 +-
flink-doris-connector/pom.xml | 134 ++--
.../apache/doris/flink/backend/BackendClient.java | 9 +-
.../doris/flink/cfg/DorisExecutionOptions.java | 2 +-
.../flink/datastream/DorisSourceFunction.java | 14 +-
.../flink/exception/ConnectedFailedException.java | 2 +-
.../flink/exception/DorisInternalException.java| 2 +-
.../exception/ShouldNeverHappenException.java | 2 +-
.../org/apache/doris/flink/rest/RestService.java | 12 +-
.../apache/doris/flink/serialization/RowBatch.java | 7 +-
.../source/reader/DorisSourceSplitReader.java | 25 ++-
.../flink/source/reader/DorisValueReader.java | 229 +
.../flink/source/split/DorisSplitRecords.java | 18 +-
.../flink/table/DorisDynamicTableFactory.java | 2 +-
.../doris/flink/table/DorisRowDataInputFormat.java | 33 +--
.../doris/flink/datastream/ScalaValueReader.scala | 222
.../apache/doris/flink/DorisSourceSinkExample.java | 1 -
.../doris/flink/serialization/TestRowBatch.java| 17 +-
18 files changed, 310 insertions(+), 425 deletions(-)
diff --git a/.github/workflows/build-extension.yml
b/.github/workflows/build-extension.yml
index b2ad6af..bc0c47c 100644
--- a/.github/workflows/build-extension.yml
+++ b/.github/workflows/build-extension.yml
@@ -46,7 +46,7 @@ jobs:
touch custom_env.sh
echo 'export THRIFT_BIN=/usr/bin/thrift' >> custom_env.sh
-- name: Build flink connector 1.14
+- name: Build flink connector 1.15
run: |
-cd flink-doris-connector/ && /bin/bash build.sh --flink 1.14.3 --scala
2.12
+cd flink-doris-connector/ && /bin/bash build.sh --flink 1.15.0
diff --git a/flink-doris-connector/pom.xml b/flink-doris-connector/pom.xml
index ddcd3de..27b6df1 100644
--- a/flink-doris-connector/pom.xml
+++ b/flink-doris-connector/pom.xml
@@ -26,7 +26,7 @@ under the License.
23
org.apache.doris
-
flink-doris-connector-${flink.minor.version}_${scala.version}
+flink-doris-connector-${flink.minor.version}
1.0.0-SNAPSHOT
Flink Doris Connector
https://doris.apache.org/
@@ -62,7 +62,6 @@ under the License.
-${env.scala.version}
${env.flink.version}
${env.flink.minor.version}
0.13.0
@@ -111,16 +110,7 @@ under the License.
flink.version
-1.14.3
-
-
-true
-
-
-
-scala.version
-
-2.12
+1.15.0
true
@@ -147,26 +137,31 @@ under the License.
org.apache.flink
-flink-java
+flink-clients
+${flink.version}
+provided
+
+
+org.apache.flink
+flink-table-planner-loader
${flink.version}
provided
org.apache.flink
-flink-streaming-java_${scala.version}
+flink-connector-base
${flink.version}
provided
org.apache.flink
-flink-clients_${scala.version}
+flink-table-api-java-bridge
${flink.version}
provided
-
org.apache.flink
-flink-table-planner_${scala.version}
+flink-table-runtime
${flink.version}
provided
@@ -235,6 +230,7 @@ under the License.
netty-common
4.1.77.Final
+
com.fasterxml.jackson.core
jackson-annotations
@@ -250,35 +246,6 @@ under the License.
jackson-databind
2.13.3
-
-org.apache.logging.log4j
-log4j-web
-${log4j2.version}
-
-
-
-org.apache.logging.log4j
-log4j-api
-${log4j2.version}
-
-
-
-org.apache.logging.log4j
-log4j-core
-${log4j2.version}
-
-
-
-org.apache.logging.log4j
-log4j-slf4j-impl
-${log4j2.version}
-
-
-
-
[GitHub] [doris] caiconghui commented on a diff in pull request #11078: [doc]Stream load doc fix
caiconghui commented on code in PR #11078: URL: https://github.com/apache/doris/pull/11078#discussion_r926523324 ## docs/zh-CN/docs/sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md: ## @@ -50,6 +50,7 @@ SHOW STREAM LOAD 说明： +1. 默认 BE 是不记录 Stream Load 的记录，如果你要查看需要再 BE 上启用记录，`enable_stream_load_record=true`，这里需要重启BE Review Comment: ```suggestion 1. 默认 BE 是不记录 Stream Load 的记录，如果你要查看需要在 BE 上启用记录，`enable_stream_load_record=true`，这里不需要重启BE ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 65fed6a8a6 fix 65fed6a8a6 is described below commit 65fed6a8a681d59cf9dcd67a4d0cb1ba0b4729a2 Author: jiafeng.zhang AuthorDate: Thu Jul 21 20:14:12 2022 +0800 fix fix --- .../sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md | 2 +- .../sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md | 9 + 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/docs/sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md b/docs/sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md index 3ac7ba14ee..d108a4b879 100644 --- a/docs/sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md +++ b/docs/sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md @@ -50,7 +50,7 @@ SHOW STREAM LOAD illustrate: -1. The default is to not record Stream Load records. If you want to view the records that need to be enabled on BE, `enable_stream_load_record=true`, you need to restart BE here +1. BE does not record Stream Load records. If you want to view the need to re-enable the configuration `enable_stream_load_record=true`, you need to restart BE here 1. If db_name is not specified, the current default db is used 2. If LABEL LIKE is used, it will match the tasks whose label of the Stream Load task contains label_matcher 3. If LABEL = is used, it will match the specified label exactly diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md index fd4fbff163..87da5910b3 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Show-Statements/SHOW-STREAM-LOAD.md @@ -50,6 +50,7 @@ SHOW STREAM LOAD 说明： +1. 默认 BE 是不记录 Stream Load 的记录，如果你要查看需要再 BE 上启用配置 `enable_stream_load_record=true`，这里需要重启BE 1. 如果不指定 db_name，使用当前默认db 2. 如果使用 LABEL LIKE，则会匹配Stream Load任务的 label 包含 label_matcher 的任务 3. 如果使用 LABEL = ，则精确匹配指定的 label @@ -67,25 +68,25 @@ SHOW STREAM LOAD ``` 2. 展示指定 db 的Stream Load任务，label 中包含字符串 "2014_01_02"，展示最老的10个 - + ```sql SHOW STREAM LOAD FROM example_db WHERE LABEL LIKE "2014_01_02" LIMIT 10; ``` 2. 展示指定 db 的Stream Load任务，指定 label 为 "load_example_db_20140102" - + ```sql SHOW STREAM LOAD FROM example_db WHERE LABEL = "load_example_db_20140102"; ``` 2. 展示指定 db 的Stream Load任务，指定 status 为 "success", 并按 StartTime 降序排序 - + ```sql SHOW STREAM LOAD FROM example_db WHERE STATUS = "success" ORDER BY StartTime DESC; ``` 2. 展示指定 db 的导入任务 并按 StartTime 降序排序,并从偏移量5开始显示10条查询结果 - + ```sql SHOW STREAM LOAD FROM example_db ORDER BY StartTime DESC limit 5,10; SHOW STREAM LOAD FROM example_db ORDER BY StartTime DESC limit 10 offset 5; - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #11085: [feature-wip] support datetimev2
Gabriel39 opened a new pull request, #11085: URL: https://github.com/apache/doris/pull/11085 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #11084: [improvement][agg]Process aggregated results in the vectorized way
github-actions[bot] commented on PR #11084: URL: https://github.com/apache/doris/pull/11084#issuecomment-1191434801 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #11084: [improvement][agg]Process aggregated results in the vectorized way
github-actions[bot] commented on PR #11084: URL: https://github.com/apache/doris/pull/11084#issuecomment-1191434864 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] caiconghui opened a new pull request, #11086: [doc](tablet-health) modify content about tablet state
caiconghui opened a new pull request, #11086: URL: https://github.com/apache/doris/pull/11086 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #11086: [doc](tablet-health) modify content about tablet state
github-actions[bot] commented on PR #11086: URL: https://github.com/apache/doris/pull/11086#issuecomment-1191442433 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #11086: [doc](tablet-health) modify content about tablet state
github-actions[bot] commented on PR #11086: URL: https://github.com/apache/doris/pull/11086#issuecomment-1191442466 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] shenxingwuying opened a new issue, #11087: [DOC] README.md at least 2 url is not working
shenxingwuying opened a new issue, #11087: URL: https://github.com/apache/doris/issues/11087 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version master ### What's Wrong? Compile and install See [Compilation](https://doris.apache.org/docs/install/source-install/compilation-with-ldb-toolchain.html) Getting start See [Basic Usage](https://doris.apache.org/docs/data-table/basic-usage.html) The 2 url is not working. eg:  ### What You Expected? Fix it. ### How to Reproduce? _No response_ ### Anything Else? 1 ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix Chinese translation bug (#15)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new c931efe615 fix Chinese translation bug (#15)
c931efe615 is described below
commit c931efe61579ea173acb63efa8c76fb96e4f93f5
Author: song7788q
AuthorDate: Thu Jul 21 21:06:03 2022 +0800
fix Chinese translation bug (#15)
---
i18n/zh-CN/code.json | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/i18n/zh-CN/code.json b/i18n/zh-CN/code.json
index 74991a5427..0fd554ddae 100644
--- a/i18n/zh-CN/code.json
+++ b/i18n/zh-CN/code.json
@@ -509,5 +509,8 @@
},
"sotry.summary.xiaomi": {
"message": "Apache Doris 在小米得到了广泛的使用，目前已经服务了数十个业务，并形成了一套以 Apache Doris
为核心的数据生态。"
+ },
+ "navbar.download": {
+"message": "下载"
}
-}
\ No newline at end of file
+}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] weizuo93 commented on a diff in pull request #10711: [Feature] Add interface to check tablet segment lost
weizuo93 commented on code in PR #10711:
URL: https://github.com/apache/doris/pull/10711#discussion_r926657603
##
be/src/olap/tablet_manager.cpp:
##
@@ -1333,4 +1333,30 @@ void
TabletManager::get_all_tablets_storage_format(TCheckStorageFormatResult* re
result->__isset.v2_tablets = true;
}
+std::set TabletManager::check_all_tablet_segment(bool repair) {
+std::set bad_tablets;
+for (const auto& tablets_shard : _tablets_shards) {
+std::shared_lock rdlock(tablets_shard.lock);
Review Comment:
> read lock but modify tablet meta?
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #7459: add iot-function to doris
jackwener commented on PR #7459: URL: https://github.com/apache/doris/pull/7459#issuecomment-1191468564 This PR has not been updated for a long time. There could be many reasons. If it's caused by there are no maintainers to push or ignore this PR, I'm sorry. If you contact maintainer or want to continue this PR, feel free to @me or @other maintainers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener closed pull request #7459: add iot-function to doris
jackwener closed pull request #7459: add iot-function to doris URL: https://github.com/apache/doris/pull/7459 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] weizuo93 commented on a diff in pull request #10711: [Feature] Add interface to check tablet segment lost
weizuo93 commented on code in PR #10711:
URL: https://github.com/apache/doris/pull/10711#discussion_r926658034
##
be/src/olap/rowset/beta_rowset.cpp:
##
@@ -228,4 +228,21 @@ bool BetaRowset::check_file_exist() {
return true;
}
+bool BetaRowset::check_current_rowset_segment() {
+auto fs = _rowset_meta->fs();
+if (!fs) {
+return false;
+}
+for (int seg_id = 0; seg_id < num_segments(); ++seg_id) {
+auto seg_path = segment_file_path(seg_id);
+std::shared_ptr segment;
+auto s = segment_v2::Segment::open(fs, seg_path, seg_id, _schema,
&segment);
+if (!s.ok()) {
+LOG(WARNING) << "Beta rowset segment Check. segment not exist.
file=" << seg_path;
Review Comment:
> not ok always mean file not exist?
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener closed pull request #6874: [Doris On Es] Fix assertion failure when docvalue fields is gone
jackwener closed pull request #6874: [Doris On Es] Fix assertion failure when docvalue fields is gone URL: https://github.com/apache/doris/pull/6874 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #6874: [Doris On Es] Fix assertion failure when docvalue fields is gone
jackwener commented on PR #6874: URL: https://github.com/apache/doris/pull/6874#issuecomment-1191470446 This PR has not been updated for a long time. There could be many reasons. If it's caused by there are no maintainers to push or ignore this PR, I'm sorry. If you contact maintainer or want to continue this PR, feel free to @me or @other maintainers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #6935: Fix the plain page size is too large when writing a large batch
jackwener commented on PR #6935: URL: https://github.com/apache/doris/pull/6935#issuecomment-1191470859 This PR has not been updated for a long time. There could be many reasons. If it's caused by there are no maintainers to push or ignore this PR, I'm sorry. If you contact maintainer or want to continue this PR, feel free to @me or @other maintainers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener closed pull request #6935: Fix the plain page size is too large when writing a large batch
jackwener closed pull request #6935: Fix the plain page size is too large when writing a large batch URL: https://github.com/apache/doris/pull/6935 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener closed pull request #6938: fix column length exception
jackwener closed pull request #6938: fix column length exception URL: https://github.com/apache/doris/pull/6938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #6938: fix column length exception
jackwener commented on PR #6938: URL: https://github.com/apache/doris/pull/6938#issuecomment-1191471070 This PR has not been updated for a long time. There could be many reasons. If it's caused by there are no maintainers to push or ignore this PR, I'm sorry. If you contact maintainer or want to continue this PR, feel free to @me or @other maintainers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener closed pull request #6884: [BUG] fix cancel all txns when be is down bug
jackwener closed pull request #6884: [BUG] fix cancel all txns when be is down bug URL: https://github.com/apache/doris/pull/6884 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #6884: [BUG] fix cancel all txns when be is down bug
jackwener commented on PR #6884: URL: https://github.com/apache/doris/pull/6884#issuecomment-1191471537 This PR has not been updated for a long time. There could be many reasons. If it's caused by there are no maintainers to push or ignore this PR, I'm sorry. If you contact maintainer or want to continue this PR, feel free to @me or @other maintainers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #6990: Doris on es 聚合下推
jackwener commented on PR #6990: URL: https://github.com/apache/doris/pull/6990#issuecomment-1191471899 This PR has not been updated for a long time. There could be many reasons. If it's caused by there are no maintainers to push or ignore this PR, I'm sorry. If you contact maintainer or want to continue this PR, feel free to @me or @other maintainers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener closed pull request #6990: Doris on es 聚合下推
jackwener closed pull request #6990: Doris on es 聚合下推 URL: https://github.com/apache/doris/pull/6990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xinyiZzz merged pull request #11052: check vm.max_map_count before starting
xinyiZzz merged PR #11052: URL: https://github.com/apache/doris/pull/11052 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Enhancement] check vm.max_map_count before starting (#11052)
This is an automated email from the ASF dual-hosted git repository.
zouxinyi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 08ebef2992 [Enhancement] check vm.max_map_count before starting
(#11052)
08ebef2992 is described below
commit 08ebef2992a5afeef7587392d44093045e31ea6a
Author: Yongqiang YANG <98214048+dataroar...@users.noreply.github.com>
AuthorDate: Thu Jul 21 21:16:48 2022 +0800
[Enhancement] check vm.max_map_count before starting (#11052)
When vectorized engine is enabled, doris uses much more vmas than before,
and it leads to core dump due to memory allocation failure.
---
bin/start_be.sh | 6 ++
1 file changed, 6 insertions(+)
diff --git a/bin/start_be.sh b/bin/start_be.sh
index 551a12e5c0..4944230252 100755
--- a/bin/start_be.sh
+++ b/bin/start_be.sh
@@ -58,6 +58,12 @@ export DORIS_HOME=$(
pwd
)
+MAX_MAP_COUNT=`sysctl -n vm.max_map_count`
+if [ $MAX_MAP_COUNT -lt 200 ]; then
+echo "Please set vm.max_map_count to be 200. sysctl -w
vm.max_map_count=200"
+exit 1
+fi
+
# add libs to CLASSPATH
for f in $DORIS_HOME/lib/*.jar; do
if [ ! -n "${DORIS_JNI_CLASSPATH_PARAMETER}" ]; then
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #7301: [Function] Add Function date_trunc() doc and FE test case
jackwener commented on PR #7301: URL: https://github.com/apache/doris/pull/7301#issuecomment-1191473672 This PR has not been updated for a long time. There could be many reasons. If it's caused by there are no maintainers to push or ignore this PR, I'm sorry. If you contact maintainer or want to continue this PR, feel free to @me or @other maintainers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener closed pull request #7301: [Function] Add Function date_trunc() doc and FE test case
jackwener closed pull request #7301: [Function] Add Function date_trunc() doc and FE test case URL: https://github.com/apache/doris/pull/7301 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #7121: [Feature] Add an OrToIn expr rewrite rule.
jackwener commented on PR #7121: URL: https://github.com/apache/doris/pull/7121#issuecomment-1191474275 This PR has not been updated for a long time. There could be many reasons. If it's caused by there are no maintainers to push or ignore this PR, I'm sorry. If you contact maintainer or want to continue this PR, feel free to @me or @other maintainers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener closed pull request #7121: [Feature] Add an OrToIn expr rewrite rule.
jackwener closed pull request #7121: [Feature] Add an OrToIn expr rewrite rule. URL: https://github.com/apache/doris/pull/7121 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] shenxingwuying commented on issue #11087: [DOC] README.md at least 2 url is not working
shenxingwuying commented on issue #11087: URL: https://github.com/apache/doris/issues/11087#issuecomment-1191477130 If you support the correct url, I can fix it as my first commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataalive commented on issue #11087: [DOC] README.md at least 2 url is not working
dataalive commented on issue #11087: URL: https://github.com/apache/doris/issues/11087#issuecomment-1191481124 > If you support the correct url, I can fix it as my first commit. correct url should be : https://doris.apache.org/docs/install/source-install/compilation-with-ldb-toolchain https://doris.apache.org/docs/data-table/basic-usage -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman commented on a diff in pull request #11081: [improvement]Remove the website build framework from the docs directory
morningman commented on code in PR #11081: URL: https://github.com/apache/doris/pull/11081#discussion_r926670797 ## .github/workflows/build-extension.yml: ## @@ -54,8 +54,3 @@ jobs: - name: Build broker run: | cd fs_brokers/apache_hdfs_broker/ && /bin/bash build.sh - -- name: Build docs - run: | -cd docs && /bin/bash build_help_zip.sh && export NODE_OPTIONS="--max-old-space-size=8192" && npm install && npm run build Review Comment: we still need `build_help_zip.sh` ## docs/.vuepress/sidebar/en/docs.js: ## @@ -1,1014 +0,0 @@ -/* Review Comment: I think we still need this sidebar? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener opened a new pull request, #11088: [enhancement](community): add ci for close outdated PR.
jackwener opened a new pull request, #11088: URL: https://github.com/apache/doris/pull/11088 # Proposed changes Issue Number: close #xxx ## Problem Summary: add ci for close outdated PR. ## Checklist(Required) 1. Does it affect the original behavior: Yes 2. Has unit tests been added: No Need 3. Has document been added or modified: No Need 4. Does it need to update dependencies: No 5. Are there any changes that cannot be rolled back: /No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman commented on pull request #11078: [doc]Stream load doc fix
morningman commented on PR #11078: URL: https://github.com/apache/doris/pull/11078#issuecomment-1191496381 After the website upgraded, we should find a way to update the document on website. Currently, it will not update automatically. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #11077: [Feature](Nereids) Add an annotation @Developing
github-actions[bot] commented on PR #11077: URL: https://github.com/apache/doris/pull/11077#issuecomment-1191501007 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] shenxingwuying commented on issue #11087: [DOC] README.md at least 2 url is not working
shenxingwuying commented on issue #11087: URL: https://github.com/apache/doris/issues/11087#issuecomment-1191523246 > > If you support the correct url, I can fix it as my first commit. > > correct url should be : https://doris.apache.org/docs/install/source-install/compilation-with-ldb-toolchain https://doris.apache.org/docs/data-table/basic-usage Get. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataalive commented on issue #10331: hive-2.1.1-cdh6.1.1 不支持吗？
dataalive commented on issue #10331: URL: https://github.com/apache/doris/issues/10331#issuecomment-1191533413 we support hive 2.3.7+ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataalive commented on issue #11062: [Bug] exec_mem_limit option can not restrict the memory strictly
dataalive commented on issue #11062: URL: https://github.com/apache/doris/issues/11062#issuecomment-1191536472 this problem will be fixed with mem-tracker in the following version -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] shenxingwuying opened a new pull request, #11091: [doc] Fix two urls at README.md
shenxingwuying opened a new pull request, #11091: URL: https://github.com/apache/doris/pull/11091 # Proposed changes https://github.com/apache/doris/issues/11087 Issue Number: close #11087 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
