[GitHub] [doris] weizhengte commented on pull request #14910: [feature-wip](Nereids)(histogram) add aggregate function histogram and collect histogram statistics
weizhengte commented on PR #14910: URL: https://github.com/apache/doris/pull/14910#issuecomment-1363715177 enhancement 👉🏻 https://github.com/apache/doris/pull/15317 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15212: [Feature](Materialized-View) support advanced Materialized-View
github-actions[bot] commented on PR #15212: URL: https://github.com/apache/doris/pull/15212#issuecomment-1363716289 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly opened a new pull request, #15318: [fix](nereids) group_bit_xxx signature update
englefly opened a new pull request, #15318: URL: https://github.com/apache/doris/pull/15318 # Proposed changes fix result error in regression test: query_p0/sql_functions/aggregate_functions/test_aggregate_bit Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #15243: [feature](nereids) Support query on specific partitions
morrySnow commented on code in PR #15243:
URL: https://github.com/apache/doris/pull/15243#discussion_r1056117739
##
fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/UnboundRelation.java:
##
@@ -44,34 +43,21 @@
*/
public class UnboundRelation extends LogicalRelation implements Unbound {
private final List nameParts;
+private final List partNames;
public UnboundRelation(RelationId id, List nameParts) {
-this(id, nameParts, Optional.empty(), Optional.empty());
+this(id, nameParts, Optional.empty(), Optional.empty(),
Collections.emptyList());
}
-public UnboundRelation(RelationId id, List nameParts,
Optional groupExpression,
-Optional logicalProperties) {
-super(id, PlanType.LOGICAL_UNBOUND_RELATION, groupExpression,
logicalProperties);
-this.nameParts = nameParts;
-}
-
-public UnboundRelation(RelationId id, TableIdentifier identifier) {
-this(id, identifier, Optional.empty(), Optional.empty());
+public UnboundRelation(RelationId id, List nameParts, List
partNames) {
+this(id, nameParts, Optional.empty(), Optional.empty(), partNames);
}
-/**
- * Constructor for UnboundRelation.
- *
- * @param identifier relation identifier
- */
-public UnboundRelation(RelationId id, TableIdentifier identifier,
Optional groupExpression,
-Optional logicalProperties) {
+public UnboundRelation(RelationId id, List nameParts,
Optional groupExpression,
+Optional logicalProperties, List
partNames) {
super(id, PlanType.LOGICAL_UNBOUND_RELATION, groupExpression,
logicalProperties);
-this.nameParts = Lists.newArrayList();
-if (identifier.getDatabaseName().isPresent()) {
-nameParts.add(identifier.getDatabaseName().get());
-}
-nameParts.add(identifier.getTableName());
+this.nameParts = nameParts;
+this.partNames = partNames;
Review Comment:
```suggestion
this.partNames =
ImmutableList.copyOf(Objects.requireNotNull(partNames, "partNames should not
null"));
```
##
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##
@@ -410,7 +412,18 @@ private LogicalPlan withCheckPolicy(LogicalPlan plan) {
@Override
public LogicalPlan visitTableName(TableNameContext ctx) {
List tableId =
visitMultipartIdentifier(ctx.multipartIdentifier());
-LogicalPlan checkedRelation = withCheckPolicy(new
UnboundRelation(RelationUtil.newRelationId(), tableId));
+List partitionNames = new ArrayList<>();
+if (ctx.specified_partition() != null) {
+if (ctx.specified_partition().identifier() != null) {
+
partitionNames.add(ctx.specified_partition().identifier().getText());
+} else {
+partitionNames.addAll(Arrays
+
.stream(ctx.specified_partition().identifierList().identifierSeq().getText().split(","))
+.collect(Collectors.toList()));
Review Comment:
```suggestion
partitionNames.addAll(visitIdentifierList(ctx.specified_partition().identifierList()));
```
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalOlapScan.java:
##
@@ -126,6 +135,7 @@ public LogicalOlapScan(RelationId id, Table table,
List qualifier,
this.selectedIndexId = selectedIndexId <= 0 ?
getTable().getBaseIndexId() : selectedIndexId;
this.indexSelected = indexSelected;
this.preAggStatus = preAggStatus;
+this.manuallySpecifiedPartitions = partitions;
Review Comment:
```suggestion
this.manuallySpecifiedPartitions = ImmutableList.copyOf(partitions);
```
##
fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4:
##
@@ -210,7 +210,7 @@ identifierSeq
;
relationPrimary
-: multipartIdentifier tableAlias
#tableName
+: multipartIdentifier specified_partition? tableAlias
#tableName
Review Comment:
```suggestion
: multipartIdentifier specifiedPartition? tableAlias
#tableName
```
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalOlapScan.java:
##
@@ -101,13 +103,19 @@ public LogicalOlapScan(RelationId id, OlapTable table,
List qualifier) {
this(id, table, qualifier, Optional.empty(), Optional.empty(),
table.getPartitionIds(), false,
ImmutableList.of(), false,
--1, false, PreAggStatus.on());
+-1, false, PreAggStatus.on(), Collections.emptyList());
+}
+
+public LogicalOlapScan(RelationId id, OlapTable table, List
qualifier, List specifiedPartitions) {
+this(id, table, qualifier, Optional.empty(), Optional.empty(),
+spec
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #15260: [enhancement](checksum) use vectorized engine in checksum
github-actions[bot] commented on code in PR #15260:
URL: https://github.com/apache/doris/pull/15260#discussion_r1056132310
##
be/src/olap/task/engine_checksum_task.cpp:
##
@@ -38,7 +39,52 @@ Status EngineChecksumTask::_compute_checksum() {
LOG(INFO) << "begin to process compute checksum."
<< "tablet_id=" << _tablet_id << ", schema_hash=" << _schema_hash
<< ", version=" << _version;
-return Status::InternalError("Not implemented yet");
+
+if (_checksum == nullptr) {
Review Comment:
warning: use of undeclared identifier '_checksum' [clang-diagnostic-error]
```cpp
if (_checksum == nullptr) {
^
```
##
be/src/olap/task/engine_checksum_task.cpp:
##
@@ -38,7 +39,52 @@
LOG(INFO) << "begin to process compute checksum."
<< "tablet_id=" << _tablet_id << ", schema_hash=" << _schema_hash
<< ", version=" << _version;
-return Status::InternalError("Not implemented yet");
+
+if (_checksum == nullptr) {
+return Status::InvalidArgument("invalid checksum which is nullptr");
+}
+
+TabletSharedPtr tablet =
StorageEngine::instance()->tablet_manager()->get_tablet(_tablet_id);
+if (nullptr == tablet) {
+return Status::InternalError("could not find tablet {}", _tablet_id);
+}
+
+std::vector input_rowsets;
+Version version(0, _version);
+Status acquire_reader_st = tablet->capture_consistent_rowsets(version,
&input_rowsets);
+if (acquire_reader_st != Status::OK()) {
+LOG(WARNING) << "fail to captute consistent rowsets. tablet=" <<
tablet->full_name()
+ << "res=" << acquire_reader_st;
+return acquire_reader_st;
+}
+vectorized::BlockReader reader;
+TabletReader::ReaderParams reader_params;
+vectorized::Block block;
+RETURN_NOT_OK(TabletReader::init_reader_params_and_create_block(
+tablet, READER_CHECKSUM, input_rowsets, &reader_params, &block))
+
+auto res = reader.init(reader_params);
+if (!res.ok()) {
+LOG(WARNING) << "initiate reader fail. res = " << res;
+return res;
+}
+
+bool eof = false;
+SipHash block_hash;
+uint64_t rows = 0;
+while (!eof) {
+RETURN_IF_ERROR(reader.next_block_with_aggregation(&block, nullptr,
nullptr, &eof));
+rows += block.rows();
+
+block.update_hash(block_hash);
+block.clear_column_data();
+}
+uint64_t checksum64 = block_hash.get64();
+*_checksum = (checksum64 >> 32) ^ (checksum64 & 0x);
+
+LOG(INFO) << "success to finish compute checksum. tablet_id = " <<
_tablet_id
+ << ", rows = " << rows << ", checksum=" << *_checksum;
Review Comment:
warning: use of undeclared identifier '_checksum' [clang-diagnostic-error]
```cpp
<< ", rows = " << rows << ", checksum=" << *_checksum;
^
```
##
be/src/olap/task/engine_checksum_task.cpp:
##
@@ -38,7 +39,52 @@
LOG(INFO) << "begin to process compute checksum."
<< "tablet_id=" << _tablet_id << ", schema_hash=" << _schema_hash
<< ", version=" << _version;
-return Status::InternalError("Not implemented yet");
+
+if (_checksum == nullptr) {
+return Status::InvalidArgument("invalid checksum which is nullptr");
+}
+
+TabletSharedPtr tablet =
StorageEngine::instance()->tablet_manager()->get_tablet(_tablet_id);
+if (nullptr == tablet) {
+return Status::InternalError("could not find tablet {}", _tablet_id);
+}
+
+std::vector input_rowsets;
+Version version(0, _version);
+Status acquire_reader_st = tablet->capture_consistent_rowsets(version,
&input_rowsets);
+if (acquire_reader_st != Status::OK()) {
+LOG(WARNING) << "fail to captute consistent rowsets. tablet=" <<
tablet->full_name()
+ << "res=" << acquire_reader_st;
+return acquire_reader_st;
+}
+vectorized::BlockReader reader;
+TabletReader::ReaderParams reader_params;
+vectorized::Block block;
+RETURN_NOT_OK(TabletReader::init_reader_params_and_create_block(
+tablet, READER_CHECKSUM, input_rowsets, &reader_params, &block))
+
+auto res = reader.init(reader_params);
+if (!res.ok()) {
+LOG(WARNING) << "initiate reader fail. res = " << res;
+return res;
+}
+
+bool eof = false;
+SipHash block_hash;
+uint64_t rows = 0;
+while (!eof) {
+RETURN_IF_ERROR(reader.next_block_with_aggregation(&block, nullptr,
nullptr, &eof));
+rows += block.rows();
+
+block.update_hash(block_hash);
+block.clear_column_data();
+}
+uint64_t checksum64 = block_hash.get64();
+*_checksum = (checksum64 >> 32) ^ (checksum64 & 0x);
Review Comment:
warning: use of undeclared identifier '_checksum' [clang-diagnostic-e
[GitHub] [doris] morningman merged pull request #15288: [fix](s3 outfile) Add the`use_path_style` parameter for s3 outfile
morningman merged PR #15288: URL: https://github.com/apache/doris/pull/15288 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](s3 outfile) Add the`use_path_style` parameter for s3 outfile (#15288)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 764b1db097 [fix](s3 outfile) Add the`use_path_style` parameter for s3
outfile (#15288)
764b1db097 is described below
commit 764b1db097f28055fc2830eeb395a3fef93095ba
Author: Tiewei Fang <43782773+bepppo...@users.noreply.github.com>
AuthorDate: Fri Dec 23 16:22:06 2022 +0800
[fix](s3 outfile) Add the`use_path_style` parameter for s3 outfile (#15288)
Currently, `outfile` did not support `use_path_style` parameter and use
`virtual-host style` by default,
however some Object-storage may only support `use_path_style` access mode.
This pr add the`use_path_style` parameter for s3 outfile, so that different
object-storage can use different access mode.
---
.../sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md | 1 +
.../sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md | 1 +
fe/fe-core/src/main/java/org/apache/doris/analysis/OutFileClause.java | 4
3 files changed, 6 insertions(+)
diff --git
a/docs/en/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md
b/docs/en/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md
index b094fcdd3f..d46b2164d7 100644
---
a/docs/en/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md
+++
b/docs/en/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md
@@ -101,6 +101,7 @@ illustrate:
AWS_ACCESS_KEY
AWS_SECRET_KEY
AWS_REGION
+use_path_stype: (optional) default false . The S3 SDK uses the
virtual-hosted style by default. However, some object storage systems may not
be enabled or support virtual-hosted style access. At this time, we can add the
use_path_style parameter to force the use of path style access method.
```
### example
diff --git
a/docs/zh-CN/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md
b/docs/zh-CN/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md
index 82f8ecec33..25b35f5894 100644
---
a/docs/zh-CN/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md
+++
b/docs/zh-CN/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md
@@ -104,6 +104,7 @@ INTO OUTFILE "file_path"
AWS_ACCESS_KEY
AWS_SECRET_KEY
AWS_REGION
+use_path_stype: (选填) 默认为false 。S3 SDK 默认使用 virtual-hosted style
方式。但某些对象存储系统可能没开启或不支持virtual-hosted style 方式的访问，此时可以添加 use_path_style 参数来强制使用
path style 访问方式。
```
### example
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/OutFileClause.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/OutFileClause.java
index 3d3f82b74b..78f57cff9c 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/OutFileClause.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/OutFileClause.java
@@ -648,6 +648,10 @@ public class OutFileClause {
}
}
if (storageType == StorageBackend.StorageType.S3) {
+if (properties.containsKey(S3Resource.USE_PATH_STYLE)) {
+brokerProps.put(S3Resource.USE_PATH_STYLE,
properties.get(S3Resource.USE_PATH_STYLE));
+processedPropKeys.add(S3Resource.USE_PATH_STYLE);
+}
S3Storage.checkS3(brokerProps);
} else if (storageType == StorageBackend.StorageType.HDFS) {
if (!brokerProps.containsKey(HdfsResource.HADOOP_FS_NAME)) {
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (764b1db097 -> 8515a03ef9)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 764b1db097 [fix](s3 outfile) Add the`use_path_style` parameter for s3 outfile (#15288) add 8515a03ef9 [fix](compile) fix compile error caused by `mysql_scan_node.cpp` not being found when enabling `WITH_MYSQL` (#15277) No new revisions were added by this update. Summary of changes: be/src/exec/CMakeLists.txt | 7 --- be/src/vec/CMakeLists.txt | 7 +++ 2 files changed, 7 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yangzhg merged pull request #15277: [fix](compile) fix compile error caused by `mysql_scan_node.cpp` not being found when enabling `WITH_MYSQL`
yangzhg merged PR #15277: URL: https://github.com/apache/doris/pull/15277 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yangzhg closed issue #15276: [Bug] compile error caused by `mysql_scan_node.cpp` not being found
yangzhg closed issue #15276: [Bug] compile error caused by `mysql_scan_node.cpp` not being found URL: https://github.com/apache/doris/issues/15276 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15308: [Improvement](meta) support rename hive and jdbc engine external table
github-actions[bot] commented on PR #15308: URL: https://github.com/apache/doris/pull/15308#issuecomment-1363731688 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15308: [Improvement](meta) support rename hive and jdbc engine external table
github-actions[bot] commented on PR #15308: URL: https://github.com/apache/doris/pull/15308#issuecomment-1363731707 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] meizizi-jy opened a new issue, #15319: [Bug]select from_unixtime() 导致 Decimal convert overflow
meizizi-jy opened a new issue, #15319: URL: https://github.com/apache/doris/issues/15319 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 1.1.3 ### What's Wrong?    在使用from_unixtime函数转换decima字段（decimal(15, 0)）的时候be直接宕掉，错误为 F1223 16:19:38.782735 5895 data_type_decimal.h:303] Decimal convert overflow ### What You Expected? 解决此问题 ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #15311: [feature-wip](nereids) Implement using join
hello-stephen commented on PR #15311: URL: https://github.com/apache/doris/pull/15311#issuecomment-1363737329 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.08 seconds load time: 683 seconds storage size: 17123273149 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221223083647_clickbench_pr_67764.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei closed issue #15295: [Bug] be core dump caused by bitmap filter
yiguolei closed issue #15295: [Bug] be core dump caused by bitmap filter URL: https://github.com/apache/doris/issues/15295 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #15296: [fix](bitmapfilter) fix core dump caused by bitmap filter
yiguolei merged PR #15296: URL: https://github.com/apache/doris/pull/15296 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (8515a03ef9 -> 8a810cd554)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 8515a03ef9 [fix](compile) fix compile error caused by `mysql_scan_node.cpp` not being found when enabling `WITH_MYSQL` (#15277) add 8a810cd554 [fix](bitmapfilter) fix core dump caused by bitmap filter (#15296) No new revisions were added by this update. Summary of changes: be/src/exprs/runtime_filter.cpp| 8 ++-- be/src/runtime/types.h | 5 + be/src/vec/exec/scan/vscan_node.cpp| 2 +- regression-test/data/query_p0/join/test_bitmap_filter.out | 9 + regression-test/suites/query_p0/join/test_bitmap_filter.groovy | 4 +++- 5 files changed, 24 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #15287: [improvement](test) add --conf option for run-regression-test.sh for custom config file
yiguolei merged PR #15287: URL: https://github.com/apache/doris/pull/15287 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [improvement](test) add --conf option for run-regression-test.sh for custom config file (#15287)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 1926239f09 [improvement](test) add --conf option for
run-regression-test.sh for custom config file (#15287)
1926239f09 is described below
commit 1926239f09da6bf3788399af3eca2c9bd6c964f6
Author: Kang
AuthorDate: Fri Dec 23 16:43:18 2022 +0800
[improvement](test) add --conf option for run-regression-test.sh for custom
config file (#15287)
* add --conf option for run-regression-test.sh for custom config file
* fix shell check error
---
run-regression-test.sh | 12 +++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/run-regression-test.sh b/run-regression-test.sh
index 41348bac95..df707c09ed 100755
--- a/run-regression-test.sh
+++ b/run-regression-test.sh
@@ -90,6 +90,11 @@ else
TEAMCITY=1
shift
;;
+--conf)
+CUSTOM_CONFIG_FILE="$2"
+shift
+shift
+;;
--run)
RUN=1
shift
@@ -121,7 +126,12 @@ fi
export MVN_CMD
CONF_DIR="${DORIS_HOME}/regression-test/conf"
-CONFIG_FILE="${CONF_DIR}/regression-conf.groovy"
+if [[ -n "${CUSTOM_CONFIG_FILE}" ]] && [[ -f "${CUSTOM_CONFIG_FILE}" ]]; then
+CONFIG_FILE="${CUSTOM_CONFIG_FILE}"
+echo "Using custom config file ${CONFIG_FILE}"
+else
+CONFIG_FILE="${CONF_DIR}/regression-conf.groovy"
+fi
LOG_CONFIG_FILE="${CONF_DIR}/logback.xml"
FRAMEWORK_SOURCE_DIR="${DORIS_HOME}/regression-test/framework"
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #15314: [fix](union)the union node should not pass through children in some case
yiguolei merged PR #15314: URL: https://github.com/apache/doris/pull/15314 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.1-lts updated: [fix](union)the union node should not pass through children in some case (#15314)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-1.1-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.1-lts by this push:
new 6a7ccfdcca [fix](union)the union node should not pass through children
in some case (#15314)
6a7ccfdcca is described below
commit 6a7ccfdcca12b2bf37e2b873484f626333caaab6
Author: starocean999 <40539150+starocean...@users.noreply.github.com>
AuthorDate: Fri Dec 23 16:43:45 2022 +0800
[fix](union)the union node should not pass through children in some case
(#15314)
---
.../src/main/java/org/apache/doris/planner/SetOperationNode.java| 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/planner/SetOperationNode.java
b/fe/fe-core/src/main/java/org/apache/doris/planner/SetOperationNode.java
index a16ef960a1..a3e3fc9792 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/planner/SetOperationNode.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/planner/SetOperationNode.java
@@ -269,8 +269,12 @@ public abstract class SetOperationNode extends PlanNode {
}
for (int i = 0; i < setOpResultExprs_.size(); ++i) {
-if (!setOpTupleDescriptor.getSlots().get(i).isMaterialized())
+if (!setOpTupleDescriptor.getSlots().get(i).isMaterialized()) {
+if (VectorizedUtil.isVectorized() &&
childTupleDescriptor.getSlots().get(i).isMaterialized()) {
+return false;
+}
continue;
+}
SlotRef setOpSlotRef =
setOpResultExprs_.get(i).unwrapSlotRef(false);
SlotRef childSlotRef = childExprList.get(i).unwrapSlotRef(false);
Preconditions.checkNotNull(setOpSlotRef);
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #15255: [fix](fe)fix bug of the bucket shuffle join is not recognized
yiguolei merged PR #15255: URL: https://github.com/apache/doris/pull/15255 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (1926239f09 -> b935fd0e7d)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
from 1926239f09 [improvement](test) add --conf option for
run-regression-test.sh for custom config file (#15287)
add b935fd0e7d [fix](fe)fix bug of the bucket shuffle join is not
recognized (#15255)
No new revisions were added by this update.
Summary of changes:
.../org/apache/doris/planner/DistributedPlanner.java| 2 +-
.../main/java/org/apache/doris/planner/PlanNode.java| 17 +
...cate_join.groovy => test_bucket_shuffle_join.groovy} | 13 +
3 files changed, 23 insertions(+), 9 deletions(-)
copy regression-test/suites/correctness_p0/{test_colocate_join.groovy =>
test_bucket_shuffle_join.groovy} (87%)
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #15226: [Fix](multi catalog)Fix VFileScanner file not found status bug.
yiguolei merged PR #15226: URL: https://github.com/apache/doris/pull/15226 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (b935fd0e7d -> e336178ef8)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from b935fd0e7d [fix](fe)fix bug of the bucket shuffle join is not recognized (#15255) add e336178ef8 [Fix](multi catalog)Fix VFileScanner file not found status bug. #15226 No new revisions were added by this update. Summary of changes: be/src/vec/exec/scan/scanner_scheduler.cpp | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #15216: [fix](jdbc catalog) fix bugs of jdbc catalog and table valued function
yiguolei merged PR #15216: URL: https://github.com/apache/doris/pull/15216 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (e336178ef8 -> e7a077a81f)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from e336178ef8 [Fix](multi catalog)Fix VFileScanner file not found status bug. #15226 add e7a077a81f [fix](jdbc catalog) fix bugs of jdbc catalog and table valued function (#15216) No new revisions were added by this update. Summary of changes: .../docker-compose/mysql/init/03-create-table.sql | 9 .../docker-compose/mysql/init/04-insert.sql| 5 + .../org/apache/doris/analysis/DescribeStmt.java| 3 ++- .../org/apache/doris/external/jdbc/JdbcClient.java | 25 +- .../java/org/apache/doris/qe/ConnectProcessor.java | 2 ++ .../ExternalFileTableValuedFunction.java | 2 +- .../doris/tablefunction/S3TableValuedFunction.java | 2 +- .../table_valued_function/test_hdfs_tvf.out| 9 .../jdbc_catalog_p0/test_mysql_jdbc_catalog.out| 4 .../table_valued_function/test_hdfs_tvf.groovy | 9 .../jdbc_catalog_p0/test_mysql_jdbc_catalog.groovy | 4 +++- 11 files changed, 69 insertions(+), 5 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #15171: [typo](doc) update Paxos spell mistake
yiguolei merged PR #15171: URL: https://github.com/apache/doris/pull/15171 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [typo](doc) update Paxos spell mistake (#15171)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 00fd5b1b1c [typo](doc) update Paxos spell mistake (#15171) 00fd5b1b1c is described below commit 00fd5b1b1cc064a45ca1d1a73c6bef45003bacae Author: gnehil AuthorDate: Fri Dec 23 16:47:12 2022 +0800 [typo](doc) update Paxos spell mistake (#15171) --- docs/en/docs/faq/install-faq.md| 2 +- docs/zh-CN/docs/faq/install-faq.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/en/docs/faq/install-faq.md b/docs/en/docs/faq/install-faq.md index 5ec4501485..0f722407e7 100644 --- a/docs/en/docs/faq/install-faq.md +++ b/docs/en/docs/faq/install-faq.md @@ -51,7 +51,7 @@ The reason why the CIDR format is used instead of specifying a specific IP direc First of all, make it clear that FE has only two roles: Follower and Observer. The Master is just an FE selected from a group of Follower nodes. Master can be regarded as a special kind of Follower. So when we were asked how many FEs a cluster had and what roles they were, the correct answer should be the number of all FE nodes, the number of Follower roles and the number of Observer roles. -All FE nodes of the Follower role will form an optional group, similar to the group concept in the Poxas consensus protocol. A Follower will be elected as the Master in the group. When the Master hangs up, a new Follower will be automatically selected as the Master. The Observer will not participate in the election, so the Observer will not be called Master. +All FE nodes of the Follower role will form an optional group, similar to the group concept in the Paxos consensus protocol. A Follower will be elected as the Master in the group. When the Master hangs up, a new Follower will be automatically selected as the Master. The Observer will not participate in the election, so the Observer will not be called Master. A metadata log needs to be successfully written in most Follower nodes to be considered successful. For example, if there are 3 FEs, only 2 can be successfully written. This is why the number of Follower roles needs to be an odd number. diff --git a/docs/zh-CN/docs/faq/install-faq.md b/docs/zh-CN/docs/faq/install-faq.md index 3b96414630..c5c73ec4f8 100644 --- a/docs/zh-CN/docs/faq/install-faq.md +++ b/docs/zh-CN/docs/faq/install-faq.md @@ -51,7 +51,7 @@ priorty_network 的值是 CIDR 格式表示的。分为两部分，第一部分 首先明确一点，FE 只有两种角色：Follower 和 Observer。而 Master 只是一组 Follower 节点中选择出来的一个 FE。Master 可以看成是一种特殊的 Follower。所以当我们被问及一个集群有多少 FE，都是什么角色时，正确的回答当时应该是所有 FE 节点的个数，以及 Follower 角色的个数和 Observer 角色的个数。 -所有 Follower 角色的 FE 节点会组成一个可选择组，类似 Poxas 一致性协议里的组概念。组内会选举出一个 Follower 作为 Master。当 Master 挂了，会自动选择新的 Follower 作为 Master。而 Observer 不会参与选举，因此 Observer 也不会称为 Master 。 +所有 Follower 角色的 FE 节点会组成一个可选择组，类似 Paxos 一致性协议里的组概念。组内会选举出一个 Follower 作为 Master。当 Master 挂了，会自动选择新的 Follower 作为 Master。而 Observer 不会参与选举，因此 Observer 也不会称为 Master 。 一条元数据日志需要在多数 Follower 节点写入成功，才算成功。比如3个 FE ，2个写入成功才可以。这也是为什么 Follower 角色的个数需要是奇数的原因。 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #15244: [DOCS](refactor) refine en docs
yiguolei merged PR #15244: URL: https://github.com/apache/doris/pull/15244 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (00fd5b1b1c -> ef3da105c9)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 00fd5b1b1c [typo](doc) update Paxos spell mistake (#15171) add ef3da105c9 [DOCS](refactor) refine en docs (#15244) No new revisions were added by this update. Summary of changes: README.md | 46 ++-- docs/en/docs/summary/basic-summary.md | 56 ++- 2 files changed, 52 insertions(+), 50 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xinyiZzz commented on a diff in pull request #13035: [improvement](index) Support bitmap index can be applied with compound predicate when enable vectorized engine query
xinyiZzz commented on code in PR #13035:
URL: https://github.com/apache/doris/pull/13035#discussion_r1056152292
##
be/src/exec/olap_common.h:
##
@@ -239,6 +244,28 @@ class ColumnValueRange {
}
}
+void to_boundary_condition(std::vector& filters) {
+for (const auto& value : _boundary_values) {
+TCondition condition;
+condition.__set_column_name(_column_name);
+condition.__set_compound_type(_compound_type);
Review Comment:
`condition.compound_type` and `get_compound_type` do not seem to be used
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15259: [Pipeline] Fix PipScannerContext::can_finish return wrong status
github-actions[bot] commented on PR #15259: URL: https://github.com/apache/doris/pull/15259#issuecomment-1363750913 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15298: [bugfix](from_unixtime) fix timezone not work for from_unixtime
github-actions[bot] commented on PR #15298: URL: https://github.com/apache/doris/pull/15298#issuecomment-1363752264 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #15312: [Bug](timediff) Fix wrong result for function `timediff`
hello-stephen commented on PR #15312: URL: https://github.com/apache/doris/pull/15312#issuecomment-1363756827 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 36.03 seconds load time: 686 seconds storage size: 17123581408 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221223090442_clickbench_pr_67934.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #15313: [refactor](non-vec)remove schema change related non-vec code
hello-stephen commented on PR #15313: URL: https://github.com/apache/doris/pull/15313#issuecomment-1363758945 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.68 seconds load time: 634 seconds storage size: 17122384698 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221223090651_clickbench_pr_67798.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] freemandealer closed pull request #14080: [fix](log) fix and clarify error msg for tablet writer write failure (#14078)
freemandealer closed pull request #14080: [fix](log) fix and clarify error msg for tablet writer write failure (#14078) URL: https://github.com/apache/doris/pull/14080 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15259: [Pipeline] Fix PipScannerContext::can_finish return wrong status
github-actions[bot] commented on PR #15259: URL: https://github.com/apache/doris/pull/15259#issuecomment-1363761728 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15260: [enhancement](checksum) use vectorized engine in checksum
github-actions[bot] commented on PR #15260: URL: https://github.com/apache/doris/pull/15260#issuecomment-1363763551 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15158: docs: fix small error
github-actions[bot] commented on PR #15158: URL: https://github.com/apache/doris/pull/15158#issuecomment-1363774016 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15312: [Bug](timediff) Fix wrong result for function `timediff`
github-actions[bot] commented on PR #15312: URL: https://github.com/apache/doris/pull/15312#issuecomment-1363784950 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zy-kkk commented on issue #15319: [Bug]select from_unixtime() 导致 Decimal convert overflow
zy-kkk commented on issue #15319: URL: https://github.com/apache/doris/issues/15319#issuecomment-1363799209 可以升级一下1.1.5再试一下 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] cambyzju opened a new issue, #15320: [Bug] load with wrong array type make be crash
cambyzju opened a new issue, #15320: URL: https://github.com/apache/doris/issues/15320 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version latest ### What's Wrong? F1221 22:08:58.347618 29624 assert_cast.h:50] Bad cast from type:doris::vectorized::IColumn const* to doris::vectorized::ColumnString const* *** Check failure stack trace: *** @ 0x118a8cbd google::LogMessage::Fail() @ 0x118aaf19 google::LogMessage::SendToLog() @ 0x118a8829 google::LogMessage::Flush() @ 0x118ab4e9 google::LogMessageFatal::~LogMessageFatal() @ 0xca77ad7 assert_cast<>() @ 0x10eed927 doris::stream_load::VOlapTableSink::_validate_column() @ 0x10eee4fc doris::stream_load::VOlapTableSink::_validate_column() @ 0x10eeeae0 doris::stream_load::VOlapTableSink::_validate_data() @ 0x10eec2ad doris::stream_load::VOlapTableSink::send() @ 0xc1285ca doris::PlanFragmentExecutor::open_vectorized_internal() @ 0xc127dff doris::PlanFragmentExecutor::open() @ 0xc0ef8a1 doris::FragmentExecState::execute() @ 0xc0f2b0e doris::FragmentMgr::_exec_actual() @ 0xc0f3816 _ZZN5doris11FragmentMgr18exec_plan_fragmentERKNS_23TExecPlanFragmentParamsESt8functionIFvPNS_12RuntimeStateEPNS_6StatusNKUlvE_clEv @ 0xc0f9db2 _ZSt13__invoke_implIvRZN5doris11FragmentMgr18exec_plan_fragmentERKNS0_23TExecPlanFragmentParamsESt8functionIFvPNS0_12RuntimeStateEPNS0_6StatusUlvE_JEET_St14__invoke_otherOT0_DpOT1_ @ 0xc0f9a0c _ZSt10__invoke_rIvRZN5doris11FragmentMgr18exec_plan_fragmentERKNS0_23TExecPlanFragmentParamsESt8functionIFvPNS0_12RuntimeStateEPNS0_6StatusUlvE_JEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESF_E4typeEOSG_DpOSH_ @ 0xc0f93fd _ZNSt17_Function_handlerIFvvEZN5doris11FragmentMgr18exec_plan_fragmentERKNS1_23TExecPlanFragmentParamsESt8functionIFvPNS1_12RuntimeStateEPNS1_6StatusUlvE_E9_M_invokeERKSt9_Any_data @ 0xc04a9a6 std::function<>::operator()() @ 0xc56257c doris::FunctionRunnable::run() @ 0xc5608d7 doris::ThreadPool::dispatch_thread() @ 0xc56e3eb std::__invoke_impl<>() @ 0xc56db3f std::__invoke<>() @ 0xc56d39c _ZNSt5_BindIFMN5doris10ThreadPoolEFvvEPS1_EE6__callIvJEJLm0T_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE @ 0xc56c6c8 std::_Bind<>::operator()<>() @ 0xc56b57a std::__invoke_impl<>() @ 0xc56a2b2 _ZSt10__invoke_rIvRSt5_BindIFMN5doris10ThreadPoolEFvvEPS2_EEJEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESA_E4typeEOSB_DpOSC_ @ 0xc568367 std::_Function_handler<>::_M_invoke() @ 0xc04a9a6 std::function<>::operator()() @ 0xc553c5e doris::Thread::supervise_thread() @ 0x7f2295ed6d05 start_thread @ 0x7f2295fe864f __GI___clone @ (nil) (unknown) *** Query id: 7e48a353d3ec8949-8598a53907663cbb *** *** Aborted at 1671631738 (unix time) try "date -d @1671631738" if you are using GNU date *** *** Current BE git commitID: 7d556e9d7 *** *** SIGABRT unkown detail explain (@0x3711065d2) received by PID 26066 (TID 0x7f21635ff640) from PID 26066; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/common/signal_handler.h:420 1# 0x7F2295F29080 in /opt/compiler/gcc-12/lib/libc.so.6 2# raise at ../sysdeps/unix/sysv/linux/raise.c:50 3# abort at /root/work/deck/devel/toolchain/glibc-2.33/stdlib/abort.c:81 4# 0x118B338C in /home/disk1/zhuxiaoli01/doris/output_for_master/be1/lib/doris_be 5# 0x118A8CBD in /home/disk1/zhuxiaoli01/doris/output_for_master/be1/lib/doris_be 6# google::LogMessage::SendToLog() in /home/disk1/zhuxiaoli01/doris/output_for_master/be1/lib/doris_be 7# google::LogMessage::Flush() in /home/disk1/zhuxiaoli01/doris/output_for_master/be1/lib/doris_be 8# google::LogMessageFatal::~LogMessageFatal() in /home/disk1/zhuxiaoli01/doris/output_for_master/be1/lib/doris_be 9# doris::vectorized::ColumnString const* assert_cast(doris::vectorized::IColumn const*&&) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/vec/common/assert_cast.h:46 10# doris::stream_load::VOlapTableSink::_validate_column(doris::RuntimeState*, doris::TypeDescriptor const&, bool, COW::immutable_ptr, unsigned long, doris::Bitmap*, bool*, fmt::v7::basic_memory_buffer >&, doris::vectorized::PODArray, 15ul, 16ul>*) at /home/disk1/zhuxiaoli01/doris/baidu/bdg/doris/core/be/src/vec/sink/vtablet_sink.c
[GitHub] [doris] xinyiZzz commented on a diff in pull request #13035: [improvement](index) Support bitmap index can be applied with compound predicate when enable vectorized engine query
xinyiZzz commented on code in PR #13035:
URL: https://github.com/apache/doris/pull/13035#discussion_r1056200555
##
be/src/vec/exprs/vectorized_fn_call.cpp:
##
@@ -104,12 +104,38 @@ doris::Status VectorizedFnCall::execute(VExprContext*
context, doris::vectorized
size_t num_columns_without_result = block->columns();
// prepare a column to save result
block->insert({nullptr, _data_type, _expr_name});
+if (_function->can_fast_execute()) {
+bool ok = fast_execute(context->fn_context(_fn_context_index), *block,
arguments,
+ num_columns_without_result, block->rows());
+if (ok) {
+*result_column_id = num_columns_without_result;
+return Status::OK();
+}
+}
+
RETURN_IF_ERROR(_function->execute(context->fn_context(_fn_context_index),
*block, arguments,
num_columns_without_result,
block->rows(), false));
*result_column_id = num_columns_without_result;
return Status::OK();
}
+bool VectorizedFnCall::fast_execute(FunctionContext* context, Block& block,
+const ColumnNumbers& arguments, size_t
result,
+size_t input_rows_count) {
+auto column_with_name = block.get_by_position(arguments[1]);
+auto query_value = column_with_name.to_string(0);
+std::string column_name = block.get_by_position(arguments[0]).name;
+auto result_column_name = column_name + "_" + _function->get_name() + "_"
+ query_value;
Review Comment:
Save the unique identifier of expr in `ColumnValueRange` -> `TCodition
filter` -> `ColumnPredicate`, so that `fast_execute` can find the result column
of `_execute_all_compound_predicates`, is it possible?
Just an implementation idea, probably doesn't need modification
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei opened a new pull request, #15321: [profile](scanner) add per scanner running time profile
yiguolei opened a new pull request, #15321: URL: https://github.com/apache/doris/pull/15321 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 opened a new pull request, #15322: [fix](nereids) Fix bugs of group by constants
Kikyou1997 opened a new pull request, #15322: URL: https://github.com/apache/doris/pull/15322 # Proposed changes Issue Number: close #xxx ## Problem summary ```sql select 2 from nation group by 1 ``` it should produce 2 would the table is not empty. Before this PR, the execution of nereids generated plan would produce empty result set ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #15321: [profile](scanner) add per scanner running time profile
github-actions[bot] commented on code in PR #15321:
URL: https://github.com/apache/doris/pull/15321#discussion_r1056204493
##
be/src/vec/exec/scan/vscanner.h:
##
@@ -64,6 +64,10 @@ class VScanner {
public:
VScanNode* get_parent() { return _parent; }
+int64_t get_time_cost_ns() { return _per_scanner_timer; }
Review Comment:
warning: method 'get_time_cost_ns' can be made const
[readability-make-member-function-const]
```suggestion
int64_t get_time_cost_ns() const { return _per_scanner_timer; }
```
##
be/src/vec/exec/scan/vscanner.h:
##
@@ -64,6 +64,10 @@
public:
VScanNode* get_parent() { return _parent; }
+int64_t get_time_cost_ns() { return _per_scanner_timer; }
+
+int64_t get_rows_read() { return _num_rows_read; }
Review Comment:
warning: method 'get_rows_read' can be made const
[readability-make-member-function-const]
```suggestion
int64_t get_rows_read() const { return _num_rows_read; }
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BePPPower opened a new pull request, #15323: [fix](s3 load) fix that `FE` can not access s3 objected-storage
BePPPower opened a new pull request, #15323: URL: https://github.com/apache/doris/pull/15323 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] AshinGau commented on pull request #15323: [fix](s3 load) fix that `FE` can not access s3 objected-storage
AshinGau commented on PR #15323: URL: https://github.com/apache/doris/pull/15323#issuecomment-1363819320 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #14657: [enhancement](Nereids) cast expression to the type with parameters
github-actions[bot] commented on PR #14657: URL: https://github.com/apache/doris/pull/14657#issuecomment-1363822710 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xinyiZzz commented on a diff in pull request #13035: [improvement](index) Support bitmap index can be applied with compound predicate when enable vectorized engine query
xinyiZzz commented on code in PR #13035:
URL: https://github.com/apache/doris/pull/13035#discussion_r1056218198
##
be/src/vec/exprs/vectorized_fn_call.cpp:
##
@@ -104,12 +104,38 @@ doris::Status VectorizedFnCall::execute(VExprContext*
context, doris::vectorized
size_t num_columns_without_result = block->columns();
// prepare a column to save result
block->insert({nullptr, _data_type, _expr_name});
+if (_function->can_fast_execute()) {
+bool ok = fast_execute(context->fn_context(_fn_context_index), *block,
arguments,
+ num_columns_without_result, block->rows());
+if (ok) {
+*result_column_id = num_columns_without_result;
+return Status::OK();
+}
+}
+
RETURN_IF_ERROR(_function->execute(context->fn_context(_fn_context_index),
*block, arguments,
num_columns_without_result,
block->rows(), false));
*result_column_id = num_columns_without_result;
return Status::OK();
}
+bool VectorizedFnCall::fast_execute(FunctionContext* context, Block& block,
+const ColumnNumbers& arguments, size_t
result,
+size_t input_rows_count) {
+auto column_with_name = block.get_by_position(arguments[1]);
+auto query_value = column_with_name.to_string(0);
+std::string column_name = block.get_by_position(arguments[0]).name;
+auto result_column_name = column_name + "_" + _function->get_name() + "_"
+ query_value;
Review Comment:
In `SegmentIterator`, a lot of codes are used in order to splicing the sign
string, and it seems that it is easier to understand with a unique identifier
of expr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15261: [feature](nereids) support bitAnd/ bitOr/ bitXor
github-actions[bot] commented on PR #15261: URL: https://github.com/apache/doris/pull/15261#issuecomment-1363825161 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15261: [feature](nereids) support bitAnd/ bitOr/ bitXor
github-actions[bot] commented on PR #15261: URL: https://github.com/apache/doris/pull/15261#issuecomment-1363825128 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow merged pull request #14657: [enhancement](Nereids) cast expression to the type with parameters
morrySnow merged PR #14657: URL: https://github.com/apache/doris/pull/14657 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangshuo128 commented on a diff in pull request #15275: [Feature](Nereids) Support hll and count for materialized index.
wangshuo128 commented on code in PR #15275:
URL: https://github.com/apache/doris/pull/15275#discussion_r1056221716
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/mv/SelectMaterializedIndexWithAggregate.java:
##
@@ -328,6 +330,13 @@ private SelectResult select(
.filter(index ->
!candidatesWithoutRewriting.contains(index))
.map(index -> rewriteAgg(index, scan,
requiredScanOutput, predicates, aggregateFunctions,
groupingExprs))
+.filter(aggRewriteResult -> checkPreAggStatus(scan,
aggRewriteResult.index.getId(),
+predicates,
+// check pre-agg status of aggregate function
that couldn't rewrite.
+//
ImmutableList.copyOf(Sets.difference(ImmutableSet.copyOf(aggregateFunctions),
+//
aggRewriteResult.exprRewriteMap.aggFuncMap.keySet())),
Review Comment:
thanks, fixed.
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/Ndv.java:
##
@@ -0,0 +1,117 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.expressions.functions.agg;
+
+import org.apache.doris.catalog.FunctionSignature;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.functions.AlwaysNotNullable;
+import
org.apache.doris.nereids.trees.expressions.functions.ExplicitlyCastableSignature;
+import org.apache.doris.nereids.trees.expressions.shape.UnaryExpression;
+import org.apache.doris.nereids.trees.expressions.visitor.ExpressionVisitor;
+import org.apache.doris.nereids.types.BigIntType;
+import org.apache.doris.nereids.types.BitmapType;
+import org.apache.doris.nereids.types.BooleanType;
+import org.apache.doris.nereids.types.DateTimeType;
+import org.apache.doris.nereids.types.DateTimeV2Type;
+import org.apache.doris.nereids.types.DateType;
+import org.apache.doris.nereids.types.DateV2Type;
+import org.apache.doris.nereids.types.DecimalV2Type;
+import org.apache.doris.nereids.types.DecimalV3Type;
+import org.apache.doris.nereids.types.DoubleType;
+import org.apache.doris.nereids.types.FloatType;
+import org.apache.doris.nereids.types.HllType;
+import org.apache.doris.nereids.types.IntegerType;
+import org.apache.doris.nereids.types.JsonType;
+import org.apache.doris.nereids.types.LargeIntType;
+import org.apache.doris.nereids.types.QuantileStateType;
+import org.apache.doris.nereids.types.SmallIntType;
+import org.apache.doris.nereids.types.StringType;
+import org.apache.doris.nereids.types.TimeType;
+import org.apache.doris.nereids.types.TimeV2Type;
+import org.apache.doris.nereids.types.TinyIntType;
+import org.apache.doris.nereids.types.VarcharType;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableList;
+
+import java.util.List;
+
+/**
+ * AggregateFunction 'ndv'. This class is generated by GenerateFunction.
+ */
+public class Ndv extends AggregateFunction
+implements UnaryExpression, ExplicitlyCastableSignature,
AlwaysNotNullable {
+
+public static final List SIGNATURES = ImmutableList.of(
+
FunctionSignature.ret(BigIntType.INSTANCE).args(TinyIntType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(SmallIntType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(IntegerType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(BigIntType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(LargeIntType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(FloatType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DoubleType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DecimalV2Type.SYSTEM_DEFAULT),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DecimalV3Type.DEFAULT_DECIMAL32),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DecimalV3Type.DEFAULT_DECIMAL64),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(De
[GitHub] [doris] github-actions[bot] commented on pull request #15313: [refactor](non-vec)remove schema change related non-vec code
github-actions[bot] commented on PR #15313: URL: https://github.com/apache/doris/pull/15313#issuecomment-1363829412 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15313: [refactor](non-vec)remove schema change related non-vec code
github-actions[bot] commented on PR #15313: URL: https://github.com/apache/doris/pull/15313#issuecomment-1363829385 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #15313: [refactor](non-vec)remove schema change related non-vec code
yiguolei merged PR #15313: URL: https://github.com/apache/doris/pull/15313 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [refactor](non-vec)remove schema change related non-vec code (#15313)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 06d0035c02 [refactor](non-vec)remove schema change related non-vec
code (#15313)
06d0035c02 is described below
commit 06d0035c0295902dd068aeb7aebe065d9f5a2719
Author: yiguolei <676222...@qq.com>
AuthorDate: Fri Dec 23 18:33:04 2022 +0800
[refactor](non-vec)remove schema change related non-vec code (#15313)
Co-authored-by: yiguolei
---
be/src/olap/memtable.cpp |3 +-
be/src/olap/row_block.h|4 -
be/src/olap/rowset/beta_rowset_reader.cpp |1 -
be/src/olap/rowset/beta_rowset_reader.h|3 -
be/src/olap/schema_change.cpp | 824 --
be/src/olap/schema_change.h| 29 -
be/src/util/tuple_row_zorder_compare.cpp | 204 -
be/src/util/tuple_row_zorder_compare.h | 29 -
be/test/CMakeLists.txt |1 -
be/test/util/tuple_row_zorder_compare_test.cpp |
10 files changed, 1 insertion(+), 2208 deletions(-)
diff --git a/be/src/olap/memtable.cpp b/be/src/olap/memtable.cpp
index 63575a3bb2..fda31df925 100644
--- a/be/src/olap/memtable.cpp
+++ b/be/src/olap/memtable.cpp
@@ -86,8 +86,7 @@ MemTable::MemTable(TabletSharedPtr tablet, Schema* schema,
const TabletSchema* t
_aggregate_two_row_fn = &MemTable::_aggregate_two_row;
}
if (tablet_schema->sort_type() == SortType::ZORDER) {
-_row_comparator = std::make_shared(
-_schema, tablet_schema->sort_col_num());
+LOG(FATAL) << "ZOrder not supported";
} else {
_row_comparator = std::make_shared(_schema);
}
diff --git a/be/src/olap/row_block.h b/be/src/olap/row_block.h
index 0d9ed4c24a..eead5cb9cf 100644
--- a/be/src/olap/row_block.h
+++ b/be/src/olap/row_block.h
@@ -49,10 +49,6 @@ struct RowBlockInfo {
// 3. 给定查询的key，在RowBlock内做二分查找，返回起点的行偏移；
// 4. 向量化的条件过滤下推到RowBlock级别进行，因此增加完成过滤的数据读取接口
class RowBlock {
-// Please keep these classes as 'friend'. They have to use lots of
private fields for
-// faster operation.
-friend class RowBlockChanger;
-
public:
RowBlock(TabletSchemaSPtr schema);
diff --git a/be/src/olap/rowset/beta_rowset_reader.cpp
b/be/src/olap/rowset/beta_rowset_reader.cpp
index e5d461210e..1c9cf8c449 100644
--- a/be/src/olap/rowset/beta_rowset_reader.cpp
+++ b/be/src/olap/rowset/beta_rowset_reader.cpp
@@ -215,7 +215,6 @@ Status BetaRowsetReader::init(RowsetReaderContext*
read_context) {
return Status::Error();
}
_iterator.reset(final_iterator);
-
return Status::OK();
}
diff --git a/be/src/olap/rowset/beta_rowset_reader.h
b/be/src/olap/rowset/beta_rowset_reader.h
index c80d258d3b..2812339b10 100644
--- a/be/src/olap/rowset/beta_rowset_reader.h
+++ b/be/src/olap/rowset/beta_rowset_reader.h
@@ -85,9 +85,6 @@ private:
std::unique_ptr _iterator;
-std::shared_ptr _input_block;
-std::unique_ptr _row;
-
// make sure this handle is initialized and valid before
// reading data.
SegmentCacheHandle _segment_cache_handle;
diff --git a/be/src/olap/schema_change.cpp b/be/src/olap/schema_change.cpp
index f4015fdcb3..4f8595b168 100644
--- a/be/src/olap/schema_change.cpp
+++ b/be/src/olap/schema_change.cpp
@@ -47,50 +47,6 @@ using namespace ErrorCode;
constexpr int ALTER_TABLE_BATCH_SIZE = 4096;
-class RowBlockSorter {
-public:
-explicit RowBlockSorter(RowBlockAllocator* allocator);
-virtual ~RowBlockSorter();
-size_t num_rows() { return _swap_row_block != nullptr ?
_swap_row_block->capacity() : 0; }
-
-bool sort(RowBlock** row_block);
-
-private:
-static bool _row_cursor_comparator(const std::unique_ptr& a,
- const std::unique_ptr& b) {
-return compare_row(*a, *b) < 0;
-}
-
-RowBlockAllocator* _row_block_allocator;
-RowBlock* _swap_row_block;
-};
-
-class RowBlockMerger {
-public:
-explicit RowBlockMerger(TabletSharedPtr tablet);
-virtual ~RowBlockMerger();
-
-bool merge(const std::vector& row_block_arr, RowsetWriter*
rowset_writer,
- uint64_t* merged_rows);
-
-private:
-struct MergeElement {
-bool operator<(const MergeElement& other) const {
-return compare_row(*row_cursor, *other.row_cursor) > 0;
-}
-
-const RowBlock* row_block;
-RowCursor* row_cursor;
-uint32_t row_block_index;
-};
-
-bool _make_heap(const std::vector& row_block_arr);
-void _pop_heap();
-
-TabletSharedPtr _tablet;
-std::priority_queue _heap;
-};
-
class MultiBlockMerger {
public:
MultiBlockMerger(TabletSharedPtr tablet) : _tablet(tablet), _cmp(tablet) {}
@@ -261,524 +
[GitHub] [doris] wangshuo128 commented on a diff in pull request #15275: [Feature](Nereids) Support hll and count for materialized index.
wangshuo128 commented on code in PR #15275:
URL: https://github.com/apache/doris/pull/15275#discussion_r1056224544
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/mv/SelectMaterializedIndexWithAggregate.java:
##
@@ -711,34 +746,62 @@ private static Expression rewrite(Expression expr,
RewriteContext context) {
/**
* count(distinct col) -> bitmap_union_count(mv_bitmap_union_col)
+ * count(col) -> sum(mv_count_col)
*/
@Override
public Expression visitCount(Count count, RewriteContext context) {
if (count.isDistinct() && count.arity() == 1) {
Review Comment:
This rewriter is a bit different.
It only handles aggregate function on a slot, e.g., rewriting `count(col)`
to `sum(mv_count_col)`. The child of the aggregate function is checked by
`ExpressionUtils.extractSlotOrCastOnSlot(count.child(0))` to ensure it's a slot.
So we don't need to rewrite children first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zy-kkk opened a new pull request, #15324: [typo](docs)fix 404 err to Monitoring and alarming doc
zy-kkk opened a new pull request, #15324: URL: https://github.com/apache/doris/pull/15324 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #15275: [Feature](Nereids) Support hll and count for materialized index.
morrySnow commented on code in PR #15275:
URL: https://github.com/apache/doris/pull/15275#discussion_r1056226751
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/Ndv.java:
##
@@ -0,0 +1,121 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.expressions.functions.agg;
+
+import org.apache.doris.catalog.FunctionSignature;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.functions.AlwaysNotNullable;
+import
org.apache.doris.nereids.trees.expressions.functions.ExplicitlyCastableSignature;
+import org.apache.doris.nereids.trees.expressions.shape.UnaryExpression;
+import org.apache.doris.nereids.trees.expressions.visitor.ExpressionVisitor;
+import org.apache.doris.nereids.types.BigIntType;
+import org.apache.doris.nereids.types.BitmapType;
+import org.apache.doris.nereids.types.BooleanType;
+import org.apache.doris.nereids.types.CharType;
+import org.apache.doris.nereids.types.DateTimeType;
+import org.apache.doris.nereids.types.DateTimeV2Type;
+import org.apache.doris.nereids.types.DateType;
+import org.apache.doris.nereids.types.DateV2Type;
+import org.apache.doris.nereids.types.DecimalV2Type;
+import org.apache.doris.nereids.types.DecimalV3Type;
+import org.apache.doris.nereids.types.DoubleType;
+import org.apache.doris.nereids.types.FloatType;
+import org.apache.doris.nereids.types.HllType;
+import org.apache.doris.nereids.types.IntegerType;
+import org.apache.doris.nereids.types.JsonType;
+import org.apache.doris.nereids.types.LargeIntType;
+import org.apache.doris.nereids.types.NullType;
+import org.apache.doris.nereids.types.QuantileStateType;
+import org.apache.doris.nereids.types.SmallIntType;
+import org.apache.doris.nereids.types.StringType;
+import org.apache.doris.nereids.types.TimeType;
+import org.apache.doris.nereids.types.TimeV2Type;
+import org.apache.doris.nereids.types.TinyIntType;
+import org.apache.doris.nereids.types.VarcharType;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableList;
+
+import java.util.List;
+
+/**
+ * AggregateFunction 'ndv'. This class is generated by GenerateFunction.
+ */
+public class Ndv extends AggregateFunction
+implements UnaryExpression, ExplicitlyCastableSignature,
AlwaysNotNullable {
+
+public static final List SIGNATURES = ImmutableList.of(
+
FunctionSignature.ret(BigIntType.INSTANCE).args(TinyIntType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(SmallIntType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(IntegerType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(BigIntType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(LargeIntType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(FloatType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DoubleType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DecimalV2Type.SYSTEM_DEFAULT),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DecimalV3Type.DEFAULT_DECIMAL32),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DecimalV3Type.DEFAULT_DECIMAL64),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DecimalV3Type.DEFAULT_DECIMAL128),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(BooleanType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(VarcharType.SYSTEM_DEFAULT),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(StringType.INSTANCE),
+FunctionSignature.ret(BigIntType.INSTANCE).args(DateType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DateTimeType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DateV2Type.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DateTimeV2Type.INSTANCE),
+FunctionSignature.ret(BigIntType.INSTANCE).args(TimeType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(TimeV2Type.INSTANCE),
+FunctionSignature.ret(BigIntType
[GitHub] [doris] morrySnow merged pull request #15261: [feature](nereids) support bitAnd/ bitOr/ bitXor
morrySnow merged PR #15261: URL: https://github.com/apache/doris/pull/15261 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (06d0035c02 -> 2f089be37e)
This is an automated email from the ASF dual-hosted git repository. morrysnow pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 06d0035c02 [refactor](non-vec)remove schema change related non-vec code (#15313) add 2f089be37e [feature](nereids) support bitAnd/ bitOr/ bitXor (#15261) No new revisions were added by this update. Summary of changes: .../main/antlr4/org/apache/doris/nereids/DorisLexer.g4 | 3 +++ .../main/antlr4/org/apache/doris/nereids/DorisParser.g4 | 2 ++ .../apache/doris/nereids/parser/LogicalPlanBuilder.java | 17 + 3 files changed, 22 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15252: [feature](Nereids): support digital_masking function
github-actions[bot] commented on PR #15252: URL: https://github.com/apache/doris/pull/15252#issuecomment-1363837554 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15252: [feature](Nereids): support digital_masking function
github-actions[bot] commented on PR #15252: URL: https://github.com/apache/doris/pull/15252#issuecomment-1363837582 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangshuo128 commented on a diff in pull request #15275: [Feature](Nereids) Support hll and count for materialized index.
wangshuo128 commented on code in PR #15275:
URL: https://github.com/apache/doris/pull/15275#discussion_r1056238303
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/Ndv.java:
##
@@ -0,0 +1,121 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.expressions.functions.agg;
+
+import org.apache.doris.catalog.FunctionSignature;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.functions.AlwaysNotNullable;
+import
org.apache.doris.nereids.trees.expressions.functions.ExplicitlyCastableSignature;
+import org.apache.doris.nereids.trees.expressions.shape.UnaryExpression;
+import org.apache.doris.nereids.trees.expressions.visitor.ExpressionVisitor;
+import org.apache.doris.nereids.types.BigIntType;
+import org.apache.doris.nereids.types.BitmapType;
+import org.apache.doris.nereids.types.BooleanType;
+import org.apache.doris.nereids.types.CharType;
+import org.apache.doris.nereids.types.DateTimeType;
+import org.apache.doris.nereids.types.DateTimeV2Type;
+import org.apache.doris.nereids.types.DateType;
+import org.apache.doris.nereids.types.DateV2Type;
+import org.apache.doris.nereids.types.DecimalV2Type;
+import org.apache.doris.nereids.types.DecimalV3Type;
+import org.apache.doris.nereids.types.DoubleType;
+import org.apache.doris.nereids.types.FloatType;
+import org.apache.doris.nereids.types.HllType;
+import org.apache.doris.nereids.types.IntegerType;
+import org.apache.doris.nereids.types.JsonType;
+import org.apache.doris.nereids.types.LargeIntType;
+import org.apache.doris.nereids.types.NullType;
+import org.apache.doris.nereids.types.QuantileStateType;
+import org.apache.doris.nereids.types.SmallIntType;
+import org.apache.doris.nereids.types.StringType;
+import org.apache.doris.nereids.types.TimeType;
+import org.apache.doris.nereids.types.TimeV2Type;
+import org.apache.doris.nereids.types.TinyIntType;
+import org.apache.doris.nereids.types.VarcharType;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableList;
+
+import java.util.List;
+
+/**
+ * AggregateFunction 'ndv'. This class is generated by GenerateFunction.
+ */
+public class Ndv extends AggregateFunction
+implements UnaryExpression, ExplicitlyCastableSignature,
AlwaysNotNullable {
+
+public static final List SIGNATURES = ImmutableList.of(
+
FunctionSignature.ret(BigIntType.INSTANCE).args(TinyIntType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(SmallIntType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(IntegerType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(BigIntType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(LargeIntType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(FloatType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DoubleType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DecimalV2Type.SYSTEM_DEFAULT),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DecimalV3Type.DEFAULT_DECIMAL32),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DecimalV3Type.DEFAULT_DECIMAL64),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DecimalV3Type.DEFAULT_DECIMAL128),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(BooleanType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(VarcharType.SYSTEM_DEFAULT),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(StringType.INSTANCE),
+FunctionSignature.ret(BigIntType.INSTANCE).args(DateType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DateTimeType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DateV2Type.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(DateTimeV2Type.INSTANCE),
+FunctionSignature.ret(BigIntType.INSTANCE).args(TimeType.INSTANCE),
+
FunctionSignature.ret(BigIntType.INSTANCE).args(TimeV2Type.INSTANCE),
+FunctionSignature.ret(BigIntTy
[GitHub] [doris] github-actions[bot] commented on pull request #15324: [typo](docs)fix 404 err to Monitoring and alarming doc
github-actions[bot] commented on PR #15324: URL: https://github.com/apache/doris/pull/15324#issuecomment-1363843224 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15324: [typo](docs)fix 404 err to Monitoring and alarming doc
github-actions[bot] commented on PR #15324: URL: https://github.com/apache/doris/pull/15324#issuecomment-1363843254 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow merged pull request #15252: [feature](Nereids): support digital_masking function
morrySnow merged PR #15252: URL: https://github.com/apache/doris/pull/15252 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [feature](Nereids) support digital_masking function (#15252)
This is an automated email from the ASF dual-hosted git repository.
morrysnow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new bfaaa2bd7c [feature](Nereids) support digital_masking function (#15252)
bfaaa2bd7c is described below
commit bfaaa2bd7cde50b674924f911a8f0028009fdeba
Author: jakevin
AuthorDate: Fri Dec 23 18:59:08 2022 +0800
[feature](Nereids) support digital_masking function (#15252)
---
.../doris/catalog/BuiltinScalarFunctions.java | 9 ++-
.../rewrite/ExpressionNormalization.java | 4 +-
.../rewrite/rules/DigitalMaskingConvert.java | 41 +
.../functions/scalar/DigitalMasking.java | 69 ++
.../expressions/visitor/ScalarFunctionVisitor.java | 5 ++
.../data/correctness_p0/test_mask_function.out | 3 +
.../correctness_p0/test_mask_function.groovy | 4 ++
7 files changed, 131 insertions(+), 4 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/catalog/BuiltinScalarFunctions.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/BuiltinScalarFunctions.java
index 08282731d9..327decc04c 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/catalog/BuiltinScalarFunctions.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/catalog/BuiltinScalarFunctions.java
@@ -81,6 +81,7 @@ import
org.apache.doris.nereids.trees.expressions.functions.scalar.Dceil;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Degrees;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Dexp;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Dfloor;
+import
org.apache.doris.nereids.trees.expressions.functions.scalar.DigitalMasking;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Dlog1;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Dlog10;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Dpow;
@@ -267,9 +268,9 @@ import java.util.List;
/**
* Builtin scalar functions.
- *
+ *
* Note: Please ensure that this class only has some lists and no procedural
code.
- * It helps to be clear and concise.
+ * It helps to be clear and concise.
*/
public class BuiltinScalarFunctions implements FunctionHelper {
public final List scalarFunctions = ImmutableList.of(
@@ -337,6 +338,7 @@ public class BuiltinScalarFunctions implements
FunctionHelper {
scalar(Degrees.class, "degrees"),
scalar(Dexp.class, "dexp"),
scalar(Dfloor.class, "dfloor"),
+scalar(DigitalMasking.class, "digital_masking"),
scalar(Dlog1.class, "dlog1"),
scalar(Dlog10.class, "dlog10"),
scalar(Dpow.class, "dpow"),
@@ -521,5 +523,6 @@ public class BuiltinScalarFunctions implements
FunctionHelper {
public static final BuiltinScalarFunctions INSTANCE = new
BuiltinScalarFunctions();
// Note: Do not add any code here!
-private BuiltinScalarFunctions() {}
+private BuiltinScalarFunctions() {
+}
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/ExpressionNormalization.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/ExpressionNormalization.java
index a8773d9642..d79a9bb62e 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/ExpressionNormalization.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/ExpressionNormalization.java
@@ -19,6 +19,7 @@ package org.apache.doris.nereids.rules.expression.rewrite;
import
org.apache.doris.nereids.rules.expression.rewrite.rules.BetweenToCompoundRule;
import
org.apache.doris.nereids.rules.expression.rewrite.rules.CharacterLiteralTypeCoercion;
+import
org.apache.doris.nereids.rules.expression.rewrite.rules.DigitalMaskingConvert;
import
org.apache.doris.nereids.rules.expression.rewrite.rules.FoldConstantRule;
import
org.apache.doris.nereids.rules.expression.rewrite.rules.InPredicateToEqualToRule;
import
org.apache.doris.nereids.rules.expression.rewrite.rules.NormalizeBinaryPredicatesRule;
@@ -44,7 +45,8 @@ public class ExpressionNormalization extends
ExpressionRewrite {
CharacterLiteralTypeCoercion.INSTANCE,
TypeCoercion.INSTANCE,
FoldConstantRule.INSTANCE,
-SimplifyCastRule.INSTANCE
+SimplifyCastRule.INSTANCE,
+DigitalMaskingConvert.INSTANCE
);
public ExpressionNormalization(ConnectContext context) {
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/DigitalMaskingConvert.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/DigitalMaskingConvert.java
new file mode 100644
index 00..e297c2f120
--- /dev/null
+++
b/fe/fe-core/src/main/java/org/apach
[GitHub] [doris] nextdreamblue commented on pull request #15304: [fix](jdbc-catalog) fix close connection when use jdbc.db which table…
nextdreamblue commented on PR #15304: URL: https://github.com/apache/doris/pull/15304#issuecomment-1363853586 fixed by https://github.com/apache/doris/pull/15216 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] nextdreamblue closed pull request #15304: [fix](jdbc-catalog) fix close connection when use jdbc.db which table…
nextdreamblue closed pull request #15304: [fix](jdbc-catalog) fix close connection when use jdbc.db which table… URL: https://github.com/apache/doris/pull/15304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15298: [bugfix](from_unixtime) fix timezone not work for from_unixtime
github-actions[bot] commented on PR #15298: URL: https://github.com/apache/doris/pull/15298#issuecomment-1363854557 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yangzhg closed issue #15284: [Bug] time_zone not work for function `from_unixtime`
yangzhg closed issue #15284: [Bug] time_zone not work for function `from_unixtime` URL: https://github.com/apache/doris/issues/15284 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yangzhg merged pull request #15298: [bugfix](from_unixtime) fix timezone not work for from_unixtime
yangzhg merged PR #15298: URL: https://github.com/apache/doris/pull/15298 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [bugfix](from_unixtime) fix timezone not work for from_unixtime (#15298)
This is an automated email from the ASF dual-hosted git repository.
yangzhg pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new a98636a970 [bugfix](from_unixtime) fix timezone not work for
from_unixtime (#15298)
a98636a970 is described below
commit a98636a97083932a39045994c7b612d029ce0be7
Author: Zhengguo Yang
AuthorDate: Fri Dec 23 19:05:09 2022 +0800
[bugfix](from_unixtime) fix timezone not work for from_unixtime (#15298)
* [bugfix](from_unixtime) fix timezone not work for from_unixtime
---
be/src/gutil/casts.h | 7 +-
be/src/gutil/charmap.h | 5 -
be/src/gutil/gscoped_ptr.h | 12 +-
be/src/gutil/once.h| 5 +-
be/src/gutil/strings/stringpiece.h | 5 -
be/src/gutil/type_traits.h | 468 -
be/src/http/action/tablet_migration_action.cpp | 2 +-
be/src/runtime/memory/mem_tracker_limiter.cpp | 14 +-
be/src/util/thread.cpp | 2 +-
.../function_utils.h => src/util/type_traits.h}| 40 +-
be/src/vec/functions/date_time_transforms.h| 29 +-
.../function_date_or_datetime_to_string.h | 2 +-
.../function_datetime_string_to_string.cpp | 7 +-
.../functions/function_datetime_string_to_string.h | 38 +-
be/test/exprs/timestamp_functions_test.cpp | 25 +-
be/test/testutil/function_utils.cpp| 22 +-
be/test/testutil/function_utils.h | 1 -
.../data/datatype_p0/date/test_from_unixtime.out | 7 +
.../datatype_p0/date/test_from_unixtime.groovy | 34 +-
19 files changed, 99 insertions(+), 626 deletions(-)
diff --git a/be/src/gutil/casts.h b/be/src/gutil/casts.h
index 71a147cfaf..8e2a1a5cad 100644
--- a/be/src/gutil/casts.h
+++ b/be/src/gutil/casts.h
@@ -14,9 +14,10 @@
#include // for enumeration casts and tests
#include // for memcpy
+#include
+
#include "gutil/macros.h"
#include "gutil/template_util.h"
-#include "gutil/type_traits.h"
// Use implicit_cast as a safe version of static_cast or const_cast
// for implicit conversions. For example:
@@ -89,8 +90,8 @@ inline To down_cast(From* f) {// so we only accept
pointers
// compiler will just bind From to const T.
template
inline To down_cast(From& f) {
-COMPILE_ASSERT(base::is_reference::value, target_type_not_a_reference);
-typedef typename base::remove_reference::type* ToAsPointer;
+COMPILE_ASSERT(std::is_reference::value, target_type_not_a_reference);
+using ToAsPointer = typename std::remove_reference::type*;
if (false) {
// Compile-time check that To inherits from From. See above for
details.
::implicit_cast(NULL);
diff --git a/be/src/gutil/charmap.h b/be/src/gutil/charmap.h
index 213d44f726..797abcce5c 100644
--- a/be/src/gutil/charmap.h
+++ b/be/src/gutil/charmap.h
@@ -17,10 +17,6 @@
#include
-#include "gutil/basictypes.h"
-#include "gutil/integral_types.h"
-#include "gutil/type_traits.h"
-
class Charmap {
public:
// Initializes with given uint32 values. For instance, the first
@@ -73,4 +69,3 @@ protected:
}
}
};
-DECLARE_POD(Charmap);
diff --git a/be/src/gutil/gscoped_ptr.h b/be/src/gutil/gscoped_ptr.h
index d1d88d1da7..01305b9294 100644
--- a/be/src/gutil/gscoped_ptr.h
+++ b/be/src/gutil/gscoped_ptr.h
@@ -102,11 +102,11 @@
#include
#include // For std::swap().
+#include
#include "gutil/basictypes.h"
#include "gutil/move.h"
#include "gutil/template_util.h"
-#include "gutil/type_traits.h"
namespace doris {
@@ -137,7 +137,7 @@ struct DefaultDeleter {
// cannot convert to T*.
enum { T_must_be_complete = sizeof(T) };
enum { U_must_be_complete = sizeof(U) };
-COMPILE_ASSERT((base::is_convertible::value),
+COMPILE_ASSERT((std::is_convertible::value),
U_ptr_must_implicitly_convert_to_T_ptr);
}
inline void operator()(T* ptr) const {
@@ -186,8 +186,8 @@ namespace internal {
template
struct IsNotRefCounted {
enum {
-value = !base::is_convertible::value &&
-!base::is_convertible::value
+value = !std::is_convertible::value &&
+!std::is_convertible::value
};
};
@@ -345,7 +345,7 @@ public:
// implementation of gscoped_ptr.
template
gscoped_ptr(gscoped_ptr other) : impl_(&other.impl_) {
-COMPILE_ASSERT(!base::is_array::value, U_cannot_be_an_array);
+COMPILE_ASSERT(!std::is_array::value, U_cannot_be_an_array);
}
// Constructor. Move constructor for C++03 move emulation of this type.
@@ -363,7 +363,7 @@ public:
// gscoped_ptr.
template
gscoped_ptr& operator=(gscoped_ptr rhs) {
-COMPILE_ASSERT(!base::is_array::value, U_ca
[GitHub] [doris] cambyzju opened a new pull request, #15325: [fix](array-type) forbid implicit cast of array type while load
cambyzju opened a new pull request, #15325: URL: https://github.com/apache/doris/pull/15325 # Proposed changes Issue Number: close https://github.com/apache/doris/issues/15320 ## Problem summary Array type now do not support cast from ARRAY to ARRAY, so we need to forbid it while load, otherwise be may crash. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] starocean999 opened a new pull request, #15326: [fix](nereids)fix some nereids bugs
starocean999 opened a new pull request, #15326: URL: https://github.com/apache/doris/pull/15326 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15323: [fix](s3 load) fix that `FE` can not access s3 objected-storage
github-actions[bot] commented on PR #15323: URL: https://github.com/apache/doris/pull/15323#issuecomment-1363874232 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15323: [fix](s3 load) fix that `FE` can not access s3 objected-storage
github-actions[bot] commented on PR #15323: URL: https://github.com/apache/doris/pull/15323#issuecomment-1363874251 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15299: [fix](iceberg-v2) fix fe iceberg split, add regression case
github-actions[bot] commented on PR #15299: URL: https://github.com/apache/doris/pull/15299#issuecomment-1363874623 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15299: [fix](iceberg-v2) fix fe iceberg split, add regression case
github-actions[bot] commented on PR #15299: URL: https://github.com/apache/doris/pull/15299#issuecomment-1363874598 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #15316: [bug](decimalv3) Fix wrong decimal scale for arithmetic expr
hello-stephen commented on PR #15316: URL: https://github.com/apache/doris/pull/15316#issuecomment-1363875597 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 36.26 seconds load time: 632 seconds storage size: 17123750883 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221223113212_clickbench_pr_67927.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman merged pull request #15299: [fix](iceberg-v2) fix fe iceberg split, add regression case
morningman merged PR #15299: URL: https://github.com/apache/doris/pull/15299 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](iceberg-v2) fix fe iceberg split, add regression case (#15299)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new ede68e075d [fix](iceberg-v2) fix fe iceberg split, add regression case
(#15299)
ede68e075d is described below
commit ede68e075ded258ec6dfdf3f2c68c55ff41dff8d
Author: slothever <18522955+w...@users.noreply.github.com>
AuthorDate: Fri Dec 23 19:33:00 2022 +0800
[fix](iceberg-v2) fix fe iceberg split, add regression case (#15299)
---
.../doris/planner/external/QueryScanProvider.java | 2 +-
.../iceberg/test_external_catalog_icebergv2.out| 18 --
.../iceberg/test_external_catalog_icebergv2.groovy | 13 +++--
3 files changed, 20 insertions(+), 13 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/QueryScanProvider.java
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/QueryScanProvider.java
index a2ce044bb7..a7453394f6 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/QueryScanProvider.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/QueryScanProvider.java
@@ -106,7 +106,7 @@ public abstract class QueryScanProvider implements
FileScanProviderIf {
TFileRangeDesc rangeDesc = createFileRangeDesc(fileSplit,
partitionValuesFromPath, pathPartitionKeys);
// external data lake table
if (split instanceof IcebergSplit) {
-IcebergScanProvider.setIcebergParams(rangeDesc,
(IcebergSplit) inputSplit);
+IcebergScanProvider.setIcebergParams(rangeDesc,
(IcebergSplit) split);
}
curLocations.getScanRange().getExtScanRange().getFileScanRange().addToRanges(rangeDesc);
LOG.debug("assign to backend {} with table split: {} ({}, {}),
location: {}",
diff --git
a/regression-test/data/external_table_emr_p2/iceberg/test_external_catalog_icebergv2.out
b/regression-test/data/external_table_emr_p2/iceberg/test_external_catalog_icebergv2.out
index f5cb0464e4..4d3a4b9176 100644
---
a/regression-test/data/external_table_emr_p2/iceberg/test_external_catalog_icebergv2.out
+++
b/regression-test/data/external_table_emr_p2/iceberg/test_external_catalog_icebergv2.out
@@ -1,27 +1,33 @@
-- This file is automatically generated. You should know what you did if you
want to edit this
-- !q01 --
-149990
+149988
-- !q02 --
-45132671
+1
+3
+4
+7
-- !q03 --
-0
+45132671
-- !q04 --
+0
+
+-- !q05 --
1 Customer#1 IVhzIApeRb ot,c,E15
25-989-741-2988 711.56 BUILDING to the even, regular platelets.
regular, ironic epitaphs nag e |
2 Customer#2 XSTf4,NCwDVaWNe6tEgvwfmRchLXak 13
23-768-687-3665 121.65 AUTOMOBILEl accounts. blithely ironic
theodolites integrate boldly: caref|
3 Customer#3 MG9kdTD2WBHm 1
11-719-748-33647498.12 AUTOMOBILE deposits eat slyly ironic, even
instructions. express foxes detect slyly. blithely even accounts abov |
--- !q05 -
+-- !q06 -
366778465
366778561
366778657
--- !q06 --
+-- !q07 --
10539361
130424833
2736865
--- !q07 --
+-- !q08 --
149990
\ No newline at end of file
diff --git
a/regression-test/suites/external_table_emr_p2/iceberg/test_external_catalog_icebergv2.groovy
b/regression-test/suites/external_table_emr_p2/iceberg/test_external_catalog_icebergv2.groovy
index 473bd76fc8..816e75b85e 100644
---
a/regression-test/suites/external_table_emr_p2/iceberg/test_external_catalog_icebergv2.groovy
+++
b/regression-test/suites/external_table_emr_p2/iceberg/test_external_catalog_icebergv2.groovy
@@ -34,12 +34,13 @@ suite("test_external_catalog_icebergv2", "p2") {
// test parquet format format
def q01 = {
qt_q01 """ select count(1) as c from customer_small;"""
-qt_q02 """ select count(1) from orders """
-qt_q03 """ select count(1) from customer_small where c_name =
'Customer#0063356' order by c_custkey limit 1; """
-qt_q04 """ select * from customer_small order by c_custkey limit 3
"""
-qt_q05 """ select o_orderkey from orders where o_orderkey > 652566
limit 3"""
-qt_q06 """ select o_orderkey from orders where o_custkey < 3357
limit 3"""
-qt_q07 """ select count(1) as c from customer;"""
+qt_q02 """ select c_custkey from customer_small group by c_custkey
limit 4;"""
+qt_q03 """ select count(1) from orders """
+qt_q04 """ select count(1) from customer_small where c_name =
'Customer#0063356' order by c_custkey limit 1; """
+qt_q05 """ select * from customer_small order by c_custkey limit 3
"""
+qt_q0
[GitHub] [doris] jackwener merged pull request #15290: [fix](Nereids): convert empty inner join to cross join
jackwener merged PR #15290: URL: https://github.com/apache/doris/pull/15290 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (ede68e075d -> 19cc65cc24)
This is an automated email from the ASF dual-hosted git repository.
jakevin pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
from ede68e075d [fix](iceberg-v2) fix fe iceberg split, add regression case
(#15299)
add 19cc65cc24 [fix](Nereids): fix bug of converting to NLJ. (#15290)
No new revisions were added by this update.
Summary of changes:
.../nereids/jobs/batch/NereidsRewriteJobExecutor.java | 2 ++
.../java/org/apache/doris/nereids/rules/RuleType.java | 1 +
.../{EliminateLimit.java => InnerToCrossJoin.java}| 15 ---
3 files changed, 11 insertions(+), 7 deletions(-)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/logical/{EliminateLimit.java
=> InnerToCrossJoin.java} (70%)
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xiaopoyu opened a new issue, #15327: [Bug] mysql external table, select be crash
xiaopoyu opened a new issue, #15327: URL: https://github.com/apache/doris/issues/15327 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version doris 1.2.0 mysql 8.0.18-X-Cluster-1.0.8 ### What's Wrong? create mysql external table , select * from table_name_external ERROR 1105 (HY000): RpcException, msg: org.apache.doris.rpc.RpcException: io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason ### What You Expected? select * rom table_name_external ### How to Reproduce? CREATE EXTERNAL RESOURCE mysql_resource properties ( "type"="jdbc", "user"="root", "password"="", "jdbc_url"="jdbc:mysql://*:3306/new?characterEncoding=utf8&serverTimezone=Asia/Shanghai&useSSL=false", "driver_url"="file:///opt/doris/connector/mysql-connector-java-8.0.21.jar", "driver_class"="com.mysql.cj.jdbc.Driver" ); CREATE EXTERNAL TABLE `doris_ods`.`table`( `id` BIGINT, `order_sn`varchar(255) , `main_order_sn` varchar(255) , `merchant_id` BIGINT, `merchant_type` INT , `merchant_name` varchar(255) , `merchant_img`varchar(255) , `fee_ratio` decimal(10,2) , `company_id` BIGINT, `order_type` INT , `goods_spu_type` INT , `pay_type`INT , `pay_status` INT , `order_status`INT , `close_type` INT , `delivery_type` INT , `delivery_price` decimal(10,2) , `delivery_sn` varchar(255) , `order_time` DATETIME , `pay_time`DATETIME , `if_can_settlement` INT , `if_settlement` INT , `settlement_time` DATETIME , `finsh_time` DATETIME , `receive_address_id` BIGINT, `receive_address_snapshot`STRING, `receive_contact` varchar(255) , `receive_mobile` varchar(50) , `member_id` BIGINT, `mobile` varchar(255) , `recharge_mobile` varchar(255) , `member_name` varchar(255) , `remark` varchar(255) , `total_price` decimal(10,2) , `goods_price` decimal(10,2) , `discount_price` decimal(10,2) , `reserve_price` decimal(10,2) , `order_discount_quota`decimal(10,2) , `real_price` decimal(10,2) , `if_integral` TINYINT , `use_integral`decimal(10,2) , `callback_integral` decimal(10,2) , `callBack_integral_ratio` varchar(255) , `if_single_card` TINYINT , `use_single_card_amount` decimal(10,2) , `single_card_service_amount` decimal(10,2) , `if_overseas_tao` INT , `real_name_authentication_id` BIGINT, `single_card_delivery_fee`decimal(10,2) , `integral_delivery_fee` decimal(10,2) , `cash_delivery_fee` decimal(10,2) , `pay_discount_quota` decimal(10,0) , `if_bill` INT , `unit`INT , `activity_type` INT , `activity_name` varchar(64) , `source` INT , `coupon_code` varchar(255) , `coupon_amount` varchar(255) , `coupon_id` INT , `activity_id` INT , `channel_id` INT , `delete_status` INT , `recharge_status` INT , `sign_id` varchar(255) , `platform_id` INT , `merchant_remark` varchar(255) , `create_time` DATETIME , `creater` varchar(255) , `update_time` DATETIME , `updator` varchar(255) , ) ENGINE=JDBC PROPERTIES( "resource" = "mysql_resource", "database" = "mysql", "table" = "table", "table_type"="mysql", "charset" = "utf8mb4" ); select * from table limit 1; ERROR 1105 (HY000): RpcException, msg: org
[GitHub] [doris] zhengshiJ opened a new pull request, #15328: [Fix](Nereids)fix scalarFunction and groupingSets
zhengshiJ opened a new pull request, #15328: URL: https://github.com/apache/doris/pull/15328 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BePPPower opened a new pull request, #15329: [fix](s3 outfile) Add the `use_path_style` parameter for s3 outfile
BePPPower opened a new pull request, #15329: URL: https://github.com/apache/doris/pull/15329 # Proposed changes Issue Number: close #xxx Currently, outfile did not support `use_path_style` parameter and use `virtual-host style` by default, however some Object-storage may only support `path_style` access mode. This pr add the `use_path_style` parameter for s3 outfile, so that different object-storage can use different access mode. ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15121: [feature](Nereids) support table generating function
github-actions[bot] commented on PR #15121: URL: https://github.com/apache/doris/pull/15121#issuecomment-1363894711 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15121: [feature](Nereids) support table generating function
github-actions[bot] commented on PR #15121: URL: https://github.com/apache/doris/pull/15121#issuecomment-1363894744 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15325: [fix](array-type) forbid implicit cast of array type while load
github-actions[bot] commented on PR #15325: URL: https://github.com/apache/doris/pull/15325#issuecomment-1363902341 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15325: [fix](array-type) forbid implicit cast of array type while load
github-actions[bot] commented on PR #15325: URL: https://github.com/apache/doris/pull/15325#issuecomment-1363902311 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow closed issue #14912: [Feature] (Nereids) lateral view
morrySnow closed issue #14912: [Feature] (Nereids) lateral view URL: https://github.com/apache/doris/issues/14912 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow merged pull request #15121: [feature](Nereids) support table generating function
morrySnow merged PR #15121: URL: https://github.com/apache/doris/pull/15121 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (19cc65cc24 -> 8c0de789e4)
This is an automated email from the ASF dual-hosted git repository.
morrysnow pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
from 19cc65cc24 [fix](Nereids): fix bug of converting to NLJ. (#15290)
add 8c0de789e4 [feature](Nereids) support table generating function
(#15121)
No new revisions were added by this update.
Summary of changes:
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 14 +-
.../catalog/BuiltinTableGeneratingFunctions.java | 63 +
.../org/apache/doris/catalog/FunctionHelper.java | 20 ++-
.../org/apache/doris/catalog/FunctionRegistry.java | 1 +
.../doris/nereids/analyzer/UnboundFunction.java| 7 +-
.../doris/nereids/analyzer/UnboundRelation.java| 21 ---
.../identifier/IdentifierWithDatabase.java | 65 -
.../apache/doris/nereids/cost/CostCalculator.java | 11 ++
.../glue/translator/ExpressionTranslator.java | 25
.../glue/translator/PhysicalPlanTranslator.java| 44 --
.../doris/nereids/parser/LogicalPlanBuilder.java | 45 +++---
.../properties/ChildOutputPropertyDeriver.java | 7 +
.../nereids/properties/RequestPropertyDeriver.java | 7 +
.../org/apache/doris/nereids/rules/RuleSet.java| 4 +
.../org/apache/doris/nereids/rules/RuleType.java | 7 +-
.../doris/nereids/rules/analysis/BindFunction.java | 28 +++-
.../nereids/rules/analysis/BindSlotReference.java | 28
.../expression/rewrite/ExpressionRewrite.java | 18 +++
.../LogicalGenerateToPhysicalGenerate.java}| 29 ++--
.../rules/rewrite/logical/MergeGenerates.java | 53 +++
.../doris/nereids/stats/StatsCalculator.java | 35 +
.../trees/expressions/functions/BoundFunction.java | 2 +-
.../expressions/functions/Function.java} | 23 ++--
.../expressions/functions/SearchSignature.java | 25 +++-
.../functions/generator/ExplodeBitmap.java | 67 +
.../functions/generator/ExplodeBitmapOuter.java| 67 +
.../generator/ExplodeJsonArrayDouble.java | 67 +
.../generator/ExplodeJsonArrayDoubleOuter.java | 67 +
.../functions/generator/ExplodeJsonArrayInt.java | 67 +
.../generator/ExplodeJsonArrayIntOuter.java| 67 +
.../generator/ExplodeJsonArrayString.java | 66 +
.../generator/ExplodeJsonArrayStringOuter.java | 66 +
.../functions/generator/ExplodeNumbers.java| 66 +
.../functions/generator/ExplodeNumbersOuter.java | 66 +
.../functions/generator/ExplodeSplit.java | 67 +
.../functions/generator/ExplodeSplitOuter.java | 67 +
.../generator/TableGeneratingFunction.java | 45 ++
.../expressions/visitor/ExpressionVisitor.java | 9 +-
.../visitor/TableGeneratingFunctionVisitor.java| 87
.../plans/algebra/Generate.java} | 23 ++--
.../trees/plans/logical/LogicalGenerate.java | 135 ++
.../nereids/trees/plans/logical/LogicalHaving.java | 2 +-
.../trees/plans/physical/PhysicalGenerate.java | 153 +
.../nereids/trees/plans/visitor/PlanVisitor.java | 12 +-
.../doris/nereids/util/TypeCoercionUtils.java | 2 +-
.../apache/doris/planner/TableFunctionNode.java| 15 ++
.../apache/doris/statistics/ColumnStatistic.java | 3 +-
.../data/nereids_syntax_p0/lateral_view.out| 91
.../data/nereids_syntax_p0/test_date_add.out | 4 +-
.../data/nereids_syntax_p0/test_date_sub.out | 4 +-
.../suites/nereids_syntax_p0/lateral_view.groovy | 66 +
51 files changed, 1858 insertions(+), 175 deletions(-)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/catalog/BuiltinTableGeneratingFunctions.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/identifier/IdentifierWithDatabase.java
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/{analyzer/identifier/TableIdentifier.java
=> rules/implementation/LogicalGenerateToPhysicalGenerate.java} (54%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/logical/MergeGenerates.java
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/{analyzer/identifier/TableIdentifier.java
=> trees/expressions/functions/Function.java} (65%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/generator/ExplodeBitmap.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/generator/ExplodeBitmapOuter.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/generator/ExplodeJsonArrayDouble.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/generator/ExplodeJsonArrayDoubleOuter.java
create mode 100644
fe/fe-core/src/mai
[GitHub] [doris] HappenLee opened a new pull request, #15330: [pipeline](opt) opt the exec performance of pipe exec engine
HappenLee opened a new pull request, #15330: URL: https://github.com/apache/doris/pull/15330 # Proposed changes Before ``` mysql> select count() from ( select l_orderkey from lineitem where l_linenumber % 3 = 0 except select l_orderkey from lineitem where l_partkey % 7 = 0 except select l_orderkey from lineitem where l_partkey % 5 = 0 )a; +--+ | count() | +--+ | 18658876 | +--+ 1 row in set (6.66 sec) ``` After ``` mysql> select count() from ( select l_orderkey from lineitem where l_linenumber % 3 = 0 except select l_orderkey from lineitem where l_partkey % 7 = 0 except select l_orderkey from lineitem where l_partkey % 5 = 0 )a; +--+ | count() | +--+ | 18658876 | +--+ 1 row in set (5.74 sec) ``` ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15330: [pipeline](opt) opt the exec performance of pipe exec engine
github-actions[bot] commented on PR #15330: URL: https://github.com/apache/doris/pull/15330#issuecomment-1363922676 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
