Re: [PR] [feature](nereids)support correlated scalar subquery without scalar agg [doris]
starocean999 merged PR #39471: URL: https://github.com/apache/doris/pull/39471 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [feature](nereids)support correlated scalar subquery without scalar agg (#39471)
This is an automated email from the ASF dual-hosted git repository.
starocean999 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new d513b11eb4e [feature](nereids)support correlated scalar subquery
without scalar agg (#39471)
d513b11eb4e is described below
commit d513b11eb4e23cc583f6048aada1baae5dcc3427
Author: starocean999 <40539150+starocean...@users.noreply.github.com>
AuthorDate: Fri Sep 20 09:34:26 2024 +0800
[feature](nereids)support correlated scalar subquery without scalar agg
(#39471)
support correlated scalar subquery without scalar agg like:
`select t1.c1 from t1
where t1.c2 > (select t2.c2 from t2 where t1.c1 = t2.c1);`
after this pr, nereids would produce a correct plan for above sql.
---
.../doris/nereids/jobs/executor/Rewriter.java | 7 +-
.../post/PushDownFilterThroughProject.java | 3 +-
.../doris/nereids/processor/post/Validator.java| 3 +-
.../nereids/rules/analysis/ExpressionAnalyzer.java | 15 --
.../nereids/rules/analysis/FillUpMissingSlots.java | 73 --
.../nereids/rules/analysis/SubExprAnalyzer.java| 259 +
.../nereids/rules/analysis/SubqueryToApply.java| 160 ++---
.../expression/rules/FoldConstantRuleOnFE.java | 19 +-
...CorrelatedFilterUnderApplyAggregateProject.java | 23 +-
.../rules/rewrite/PullUpProjectUnderApply.java | 10 +-
.../rewrite/PushDownFilterThroughProject.java | 4 +-
.../nereids/trees/expressions/ScalarSubquery.java | 62 -
.../expressions/functions/AlwaysNotNullable.java | 9 +
...ysNotNullable.java => NoneMovableFunction.java} | 11 +-
.../trees/expressions/functions/agg/ArrayAgg.java | 7 +
.../trees/expressions/functions/agg/BitmapAgg.java | 6 +
.../expressions/functions/agg/BitmapIntersect.java | 6 +
.../expressions/functions/agg/BitmapUnion.java | 6 +
.../functions/agg/BitmapUnionCount.java| 7 +
.../expressions/functions/agg/BitmapUnionInt.java | 6 +
.../expressions/functions/agg/CollectList.java | 7 +
.../expressions/functions/agg/CollectSet.java | 7 +
.../trees/expressions/functions/agg/Count.java | 6 +
.../expressions/functions/agg/CountByEnum.java | 6 +
.../functions/agg/GroupArrayIntersect.java | 7 +
.../trees/expressions/functions/agg/Histogram.java | 6 +
.../trees/expressions/functions/agg/HllUnion.java | 6 +
.../expressions/functions/agg/HllUnionAgg.java | 6 +
.../expressions/functions/agg/IntersectCount.java | 6 +
.../trees/expressions/functions/agg/MapAgg.java| 7 +
.../functions/agg/MultiDistinctCount.java | 6 +
.../functions/agg/MultiDistinctSum0.java | 23 ++
.../trees/expressions/functions/agg/Ndv.java | 6 +
.../expressions/functions/agg/PercentileArray.java | 7 +
.../expressions/functions/agg/QuantileUnion.java | 6 +
.../expressions/functions/agg/SequenceCount.java | 6 +
.../trees/expressions/functions/agg/Sum0.java | 22 ++
.../expressions/functions/scalar/AssertTrue.java | 3 +-
.../org/apache/doris/nereids/trees/plans/Plan.java | 4 +-
.../doris/nereids/trees/plans/algebra/Project.java | 22 +-
.../trees/plans/logical/LogicalProject.java| 12 +-
.../nereids/rules/expression/FoldConstantTest.java | 16 ++
.../data/nereids_hint_tpcds_p0/shape/query41.out | 2 +-
.../subquery/correlated_scalar_subquery.out| 108 +
.../shape/query41.out | 2 +-
.../noStatsRfPrune/query41.out | 2 +-
.../no_stats_shape/query41.out | 2 +-
.../rf_prune/query41.out | 2 +-
.../nereids_tpcds_shape_sf100_p0/shape/query41.out | 2 +-
.../tpcds_sf100/noStatsRfPrune/query41.out | 2 +-
.../tpcds_sf100/no_stats_shape/query41.out | 2 +-
.../new_shapes_p0/tpcds_sf100/rf_prune/query41.out | 2 +-
.../new_shapes_p0/tpcds_sf100/shape/query41.out| 2 +-
.../new_shapes_p0/tpcds_sf1000/shape/query41.out | 2 +-
.../subquery/correlated_scalar_subquery.groovy | 223 ++
55 files changed, 1094 insertions(+), 152 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java
index 4ab4165a446..9c3d3ebecba 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java
@@ -188,7 +188,12 @@ public class Rewriter extends AbstractBatchJobExecutor {
// after doing NormalizeAggregate in analysis job
// we need run the following 2 rules to make
AGG_SCALAR_SUBQUERY_TO_WINDOW_FUNCTION work
bottomUp
(doris) branch master updated (34ab55ff860 -> c82165ee16d)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 34ab55ff860 [improve](load) change memtable memory limiter policy (#41018) add c82165ee16d [feature](functions) impl scalar functions normal_cdf,to_iso8601,from_iso8601_date (#40695) No new revisions were added by this update. Summary of changes: be/src/vec/functions/date_time_transforms.h| 23 ++ .../function_date_or_datetime_to_string.cpp| 5 + be/src/vec/functions/function_timestamp.cpp| 244 - be/src/vec/functions/math.cpp | 69 ++ .../doris/catalog/BuiltinScalarFunctions.java | 6 + .../functions/scalar/FromIso8601Date.java | 72 ++ .../expressions/functions/scalar/NormalCdf.java| 69 ++ .../expressions/functions/scalar/ToIso8601.java| 76 +++ .../expressions/visitor/ScalarFunctionVisitor.java | 15 ++ .../datetime_functions/test_from_iso8601_date.out | 244 + .../datetime_functions/test_to_iso8601.out | 194 .../math_functions/test_normal_cdf.out | 201 + .../test_from_iso8601_date.groovy | 149 + .../datetime_functions/test_to_iso8601.groovy | 148 + .../math_functions/test_normal_cdf.groovy | 93 15 files changed, 1607 insertions(+), 1 deletion(-) create mode 100644 fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/FromIso8601Date.java create mode 100644 fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/NormalCdf.java create mode 100644 fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/ToIso8601.java create mode 100644 regression-test/data/query_p0/sql_functions/datetime_functions/test_from_iso8601_date.out create mode 100644 regression-test/data/query_p0/sql_functions/datetime_functions/test_to_iso8601.out create mode 100644 regression-test/data/query_p0/sql_functions/math_functions/test_normal_cdf.out create mode 100644 regression-test/suites/query_p0/sql_functions/datetime_functions/test_from_iso8601_date.groovy create mode 100644 regression-test/suites/query_p0/sql_functions/datetime_functions/test_to_iso8601.groovy create mode 100644 regression-test/suites/query_p0/sql_functions/math_functions/test_normal_cdf.groovy - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
englefly commented on code in PR #40349:
URL: https://github.com/apache/doris/pull/40349#discussion_r1768043925
##
fe/fe-core/src/main/java/org/apache/doris/nereids/stats/StatsCalculator.java:
##
@@ -205,6 +210,23 @@ public Map
getTotalColumnStatisticMap() {
return totalColumnStatisticMap;
}
+/**
+ * disable join reorder if any table row count is not available.
+ */
+public static void disableJoinReorderIfTableRowCountNotAvailable(
+List scans, CascadesContext context) {
+StatsCalculator calculator = new StatsCalculator(context);
+for (LogicalOlapScan scan : scans) {
+double rowCount = calculator.getOlapTableRowCount(scan);
+if (rowCount == -1 && ConnectContext.get() != null) {
Review Comment:
this is the return value of OlapTable.getRowCountForIndex().
explained the "-1" at the spec of OlapTable.getRowCountForIndex()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](functions) impl scalar functions normal_cdf,to_iso8601,from_iso8601_date [doris]
morningman merged PR #40695: URL: https://github.com/apache/doris/pull/40695 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](table_options) fix potential NPE when quering table_options systable (#40944) [doris]
doris-robot commented on PR #41040: URL: https://github.com/apache/doris/pull/41040#issuecomment-2362916434 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
englefly commented on PR #40349: URL: https://github.com/apache/doris/pull/40349#issuecomment-2362915634 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](opt) improve BE code readability of multi_match_any function [doris]
superdiaodiao commented on PR #39354: URL: https://github.com/apache/doris/pull/39354#issuecomment-2362916076 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Fix] (jdbc-scanner) Fix jdbc scanner memory leak because it didn't close `outputTable`. [doris]
kaka11chen opened a new pull request, #41041: URL: https://github.com/apache/doris/pull/41041 ## Proposed changes [Fix] (jdbc-scanner) Fix jdbc scanner memory leak because it didn't close `outputTable`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix] (jdbc-scanner) Fix jdbc scanner memory leak because it didn't close `outputTable`. [doris]
doris-robot commented on PR #41041: URL: https://github.com/apache/doris/pull/41041#issuecomment-2362917386 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix] (jdbc-scanner) Fix jdbc scanner memory leak because it didn't close `outputTable`. [doris]
kaka11chen commented on PR #41041: URL: https://github.com/apache/doris/pull/41041#issuecomment-2362917551 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [test](branch2.1) test memory [doris]
xinyiZzz opened a new pull request, #41042: URL: https://github.com/apache/doris/pull/41042 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](branch2.1) test memory [doris]
xinyiZzz commented on PR #41042: URL: https://github.com/apache/doris/pull/41042#issuecomment-2362918027 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](systable) fix unstable case for partitions table (#40553) [doris]
doris-robot commented on PR #41043: URL: https://github.com/apache/doris/pull/41043#issuecomment-2362919035 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](systable) fix unstable case for partitions table (#40553) [doris]
morningman opened a new pull request, #41043: URL: https://github.com/apache/doris/pull/41043 Followup #40153 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](branch2.1) test memory [doris]
doris-robot commented on PR #41042: URL: https://github.com/apache/doris/pull/41042#issuecomment-2362917794 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](systable) fix unstable case for partitions table (#40553) [doris]
morningman commented on PR #41043: URL: https://github.com/apache/doris/pull/41043#issuecomment-2362919439 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](test) remove enable_nereids_planner=false in cases [doris]
morrySnow merged PR #41035: URL: https://github.com/apache/doris/pull/41035 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
englefly commented on code in PR #40349:
URL: https://github.com/apache/doris/pull/40349#discussion_r1768045163
##
fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java:
##
@@ -256,6 +259,17 @@ private Plan planWithoutLock(
}
}
+// if we cannot get table row count, skip join reorder
+// except:
+// 1. user set leading hint
+// 2. ut test. In ut test, FeConstants.enableInternalSchemaDb is
false or FeConstants.runningUnitTest is true
+if (FeConstants.enableInternalSchemaDb && !FeConstants.runningUnitTest
+&& !cascadesContext.isLeadingDisableJoinReorder()) {
+List scans = cascadesContext.getRewritePlan()
+.collectToList(LogicalOlapScan.class::isInstance);
Review Comment:
this function depends on OlapTable.getRowCountForIndex().
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](table_options) fix potential NPE when quering table_options systable (#40944) [doris]
morningman opened a new pull request, #41040: URL: https://github.com/apache/doris/pull/41040 bp #40944 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](table_options) fix potential NPE when quering table_options systable (#40944) [doris]
morningman commented on PR #41040: URL: https://github.com/apache/doris/pull/41040#issuecomment-2362916532 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](function) hour/minute/second functions support time as an argument. [doris]
Mryange commented on PR #41008: URL: https://github.com/apache/doris/pull/41008#issuecomment-2362920983 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](cpu-profile) enable cpu profile in BE webui (#40330) [doris]
doris-robot commented on PR #41044: URL: https://github.com/apache/doris/pull/41044#issuecomment-2362924535 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [opt](cpu-profile) enable cpu profile in BE webui (#40330) [doris]
morningman opened a new pull request, #41044: URL: https://github.com/apache/doris/pull/41044 bp #40330 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](cpu-profile) enable cpu profile in BE webui (#40330) [doris]
morningman commented on PR #41044: URL: https://github.com/apache/doris/pull/41044#issuecomment-2362924563 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [chore](test) remove enable_nereids_planner=false in cases (#41035)
This is an automated email from the ASF dual-hosted git repository.
morrysnow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 04223de2eb4 [chore](test) remove enable_nereids_planner=false in cases
(#41035)
04223de2eb4 is described below
commit 04223de2eb4b9bcc8b3ec9d52ca158a3e6f870a2
Author: morrySnow <101034200+morrys...@users.noreply.github.com>
AuthorDate: Fri Sep 20 14:24:14 2024 +0800
[chore](test) remove enable_nereids_planner=false in cases (#41035)
- test_cloud_mow_insert_with_retry
---
.../cloud/test_cloud_mow_insert_with_retry.out | 7 ---
.../cloud/test_cloud_mow_insert_with_retry.groovy | 70 ++
2 files changed, 30 insertions(+), 47 deletions(-)
diff --git
a/regression-test/data/fault_injection_p0/cloud/test_cloud_mow_insert_with_retry.out
b/regression-test/data/fault_injection_p0/cloud/test_cloud_mow_insert_with_retry.out
index 979483692d3..f3a322f2b9f 100644
---
a/regression-test/data/fault_injection_p0/cloud/test_cloud_mow_insert_with_retry.out
+++
b/regression-test/data/fault_injection_p0/cloud/test_cloud_mow_insert_with_retry.out
@@ -6,10 +6,3 @@
-- !sql --
1 1 1
--- !sql --
-1 1 1
-2 2 2
-
--- !sql --
-1 1 1
-
diff --git
a/regression-test/suites/fault_injection_p0/cloud/test_cloud_mow_insert_with_retry.groovy
b/regression-test/suites/fault_injection_p0/cloud/test_cloud_mow_insert_with_retry.groovy
index f7038b80e42..99eb18a4917 100644
---
a/regression-test/suites/fault_injection_p0/cloud/test_cloud_mow_insert_with_retry.groovy
+++
b/regression-test/suites/fault_injection_p0/cloud/test_cloud_mow_insert_with_retry.groovy
@@ -29,11 +29,10 @@ suite("test_cloud_mow_insert_with_retry", "nonConcurrent") {
def dbName = "regression_test_fault_injection_p0_cloud"
def table1 = dbName + ".test_cloud_mow_insert_with_retry"
setFeConfigTemporary(customFeConfig) {
-for (item in ["legacy", "nereids"]) {
-try {
-
GetDebugPoint().enableDebugPointForAllBEs("CloudEngineCalcDeleteBitmapTask.execute.enable_wait")
-sql "DROP TABLE IF EXISTS ${table1} FORCE;"
-sql """ CREATE TABLE IF NOT EXISTS ${table1} (
+try {
+
GetDebugPoint().enableDebugPointForAllBEs("CloudEngineCalcDeleteBitmapTask.execute.enable_wait")
+sql "DROP TABLE IF EXISTS ${table1} FORCE;"
+sql """ CREATE TABLE IF NOT EXISTS ${table1} (
`k1` int NOT NULL,
`c1` int,
`c2` int
@@ -43,44 +42,35 @@ suite("test_cloud_mow_insert_with_retry", "nonConcurrent") {
"enable_unique_key_merge_on_write" = "true",
"disable_auto_compaction" = "true",
"replication_num" = "1"); """
-connect(user = context.config.jdbcUser, password =
context.config.jdbcPassword, url = context.config.jdbcUrl +
"&useLocalSessionState=true") {
-if (item == "nereids") {
-sql """ set enable_nereids_planner=true; """
-sql """ set enable_fallback_to_original_planner=false;
"""
-} else {
-sql """ set enable_nereids_planner = false; """
-}
-def timeout = 2000
-def now = System.currentTimeMillis()
-sql "insert into ${table1} values(1,1,1);"
-def time_diff = System.currentTimeMillis() - now
-logger.info("time_diff:" + time_diff)
-assertTrue(time_diff > timeout, "insert or delete should
take over " + timeout + " ms")
+connect(user = context.config.jdbcUser, password =
context.config.jdbcPassword, url = context.config.jdbcUrl +
"&useLocalSessionState=true") {
+def timeout = 2000
+def now = System.currentTimeMillis()
+sql "insert into ${table1} values(1,1,1);"
+def time_diff = System.currentTimeMillis() - now
+logger.info("time_diff:" + time_diff)
+assertTrue(time_diff > timeout, "insert or delete should take
over " + timeout + " ms")
-now = System.currentTimeMillis()
-sql "insert into ${table1} values(2,2,2);"
-time_diff = System.currentTimeMillis() - now
-logger.info("time_diff:" + time_diff)
-assertTrue(time_diff > timeout, "insert or delete should
take over " + timeout + " ms")
-order_qt_sql "select * from ${table1};"
+now = System.currentTimeMillis()
+sql "insert into ${table1} values(2,2,2);"
+time_diff = System.currentTimeMillis() - now
+
Re: [PR] add log to debug refresh catalog [doris]
doris-robot commented on PR #41045: URL: https://github.com/apache/doris/pull/41045#issuecomment-2362926793 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](branch2.1) test memory [doris]
github-actions[bot] commented on PR #41042: URL: https://github.com/apache/doris/pull/41042#issuecomment-2362926173 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](jdbc-scanner) Fix jdbc scanner memory leak because it didn't close `outputTable`. [doris]
github-actions[bot] commented on PR #41041: URL: https://github.com/apache/doris/pull/41041#issuecomment-2362927785 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](jdbc-scanner) Fix jdbc scanner memory leak because it didn't close `outputTable`. [doris]
github-actions[bot] commented on PR #41041: URL: https://github.com/apache/doris/pull/41041#issuecomment-2362927828 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](systable) fix unstable case for partitions table (#40553) [doris]
github-actions[bot] commented on PR #41043: URL: https://github.com/apache/doris/pull/41043#issuecomment-2362928165 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] add log to debug refresh catalog [doris]
wuwenchi opened a new pull request, #41045: URL: https://github.com/apache/doris/pull/41045 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [Feature](json) Support json_search function (#40948)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 7a00dd4a4ac [Feature](json) Support json_search function (#40948)
7a00dd4a4ac is described below
commit 7a00dd4a4ac6d71201d35e8666f2f3f56fda845f
Author: Lijia Liu
AuthorDate: Fri Sep 20 14:30:07 2024 +0800
[Feature](json) Support json_search function (#40948)
Like mysql, json_search returns the path which point to a json string
witch match the pattern.
`SELECT JSON_SEARCH('["A",[{"B":"1"}],{"C":"AB"},{"D":"BC"}]', 'one',
'A_') as res;`
```
+--+
| res |
+--+
| "$[2].C" |
+--+
```
---
be/src/util/jsonb_document.h | 29 ++
be/src/vec/functions/function_jsonb.cpp| 354 +
be/src/vec/functions/like.cpp | 204 ++--
be/src/vec/functions/like.h| 4 +
.../doris/catalog/BuiltinScalarFunctions.java | 2 +
.../expressions/functions/scalar/JsonSearch.java | 62
.../expressions/visitor/ScalarFunctionVisitor.java | 5 +
gensrc/script/doris_builtins_functions.py | 2 +
gensrc/script/gen_builtins_functions.py| 11 +-
.../sql_functions/json_functions/json_search.out | 139
.../json_functions/json_search.groovy | 121 +++
11 files changed, 827 insertions(+), 106 deletions(-)
diff --git a/be/src/util/jsonb_document.h b/be/src/util/jsonb_document.h
index 8a95ccef8d9..2a9cf8a8191 100644
--- a/be/src/util/jsonb_document.h
+++ b/be/src/util/jsonb_document.h
@@ -345,6 +345,22 @@ struct leg_info {
///type: 0 is member 1 is array
unsigned int type;
+
+bool to_string(std::string* str) const {
+if (type == MEMBER_CODE) {
+str->push_back(BEGIN_MEMBER);
+str->append(leg_ptr, leg_len);
+return true;
+} else if (type == ARRAY_CODE) {
+str->push_back(BEGIN_ARRAY);
+std::string int_str = std::to_string(array_index);
+str->append(int_str);
+str->push_back(END_ARRAY);
+return true;
+} else {
+return false;
+}
+}
};
class JsonbPath {
@@ -362,6 +378,19 @@ public:
leg_vector.emplace_back(leg.release());
}
+void pop_leg_from_leg_vector() { leg_vector.pop_back(); }
+
+bool to_string(std::string* res) const {
+res->push_back(SCOPE);
+for (const auto& leg : leg_vector) {
+auto valid = leg->to_string(res);
+if (!valid) {
+return false;
+}
+}
+return true;
+}
+
size_t get_leg_vector_size() { return leg_vector.size(); }
leg_info* get_leg_from_leg_vector(size_t i) { return leg_vector[i].get(); }
diff --git a/be/src/vec/functions/function_jsonb.cpp
b/be/src/vec/functions/function_jsonb.cpp
index 53ccec756fd..45864a07bdd 100644
--- a/be/src/vec/functions/function_jsonb.cpp
+++ b/be/src/vec/functions/function_jsonb.cpp
@@ -61,7 +61,9 @@
#include "vec/data_types/data_type_string.h"
#include "vec/functions/function.h"
#include "vec/functions/function_string.h"
+#include "vec/functions/like.h"
#include "vec/functions/simple_function_factory.h"
+#include "vec/json/simd_json_parser.h"
#include "vec/utils/util.hpp"
namespace doris::vectorized {
@@ -1598,6 +1600,356 @@ struct JsonbContainsAndPathImpl {
}
};
+class FunctionJsonSearch : public IFunction {
+private:
+using OneFun = std::function;
+static Status always_one(size_t i, bool* res) {
+*res = true;
+return Status::OK();
+}
+static Status always_all(size_t i, bool* res) {
+*res = false;
+return Status::OK();
+}
+
+using CheckNullFun = std::function;
+static bool always_not_null(size_t) { return false; }
+static bool always_null(size_t) { return true; }
+
+using GetJsonStringRefFun = std::function;
+
+Status matched(const std::string_view& str, LikeState* state, unsigned
char* res) const {
+StringRef pattern; // not used
+StringRef value_val(str.data(), str.size());
+return (state->scalar_function)(&state->search_state, value_val,
pattern, res);
+}
+
+/**
+ * Recursive search for matching string, if found, the result will be
added to a vector
+ * @param element json element
+ * @param one_match
+ * @param search_str
+ * @param cur_path
+ * @param matches The path that has already been matched
+ * @return true if matched else false
+ */
+bool find_matches(const SimdJSONParser::Element& element, const bool&
one_match,
+ LikeState* state, JsonbPath* cur_path,
+ std::unordered_set* matches) const {
+if (element.isString
Re: [PR] [Feat](json) Support json_search function in 2.0 [doris]
github-actions[bot] commented on PR #40962: URL: https://github.com/apache/doris/pull/40962#issuecomment-2362931331 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [Feature](json) Support json_search function in 2.0 (#40962)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 8e6e82d91ed [Feature](json) Support json_search function in 2.0
(#40962)
8e6e82d91ed is described below
commit 8e6e82d91edd5b68c40d80d514d9cb386e1e32d1
Author: Lijia Liu
AuthorDate: Fri Sep 20 14:30:57 2024 +0800
[Feature](json) Support json_search function in 2.0 (#40962)
---
be/src/util/jsonb_document.h | 29 ++
be/src/vec/functions/function_jsonb.cpp| 354 +
be/src/vec/functions/like.cpp | 204 ++--
be/src/vec/functions/like.h| 4 +
.../doris/catalog/BuiltinScalarFunctions.java | 2 +
.../expressions/functions/scalar/JsonSearch.java | 62
.../expressions/visitor/ScalarFunctionVisitor.java | 5 +
gensrc/script/doris_builtins_functions.py | 2 +
gensrc/script/gen_builtins_functions.py| 11 +-
.../sql_functions/json_functions/json_search.out | 139
.../json_functions/json_search.groovy | 121 +++
11 files changed, 827 insertions(+), 106 deletions(-)
diff --git a/be/src/util/jsonb_document.h b/be/src/util/jsonb_document.h
index 041fdc14541..55128a5d121 100644
--- a/be/src/util/jsonb_document.h
+++ b/be/src/util/jsonb_document.h
@@ -340,6 +340,22 @@ struct leg_info {
///type: 0 is member 1 is array
unsigned int type;
+
+bool to_string(std::string* str) const {
+if (type == MEMBER_CODE) {
+str->push_back(BEGIN_MEMBER);
+str->append(leg_ptr, leg_len);
+return true;
+} else if (type == ARRAY_CODE) {
+str->push_back(BEGIN_ARRAY);
+std::string int_str = std::to_string(array_index);
+str->append(int_str);
+str->push_back(END_ARRAY);
+return true;
+} else {
+return false;
+}
+}
};
class JsonbPath {
@@ -357,6 +373,19 @@ public:
leg_vector.emplace_back(leg.release());
}
+void pop_leg_from_leg_vector() { leg_vector.pop_back(); }
+
+bool to_string(std::string* res) const {
+res->push_back(SCOPE);
+for (const auto& leg : leg_vector) {
+auto valid = leg->to_string(res);
+if (!valid) {
+return false;
+}
+}
+return true;
+}
+
size_t get_leg_vector_size() { return leg_vector.size(); }
leg_info* get_leg_from_leg_vector(size_t i) { return leg_vector[i].get(); }
diff --git a/be/src/vec/functions/function_jsonb.cpp
b/be/src/vec/functions/function_jsonb.cpp
index 37f2a5d59fd..b6029dc790f 100644
--- a/be/src/vec/functions/function_jsonb.cpp
+++ b/be/src/vec/functions/function_jsonb.cpp
@@ -62,7 +62,9 @@
#include "vec/data_types/data_type_string.h"
#include "vec/functions/function.h"
#include "vec/functions/function_string.h"
+#include "vec/functions/like.h"
#include "vec/functions/simple_function_factory.h"
+#include "vec/json/simd_json_parser.h"
#include "vec/utils/util.hpp"
namespace doris::vectorized {
@@ -1426,6 +1428,356 @@ struct JsonbContainsAndPathImpl {
}
};
+class FunctionJsonSearch : public IFunction {
+private:
+using OneFun = std::function;
+static Status always_one(size_t i, bool* res) {
+*res = true;
+return Status::OK();
+}
+static Status always_all(size_t i, bool* res) {
+*res = false;
+return Status::OK();
+}
+
+using CheckNullFun = std::function;
+static bool always_not_null(size_t) { return false; }
+static bool always_null(size_t) { return true; }
+
+using GetJsonStringRefFun = std::function;
+
+Status matched(const std::string_view& str, LikeState* state, unsigned
char* res) const {
+StringRef pattern; // not used
+StringRef value_val(str.data(), str.size());
+return (state->scalar_function)(&state->search_state, value_val,
pattern, res);
+}
+
+/**
+ * Recursive search for matching string, if found, the result will be
added to a vector
+ * @param element json element
+ * @param one_match
+ * @param search_str
+ * @param cur_path
+ * @param matches The path that has already been matched
+ * @return true if matched else false
+ */
+bool find_matches(const SimdJSONParser::Element& element, const bool&
one_match,
+ LikeState* state, JsonbPath* cur_path,
+ std::unordered_set* matches) const {
+if (element.isString()) {
+const std::string_view str = element.getString();
+unsigned char res;
+RETURN_IF_ERROR(matched(str, state, &res));
+if (res) {
+std::string str;
+auto valid = cur_path->to_string(&str);
+
Re: [PR] [Feat](json) Support json_search function [doris]
lide-reed merged PR #40948: URL: https://github.com/apache/doris/pull/40948 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](skew & kurt) New aggregate function skew & kurt [doris]
zhiqiang- commented on code in PR #40945:
URL: https://github.com/apache/doris/pull/40945#discussion_r1768056640
##
be/src/vec/aggregate_functions/aggregate_function_statistic.h:
##
@@ -0,0 +1,162 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+#include
+#include
+#include
+#include
+
+#include "common/exception.h"
+#include "common/status.h"
+#include "moments.h"
+#include "vec/aggregate_functions/aggregate_function.h"
+#include "vec/aggregate_functions/moments.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/columns/column_vector.h"
+#include "vec/common/assert_cast.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_nullable.h"
+#include "vec/data_types/data_type_number.h"
+
+namespace doris::vectorized {
+
+enum class StatisticsFunctionKind : uint8_t { skewPop, kurtPop };
+
+inline std::string to_string(StatisticsFunctionKind kind) {
+switch (kind) {
+case StatisticsFunctionKind::skewPop:
+return "skewness";
+case StatisticsFunctionKind::kurtPop:
+return "kurtosis";
+default:
+return "Unknown";
+}
+}
+
+template
+struct StatFuncOneArg {
+using Type1 = T;
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feat](json) Support json_search function in 2.0 [doris]
github-actions[bot] commented on PR #40962: URL: https://github.com/apache/doris/pull/40962#issuecomment-2362931294 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](skew & kurt) New aggregate function skew & kurt [doris]
zhiqiang- commented on code in PR #40945:
URL: https://github.com/apache/doris/pull/40945#discussion_r1768055749
##
be/src/vec/aggregate_functions/aggregate_function_statistic.h:
##
@@ -0,0 +1,162 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+#include
+#include
+#include
+#include
+
+#include "common/exception.h"
+#include "common/status.h"
+#include "moments.h"
+#include "vec/aggregate_functions/aggregate_function.h"
+#include "vec/aggregate_functions/moments.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/columns/column_vector.h"
+#include "vec/common/assert_cast.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_nullable.h"
+#include "vec/data_types/data_type_number.h"
+
+namespace doris::vectorized {
+
+enum class StatisticsFunctionKind : uint8_t { skewPop, kurtPop };
Review Comment:
renamed to `STATISTICS_FUNCTION_KIND`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feat](json) Support json_search function in 2.0 [doris]
lide-reed merged PR #40962: URL: https://github.com/apache/doris/pull/40962 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](cpu-profile) enable cpu profile in BE webui (#40330) [doris]
github-actions[bot] commented on PR #41044: URL: https://github.com/apache/doris/pull/41044#issuecomment-2362934119 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [nereids](enhance) Added order by and limit expr for show routine load [doris]
Toms1999 commented on PR #40938: URL: https://github.com/apache/doris/pull/40938#issuecomment-2362932867 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](skew & kurt) New aggregate function skew & kurt [doris]
zhiqiang- commented on code in PR #40945:
URL: https://github.com/apache/doris/pull/40945#discussion_r1768061359
##
be/src/vec/aggregate_functions/moments.h:
##
@@ -0,0 +1,112 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include
+
+#include "common/exception.h"
+#include "common/status.h"
+#include "vec/io/io_helper.h"
+
+namespace doris::vectorized {
+
+class BufferReadable;
+class BufferWritable;
+
+template
+struct VarMoments {
+// m[1] = sum(x)
+// m[2] = sum(x^2)
+// m[3] = sum(x^3)
+// m[4] = sum(x^4)
+T m[_level + 1] {};
+
+void add(T x) {
+++m[0];
+m[1] += x;
+m[2] += x * x;
+if constexpr (_level >= 3) m[3] += x * x * x;
+if constexpr (_level >= 4) m[4] += x * x * x * x;
+}
+
+void merge(const VarMoments& rhs) {
+m[0] += rhs.m[0];
+m[1] += rhs.m[1];
+m[2] += rhs.m[2];
+if constexpr (_level >= 3) m[3] += rhs.m[3];
+if constexpr (_level >= 4) m[4] += rhs.m[4];
+}
+
+void write(BufferWritable& buf) const { write_binary(*this, buf); }
+
+void read(BufferReadable& buf) { read_binary(*this, buf); }
+
+T get() const {
+throw doris::Exception(ErrorCode::INTERNAL_ERROR,
+ "Variation moments should be obtained by either
'getSample' or "
+ "'getPopulation' method");
+}
+
+T getPopulation() const {
+if (m[0] == 0) return std::numeric_limits::quiet_NaN();
+
+/// Due to numerical errors, the result can be slightly less than zero,
+/// but it should be impossible. Trim to zero.
+
+return std::max(T {}, (m[2] - m[1] * m[1] / m[0]) / m[0]);
+}
+
+T getSample() const {
+if (m[0] <= 1) return std::numeric_limits::quiet_NaN();
+return std::max(T {}, (m[2] - m[1] * m[1] / m[0]) / (m[0] - 1));
+}
+
+T getMoment3() const {
+if constexpr (_level < 3) {
+throw doris::Exception(ErrorCode::INTERNAL_ERROR,
+ "Variation moments should be obtained by
either 'getSample' or "
+ "'getPopulation' method");
+} else {
+if (m[0] == 0) return std::numeric_limits::quiet_NaN();
+// to avoid accuracy problem
+if (m[0] == 1) return 0;
+/// \[ \frac{1}{m_0} (m_3 - (3 * m_2 - \frac{2 * {m_1}^2}{m_0}) *
\frac{m_1}{m_0});\]
+return (m[3] - (3 * m[2] - 2 * m[1] * m[1] / m[0]) * m[1] / m[0])
/ m[0];
+}
+}
+
+T getMoment4() const {
+if constexpr (_level < 4) {
+throw doris::Exception(ErrorCode::INTERNAL_ERROR,
+ "Variation moments should be obtained by
either 'getSample' or "
+ "'getPopulation' method");
+} else {
+if (m[0] == 0) return std::numeric_limits::quiet_NaN();
+// to avoid accuracy problem
+if (m[0] == 1) return 0;
+/// \[ \frac{1}{m_0}(m_4 - (4 * m_3 - (6 * m_2 - \frac{3 *
m_1^2}{m_0} ) \frac{m_1}{m_0})\frac{m_1}{m_0})\]
+return (m[4] -
+(4 * m[3] - (6 * m[2] - 3 * m[1] * m[1] / m[0]) * m[1] /
m[0]) * m[1] / m[0]) /
+ m[0];
+}
+}
+
+void reset() { return; }
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feat](json) Support json_search function [doris]
github-actions[bot] commented on PR #40948: URL: https://github.com/apache/doris/pull/40948#issuecomment-2362570918 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[I] [Feature] Partition balance considers disk usage [doris]
tangbinyeer opened a new issue, #41029: URL: https://github.com/apache/doris/issues/41029 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description Partition Balance should combine disk usage when allocating buckets to make data distribution more even. ### Use case _No response_ ### Related issues _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](partial update) Support flexible partial update in stream load with json files [doris]
bobhan1 commented on PR #39756: URL: https://github.com/apache/doris/pull/39756#issuecomment-2362567388 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-flink-connector) branch master updated: [Improve](case) add customer doris container cluster (#491)
This is an automated email from the ASF dual-hosted git repository.
diwu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-flink-connector.git
The following commit(s) were added to refs/heads/master by this push:
new c61342f9 [Improve](case) add customer doris container cluster (#491)
c61342f9 is described below
commit c61342f9651829c72f7828480c831dba52ad8ef3
Author: wudi <676366...@qq.com>
AuthorDate: Fri Sep 20 10:16:09 2024 +0800
[Improve](case) add customer doris container cluster (#491)
---
.../flink/container/AbstractContainerTestBase.java | 8 +-
.../flink/container/instance/ContainerService.java | 2 +
.../flink/container/instance/DorisContainer.java | 5 +
.../container/instance/DorisCustomerContainer.java | 132 +
.../flink/container/instance/MySQLContainer.java | 5 +
.../apache/doris/flink/sink/DorisSinkITCase.java | 51 +++-
6 files changed, 196 insertions(+), 7 deletions(-)
diff --git
a/flink-doris-connector/src/test/java/org/apache/doris/flink/container/AbstractContainerTestBase.java
b/flink-doris-connector/src/test/java/org/apache/doris/flink/container/AbstractContainerTestBase.java
index 61e0faac..5c7c151e 100644
---
a/flink-doris-connector/src/test/java/org/apache/doris/flink/container/AbstractContainerTestBase.java
+++
b/flink-doris-connector/src/test/java/org/apache/doris/flink/container/AbstractContainerTestBase.java
@@ -19,6 +19,7 @@ package org.apache.doris.flink.container;
import org.apache.doris.flink.container.instance.ContainerService;
import org.apache.doris.flink.container.instance.DorisContainer;
+import org.apache.doris.flink.container.instance.DorisCustomerContainer;
import org.junit.BeforeClass;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -48,7 +49,8 @@ public abstract class AbstractContainerTestBase {
LOG.info("The doris container has been started and is running
status.");
return;
}
-dorisContainerService = new DorisContainer();
+Boolean customerEnv =
Boolean.valueOf(System.getProperty("customer_env", "false"));
+dorisContainerService = customerEnv ? new DorisCustomerContainer() :
new DorisContainer();
dorisContainerService.startContainer();
LOG.info("Doris container was started.");
}
@@ -74,9 +76,7 @@ public abstract class AbstractContainerTestBase {
}
protected String getDorisQueryUrl() {
-return String.format(
-"jdbc:mysql://%s:%s",
-getDorisInstanceHost(),
dorisContainerService.getMappedPort(9030));
+return dorisContainerService.getJdbcUrl();
}
protected String getDorisInstanceHost() {
diff --git
a/flink-doris-connector/src/test/java/org/apache/doris/flink/container/instance/ContainerService.java
b/flink-doris-connector/src/test/java/org/apache/doris/flink/container/instance/ContainerService.java
index 6ad1e3cd..684de5a0 100644
---
a/flink-doris-connector/src/test/java/org/apache/doris/flink/container/instance/ContainerService.java
+++
b/flink-doris-connector/src/test/java/org/apache/doris/flink/container/instance/ContainerService.java
@@ -29,6 +29,8 @@ public interface ContainerService {
Connection getQueryConnection();
+String getJdbcUrl();
+
String getInstanceHost();
Integer getMappedPort(int originalPort);
diff --git
a/flink-doris-connector/src/test/java/org/apache/doris/flink/container/instance/DorisContainer.java
b/flink-doris-connector/src/test/java/org/apache/doris/flink/container/instance/DorisContainer.java
index 6af827b8..ef399d0d 100644
---
a/flink-doris-connector/src/test/java/org/apache/doris/flink/container/instance/DorisContainer.java
+++
b/flink-doris-connector/src/test/java/org/apache/doris/flink/container/instance/DorisContainer.java
@@ -115,6 +115,11 @@ public class DorisContainer implements ContainerService {
}
}
+@Override
+public String getJdbcUrl() {
+return String.format(JDBC_URL, dorisContainer.getHost());
+}
+
@Override
public String getInstanceHost() {
return dorisContainer.getHost();
diff --git
a/flink-doris-connector/src/test/java/org/apache/doris/flink/container/instance/DorisCustomerContainer.java
b/flink-doris-connector/src/test/java/org/apache/doris/flink/container/instance/DorisCustomerContainer.java
new file mode 100644
index ..3d417303
--- /dev/null
+++
b/flink-doris-connector/src/test/java/org/apache/doris/flink/container/instance/DorisCustomerContainer.java
@@ -0,0 +1,132 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.
(doris) branch master updated (99ef5236eaa -> 8b154db1753)
This is an automated email from the ASF dual-hosted git repository. englefly pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 99ef5236eaa [minor](refactor) Remove duplicate code (#40952) add 8b154db1753 [fix](Nereids) fix fold constant by be return type mismatched (#39723) No new revisions were added by this update. Summary of changes: .../nereids/rules/expression/rules/FoldConstantRuleOnBE.java | 11 ++- .../expression/fold_constant/fold_constant_by_be.groovy | 5 + 2 files changed, 15 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Nereids) fix date and date time arithmatic [doris]
github-actions[bot] commented on PR #40745: URL: https://github.com/apache/doris/pull/40745#issuecomment-2362571216 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improve](case) add customer doris container cluster [doris-flink-connector]
JNSimba merged PR #491: URL: https://github.com/apache/doris-flink-connector/pull/491 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](test) remove enable_nereids_planner=false in cases [doris]
morrySnow merged PR #40984: URL: https://github.com/apache/doris/pull/40984 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [chore](test) remove enable_nereids_planner=false in cases (#40984)
This is an automated email from the ASF dual-hosted git repository.
morrysnow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new d8277526697 [chore](test) remove enable_nereids_planner=false in cases
(#40984)
d8277526697 is described below
commit d827752669772530cf6470294acec177d4770378
Author: morrySnow <101034200+morrys...@users.noreply.github.com>
AuthorDate: Fri Sep 20 10:24:20 2024 +0800
[chore](test) remove enable_nereids_planner=false in cases (#40984)
---
.../test_partial_update_seq_map_col.out| 94 --
.../test_partial_update_mow_with_sync_mv.groovy| 19 +--
.../test_partial_update_seq_map_col.groovy | 144 ++---
3 files changed, 71 insertions(+), 186 deletions(-)
diff --git
a/regression-test/data/unique_with_mow_p0/partial_update/test_partial_update_seq_map_col.out
b/regression-test/data/unique_with_mow_p0/partial_update/test_partial_update_seq_map_col.out
index 5bdfef02182..4fab559a433 100644
---
a/regression-test/data/unique_with_mow_p0/partial_update/test_partial_update_seq_map_col.out
+++
b/regression-test/data/unique_with_mow_p0/partial_update/test_partial_update_seq_map_col.out
@@ -93,97 +93,3 @@
3 3 \N \N
4 4 \N \N
--- !sql1 --
-1 1 \N
-2 2 \N
-3 3 \N
-4 4 \N
-
--- !sql1 --
-1 1 20
-2 2 20
-3 3 \N
-4 4 \N
-
--- !sql1 --
-1 1 20
-2 2 20
-3 3 \N
-4 4 \N
-
--- !sql1 --
-3 3 2099-09-10T12:00:00.977174 \N
2099-09-10T12:00:00.977174
-4 4 2099-09-10T12:00:00.977174 \N
2099-09-10T12:00:00.977174
-
--- !sql2 --
-1 1
-2 2
-3 3
-4 4
-
--- !sql3 --
-1 1 999 999
-2 2 999 999
-3 3 999 999
-4 4 999 999
-
--- !sql3 --
-1 99
-2 99
-3 3 999 999
-4 4 999 999
-5 99
-
--- !sql4 --
-1 1 \N \N
-2 2 \N \N
-3 3 \N \N
-4 4 \N \N
-
--- !sql1 --
-1 1 \N
-2 2 \N
-3 3 \N
-4 4 \N
-
--- !sql1 --
-1 1 20
-2 2 20
-3 3 \N
-4 4 \N
-
--- !sql1 --
-1 1 20
-2 2 20
-3 3 \N
-4 4 \N
-
--- !sql1 --
-3 3 2099-09-10T12:00:00.977174 \N
2099-09-10T12:00:00.977174
-4 4 2099-09-10T12:00:00.977174 \N
2099-09-10T12:00:00.977174
-
--- !sql2 --
-1 1
-2 2
-3 3
-4 4
-
--- !sql3 --
-1 1 999 999
-2 2 999 999
-3 3 999 999
-4 4 999 999
-
--- !sql3 --
-1 99
-2 99
-3 3 999 999
-4 4 999 999
-5 99
-
--- !sql4 --
-1 1 \N \N
-2 2 \N \N
-3 3 \N \N
-4 4 \N \N
-
diff --git
a/regression-test/suites/unique_with_mow_p0/partial_update/test_partial_update_mow_with_sync_mv.groovy
b/regression-test/suites/unique_with_mow_p0/partial_update/test_partial_update_mow_with_sync_mv.groovy
index 34405843269..972542dcd34 100644
---
a/regression-test/suites/unique_with_mow_p0/partial_update/test_partial_update_mow_with_sync_mv.groovy
+++
b/regression-test/suites/unique_with_mow_p0/partial_update/test_partial_update_mow_with_sync_mv.groovy
@@ -72,22 +72,13 @@ suite("test_partial_update_mow_with_sync_mv") {
from test_partial_update_mow_with_sync_mv;
""")
-for (def use_nereids : [true, false]) {
-if (use_nereids) {
-sql "set enable_nereids_planner=true"
-sql "set enable_fallback_to_original_planner=false"
-} else {
-sql "set enable_nereids_planner=false"
-}
-
-sql "set enable_unique_key_partial_update=true;"
-sql "sync;"
+sql "set enable_unique_key_partial_update=true;"
+sql "sync;"
-test {
-sql """insert into
test_partial_update_mow_with_sync_mv(l_orderkey, l_linenumber, l_partkey,
l_suppkey, l_shipdate, l_returnflag) values
+test {
+sql """insert into test_partial_update_mow_with_sync_mv(l_orderkey,
l_linenumber, l_partkey, l_suppkey, l_shipdate, l_returnflag) values
(2, 3, 2, 1, '2023-10-18', 'k'); """
-exception "Can't do partial update on merge-on-write Unique table
with sync materialized view."
-}
+exception "Can't do partial update on merge-on-write Unique table with
sync materialized view."
}
streamLoad {
diff --git
a/regression-test/suites/unique_with_mow_p0/partial_update/test_partial_update_seq_map_col.groovy
b/regre
Re: [PR] [fix](nereids)adjust conjunct's nullable info in LogicalCatalogRelation [doris]
doris-robot commented on PR #41030: URL: https://github.com/apache/doris/pull/41030#issuecomment-2362582473 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids)adjust conjunct's nullable info in LogicalCatalogRelation [doris]
starocean999 commented on PR #41030: URL: https://github.com/apache/doris/pull/41030#issuecomment-2362582541 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](nereids)adjust conjunct's nullable info in LogicalCatalogRelation [doris]
starocean999 opened a new pull request, #41030: URL: https://github.com/apache/doris/pull/41030 ## Proposed changes pick from master https://github.com/apache/doris/pull/41014 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids)adjust conjunct's nullable info in LogicalExternalRelation [doris]
starocean999 commented on PR #41031: URL: https://github.com/apache/doris/pull/41031#issuecomment-2362582931 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](nereids)adjust conjunct's nullable info in LogicalExternalRelation [doris]
starocean999 opened a new pull request, #41031: URL: https://github.com/apache/doris/pull/41031 ## Proposed changes pick from master https://github.com/apache/doris/pull/41014 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](decimal) throw overflow exception if result is NaN of Infinit when converting from decimal to float (#40290) [doris]
jacktengg commented on PR #41007: URL: https://github.com/apache/doris/pull/41007#issuecomment-2362587423 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](cloud-mow) merge and remove old version of delete bitmap when cumulative compaction is done [doris]
github-actions[bot] commented on PR #40204: URL: https://github.com/apache/doris/pull/40204#issuecomment-2362865323 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](skew & kurt) New aggregate function skew & kurt [doris]
zhangstar333 commented on code in PR #40945:
URL: https://github.com/apache/doris/pull/40945#discussion_r1768005984
##
be/src/vec/aggregate_functions/aggregate_function_statistic.h:
##
@@ -0,0 +1,162 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+#include
+#include
+#include
+#include
+
+#include "common/exception.h"
+#include "common/status.h"
+#include "moments.h"
+#include "vec/aggregate_functions/aggregate_function.h"
+#include "vec/aggregate_functions/moments.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/columns/column_vector.h"
+#include "vec/common/assert_cast.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_nullable.h"
+#include "vec/data_types/data_type_number.h"
+
+namespace doris::vectorized {
+
+enum class StatisticsFunctionKind : uint8_t { skewPop, kurtPop };
+
+inline std::string to_string(StatisticsFunctionKind kind) {
+switch (kind) {
+case StatisticsFunctionKind::skewPop:
+return "skewness";
+case StatisticsFunctionKind::kurtPop:
+return "kurtosis";
+default:
+return "Unknown";
+}
+}
+
+template
+struct StatFuncOneArg {
+using Type1 = T;
+using Type2 = T;
+using ResultType = Float64;
+using Data = VarMoments;
+
+static constexpr UInt32 num_args = 1;
+};
+
+template
+class AggregateFunctionVarianceSimple
+: public IAggregateFunctionDataHelper<
+ typename StatFunc::Data,
+ AggregateFunctionVarianceSimple> {
+public:
+using T1 = typename StatFunc::Type1;
+using T2 = typename StatFunc::Type2;
+using ColVecT1 = ColumnVectorOrDecimal;
+using ColVecT2 = ColumnVectorOrDecimal;
+using ResultType = typename StatFunc::ResultType;
+using ColVecResult = ColumnVector;
Review Comment:
here seems could write more simple code,
as the two function return type is ColumnFloat64
##
be/src/vec/aggregate_functions/aggregate_function_statistic.h:
##
@@ -0,0 +1,162 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+#include
+#include
+#include
+#include
+
+#include "common/exception.h"
+#include "common/status.h"
+#include "moments.h"
+#include "vec/aggregate_functions/aggregate_function.h"
+#include "vec/aggregate_functions/moments.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/columns/column_vector.h"
+#include "vec/common/assert_cast.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_nullable.h"
+#include "vec/data_types/data_type_number.h"
+
+namespace doris::vectorized {
+
+enum class StatisticsFunctionKind : uint8_t { skewPop, kurtPop };
+
+inline std::string to_string(StatisticsFunctionKind kind) {
+switch (kind) {
+case StatisticsFunctionKind::skewPop:
+return "skewness";
+case StatisticsFunctionKind::kurtPop:
+return "kurtosis";
+default:
+return "Unknown";
+}
+}
+
+template
+struct StatFuncOneArg {
+using Type1 = T;
+using Type2 = T;
+using ResultType = Float64;
+using Data = VarMoments;
+
+static constexpr UInt32 num_args = 1;
Review Comment:
seems not use this var?
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/Kurt.java:
##
@@ -0,0 +1,81 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or m
Re: [PR] [feat](skew & kurt) New aggregate function skew & kurt [doris]
HappenLee commented on code in PR #40945:
URL: https://github.com/apache/doris/pull/40945#discussion_r1768032888
##
be/src/vec/aggregate_functions/aggregate_function_statistic.h:
##
@@ -0,0 +1,162 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+#include
+#include
+#include
+#include
+
+#include "common/exception.h"
+#include "common/status.h"
+#include "moments.h"
+#include "vec/aggregate_functions/aggregate_function.h"
+#include "vec/aggregate_functions/moments.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/columns/column_vector.h"
+#include "vec/common/assert_cast.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_nullable.h"
+#include "vec/data_types/data_type_number.h"
+
+namespace doris::vectorized {
+
+enum class StatisticsFunctionKind : uint8_t { skewPop, kurtPop };
+
+inline std::string to_string(StatisticsFunctionKind kind) {
+switch (kind) {
+case StatisticsFunctionKind::skewPop:
+return "skewness";
+case StatisticsFunctionKind::kurtPop:
+return "kurtosis";
+default:
+return "Unknown";
+}
+}
+
+template
+struct StatFuncOneArg {
+using Type1 = T;
+using Type2 = T;
+using ResultType = Float64;
+using Data = VarMoments;
+
+static constexpr UInt32 num_args = 1;
+};
+
+template
+class AggregateFunctionVarianceSimple
+: public IAggregateFunctionDataHelper<
+ typename StatFunc::Data,
+ AggregateFunctionVarianceSimple> {
+public:
+using T1 = typename StatFunc::Type1;
+using T2 = typename StatFunc::Type2;
+using ColVecT1 = ColumnVectorOrDecimal;
+using ColVecT2 = ColumnVectorOrDecimal;
+using ResultType = typename StatFunc::ResultType;
+using ColVecResult = ColumnVector;
+
+explicit AggregateFunctionVarianceSimple(StatisticsFunctionKind kind_,
+ const DataTypes& argument_types_)
+: IAggregateFunctionDataHelper<
+ typename StatFunc::Data,
+ AggregateFunctionVarianceSimple>(argument_types_),
+ kind(kind_) {
+DCHECK(!argument_types_.empty());
+}
+
+String get_name() const override { return to_string(kind); }
+
+DataTypePtr get_return_type() const override {
+return make_nullable(std::make_shared());
+}
+
+void add(AggregateDataPtr __restrict place, const IColumn** columns,
ssize_t row_num,
+ Arena*) const override {
+if constexpr (NullableInput) {
Review Comment:
no need do the work, should aggregate_function_nullable do the work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix]unified cancel method in coordinator [doris]
wangbo commented on PR #41038: URL: https://github.com/apache/doris/pull/41038#issuecomment-2362894766 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](test) remove enable_nereids_planner=false in cases [doris]
github-actions[bot] commented on PR #41035: URL: https://github.com/apache/doris/pull/41035#issuecomment-2362895566 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](test) remove enable_nereids_planner=false in cases [doris]
github-actions[bot] commented on PR #41035: URL: https://github.com/apache/doris/pull/41035#issuecomment-2362895630 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [fix](decimal) throw overflow exception if result is NaN of Infinit when converting from decimal to float (#40290) (#41007)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new 6539c8fd35d [fix](decimal) throw overflow exception if result is NaN
of Infinit when converting from decimal to float (#40290) (#41007)
6539c8fd35d is described below
commit 6539c8fd35da4647e5461417198ce45f8201387c
Author: TengJianPing <18241664+jackte...@users.noreply.github.com>
AuthorDate: Fri Sep 20 14:05:51 2024 +0800
[fix](decimal) throw overflow exception if result is NaN of Infinit when
converting from decimal to float (#40290) (#41007)
## Proposed changes
Issue Number: close #xxx
BP #40290
---
be/src/vec/data_types/data_type_decimal.h | 5 +-
.../datatype_p0/decimalv3/test_decimal256_cast.out | 10 +++
.../decimalv3/test_decimal256_cast.groovy | 71 ++
3 files changed, 84 insertions(+), 2 deletions(-)
diff --git a/be/src/vec/data_types/data_type_decimal.h
b/be/src/vec/data_types/data_type_decimal.h
index 8b12010e159..2acae5800d0 100644
--- a/be/src/vec/data_types/data_type_decimal.h
+++ b/be/src/vec/data_types/data_type_decimal.h
@@ -642,10 +642,11 @@ void convert_from_decimal(typename ToDataType::FieldType*
dst,
dst[i] = static_cast(src[i].value) /
multiplier.value;
}
}
-FromDataType from_data_type(precision, scale);
if constexpr (narrow_integral) {
+FromDataType from_data_type(precision, scale);
for (size_t i = 0; i < size; i++) {
-if (dst[i] < min_result || dst[i] > max_result) {
+if (std::isnan(dst[i]) || std::isinf(dst[i]) || dst[i] <
min_result ||
+dst[i] > max_result) {
THROW_DECIMAL_CONVERT_OVERFLOW_EXCEPTION(from_data_type.to_string(src[i]),
from_data_type.get_name(),
ToDataType
{}.get_name());
diff --git
a/regression-test/data/datatype_p0/decimalv3/test_decimal256_cast.out
b/regression-test/data/datatype_p0/decimalv3/test_decimal256_cast.out
index 0d79c060690..ac1f3da16dd 100644
--- a/regression-test/data/datatype_p0/decimalv3/test_decimal256_cast.out
+++ b/regression-test/data/datatype_p0/decimalv3/test_decimal256_cast.out
@@ -26,3 +26,13 @@
-- !decimal256_cast8 --
0
+-- !decimal256_cast9 --
+-9
-99.99
+9
99.99
+
+-- !decimal256_cast10 --
+10 0
+
+-- !decimal256_cast_to_double_1 --
+1.2345678E7
+
diff --git
a/regression-test/suites/datatype_p0/decimalv3/test_decimal256_cast.groovy
b/regression-test/suites/datatype_p0/decimalv3/test_decimal256_cast.groovy
index ea3001232f1..3ddfbec3975 100644
--- a/regression-test/suites/datatype_p0/decimalv3/test_decimal256_cast.groovy
+++ b/regression-test/suites/datatype_p0/decimalv3/test_decimal256_cast.groovy
@@ -18,6 +18,9 @@
suite("test_decimal256_cast") {
sql "set enable_nereids_planner = true;"
sql "set enable_decimal256 = true;"
+// sql """
+// set debug_skip_fold_constant=true;
+// """
qt_decimal256_cast0 """SELECT /*+ SET_VAR(enable_fold_constant_by_be =
false) */
cast(99.99
as decimalv3(76,10));"""
@@ -41,4 +44,72 @@ suite("test_decimal256_cast") {
select
cast('0.0012345678901'
as decimalv3(76,0));
"""
+sql """
+drop table if exists cast_to_dec256;
+"""
+sql """
+create table cast_to_dec256 (
+k1 int,
+v1 varchar(128)
+) distributed by hash(k1)
+properties (
+'replication_num' = '1'
+);
+"""
+sql """
+insert into cast_to_dec256 values(9,
"99.99"),
+(-9,
"-99.99");
+"""
+qt_decimal256_cast9 """
+select k1, cast(v1 as decimalv3(76,10)) from cast_to_dec256 order by
k1, v1;
+"""
+
+sql """
+truncate table cast_to_dec256;
+"""
+sql """
+insert into cast_to_dec256 values(10,
"0.0012345678901");
+"""
+qt_decimal256_cast10 """
+select k1, cast(v1 as decimalv3(76, 0)) from cast_to_dec256 order by
k1, v1;
+"""
+
+// test {
+// sql """
+// select /*+SET_VAR(enable_fold_constant_by_be = true)
*/cast(cast("12345678.1" as decimalv3(76, 60))
as float);
+
Re: [PR] [feat](skew & kurt) New aggregate function skew & kurt [doris]
zhiqiang- commented on code in PR #40945:
URL: https://github.com/apache/doris/pull/40945#discussion_r1768034382
##
be/src/vec/aggregate_functions/aggregate_function_statistic.h:
##
@@ -0,0 +1,162 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+#include
+#include
+#include
+#include
+
+#include "common/exception.h"
+#include "common/status.h"
+#include "moments.h"
+#include "vec/aggregate_functions/aggregate_function.h"
+#include "vec/aggregate_functions/moments.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/columns/column_vector.h"
+#include "vec/common/assert_cast.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_nullable.h"
+#include "vec/data_types/data_type_number.h"
+
+namespace doris::vectorized {
+
+enum class StatisticsFunctionKind : uint8_t { skewPop, kurtPop };
+
+inline std::string to_string(StatisticsFunctionKind kind) {
+switch (kind) {
+case StatisticsFunctionKind::skewPop:
+return "skewness";
+case StatisticsFunctionKind::kurtPop:
+return "kurtosis";
+default:
+return "Unknown";
+}
+}
+
+template
+struct StatFuncOneArg {
+using Type1 = T;
+using Type2 = T;
+using ResultType = Float64;
+using Data = VarMoments;
+
+static constexpr UInt32 num_args = 1;
+};
+
+template
+class AggregateFunctionVarianceSimple
+: public IAggregateFunctionDataHelper<
+ typename StatFunc::Data,
+ AggregateFunctionVarianceSimple> {
+public:
+using T1 = typename StatFunc::Type1;
+using T2 = typename StatFunc::Type2;
+using ColVecT1 = ColumnVectorOrDecimal;
+using ColVecT2 = ColumnVectorOrDecimal;
+using ResultType = typename StatFunc::ResultType;
+using ColVecResult = ColumnVector;
+
+explicit AggregateFunctionVarianceSimple(StatisticsFunctionKind kind_,
+ const DataTypes& argument_types_)
+: IAggregateFunctionDataHelper<
+ typename StatFunc::Data,
+ AggregateFunctionVarianceSimple>(argument_types_),
+ kind(kind_) {
+DCHECK(!argument_types_.empty());
+}
+
+String get_name() const override { return to_string(kind); }
+
+DataTypePtr get_return_type() const override {
+return make_nullable(std::make_shared());
+}
+
+void add(AggregateDataPtr __restrict place, const IColumn** columns,
ssize_t row_num,
+ Arena*) const override {
+if constexpr (NullableInput) {
Review Comment:
this function is using `creator_without_type::create_ignore_nullable`,
aggregate_function_null will not be used since this return type is always
nullable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](decimal) throw overflow exception if result is NaN of Infinit when converting from decimal to float (#40290) [doris]
yiguolei merged PR #41007: URL: https://github.com/apache/doris/pull/41007 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [Cherry-pick](branch 2.1)[Enhancement] support last_query_id function (#41025)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new d7f60c5ef1d [Cherry-pick](branch 2.1)[Enhancement] support
last_query_id function (#41025)
d7f60c5ef1d is described below
commit d7f60c5ef1df5d916e350938e243e2c6382a3821
Author: Vallish Pai
AuthorDate: Fri Sep 20 11:37:39 2024 +0530
[Cherry-pick](branch 2.1)[Enhancement] support last_query_id function
(#41025)
## Proposed changes
Issue Number: close #xxx
backport #40739
---
.../doris/catalog/BuiltinScalarFunctions.java | 4 +-
.../expression/rules/FoldConstantRuleOnFE.java | 16 +-
.../expressions/functions/scalar/LastQueryId.java | 59 ++
.../expressions/visitor/ScalarFunctionVisitor.java | 5 ++
.../java/org/apache/doris/qe/ConnectContext.java | 8 +++
.../suites/query_p0/system/test_query_sys.groovy | 2 +
6 files changed, 92 insertions(+), 2 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/catalog/BuiltinScalarFunctions.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/BuiltinScalarFunctions.java
index 306870fa666..ded6f764e0f 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/catalog/BuiltinScalarFunctions.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/catalog/BuiltinScalarFunctions.java
@@ -253,6 +253,7 @@ import
org.apache.doris.nereids.trees.expressions.functions.scalar.JsonbValid;
import org.apache.doris.nereids.trees.expressions.functions.scalar.L1Distance;
import org.apache.doris.nereids.trees.expressions.functions.scalar.L2Distance;
import org.apache.doris.nereids.trees.expressions.functions.scalar.LastDay;
+import org.apache.doris.nereids.trees.expressions.functions.scalar.LastQueryId;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Least;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Left;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Length;
@@ -906,7 +907,8 @@ public class BuiltinScalarFunctions implements
FunctionHelper {
scalar(YearsAdd.class, "years_add"),
scalar(YearsDiff.class, "years_diff"),
scalar(YearsSub.class, "years_sub"),
-scalar(MultiMatch.class, "multi_match"));
+scalar(MultiMatch.class, "multi_match"),
+scalar(LastQueryId.class, "last_query_id"));
public static final BuiltinScalarFunctions INSTANCE = new
BuiltinScalarFunctions();
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/FoldConstantRuleOnFE.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/FoldConstantRuleOnFE.java
index 8c1b5538205..54ea6969988 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/FoldConstantRuleOnFE.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/FoldConstantRuleOnFE.java
@@ -20,6 +20,7 @@ package org.apache.doris.nereids.rules.expression.rules;
import org.apache.doris.catalog.EncryptKey;
import org.apache.doris.catalog.Env;
import org.apache.doris.cluster.ClusterNamespace;
+import org.apache.doris.common.util.DebugUtil;
import org.apache.doris.nereids.exceptions.AnalysisException;
import org.apache.doris.nereids.rules.expression.AbstractExpressionRewriteRule;
import org.apache.doris.nereids.rules.expression.ExpressionListenerMatcher;
@@ -62,6 +63,7 @@ import
org.apache.doris.nereids.trees.expressions.functions.scalar.Database;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Date;
import
org.apache.doris.nereids.trees.expressions.functions.scalar.EncryptKeyRef;
import org.apache.doris.nereids.trees.expressions.functions.scalar.If;
+import org.apache.doris.nereids.trees.expressions.functions.scalar.LastQueryId;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Password;
import org.apache.doris.nereids.trees.expressions.functions.scalar.User;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Version;
@@ -83,6 +85,7 @@ import org.apache.doris.nereids.types.coercion.DateLikeType;
import org.apache.doris.nereids.util.ExpressionUtils;
import org.apache.doris.qe.ConnectContext;
import org.apache.doris.qe.GlobalVariable;
+import org.apache.doris.thrift.TUniqueId;
import com.google.common.base.Preconditions;
import com.google.common.base.Strings;
@@ -164,7 +167,8 @@ public class FoldConstantRuleOnFE extends
AbstractExpressionRewriteRule
matches(Password.class, this::visitPassword),
matches(Array.class, this::visitArray),
matches(Date.class, this::visitDate),
-matches(Version.class, this::visitVersion)
+matches(Version.class, this::visitVersion),
+matches(LastQueryId.class, th
Re: [PR] [feat](skew & kurt) New aggregate function skew & kurt [doris]
HappenLee commented on code in PR #40945:
URL: https://github.com/apache/doris/pull/40945#discussion_r1768037269
##
be/src/vec/aggregate_functions/moments.h:
##
@@ -0,0 +1,112 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include
+
+#include "common/exception.h"
+#include "common/status.h"
+#include "vec/io/io_helper.h"
+
+namespace doris::vectorized {
+
+class BufferReadable;
+class BufferWritable;
+
+template
+struct VarMoments {
+// m[1] = sum(x)
+// m[2] = sum(x^2)
+// m[3] = sum(x^3)
+// m[4] = sum(x^4)
+T m[_level + 1] {};
+
+void add(T x) {
+++m[0];
+m[1] += x;
+m[2] += x * x;
+if constexpr (_level >= 3) m[3] += x * x * x;
+if constexpr (_level >= 4) m[4] += x * x * x * x;
+}
+
+void merge(const VarMoments& rhs) {
+m[0] += rhs.m[0];
+m[1] += rhs.m[1];
+m[2] += rhs.m[2];
+if constexpr (_level >= 3) m[3] += rhs.m[3];
+if constexpr (_level >= 4) m[4] += rhs.m[4];
+}
+
+void write(BufferWritable& buf) const { write_binary(*this, buf); }
+
+void read(BufferReadable& buf) { read_binary(*this, buf); }
+
+T get() const {
+throw doris::Exception(ErrorCode::INTERNAL_ERROR,
+ "Variation moments should be obtained by either
'getSample' or "
+ "'getPopulation' method");
+}
+
+T getPopulation() const {
Review Comment:
get_population
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) change memtable memory limiter policy [doris]
yiguolei merged PR #41018: URL: https://github.com/apache/doris/pull/41018 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Cherry-pick](branch 2.1)[Enhancement] support last_query_id function [doris]
yiguolei merged PR #41025: URL: https://github.com/apache/doris/pull/41025 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (17da0969ece -> 34ab55ff860)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 17da0969ece [fix](tvf) fix that FE OOM when using tvf (#40981) add 34ab55ff860 [improve](load) change memtable memory limiter policy (#41018) No new revisions were added by this update. Summary of changes: be/src/common/config.cpp| 6 -- be/src/common/config.h | 6 -- be/src/olap/memtable_memory_limiter.cpp | 169 be/src/olap/memtable_memory_limiter.h | 8 +- 4 files changed, 88 insertions(+), 101 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
englefly commented on PR #40349: URL: https://github.com/apache/doris/pull/40349#issuecomment-2362906146 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](skew & kurt) New aggregate function skew & kurt [doris]
HappenLee commented on code in PR #40945:
URL: https://github.com/apache/doris/pull/40945#discussion_r1768032888
##
be/src/vec/aggregate_functions/aggregate_function_statistic.h:
##
@@ -0,0 +1,162 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+#include
+#include
+#include
+#include
+
+#include "common/exception.h"
+#include "common/status.h"
+#include "moments.h"
+#include "vec/aggregate_functions/aggregate_function.h"
+#include "vec/aggregate_functions/moments.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/columns/column_vector.h"
+#include "vec/common/assert_cast.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_nullable.h"
+#include "vec/data_types/data_type_number.h"
+
+namespace doris::vectorized {
+
+enum class StatisticsFunctionKind : uint8_t { skewPop, kurtPop };
+
+inline std::string to_string(StatisticsFunctionKind kind) {
+switch (kind) {
+case StatisticsFunctionKind::skewPop:
+return "skewness";
+case StatisticsFunctionKind::kurtPop:
+return "kurtosis";
+default:
+return "Unknown";
+}
+}
+
+template
+struct StatFuncOneArg {
+using Type1 = T;
+using Type2 = T;
+using ResultType = Float64;
+using Data = VarMoments;
+
+static constexpr UInt32 num_args = 1;
+};
+
+template
+class AggregateFunctionVarianceSimple
+: public IAggregateFunctionDataHelper<
+ typename StatFunc::Data,
+ AggregateFunctionVarianceSimple> {
+public:
+using T1 = typename StatFunc::Type1;
+using T2 = typename StatFunc::Type2;
+using ColVecT1 = ColumnVectorOrDecimal;
+using ColVecT2 = ColumnVectorOrDecimal;
+using ResultType = typename StatFunc::ResultType;
+using ColVecResult = ColumnVector;
+
+explicit AggregateFunctionVarianceSimple(StatisticsFunctionKind kind_,
+ const DataTypes& argument_types_)
+: IAggregateFunctionDataHelper<
+ typename StatFunc::Data,
+ AggregateFunctionVarianceSimple>(argument_types_),
+ kind(kind_) {
+DCHECK(!argument_types_.empty());
+}
+
+String get_name() const override { return to_string(kind); }
+
+DataTypePtr get_return_type() const override {
+return make_nullable(std::make_shared());
+}
+
+void add(AggregateDataPtr __restrict place, const IColumn** columns,
ssize_t row_num,
+ Arena*) const override {
+if constexpr (NullableInput) {
Review Comment:
should skip the null value
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
englefly commented on code in PR #40349:
URL: https://github.com/apache/doris/pull/40349#discussion_r1767909315
##
fe/fe-core/src/main/java/org/apache/doris/nereids/stats/StatsCalculator.java:
##
@@ -205,6 +210,23 @@ public Map
getTotalColumnStatisticMap() {
return totalColumnStatisticMap;
}
+/**
+ * disable join reorder if any table row count is not available.
+ */
+public static void disableJoinReorderIfTableRowCountNotAvailable(
+List scans, CascadesContext context) {
+StatsCalculator calculator = new StatsCalculator(context);
+for (LogicalOlapScan scan : scans) {
+double rowCount = calculator.getOlapTableRowCount(scan);
+if (rowCount == -1 && ConnectContext.get() != null) {
+LOG.info("disable join reorder since row count not available: "
++ scan.getTable().getNameWithFullQualifiers());
+
ConnectContext.get().getSessionVariable().setDisableJoinReorder(true);
Review Comment:
this is statement level. it has no impact on other queries.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](test) fix flake case in both test_insert_strict_fail_url and test_insert_partition_fail_url #40998 [doris]
liaoxin01 merged PR #41024: URL: https://github.com/apache/doris/pull/41024 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated (64880a10d67 -> 3249931fa67)
This is an automated email from the ASF dual-hosted git repository. liaoxin pushed a change to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git from 64880a10d67 [branch-2.1] Picks "[Fix](partial update) Fix partial update failed when merge_type=MERGE #40730" (#40951) add 3249931fa67 [Fix](test) fix flake case in both test_insert_strict_fail_url and test_insert_partition_fail_url #40998 (#41024) No new revisions were added by this update. Summary of changes: .../suites/insert_p0/test_insert_partition_fail_url.groovy| 11 +-- .../suites/insert_p0/test_insert_strict_fail_url.groovy | 11 +-- 2 files changed, 18 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improve]Support modify column type without default when column exists default value [doris-flink-connector]
JNSimba merged PR #490: URL: https://github.com/apache/doris-flink-connector/pull/490 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-flink-connector) branch master updated: [Improve]Support modify column type without default when column exists default value (#490)
This is an automated email from the ASF dual-hosted git repository.
diwu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-flink-connector.git
The following commit(s) were added to refs/heads/master by this push:
new c4ae0512 [Improve]Support modify column type without default when
column exists default value (#490)
c4ae0512 is described below
commit c4ae0512941f98108caaa13fe4cf2dfcece61b6e
Author: North Lin <37775475+qg-...@users.noreply.github.com>
AuthorDate: Fri Sep 20 09:57:07 2024 +0800
[Improve]Support modify column type without default when column exists
default value (#490)
---
.../flink/exception/IllegalArgumentException.java | 4 ++
.../flink/sink/schema/SchemaChangeHelper.java | 11
.../flink/sink/schema/SchemaChangeManager.java | 55 +++--
.../flink/sink/schema/SchemaManagerITCase.java | 70 +++---
4 files changed, 127 insertions(+), 13 deletions(-)
diff --git
a/flink-doris-connector/src/main/java/org/apache/doris/flink/exception/IllegalArgumentException.java
b/flink-doris-connector/src/main/java/org/apache/doris/flink/exception/IllegalArgumentException.java
index 4c0ae093..7b242891 100644
---
a/flink-doris-connector/src/main/java/org/apache/doris/flink/exception/IllegalArgumentException.java
+++
b/flink-doris-connector/src/main/java/org/apache/doris/flink/exception/IllegalArgumentException.java
@@ -25,4 +25,8 @@ public class IllegalArgumentException extends DorisException {
public IllegalArgumentException(String arg, String value) {
super("argument '" + arg + "' is illegal, value is '" + value + "'.");
}
+
+public IllegalArgumentException(String msg) {
+super(msg);
+}
}
diff --git
a/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/schema/SchemaChangeHelper.java
b/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/schema/SchemaChangeHelper.java
index 74b57417..d0630b03 100644
---
a/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/schema/SchemaChangeHelper.java
+++
b/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/schema/SchemaChangeHelper.java
@@ -44,6 +44,7 @@ public class SchemaChangeHelper {
private static final String CREATE_DATABASE_DDL = "CREATE DATABASE IF NOT
EXISTS %s";
private static final String MODIFY_TYPE_DDL = "ALTER TABLE %s MODIFY
COLUMN %s %s";
private static final String MODIFY_COMMENT_DDL = "ALTER TABLE %s MODIFY
COLUMN %s COMMENT '%s'";
+private static final String SHOW_FULL_COLUMN_DDL = "SHOW FULL COLUMNS FROM
`%s`.`%s`";
public static void compareSchema(
Map updateFiledSchemaMap,
@@ -166,6 +167,7 @@ public class SchemaChangeHelper {
String columnName = fieldSchema.getName();
String dataType = fieldSchema.getTypeString();
String comment = fieldSchema.getComment();
+String defaultValue = fieldSchema.getDefaultValue();
StringBuilder modifyDDL =
new StringBuilder(
String.format(
@@ -173,6 +175,11 @@ public class SchemaChangeHelper {
DorisSchemaFactory.quoteTableIdentifier(tableIdentifier),
DorisSchemaFactory.identifier(columnName),
dataType));
+if (StringUtils.isNotBlank(defaultValue)) {
+modifyDDL
+.append(" DEFAULT ")
+
.append(DorisSchemaFactory.quoteDefaultValue(defaultValue));
+}
commentColumn(modifyDDL, comment);
return modifyDDL.toString();
}
@@ -183,6 +190,10 @@ public class SchemaChangeHelper {
}
}
+public static String buildShowFullColumnDDL(String database, String table)
{
+return String.format(SHOW_FULL_COLUMN_DDL, database, table);
+}
+
public static List getDdlSchemas() {
return ddlSchemas;
}
diff --git
a/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/schema/SchemaChangeManager.java
b/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/schema/SchemaChangeManager.java
index c946bee7..50ec1d34 100644
---
a/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/schema/SchemaChangeManager.java
+++
b/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/schema/SchemaChangeManager.java
@@ -23,6 +23,7 @@ import org.apache.flink.util.StringUtils;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.NullNode;
import org.apache.commons.codec.binary.Base64;
import org.apache.doris.flink.catalog.doris.DorisSystem;
import org.apache.doris.flink.catalog.doris.FieldSchema;
@@ -123,11 +124,30 @@ public class SchemaChangeManager implements Serializable {
throws IOException, IllegalArgumentE
Re: [PR] [minor](refactor) Remove duplicate code [doris]
Gabriel39 merged PR #40952: URL: https://github.com/apache/doris/pull/40952 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (d513b11eb4e -> 99ef5236eaa)
This is an automated email from the ASF dual-hosted git repository. gabriellee pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from d513b11eb4e [feature](nereids)support correlated scalar subquery without scalar agg (#39471) add 99ef5236eaa [minor](refactor) Remove duplicate code (#40952) No new revisions were added by this update. Summary of changes: be/src/pipeline/exec/exchange_sink_operator.cpp| 2 +- be/src/pipeline/exec/result_file_sink_operator.cpp | 2 +- be/src/vec/sink/vdata_stream_sender.cpp| 8 +--- be/src/vec/sink/vdata_stream_sender.h | 2 +- 4 files changed, 4 insertions(+), 10 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feat](Nereids) add numeric functions [doris]
LiBinfeng-01 commented on PR #40744: URL: https://github.com/apache/doris/pull/40744#issuecomment-2362563898 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feat](Nereids) add numeric functions [doris]
LiBinfeng-01 commented on PR #40744: URL: https://github.com/apache/doris/pull/40744#issuecomment-2362564010 run cloud_p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.1](session-variable) Add a debug variable of constant fold (#35584) [doris]
doris-robot commented on PR #41032: URL: https://github.com/apache/doris/pull/41032#issuecomment-2362588893 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.1](session-variable) Add a debug variable of constant fold (#35584) [doris]
zclllyybb commented on PR #41032: URL: https://github.com/apache/doris/pull/41032#issuecomment-2362588925 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](mem-tracker) Use thread local mem tracker to track s3 file buffer memory usage [doris]
github-actions[bot] commented on PR #40597: URL: https://github.com/apache/doris/pull/40597#issuecomment-2362588989 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) fix fold constant by be return type mismatched [doris]
englefly merged PR #39723: URL: https://github.com/apache/doris/pull/39723 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
englefly commented on PR #40349: URL: https://github.com/apache/doris/pull/40349#issuecomment-2362591572 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (d8277526697 -> 107856e83a6)
This is an automated email from the ASF dual-hosted git repository. w41ter pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from d8277526697 [chore](test) remove enable_nereids_planner=false in cases (#40984) add 107856e83a6 [fix](restore) Ignore base tablet if it is migrated (#41017) No new revisions were added by this update. Summary of changes: be/src/olap/snapshot_manager.cpp | 13 + 1 file changed, 9 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](restore) Ignore base tablet if it is migrated [doris]
w41ter merged PR #41017: URL: https://github.com/apache/doris/pull/41017 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](decimal) throw overflow exception if result is NaN of Infinit when converting from decimal to float (#40290) [doris]
github-actions[bot] commented on PR #41007: URL: https://github.com/apache/doris/pull/41007#issuecomment-2362592815 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](decimal) throw overflow exception if result is NaN of Infinit when converting from decimal to float (#40290) [doris]
github-actions[bot] commented on PR #41007: URL: https://github.com/apache/doris/pull/41007#issuecomment-2362593075 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) merge request properties map of group should compare cost [doris]
morrySnow merged PR #40819: URL: https://github.com/apache/doris/pull/40819 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (107856e83a6 -> fc7f8f3c09a)
This is an automated email from the ASF dual-hosted git repository. morrysnow pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 107856e83a6 [fix](restore) Ignore base tablet if it is migrated (#41017) add fc7f8f3c09a [fix](Nereids) merge request properties map of group should compare cost (#40819) No new revisions were added by this update. Summary of changes: .../apache/doris/nereids/memo/GroupExpression.java | 22 +++- .../doris/nereids/memo/GroupExpressionTest.java| 62 ++ 2 files changed, 83 insertions(+), 1 deletion(-) create mode 100644 fe/fe-core/src/test/java/org/apache/doris/nereids/memo/GroupExpressionTest.java - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](profile) Print changed session var in profile [doris]
zhiqiang- commented on PR #41016: URL: https://github.com/apache/doris/pull/41016#issuecomment-2362628584 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](nereids)show user friendly error msg when window function contains order by expression [doris]
github-actions[bot] commented on PR #40937: URL: https://github.com/apache/doris/pull/40937#issuecomment-2362641393 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](nereids)show user friendly error msg when window function contains order by expression [doris]
github-actions[bot] commented on PR #40937: URL: https://github.com/apache/doris/pull/40937#issuecomment-2362641195 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) merge request properties map of group should compare cost [doris]
github-actions[bot] commented on PR #40819: URL: https://github.com/apache/doris/pull/40819#issuecomment-2362584561 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](profile) Print changed session var in profile [doris]
zhiqiang- commented on PR #41016: URL: https://github.com/apache/doris/pull/41016#issuecomment-2362641647 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](opt) improve BE code readability of multi_match_any function [doris]
github-actions[bot] commented on PR #39354: URL: https://github.com/apache/doris/pull/39354#issuecomment-2362644361 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
