(doris-website) branch asf-site updated (75699a0e25 -> e145e5da32)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/doris-website.git discard 75699a0e25 Automated deployment with doris branch @ 3bedaa0125afce741623ff247b636f9a2872c2a2 new e145e5da32 Automated deployment with doris branch @ 3bedaa0125afce741623ff247b636f9a2872c2a2 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (75699a0e25) \ N -- N -- N refs/heads/asf-site (e145e5da32) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/1.2/search-index.json | 2 +- docs/2.0/search-index.json | 2 +- docs/3.0/search-index.json | 2 +- docs/dev/search-index.json | 2 +- search-index.json| 2 +- zh-CN/docs/1.2/search-index.json | 2 +- zh-CN/docs/2.0/search-index.json | 2 +- zh-CN/docs/3.0/search-index.json | 2 +- zh-CN/docs/dev/search-index.json | 2 +- zh-CN/search-index.json | 2 +- 10 files changed, 10 insertions(+), 10 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) 01/05: [improvement](statistics)Remove analyze retry logic. (#41224)
This is an automated email from the ASF dual-hosted git repository.
lijibing pushed a commit to branch 2.0.15-tebu
in repository https://gitbox.apache.org/repos/asf/doris.git
commit b83949025cd7c880bf41162045120bd40016aa74
Author: Jibing-Li <64681310+jibing...@users.noreply.github.com>
AuthorDate: Tue Sep 24 20:25:54 2024 +0800
[improvement](statistics)Remove analyze retry logic. (#41224)
Remove analyze retry logic when task failed. Because usually retry would
fail again and retry would bring a long time of sleep, which cause the
analyze job running too slow.
Master pr: https://github.com/apache/doris/pull/33703
---
.../apache/doris/statistics/AnalysisManager.java | 22 +++---
.../apache/doris/statistics/BaseAnalysisTask.java | 34 --
.../org/apache/doris/statistics/HistogramTask.java | 5
.../doris/statistics/StatisticConstants.java | 2 --
4 files changed, 16 insertions(+), 47 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisManager.java
b/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisManager.java
index 5d3debb8ddd..2446686d46e 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisManager.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisManager.java
@@ -90,6 +90,7 @@ import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.NavigableMap;
+import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.StringJoiner;
@@ -949,7 +950,7 @@ public class AnalysisManager implements Writable {
public List findTasksByTaskIds(long jobId) {
AnalysisInfo jobInfo = analysisJobInfoMap.get(jobId);
if (jobInfo != null && jobInfo.taskIds != null) {
-return
jobInfo.taskIds.stream().map(analysisTaskInfoMap::get).filter(i -> i != null)
+return
jobInfo.taskIds.stream().map(analysisTaskInfoMap::get).filter(Objects::nonNull)
.collect(Collectors.toList());
}
return null;
@@ -966,7 +967,7 @@ public class AnalysisManager implements Writable {
public void dropAnalyzeJob(DropAnalyzeJobStmt analyzeJobStmt) throws
DdlException {
AnalysisInfo jobInfo =
analysisJobInfoMap.get(analyzeJobStmt.getJobId());
if (jobInfo == null) {
-throw new DdlException(String.format("Analyze job [%d] not
exists", jobInfo.jobId));
+throw new DdlException(String.format("Analyze job [%d] not
exists", analyzeJobStmt.getJobId()));
}
checkPriv(jobInfo);
long jobId = analyzeJobStmt.getJobId();
@@ -1003,12 +1004,12 @@ public class AnalysisManager implements Writable {
if (analysisInfo == null) {
return true;
}
-if ((AnalysisState.PENDING.equals(analysisInfo.state) ||
AnalysisState.RUNNING.equals(analysisInfo.state))
-&& ScheduleType.ONCE.equals(analysisInfo.scheduleType)
-&& JobType.MANUAL.equals(analysisInfo.jobType)) {
+if (analysisInfo.scheduleType == null || analysisInfo.jobType == null)
{
return true;
}
-return false;
+return (AnalysisState.PENDING.equals(analysisInfo.state) ||
AnalysisState.RUNNING.equals(analysisInfo.state))
+&& ScheduleType.ONCE.equals(analysisInfo.scheduleType)
+&& JobType.MANUAL.equals(analysisInfo.jobType);
}
private static void readIdToTblStats(DataInput in, Map map) throws IOException {
@@ -1177,17 +1178,14 @@ public class AnalysisManager implements Writable {
/**
* Only OlapTable and Hive HMSExternalTable can sample for now.
- * @param table
+ * @param table Table to check
* @return Return true if the given table can do sample analyze. False
otherwise.
*/
public boolean canSample(TableIf table) {
if (table instanceof OlapTable) {
return true;
}
-if (table instanceof HMSExternalTable
-&& ((HMSExternalTable)
table).getDlaType().equals(HMSExternalTable.DLAType.HIVE)) {
-return true;
-}
-return false;
+return table instanceof HMSExternalTable
+&& ((HMSExternalTable)
table).getDlaType().equals(HMSExternalTable.DLAType.HIVE);
}
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/statistics/BaseAnalysisTask.java
b/fe/fe-core/src/main/java/org/apache/doris/statistics/BaseAnalysisTask.java
index 85a2fd0de3f..52c8ce932a5 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/statistics/BaseAnalysisTask.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/statistics/BaseAnalysisTask.java
@@ -40,7 +40,6 @@ import org.apache.logging.log4j.Logger;
import java.text.MessageFormat;
import java.util.Collections;
-import java.util.concurrent.TimeUnit;
public abstract class BaseAnalysisTask {
@@ -195,9 +194,9 @@ public abstract c
(doris) 02/05: [improvement](statistics)Return -1 to neredis if report olap table row count for new table is not done for all tablets. (#40457) (#40973)
This is an automated email from the ASF dual-hosted git repository.
lijibing pushed a commit to branch 2.0.15-tebu
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 769f2c052e2097f185d98da9849402e765ef780a
Author: Jibing-Li <64681310+jibing...@users.noreply.github.com>
AuthorDate: Thu Sep 26 10:30:47 2024 +0800
[improvement](statistics)Return -1 to neredis if report olap table row
count for new table is not done for all tablets. (#40457) (#40973)
backport: https://github.com/apache/doris/pull/40457
---
be/src/olap/tablet_manager.cpp | 1 +
.../apache/doris/analysis/ShowTableStatsStmt.java | 51 +++--
.../apache/doris/catalog/MaterializedIndex.java| 11 +++
.../java/org/apache/doris/catalog/OlapTable.java | 13 ++--
.../java/org/apache/doris/catalog/Replica.java | 10 +++
.../org/apache/doris/catalog/TabletStatMgr.java| 29 +++-
.../java/org/apache/doris/qe/ShowExecutor.java | 13 +---
.../apache/doris/statistics/AnalysisManager.java | 6 +-
.../apache/doris/statistics/OlapAnalysisTask.java | 2 +-
.../apache/doris/statistics/TableStatsMeta.java| 6 +-
.../doris/transaction/DatabaseTransactionMgr.java | 3 +
gensrc/thrift/BackendService.thrift| 1 +
.../suites/statistics/analyze_stats.groovy | 2 +-
.../suites/statistics/test_analyze_mv.groovy | 83 --
14 files changed, 173 insertions(+), 58 deletions(-)
diff --git a/be/src/olap/tablet_manager.cpp b/be/src/olap/tablet_manager.cpp
index 573ec920160..c7320cdaa3b 100644
--- a/be/src/olap/tablet_manager.cpp
+++ b/be/src/olap/tablet_manager.cpp
@@ -1049,6 +1049,7 @@ Status
TabletManager::build_all_report_tablets_info(std::map
t_tablet_stat.__set_remote_data_size(tablet_info.remote_data_size);
t_tablet_stat.__set_row_num(tablet_info.row_count);
t_tablet_stat.__set_version_count(tablet_info.version_count);
+t_tablet_stat.__set_visible_version(tablet_info.version);
};
for_each_tablet(handler, filter_all_tablets);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowTableStatsStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowTableStatsStmt.java
index 8e9800fc410..12f097729d2 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowTableStatsStmt.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowTableStatsStmt.java
@@ -63,7 +63,9 @@ public class ShowTableStatsStmt extends ShowStmt {
new ImmutableList.Builder()
.add("table_name")
.add("index_name")
-.add("row_count")
+.add("analyze_row_count")
+.add("report_row_count")
+.add("report_row_count_for_nereids")
.build();
private final TableName tableName;
@@ -166,37 +168,33 @@ public class ShowTableStatsStmt extends ShowStmt {
return tableId;
}
-public ShowResultSet constructResultSet(TableStatsMeta tableStatistic) {
+public ShowResultSet constructResultSet(TableStatsMeta tableStatistic,
TableIf table) {
if (indexName != null) {
-return constructIndexResultSet(tableStatistic);
+return constructIndexResultSet(tableStatistic, table);
}
-return constructTableResultSet(tableStatistic);
+return constructTableResultSet(tableStatistic, table);
}
public ShowResultSet constructEmptyResultSet() {
return new ShowResultSet(getMetaData(), new ArrayList<>());
}
-public ShowResultSet constructResultSet(long rowCount) {
-List> result = Lists.newArrayList();
-List row = Lists.newArrayList();
-row.add("");
-row.add("");
-row.add(String.valueOf(rowCount));
-row.add("");
-row.add("");
-row.add("");
-row.add("");
-row.add("");
-result.add(row);
-return new ShowResultSet(getMetaData(), result);
-}
-
-public ShowResultSet constructTableResultSet(TableStatsMeta
tableStatistic) {
-DateTimeFormatter formatter = DateTimeFormatter.ofPattern("-MM-dd
HH:mm:ss");
+public ShowResultSet constructTableResultSet(TableStatsMeta
tableStatistic, TableIf table) {
if (tableStatistic == null) {
-return new ShowResultSet(getMetaData(), new ArrayList<>());
+List> result = Lists.newArrayList();
+List row = Lists.newArrayList();
+row.add("");
+row.add("");
+row.add(String.valueOf(table.getRowCount()));
+row.add("");
+row.add("");
+row.add("");
+row.add("");
+row.add("");
+result.add(row);
+return new ShowResultSet(getMetaData(), result);
}
+DateTimeFormatter formatter = DateTimeFormatter.ofPattern("-MM-dd
HH:mm:ss");
List> result = Lists.newArrayList();
List row = Lists.n
(doris) 05/05: [improvement](statistics)Support partition row count return -1 when it is not fully reported. (#41348) (#41410)
This is an automated email from the ASF dual-hosted git repository.
lijibing pushed a commit to branch 2.0.15-tebu
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 1a67fe0d0ae02dbd52a6b0d639359a667cce7783
Author: Jibing-Li <64681310+jibing...@users.noreply.github.com>
AuthorDate: Tue Oct 8 11:21:36 2024 +0800
[improvement](statistics)Support partition row count return -1 when it is
not fully reported. (#41348) (#41410)
backport: https://github.com/apache/doris/pull/41348
---
.../java/org/apache/doris/catalog/OlapTable.java | 32 ++-
.../org/apache/doris/catalog/OlapTableTest.java| 47 ++
2 files changed, 78 insertions(+), 1 deletion(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
index 8b516b5ab5d..0d585e6271a 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
@@ -1277,18 +1277,48 @@ public class OlapTable extends Table {
return getRowCountForIndex(baseIndexId, false);
}
+/**
+ * @return If strict is true, -1 if there are some tablets whose row count
is not reported to FE
+ * If strict is false, return the sum of all partition's index
current reported row count.
+ */
public long getRowCountForIndex(long indexId, boolean strict) {
long rowCount = 0;
for (Map.Entry entry : idToPartition.entrySet()) {
MaterializedIndex index = entry.getValue().getIndex(indexId);
+if (index == null) {
+LOG.warn("Index {} not exist in partition {}, table {}, {}",
+indexId, entry.getValue().getName(), id, name);
+return -1;
+}
if (strict && !index.getRowCountReported()) {
return -1;
}
-rowCount += (index == null || index.getRowCount() == -1) ? 0 :
index.getRowCount();
+rowCount += index.getRowCount() == -1 ? 0 : index.getRowCount();
}
return rowCount;
}
+/**
+ * @return If strict is true, -1 if there are some tablets whose row count
is not reported to FE.
+ * If strict is false, return the sum of partition's all indexes
current reported row count.
+ */
+public long getRowCountForPartitionIndex(long partitionId, long indexId,
boolean strict) {
+Partition partition = idToPartition.get(partitionId);
+if (partition == null) {
+LOG.warn("Partition {} not exist in table {}, {}", partitionId,
id, name);
+return -1;
+}
+MaterializedIndex index = partition.getIndex(indexId);
+if (index == null) {
+LOG.warn("Index {} not exist in partition {}, table {}, {}",
indexId, partitionId, id, name);
+return -1;
+}
+if (strict && !index.getRowCountReported()) {
+return -1;
+}
+return index.getRowCount() == -1 ? 0 : index.getRowCount();
+}
+
@Override
public long getAvgRowLength() {
long rowCount = 0;
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/catalog/OlapTableTest.java
b/fe/fe-core/src/test/java/org/apache/doris/catalog/OlapTableTest.java
index aa85178c08e..fd394353cd2 100644
--- a/fe/fe-core/src/test/java/org/apache/doris/catalog/OlapTableTest.java
+++ b/fe/fe-core/src/test/java/org/apache/doris/catalog/OlapTableTest.java
@@ -121,4 +121,51 @@ public class OlapTableTest {
Assert.assertFalse(olapTable.getTableProperty().getDynamicPartitionProperty().getEnable());
Assert.assertEquals((short) 3,
olapTable.getDefaultReplicaAllocation().getTotalReplicaNum());
}
+
+@Test
+public void testGetPartitionRowCount() {
+OlapTable olapTable = new OlapTable();
+// Partition is null.
+long row = olapTable.getRowCountForPartitionIndex(0, 0, true);
+Assert.assertEquals(-1, row);
+
+// Index is null.
+MaterializedIndex index = new MaterializedIndex(10,
MaterializedIndex.IndexState.NORMAL);
+Partition partition = new Partition(11, "p1", index, null);
+olapTable.addPartition(partition);
+row = olapTable.getRowCountForPartitionIndex(11, 0, true);
+Assert.assertEquals(-1, row);
+
+// Strict is true and index is not reported.
+index.setRowCountReported(false);
+index.setRowCount(100);
+row = olapTable.getRowCountForPartitionIndex(11, 10, true);
+Assert.assertEquals(-1, row);
+
+// Strict is true and index is reported.
+index.setRowCountReported(true);
+index.setRowCount(101);
+row = olapTable.getRowCountForPartitionIndex(11, 10, true);
+Assert.assertEquals(101, row);
+
+// Strict is false and index is not reported.
+index.setRowCountReported(false);
+
(doris) branch 2.0.15-tebu updated (541880788f0 -> 1a67fe0d0ae)
This is an automated email from the ASF dual-hosted git repository. lijibing pushed a change to branch 2.0.15-tebu in repository https://gitbox.apache.org/repos/asf/doris.git from 541880788f0 [Fix](inverted index) fix wrong opt for count_on_index #41127 (#41153) new b83949025cd [improvement](statistics)Remove analyze retry logic. (#41224) new 769f2c052e2 [improvement](statistics)Return -1 to neredis if report olap table row count for new table is not done for all tablets. (#40457) (#40973) new 1515e17127c [fix](statistics)Fix drop stats log editlog bug. Catch drop stats exception while truncate table. (#40738) (#40979) new 6e59d8720c7 [fix](statistics)Fix empty table keep auto analyze bug. (#40811) #40982 (#40982) new 1a67fe0d0ae [improvement](statistics)Support partition row count return -1 when it is not fully reported. (#41348) (#41410) The 5 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: be/src/olap/tablet_manager.cpp | 1 + .../apache/doris/alter/SchemaChangeHandler.java| 12 +- .../org/apache/doris/alter/SchemaChangeJobV2.java | 6 +- .../apache/doris/analysis/ShowTableStatsStmt.java | 51 +++-- .../main/java/org/apache/doris/catalog/Env.java| 6 +- .../apache/doris/catalog/MaterializedIndex.java| 11 + .../java/org/apache/doris/catalog/OlapTable.java | 43 +++- .../java/org/apache/doris/catalog/Replica.java | 10 + .../org/apache/doris/catalog/TabletStatMgr.java| 29 ++- .../java/org/apache/doris/qe/ShowExecutor.java | 13 +- .../apache/doris/service/FrontendServiceImpl.java | 3 - .../doris/statistics/AnalysisInfoBuilder.java | 2 +- .../apache/doris/statistics/AnalysisManager.java | 92 + .../apache/doris/statistics/BaseAnalysisTask.java | 34 +-- .../org/apache/doris/statistics/HistogramTask.java | 5 - .../apache/doris/statistics/OlapAnalysisTask.java | 2 +- .../doris/statistics/StatisticConstants.java | 2 - .../apache/doris/statistics/TableStatsMeta.java| 18 +- .../doris/statistics/util/StatisticsUtil.java | 35 ++-- .../doris/transaction/DatabaseTransactionMgr.java | 3 + .../org/apache/doris/catalog/OlapTableTest.java| 47 + gensrc/thrift/BackendService.thrift| 1 + .../suites/statistics/analyze_stats.groovy | 2 +- .../suites/statistics/test_analyze_mv.groovy | 83 +++- .../statistics/test_drop_stats_and_truncate.groovy | 227 + 25 files changed, 550 insertions(+), 188 deletions(-) create mode 100644 regression-test/suites/statistics/test_drop_stats_and_truncate.groovy - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) 04/05: [fix](statistics)Fix empty table keep auto analyze bug. (#40811) #40982 (#40982)
This is an automated email from the ASF dual-hosted git repository.
lijibing pushed a commit to branch 2.0.15-tebu
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 6e59d8720c71af3eee1220b6bb65a835f3f0eaef
Author: Jibing-Li <64681310+jibing...@users.noreply.github.com>
AuthorDate: Thu Sep 26 14:04:17 2024 +0800
[fix](statistics)Fix empty table keep auto analyze bug. (#40811) #40982
(#40982)
backport: https://github.com/apache/doris/pull/40811
---
.../doris/statistics/AnalysisInfoBuilder.java | 2 +-
.../apache/doris/statistics/TableStatsMeta.java| 12 +++---
.../doris/statistics/util/StatisticsUtil.java | 35 ++--
.../statistics/test_drop_stats_and_truncate.groovy | 48 ++
4 files changed, 68 insertions(+), 29 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisInfoBuilder.java
b/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisInfoBuilder.java
index ddef30ee4de..17a32900be3 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisInfoBuilder.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisInfoBuilder.java
@@ -63,7 +63,7 @@ public class AnalysisInfoBuilder {
private boolean usingSqlForPartitionColumn;
private long tblUpdateTime;
private boolean emptyJob;
-private boolean userInject;
+private boolean userInject = false;
private long rowCount;
public AnalysisInfoBuilder() {
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/statistics/TableStatsMeta.java
b/fe/fe-core/src/main/java/org/apache/doris/statistics/TableStatsMeta.java
index a7701f06ef1..d0673998b54 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/statistics/TableStatsMeta.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/statistics/TableStatsMeta.java
@@ -24,7 +24,6 @@ import org.apache.doris.common.io.Text;
import org.apache.doris.common.io.Writable;
import org.apache.doris.persist.gson.GsonPostProcessable;
import org.apache.doris.persist.gson.GsonUtils;
-import org.apache.doris.statistics.AnalysisInfo.AnalysisMethod;
import org.apache.doris.statistics.AnalysisInfo.JobType;
import org.apache.doris.statistics.util.StatisticsUtil;
@@ -167,7 +166,9 @@ public class TableStatsMeta implements Writable,
GsonPostProcessable {
public void update(AnalysisInfo analyzedJob, TableIf tableIf) {
updatedTime = analyzedJob.tblUpdateTime;
-userInjected = analyzedJob.userInject;
+if (analyzedJob.userInject) {
+userInjected = true;
+}
String colNameStr = analyzedJob.colName;
// colName field AnalyzeJob's format likes: "[col1, col2]", we need to
remove brackets here
// TODO: Refactor this later
@@ -195,9 +196,6 @@ public class TableStatsMeta implements Writable,
GsonPostProcessable {
indexesRowCount.putAll(analyzedJob.indexesRowCount);
clearStaleIndexRowCount((OlapTable) tableIf);
}
-if (analyzedJob.emptyJob &&
AnalysisMethod.SAMPLE.equals(analyzedJob.analysisMethod)) {
-return;
-}
if (analyzedJob.colToPartitions.keySet()
.containsAll(tableIf.getBaseSchema().stream()
.filter(c ->
!StatisticsUtil.isUnsupportedType(c.getType()))
@@ -205,6 +203,10 @@ public class TableStatsMeta implements Writable,
GsonPostProcessable {
updatedRows.set(0);
newPartitionLoaded.set(false);
}
+// Set userInject back to false after manual analyze.
+if (JobType.MANUAL.equals(jobType) && !analyzedJob.userInject) {
+userInjected = false;
+}
}
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/statistics/util/StatisticsUtil.java
b/fe/fe-core/src/main/java/org/apache/doris/statistics/util/StatisticsUtil.java
index e0eef39a217..fe32755be45 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/statistics/util/StatisticsUtil.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/statistics/util/StatisticsUtil.java
@@ -976,37 +976,26 @@ public class StatisticsUtil {
}
public static boolean isEmptyTable(TableIf table,
AnalysisInfo.AnalysisMethod method) {
-int waitRowCountReportedTime = 90;
+int waitRowCountReportedTime = 120;
if (!(table instanceof OlapTable) ||
method.equals(AnalysisInfo.AnalysisMethod.FULL)) {
return false;
}
OlapTable olapTable = (OlapTable) table;
+long rowCount = 0;
for (int i = 0; i < waitRowCountReportedTime; i++) {
-if (olapTable.getRowCount() > 0) {
-return false;
-}
-boolean allInitVersion = true;
-// If all partitions' visible version are PARTITION_INIT_VERSION,
return true.
-// If any partition's visible version is greater than 2, r
(doris) 03/05: [fix](statistics)Fix drop stats log editlog bug. Catch drop stats exception while truncate table. (#40738) (#40979)
This is an automated email from the ASF dual-hosted git repository.
lijibing pushed a commit to branch 2.0.15-tebu
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 1515e17127ce0d60c0eaf714a18ef6eeb6379e98
Author: Jibing-Li <64681310+jibing...@users.noreply.github.com>
AuthorDate: Thu Sep 26 11:46:12 2024 +0800
[fix](statistics)Fix drop stats log editlog bug. Catch drop stats exception
while truncate table. (#40738) (#40979)
backport: https://github.com/apache/doris/pull/40738
---
.../apache/doris/alter/SchemaChangeHandler.java| 12 +-
.../org/apache/doris/alter/SchemaChangeJobV2.java | 6 +-
.../main/java/org/apache/doris/catalog/Env.java| 6 +-
.../apache/doris/service/FrontendServiceImpl.java | 3 -
.../apache/doris/statistics/AnalysisManager.java | 70
.../statistics/test_drop_stats_and_truncate.groovy | 179 +
6 files changed, 219 insertions(+), 57 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java
b/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java
index f60352f6e6b..1fd46769fbf 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java
@@ -2693,11 +2693,7 @@ public class SchemaChangeHandler extends AlterHandler {
LOG.debug("logModifyTableAddOrDropInvertedIndices info:{}",
info);
Env.getCurrentEnv().getEditLog().logModifyTableAddOrDropInvertedIndices(info);
// Drop table column stats after light schema change finished.
-try {
-
Env.getCurrentEnv().getAnalysisManager().dropStats(olapTable);
-} catch (Exception e) {
-LOG.info("Failed to drop stats after light schema change.
Reason: {}", e.getMessage());
-}
+Env.getCurrentEnv().getAnalysisManager().dropStats(olapTable);
if (isDropIndex) {
// send drop rpc to be
@@ -2723,11 +2719,7 @@ public class SchemaChangeHandler extends AlterHandler {
LOG.debug("logModifyTableAddOrDropColumns info:{}", info);
Env.getCurrentEnv().getEditLog().logModifyTableAddOrDropColumns(info);
// Drop table column stats after light schema change finished.
-try {
-
Env.getCurrentEnv().getAnalysisManager().dropStats(olapTable);
-} catch (Exception e) {
-LOG.info("Failed to drop stats after light schema change.
Reason: {}", e.getMessage());
-}
+Env.getCurrentEnv().getAnalysisManager().dropStats(olapTable);
}
LOG.info("finished modify table's add or drop or modify columns.
table: {}, job: {}, is replay: {}",
olapTable.getName(), jobId, isReplay);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeJobV2.java
b/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeJobV2.java
index 2591d3106e0..a4820022a07 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeJobV2.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeJobV2.java
@@ -605,11 +605,7 @@ public class SchemaChangeJobV2 extends AlterJobV2 {
changeTableState(dbId, tableId, OlapTableState.NORMAL);
LOG.info("set table's state to NORMAL, table id: {}, job id: {}",
tableId, jobId);
// Drop table column stats after schema change finished.
-try {
-Env.getCurrentEnv().getAnalysisManager().dropStats(tbl);
-} catch (Exception e) {
-LOG.info("Failed to drop stats after schema change finished.
Reason: {}", e.getMessage());
-}
+Env.getCurrentEnv().getAnalysisManager().dropStats(tbl);
}
private void onFinished(OlapTable tbl) {
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
index 91154b8d76c..cb66fe49559 100755
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
@@ -4528,11 +4528,7 @@ public class Env {
indexIdToSchemaVersion);
editLog.logColumnRename(info);
LOG.info("rename coloumn[{}] to {}", colName, newColName);
-try {
-Env.getCurrentEnv().getAnalysisManager().dropStats(table);
-} catch (Exception e) {
-LOG.info("Failed to drop stats after rename column. Reason:
{}", e.getMessage());
-}
+Env.getCurrentEnv().getAnalysisManager().dropStats(table);
}
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
b/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
index 7f
(doris) branch master updated: [fix](nereids)do not add delta row count to BE reported row count (#41464)
This is an automated email from the ASF dual-hosted git repository.
englefly pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 958d63831cd [fix](nereids)do not add delta row count to BE reported
row count (#41464)
958d63831cd is described below
commit 958d63831cddb99800842dc14cc4cd83c4498345
Author: minghong
AuthorDate: Tue Oct 8 15:22:26 2024 +0800
[fix](nereids)do not add delta row count to BE reported row count (#41464)
## Proposed changes
the table row count is in 3 cases:
1. injected row count
2. analyzed row count + delta row count
3. BE reported row count.
in previous pr #40529, we added delta row count in all 3 cases
Issue Number: close #xxx
---
.../main/java/org/apache/doris/nereids/stats/StatsCalculator.java | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/stats/StatsCalculator.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/stats/StatsCalculator.java
index 79a574dc3f7..3c70d4cd518 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/stats/StatsCalculator.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/stats/StatsCalculator.java
@@ -396,7 +396,7 @@ public class StatsCalculator extends
DefaultPlanVisitor {
rowCount =
olapTable.getRowCountForIndex(olapScan.getSelectedIndexId(), true);
if (rowCount == -1) {
if (tableMeta != null) {
-rowCount =
tableMeta.getRowCount(olapScan.getSelectedIndexId());
+rowCount =
tableMeta.getRowCount(olapScan.getSelectedIndexId()) +
computeDeltaRowCount(olapScan);
}
}
}
@@ -489,7 +489,7 @@ public class StatsCalculator extends
DefaultPlanVisitor {
builder.putColumnStatistics(slot, colStatsBuilder.build());
}
checkIfUnknownStatsUsedAsKey(builder);
-builder.setRowCount(selectedPartitionsRowCount +
deltaRowCount);
+builder.setRowCount(selectedPartitionsRowCount);
}
}
// 1. no partition is pruned, or
@@ -503,7 +503,7 @@ public class StatsCalculator extends
DefaultPlanVisitor {
builder.putColumnStatistics(slot, colStatsBuilder.build());
}
checkIfUnknownStatsUsedAsKey(builder);
-builder.setRowCount(tableRowCount + deltaRowCount);
+builder.setRowCount(tableRowCount);
}
return builder.build();
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [fix](fe) Fix `PluginMgr.getActivePluginList` npt (#41466)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 5fd2ef7d66c [fix](fe) Fix `PluginMgr.getActivePluginList` npt (#41466)
5fd2ef7d66c is described below
commit 5fd2ef7d66cc9ffa0c1bbd3d1c0794c0b8912de7
Author: Lei Zhang <27994433+swjtu-zhang...@users.noreply.github.com>
AuthorDate: Tue Oct 8 15:23:05 2024 +0800
[fix](fe) Fix `PluginMgr.getActivePluginList` npt (#41466)
```
Exception in thread "AuditEventProcessor" java.lang.NullPointerException:
at index 2
at
com.google.common.collect.ObjectArrays.checkElementNotNull(ObjectArrays.java:232)
at
com.google.common.collect.ObjectArrays.checkElementsNotNull(ObjectArrays.java:222)
at
com.google.common.collect.ObjectArrays.checkElementsNotNull(ObjectArrays.java:216)
at
com.google.common.collect.ImmutableList.construct(ImmutableList.java:353)
at
com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:265)
at
org.apache.doris.plugin.PluginMgr.getActivePluginList(PluginMgr.java:300)
at
org.apache.doris.qe.AuditEventProcessor$Worker.run(AuditEventProcessor.java:120)
at java.base/java.lang.Thread.run(Thread.java:833)
```
## Proposed changes
Issue Number: close #xxx
---
fe/fe-core/src/main/java/org/apache/doris/plugin/PluginMgr.java | 4
1 file changed, 4 insertions(+)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/plugin/PluginMgr.java
b/fe/fe-core/src/main/java/org/apache/doris/plugin/PluginMgr.java
index ea69b247e66..daf93f44d96 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/plugin/PluginMgr.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/plugin/PluginMgr.java
@@ -293,6 +293,10 @@ public class PluginMgr implements Writable {
m.values().forEach(d -> {
if (d.getStatus() == PluginStatus.INSTALLED) {
+if (d.getPlugin() == null) {
+LOG.warn("PluginLoader({}) status is INSTALLED, but plugin
is null", d);
+return;
+}
l.add(d.getPlugin());
}
});
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [Chore](checks) Clang Tidy Preparation do not open build_cloud (#41543)
This is an automated email from the ASF dual-hosted git repository.
panxiaolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new bdd738279c1 [Chore](checks) Clang Tidy Preparation do not open
build_cloud (#41543)
bdd738279c1 is described below
commit bdd738279c12744d932e9632e557be4969f6c14a
Author: Pxl
AuthorDate: Tue Oct 8 15:24:08 2024 +0800
[Chore](checks) Clang Tidy Preparation do not open build_cloud (#41543)
## Proposed changes
Clang Tidy Preparation do not open build_cloud
https://github.com/user-attachments/assets/cfdfcb16-f725-4cbd-af60-e15dd4bcc18f";>
---
.github/workflows/code-checks.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/code-checks.yml
b/.github/workflows/code-checks.yml
index 4fe4090b516..6aaa83f47cd 100644
--- a/.github/workflows/code-checks.yml
+++ b/.github/workflows/code-checks.yml
@@ -105,7 +105,7 @@ jobs:
popd
export
PATH="${DEFAULT_DIR}/ldb-toolchain/bin/:$(pwd)/thirdparty/installed/bin/:${PATH}"
-DISABLE_BE_JAVA_EXTENSIONS=ON DO_NOT_CHECK_JAVA_ENV=ON
DORIS_TOOLCHAIN=clang ENABLE_PCH=OFF OUTPUT_BE_BINARY=0 ./build.sh --be --cloud
+DISABLE_BE_JAVA_EXTENSIONS=ON DO_NOT_CHECK_JAVA_ENV=ON
DORIS_TOOLCHAIN=clang ENABLE_PCH=OFF OUTPUT_BE_BINARY=0 ./build.sh --be
fi
echo "should_check=${{ steps.filter.outputs.cpp_changes }}"
>>${GITHUB_OUTPUT}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (bdd738279c1 -> e8a1a16f3ca)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from bdd738279c1 [Chore](checks) Clang Tidy Preparation do not open build_cloud (#41543) add e8a1a16f3ca [Enhancement](maxCompute)Implement compatibility with existing maxcompute catalogs from previous versions. (#41386) No new revisions were added by this update. Summary of changes: be/src/vec/exec/jni_connector.cpp | 1 + .../doris/maxcompute/MaxComputeColumnValue.java| 50 +++ .../doris/maxcompute/MaxComputeJniScanner.java | 15 +- .../maxcompute/MaxComputeExternalCatalog.java | 151 + .../maxcompute/source/MaxComputeScanNode.java | 33 - .../property/constants/MCProperties.java | 10 +- fe/pom.xml | 2 +- .../test_external_catalog_maxcompute.out | 30 .../maxcompute/test_max_compute_all_type.out | 26 .../maxcompute/test_max_compute_complex_type.out | 19 ++- .../test_external_catalog_maxcompute.groovy| 47 ++- .../maxcompute/test_max_compute_all_type.groovy| 13 +- .../test_max_compute_complex_type.groovy | 33 +++-- 13 files changed, 342 insertions(+), 88 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Refactor](exec) refactor schema scanner del unless code [doris]
HappenLee commented on PR #41540: URL: https://github.com/apache/doris/pull/41540#issuecomment-2401067319 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](compatibility) make some agg function restrict [doris]
github-actions[bot] commented on PR #41459: URL: https://github.com/apache/doris/pull/41459#issuecomment-2401069048 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](inverted index) Resolve null processing issue in arrays_overlap [doris]
github-actions[bot] commented on PR #41495: URL: https://github.com/apache/doris/pull/41495#issuecomment-2401045440 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](inverted index) Resolve null processing issue in arrays_overlap [doris]
airborne12 merged PR #41495: URL: https://github.com/apache/doris/pull/41495 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (17422ff540b -> 4562b9a6821)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 17422ff540b [test](inverted index) fix test case for no need read data (#41564) add 4562b9a6821 [bugfix](arrow) Fix time zone issues and accuracy issues (#38215) No new revisions were added by this update. Summary of changes: .../pipeline/exec/memory_scratch_sink_operator.cpp | 2 +- be/src/pipeline/exec/result_sink_operator.cpp | 5 ++- be/src/util/arrow/row_batch.cpp| 37 -- be/src/util/arrow/row_batch.h | 11 --- be/src/vec/runtime/vparquet_transformer.cpp| 3 +- .../serde/data_type_serde_arrow_test.cpp | 4 +-- .../data/arrow_flight_sql_p0/test_select.out | 4 +++ regression-test/framework/pom.xml | 2 +- .../suites/arrow_flight_sql_p0/test_select.groovy | 12 +++ 9 files changed, 54 insertions(+), 26 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (4562b9a6821 -> 5197843ffd0)
This is an automated email from the ASF dual-hosted git repository. panxiaolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 4562b9a6821 [bugfix](arrow) Fix time zone issues and accuracy issues (#38215) add 5197843ffd0 [fix](FoldConst)The length function returns incorrect results under FoldConst. (#40947) No new revisions were added by this update. Summary of changes: .../trees/expressions/functions/executable/StringArithmetic.java| 2 +- .../expression/fold_constant/fold_constant_string_arithmatic.groovy | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](catalog)Allow parallel running of insert overwrite on the external table [doris]
zddr commented on PR #41575: URL: https://github.com/apache/doris/pull/41575#issuecomment-2401204957 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Enhancement](inverted index) return OK instead of not supported in expr evaluate_inverted_index #41567 [doris]
airborne12 opened a new pull request, #41578: URL: https://github.com/apache/doris/pull/41578 cherry pick from #41567 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) return OK instead of not supported in expr evaluate_inverted_index #41567 [doris]
airborne12 commented on PR #41578: URL: https://github.com/apache/doris/pull/41578#issuecomment-2401210384 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Enhancement](inverted index) return OK instead of not supported in expr evaluate_inverted_index #41567 [doris]
airborne12 opened a new pull request, #41577: URL: https://github.com/apache/doris/pull/41577 cherry pick from #41567 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) return OK instead of not supported in expr evaluate_inverted_index #41567 [doris]
airborne12 commented on PR #41577: URL: https://github.com/apache/doris/pull/41577#issuecomment-2401209986 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) return OK instead of not supported in expr evaluate_inverted_index #41567 [doris]
doris-robot commented on PR #41578: URL: https://github.com/apache/doris/pull/41578#issuecomment-2401210417 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) return OK instead of not supported in expr evaluate_inverted_index #41567 [doris]
doris-robot commented on PR #41577: URL: https://github.com/apache/doris/pull/41577#issuecomment-2401210022 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](protocol) only return multi result when CLIENT_MULTI_STATEMENTS been set [doris]
morrySnow commented on PR #36759: URL: https://github.com/apache/doris/pull/36759#issuecomment-2401201485 @T-M-L-C https://dev.mysql.com/doc/connector-j/en/connector-j-connp-props-security.html#cj-conn-prop_allowMultiQueries -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improve](array-overlaps) improve array overlaps [doris]
github-actions[bot] commented on PR #41571: URL: https://github.com/apache/doris/pull/41571#issuecomment-2401201791 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improve](array-overlaps) improve array overlaps [doris]
github-actions[bot] commented on PR #41571: URL: https://github.com/apache/doris/pull/41571#issuecomment-2401201817 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](nereids)adjust agg function nullability in PhysicalHashAggregate [doris]
starocean999 opened a new pull request, #41576: URL: https://github.com/apache/doris/pull/41576 ## Proposed changes `select sum(distinct c1) from t;` assume c1 is not null, because there is no group by, sum(distinct c1)'s nullable is alwasNullable in rewritten phase. But in implementation phase, we may create 3 phase agg with group by key c1. And the sum(distinct c1)'s nullability should be changed depending on if there is any group by expressions. This pr update the agg function's nullability accordingly -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids)adjust agg function nullability in PhysicalHashAggregate [doris]
doris-robot commented on PR #41576: URL: https://github.com/apache/doris/pull/41576#issuecomment-2401209555 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](wg&memtracker) using weak ptr to delete memtracker and query context automatically [doris]
doris-robot commented on PR #41549: URL: https://github.com/apache/doris/pull/41549#issuecomment-2401209877 TPC-DS: Total hot run time: 192566 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit c4b9ed1d1f820c07345f6c538724a93d4e185410, data reload: false query1 878 405 418 405 query2 6254209519901990 query3 8692193 199 193 query4 33984 23575 23979 23575 query5 3324492 471 471 query6 258 173 167 167 query7 4197311 304 304 query8 277 220 222 220 query9 9573268226852682 query10 464 282 291 282 query11 17876 15291 15186 15186 query12 150 105 100 100 query13 1577474 452 452 query14 9754749075667490 query15 254 172 178 172 query16 8006477 512 477 query17 1542619 588 588 query18 2211313 317 313 query19 374 156 165 156 query20 123 112 111 111 query21 211 114 110 110 query22 4782476445214521 query23 34966 34257 34168 34168 query24 10975 285329342853 query25 607 412 405 405 query26 770 162 166 162 query27 2071303 311 303 query28 6517243624052405 query29 777 451 445 445 query30 306 155 153 153 query31 1015805 821 805 query32 96 53 60 53 query33 778 303 320 303 query34 903 511 516 511 query35 902 754 729 729 query36 1127941 965 941 query37 146 91 88 88 query38 4129392838903890 query39 1489146814481448 query40 206 99 98 98 query41 48 43 44 43 query42 116 96 94 94 query43 527 493 476 476 query44 1237828 811 811 query45 213 167 166 166 query46 1130700 711 700 query47 1933184918181818 query48 463 347 356 347 query49 968 414 421 414 query50 817 422 408 408 query51 7073692569376925 query52 98 85 89 85 query53 255 182 187 182 query54 1169476 470 470 query55 79 76 81 76 query56 286 278 253 253 query57 12311131 query58 237 232 231 231 query59 3112295528652865 query60 296 291 279 279 query61 118 106 107 106 query62 863 685 665 665 query63 222 187 190 187 query64 3723655 625 625 query65 3271318932303189 query66 816 310 307 307 query67 15892 15487 15857 15487 query68 4445576 571 571 query69 499 307 290 290 query70 1203106111401061 query71 351 280 273 273 query72 7299398940123989 query73 758 343 347 343 query74 10193 896389398939 query75 3421267126832671 query76 2698938 914 914 query77 636 300 307 300 query78 10379 962695379537 query79 1695601 598 598 query80 2450446 467 446 query81 589 237 235 235 query82 725 136 141 136 query83 296 140 132 132 query84 280 85 74 74 query85 1631293 292 292 query86 432 295 302 295 query87 4488434643964346 query88 3493246624092409 query89 410 290 291 290 query90 2121192 190 190 query91 151 112 113 112 query92 62 45 48 45 query93 1097548 557 548 query94 1197301 302 301 query95 367 263 261 261 query96 622 280 288 280 query97 3287316831323132 query98 226 202 196 196 query99 1722131713151315 Total cold run time: 297957 ms Total hot run time: 192566 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t
Re: [PR] [fix](nereids)adjust agg function nullability in PhysicalHashAggregate [doris]
starocean999 commented on PR #41576: URL: https://github.com/apache/doris/pull/41576#issuecomment-2401209610 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](inverted index) add DATEV2 and DATETIMEV2 for inverted index reader #41565 [doris]
airborne12 commented on PR #41579: URL: https://github.com/apache/doris/pull/41579#issuecomment-2401212355 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Fix](inverted index) add DATEV2 and DATETIMEV2 for inverted index reader #41565 [doris]
airborne12 opened a new pull request, #41579: URL: https://github.com/apache/doris/pull/41579 cherry pick from #41565 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Bug](materialized-view) Fixed the problem of using drop table force and create mv stmt at the… [doris]
BiteThet opened a new pull request, #41580: URL: https://github.com/apache/doris/pull/41580 … same time causing fe startup failure. ## Proposed changes Fixed the problem of using drop table force and create mv stmt at the same time causing fe startup failure. Consider this scenario： create mv -> drop table force -> create mv stmt cancelled At this time there is a checkpoint like this： [checkpoint ] [editlog ] create mv -> drop table force -> create mv stmt cancelled In this case, when read meta and parse create mv stmt, it will fail because there is no corresponding table, further causing fe startup failure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](inverted index) add DATEV2 and DATETIMEV2 for inverted index reader #41565 [doris]
doris-robot commented on PR #41579: URL: https://github.com/apache/doris/pull/41579#issuecomment-2401212384 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](materialized-view) Fixed the problem of using drop table force and create mv stmt at the… [doris]
BiteThet commented on PR #41580: URL: https://github.com/apache/doris/pull/41580#issuecomment-2401212715 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](inverted index) add DATEV2 and DATETIMEV2 for inverted index reader #41565 [doris]
doris-robot commented on PR #41581: URL: https://github.com/apache/doris/pull/41581#issuecomment-2401213971 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Fix](inverted index) add DATEV2 and DATETIMEV2 for inverted index reader #41565 [doris]
airborne12 opened a new pull request, #41581: URL: https://github.com/apache/doris/pull/41581 cherry pick from #41565 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](inverted index) add DATEV2 and DATETIMEV2 for inverted index reader #41565 [doris]
airborne12 commented on PR #41581: URL: https://github.com/apache/doris/pull/41581#issuecomment-2401213910 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](wg&memtracker) using weak ptr to delete memtracker and query context automatically [doris]
doris-robot commented on PR #41549: URL: https://github.com/apache/doris/pull/41549#issuecomment-2401214433 ClickBench: Total hot run time: 32.24 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit c4b9ed1d1f820c07345f6c538724a93d4e185410, data reload: false query1 0.050.050.04 query2 0.070.030.02 query3 0.230.060.06 query4 1.640.100.09 query5 0.520.520.51 query6 1.140.750.73 query7 0.010.010.01 query8 0.040.030.04 query9 0.570.510.50 query10 0.540.550.54 query11 0.140.110.10 query12 0.130.120.11 query13 0.600.590.61 query14 2.682.772.73 query15 0.890.830.83 query16 0.390.390.39 query17 1.061.091.06 query18 0.190.190.19 query19 2.001.872.02 query20 0.010.000.01 query21 15.36 0.580.58 query22 2.662.801.86 query23 17.07 1.160.78 query24 2.581.350.47 query25 0.150.060.20 query26 0.460.140.13 query27 0.040.040.04 query28 11.30 1.081.07 query29 12.57 3.273.27 query30 0.250.060.06 query31 2.860.380.38 query32 3.260.480.45 query33 3.032.993.05 query34 17.12 4.454.47 query35 4.604.474.47 query36 0.700.480.48 query37 0.080.060.06 query38 0.040.040.03 query39 0.030.020.02 query40 0.160.120.13 query41 0.080.020.02 query42 0.040.020.02 query43 0.030.030.03 Total cold run time: 107.37 s Total hot run time: 32.24 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [test](inverted index) fix test case for no need read data #41564 [doris]
airborne12 opened a new pull request, #41582: URL: https://github.com/apache/doris/pull/41582 cherry pick from #41564 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](inverted index) fix test case for no need read data #41564 [doris]
airborne12 commented on PR #41582: URL: https://github.com/apache/doris/pull/41582#issuecomment-2401214585 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](inverted index) fix test case for no need read data #41564 [doris]
doris-robot commented on PR #41582: URL: https://github.com/apache/doris/pull/41582#issuecomment-2401214627 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](materialized-view) Fixed the problem of using drop table force and create mv stmt at the… [doris]
BiteThet commented on PR #41580: URL: https://github.com/apache/doris/pull/41580#issuecomment-2401221317 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [test](inverted index) fix test case for no need read data #41564 [doris]
airborne12 opened a new pull request, #41583: URL: https://github.com/apache/doris/pull/41583 cherry pick from #41564 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) return OK instead of not supported in expr evaluate_inverted_index #41567 [doris]
github-actions[bot] commented on PR #41578: URL: https://github.com/apache/doris/pull/41578#issuecomment-2401217990 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](inverted index) fix test case for no need read data #41564 [doris]
airborne12 commented on PR #41583: URL: https://github.com/apache/doris/pull/41583#issuecomment-2401217943 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test) fix unstable test_export_external_table cases (#41523) [doris]
morningman merged PR #41570: URL: https://github.com/apache/doris/pull/41570 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](inverted index) fix test case for no need read data #41564 [doris]
doris-robot commented on PR #41583: URL: https://github.com/apache/doris/pull/41583#issuecomment-2401218026 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) apply inverted index when has any #41547 [doris]
doris-robot commented on PR #41584: URL: https://github.com/apache/doris/pull/41584#issuecomment-2401221240 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated (25684f487bc -> 308700f0cab)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git from 25684f487bc [2.1][improvement](jdbc catalog) Improve JdbcClientException to accommodate various identifier formats (#41530) add 308700f0cab [fix](test) fix unstable test_export_external_table cases (#41523) (#41570) No new revisions were added by this update. Summary of changes: .../data/external_table_p0/export/test_export_external_table.out| 2 +- .../suites/external_table_p0/export/test_export_external_table.groovy | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) apply inverted index when has any #41547 [doris]
airborne12 commented on PR #41584: URL: https://github.com/apache/doris/pull/41584#issuecomment-2401221110 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) apply inverted index when has any #41547 [doris]
doris-robot commented on PR #41585: URL: https://github.com/apache/doris/pull/41585#issuecomment-2401222665 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](glue) support glue on aws [doris]
github-actions[bot] commented on PR #41084: URL: https://github.com/apache/doris/pull/41084#issuecomment-2401222900 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](cases) log url for jdbc [doris]
doris-robot commented on PR #41586: URL: https://github.com/apache/doris/pull/41586#issuecomment-2401227176 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](inverted index) add DATEV2 and DATETIMEV2 for inverted index reader #41565 [doris]
github-actions[bot] commented on PR #41579: URL: https://github.com/apache/doris/pull/41579#issuecomment-2401224269 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](cases) log url for jdbc [doris]
dataroaring commented on PR #41586: URL: https://github.com/apache/doris/pull/41586#issuecomment-2401227294 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) apply inverted index when has any #41547 [doris]
airborne12 commented on PR #41585: URL: https://github.com/apache/doris/pull/41585#issuecomment-2401222575 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [enhancement](cases) log url for jdbc [doris]
dataroaring opened a new pull request, #41586: URL: https://github.com/apache/doris/pull/41586 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) return OK instead of not supported in expr evaluate_inverted_index [doris]
xiaokang commented on code in PR #41567:
URL: https://github.com/apache/doris/pull/41567#discussion_r1792645757
##
be/src/vec/exprs/vexpr.h:
##
@@ -120,7 +120,7 @@ class VExpr {
// execute current expr with inverted index to filter block. Given a
roaring bitmap of match rows
virtual Status evaluate_inverted_index(VExprContext* context, uint32_t
segment_num_rows) {
-return Status::NotSupported("Not supported
execute_with_inverted_index");
+return Status::OK();
Review Comment:
add comment for semantics of return OK()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](inverted index) fix test case for no need read data [doris]
airborne12 merged PR #41564: URL: https://github.com/apache/doris/pull/41564 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](plugin)It's allowed for the plugin directory to be empty when loading plugins [doris]
github-actions[bot] commented on PR #41574: URL: https://github.com/apache/doris/pull/41574#issuecomment-2401141386 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](scanner) Fix incorrect _max_thread_num in scanner context when many queries are running. [doris]
github-actions[bot] commented on code in PR #41273:
URL: https://github.com/apache/doris/pull/41273#discussion_r1792700114
##
be/src/vec/exec/scan/scanner_context.cpp:
##
@@ -54,53 +56,140 @@ ScannerContext::ScannerContext(
_output_row_descriptor(output_row_descriptor),
_batch_size(state->batch_size()),
limit(limit_),
- _max_bytes_in_queue(std::max(max_bytes_in_blocks_queue,
(int64_t)1024) *
- num_parallel_instances),
- _scanner_scheduler(state->exec_env()->scanner_scheduler()),
+ _scanner_scheduler_global(state->exec_env()->scanner_scheduler()),
_all_scanners(scanners.begin(), scanners.end()),
- _num_parallel_instances(num_parallel_instances) {
+ _ignore_data_distribution(ignore_data_distribution) {
DCHECK(_output_row_descriptor == nullptr ||
_output_row_descriptor->tuple_descriptors().size() == 1);
_query_id = _state->get_query_ctx()->query_id();
ctx_id = UniqueId::gen_uid().to_string();
+_scanners.enqueue_bulk(scanners.begin(), scanners.size());
+if (limit < 0) {
+limit = -1;
+}
+MAX_SCALE_UP_RATIO = _state->scanner_scale_up_ratio();
+_query_thread_context = {_query_id, _state->query_mem_tracker(),
+ _state->get_query_ctx()->workload_group()};
+_dependency = dependency;
+
+DorisMetrics::instance()->scanner_ctx_cnt->increment(1);
+}
+
+// After init function call, should not access _parent
+Status ScannerContext::init() {
Review Comment:
warning: function 'init' exceeds recommended size/complexity thresholds
[readability-function-size]
```cpp
Status ScannerContext::init() {
^
```
Additional context
**be/src/vec/exec/scan/scanner_context.cpp:78:** 143 lines including
whitespace and comments (threshold 80)
```cpp
Status ScannerContext::init() {
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](dependencies)upgrade fe dependencies [doris]
github-actions[bot] commented on PR #41142: URL: https://github.com/apache/doris/pull/41142#issuecomment-2401140290 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [2.1][improvement](jdbc catalog) Improve JdbcClientException to accommodate various identifier formats (#41530)
This is an automated email from the ASF dual-hosted git repository.
zykkk pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new 25684f487bc [2.1][improvement](jdbc catalog) Improve
JdbcClientException to accommodate various identifier formats (#41530)
25684f487bc is described below
commit 25684f487bc1c372800b00577d2c07adebaddfde
Author: zy-kkk
AuthorDate: Wed Oct 9 10:32:41 2024 +0800
[2.1][improvement](jdbc catalog) Improve JdbcClientException to accommodate
various identifier formats (#41530)
pick (#40931)
In some cases, JDBC returns exceptions with various identifiers that
cannot be formatted correctly, such as `%`. This PR optimizes this.
---
.../jdbc/client/JdbcClientException.java | 27 -
.../datasource/jdbc/JdbcClientExceptionTest.java | 132 +
2 files changed, 157 insertions(+), 2 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/client/JdbcClientException.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/client/JdbcClientException.java
index f9c1e53c2ab..7fcea7aa61a 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/client/JdbcClientException.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/client/JdbcClientException.java
@@ -19,10 +19,33 @@ package org.apache.doris.datasource.jdbc.client;
public class JdbcClientException extends RuntimeException {
public JdbcClientException(String format, Throwable cause, Object... msg) {
-super(String.format(format, msg), cause);
+super(formatMessage(format, msg), cause);
}
public JdbcClientException(String format, Object... msg) {
-super(String.format(format, msg));
+super(formatMessage(format, msg));
+}
+
+private static String formatMessage(String format, Object... msg) {
+if (msg == null || msg.length == 0) {
+return format;
+} else {
+return String.format(format, escapePercentInArgs(msg));
+}
+}
+
+private static Object[] escapePercentInArgs(Object... args) {
+if (args == null) {
+return null;
+}
+Object[] escapedArgs = new Object[args.length];
+for (int i = 0; i < args.length; i++) {
+if (args[i] instanceof String) {
+escapedArgs[i] = ((String) args[i]).replace("%", "%%");
+} else {
+escapedArgs[i] = args[i];
+}
+}
+return escapedArgs;
}
}
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/datasource/jdbc/JdbcClientExceptionTest.java
b/fe/fe-core/src/test/java/org/apache/doris/datasource/jdbc/JdbcClientExceptionTest.java
new file mode 100644
index 000..1bbf54e9438
--- /dev/null
+++
b/fe/fe-core/src/test/java/org/apache/doris/datasource/jdbc/JdbcClientExceptionTest.java
@@ -0,0 +1,132 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.datasource.jdbc;
+
+import org.apache.doris.datasource.jdbc.client.JdbcClientException;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+public class JdbcClientExceptionTest {
+
+@Test
+public void testExceptionWithoutArgs() {
+String message = "An error occurred.";
+JdbcClientException exception = new JdbcClientException(message);
+
+Assert.assertEquals(message, exception.getMessage());
+Assert.assertNull(exception.getCause());
+}
+
+@Test
+public void testExceptionWithFormattingArgs() {
+String format = "Error code: %d, message: %s";
+int errorCode = 404;
+String errorMsg = "Not Found";
+JdbcClientException exception = new JdbcClientException(format,
errorCode, errorMsg);
+
+String expectedMessage = String.format(format, errorCode, errorMsg);
+Assert.assertEquals(expectedMessage, exception.getMessage());
+Assert.assertNull(exception.getCause());
+}
+
+@Test
+public void testExceptionWithPercentInFormatString() {
+String format = "Usage is at 80%%, threshold is %
[PR] pick 40084 40200 40178 [doris]
englefly opened a new pull request, #41573: URL: https://github.com/apache/doris/pull/41573 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) fix fold constant by be return type mismatched (#39723)(#41164) [doris]
LiBinfeng-01 commented on PR #41537: URL: https://github.com/apache/doris/pull/41537#issuecomment-2401129591 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](scanner) Fix incorrect _max_thread_num in scanner context when many queries are running. [doris]
github-actions[bot] commented on code in PR #41273:
URL: https://github.com/apache/doris/pull/41273#discussion_r1792700984
##
be/src/vec/exec/scan/scanner_context.cpp:
##
@@ -54,53 +56,140 @@ ScannerContext::ScannerContext(
_output_row_descriptor(output_row_descriptor),
_batch_size(state->batch_size()),
limit(limit_),
- _max_bytes_in_queue(std::max(max_bytes_in_blocks_queue,
(int64_t)1024) *
- num_parallel_instances),
- _scanner_scheduler(state->exec_env()->scanner_scheduler()),
+ _scanner_scheduler_global(state->exec_env()->scanner_scheduler()),
_all_scanners(scanners.begin(), scanners.end()),
- _num_parallel_instances(num_parallel_instances) {
+ _ignore_data_distribution(ignore_data_distribution) {
DCHECK(_output_row_descriptor == nullptr ||
_output_row_descriptor->tuple_descriptors().size() == 1);
_query_id = _state->get_query_ctx()->query_id();
ctx_id = UniqueId::gen_uid().to_string();
+_scanners.enqueue_bulk(scanners.begin(), scanners.size());
+if (limit < 0) {
+limit = -1;
+}
+MAX_SCALE_UP_RATIO = _state->scanner_scale_up_ratio();
+_query_thread_context = {_query_id, _state->query_mem_tracker(),
+ _state->get_query_ctx()->workload_group()};
+_dependency = dependency;
+
+DorisMetrics::instance()->scanner_ctx_cnt->increment(1);
+}
+
+// After init function call, should not access _parent
+Status ScannerContext::init() {
Review Comment:
warning: function 'init' exceeds recommended size/complexity thresholds
[readability-function-size]
```cpp
Status ScannerContext::init() {
^
```
Additional context
**be/src/vec/exec/scan/scanner_context.cpp:78:** 144 lines including
whitespace and comments (threshold 80)
```cpp
Status ScannerContext::init() {
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Use DorisVector to avoid memory usage from not being traced [doris]
yiguolei commented on code in PR #41557: URL: https://github.com/apache/doris/pull/41557#discussion_r1792692054 ## be/src/vec/common/custom_allocator.h: ## @@ -23,8 +23,8 @@ template > class CustomStdAllocator; -template -using DorisVector = std::vector>; +template Review Comment: rename to clear_memory -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Use DorisVector to avoid memory usage from not being traced [doris]
yiguolei commented on code in PR #41557:
URL: https://github.com/apache/doris/pull/41557#discussion_r1792692320
##
be/src/pipeline/exec/aggregation_source_operator.cpp:
##
@@ -53,6 +53,9 @@ Status AggLocalState::init(RuntimeState* state,
LocalStateInfo& info) {
_hash_table_size_counter =
ADD_COUNTER_WITH_LEVEL(Base::profile(), "HashTableSize",
TUnit::UNIT, 1);
+_container_memory_usage = ADD_COUNTER(profile(), "ContainerMemoryUsage",
TUnit::BYTES);
Review Comment:
rename to MemoryUsageContainer MemoryUsageArena
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Compile](fix) Fix mac complie BE error because codecvt_utf8_utf16 deprecated in C++17 and atomic_long not support in mac [doris]
HappenLee merged PR #41569: URL: https://github.com/apache/doris/pull/41569 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](Nereids) use 1 instead narrowest column when do column pruning [doris]
morrySnow commented on PR #41548: URL: https://github.com/apache/doris/pull/41548#issuecomment-2401131144 run cloud_p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) return OK instead of not supported in expr evaluate_inverted_index [doris]
github-actions[bot] commented on PR #41567: URL: https://github.com/apache/doris/pull/41567#issuecomment-2401166259 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](scan) remove useless enable_scan_node_run_serial [doris]
github-actions[bot] commented on PR #41559: URL: https://github.com/apache/doris/pull/41559#issuecomment-2401149752 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](inverted index) add DATEV2 and DATETIMEV2 for inverted index reader [doris]
airborne12 merged PR #41565: URL: https://github.com/apache/doris/pull/41565 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [cherry-pick](branch2.1) fix hudi jni scanner (#41566)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new a0aed772186 [cherry-pick](branch2.1) fix hudi jni scanner (#41566)
a0aed772186 is described below
commit a0aed772186026b482f1edc604943cdff4137e33
Author: Socrates
AuthorDate: Wed Oct 9 10:31:50 2024 +0800
[cherry-pick](branch2.1) fix hudi jni scanner (#41566)
pick from https://github.com/apache/doris/pull/41316
---
fe/fe-common/pom.xml | 9 +
fe/fe-core/pom.xml | 4
fe/pom.xml | 1 +
3 files changed, 10 insertions(+), 4 deletions(-)
diff --git a/fe/fe-common/pom.xml b/fe/fe-common/pom.xml
index 4bb86c8e42b..e44546eef44 100644
--- a/fe/fe-common/pom.xml
+++ b/fe/fe-common/pom.xml
@@ -128,6 +128,15 @@ under the License.
org.apache.logging.log4j
log4j-core
+
+com.aliyun.oss
+aliyun-sdk-oss
+${aliyun-sdk-oss.version}
+
+
+org.apache.hadoop
+hadoop-aliyun
+
doris-fe-common
diff --git a/fe/fe-core/pom.xml b/fe/fe-core/pom.xml
index 1c5ee1990e1..191ded302be 100644
--- a/fe/fe-core/pom.xml
+++ b/fe/fe-core/pom.xml
@@ -418,10 +418,6 @@ under the License.
-
-org.apache.hadoop
-hadoop-aliyun
-
com.qcloud.cos
hadoop-cos
diff --git a/fe/pom.xml b/fe/pom.xml
index 8e1dfaa330f..e207344f908 100644
--- a/fe/pom.xml
+++ b/fe/pom.xml
@@ -367,6 +367,7 @@ under the License.
0.8.10
202
5.3.0
+3.15.0
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](scan) remove useless enable_scan_node_run_serial [doris]
github-actions[bot] commented on PR #41559: URL: https://github.com/apache/doris/pull/41559#issuecomment-2401149723 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](scan) remove useless enable_scan_node_run_serial [doris]
yiguolei merged PR #41559: URL: https://github.com/apache/doris/pull/41559 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) return OK instead of not supported in expr evaluate_inverted_index [doris]
github-actions[bot] commented on PR #41567: URL: https://github.com/apache/doris/pull/41567#issuecomment-2401166234 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](external) Fix case-insensitive table name mapping retrieval rules [doris]
morningman commented on PR #38227: URL: https://github.com/apache/doris/pull/38227#issuecomment-2401168642 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](external) Fix case-insensitive table name mapping retrieval rules [doris]
github-actions[bot] commented on PR #38227: URL: https://github.com/apache/doris/pull/38227#issuecomment-2401169052 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [function](cast)Make string casting to integers more like MySQL's beh… [doris]
Mryange commented on PR #41541: URL: https://github.com/apache/doris/pull/41541#issuecomment-2401001388 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cases) remove all unstable cases in hint directory [doris]
dataroaring merged PR #41383: URL: https://github.com/apache/doris/pull/41383 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [local exchange](fix) Fix correctness caused by local exchange [doris]
github-actions[bot] commented on PR #41555: URL: https://github.com/apache/doris/pull/41555#issuecomment-2401121474 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [local exchange](fix) Fix correctness caused by local exchange [doris]
github-actions[bot] commented on PR #41555: URL: https://github.com/apache/doris/pull/41555#issuecomment-2401121504 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Nereids) ignore match expression logging warning message [doris]
LiBinfeng-01 commented on PR #41546: URL: https://github.com/apache/doris/pull/41546#issuecomment-2401128407 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [Fix](inverted index) Resolve null processing issue in arrays_overlap (#41495)
This is an automated email from the ASF dual-hosted git repository.
airborne pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new bc9b87d2f72 [Fix](inverted index) Resolve null processing issue in
arrays_overlap (#41495)
bc9b87d2f72 is described below
commit bc9b87d2f72939a21f2cd8cfd8562841e2a8caf1
Author: airborne12
AuthorDate: Wed Oct 9 09:28:37 2024 +0800
[Fix](inverted index) Resolve null processing issue in arrays_overlap
(#41495)
## Proposed changes
Fix problem "Runtime Error: Null pointer passed to
'StringTypeInvertedIndexReader::query', which requires a non-null
argument."
```
Stack Trace:
#0 doris::segment_v2::StringTypeInvertedIndexReader::query(...)
inverted_index_reader.cpp:473
#1 doris::segment_v2::InvertedIndexIterator::read_from_inverted_index(...)
inverted_index_reader.cpp:1237
#2 doris::vectorized::FunctionArraysOverlap::evaluate_inverted_index(...)
function_arrays_overlap.h:192
#3 doris::vectorized::DefaultFunction::evaluate_inverted_index(...)
function.h:532
#4 doris::vectorized::VExpr::_evaluate_inverted_index(...) vexpr.cpp:708
#5 doris::vectorized::VectorizedFnCall::evaluate_inverted_index(...)
vectorized_fn_call.cpp:143
#6 doris::vectorized::VExprContext::evaluate_inverted_index(...)
vexpr_context.cpp:126
```
---
.../vec/functions/array/function_arrays_overlap.h | 20 +-
.../test_array_contains_with_inverted_index.out| 615 +
.../test_array_contains_with_inverted_index.groovy | 29 +-
3 files changed, 642 insertions(+), 22 deletions(-)
diff --git a/be/src/vec/functions/array/function_arrays_overlap.h
b/be/src/vec/functions/array/function_arrays_overlap.h
index c06af8b05bd..5556b92c685 100644
--- a/be/src/vec/functions/array/function_arrays_overlap.h
+++ b/be/src/vec/functions/array/function_arrays_overlap.h
@@ -184,22 +184,18 @@ public:
}
std::unique_ptr query_param = nullptr;
const Array& query_val = param_value.get();
-for (size_t i = 0; i < query_val.size(); ++i) {
-Field nested_query_val = query_val[i];
+for (auto nested_query_val : query_val) {
+// any element inside array is NULL, return NULL
+// by current arrays_overlap execute logic.
+if (nested_query_val.is_null()) {
+return Status::OK();
+}
std::shared_ptr single_res =
std::make_shared();
RETURN_IF_ERROR(InvertedIndexQueryParamFactory::create_query_value(
nested_param_type, &nested_query_val, query_param));
-Status st = iter->read_from_inverted_index(
+RETURN_IF_ERROR(iter->read_from_inverted_index(
data_type_with_name.first, query_param->get_value(),
-segment_v2::InvertedIndexQueryType::EQUAL_QUERY, num_rows,
single_res);
-if (st.code() == ErrorCode::INVERTED_INDEX_NO_TERMS) {
-// if analyzed param with no term, we do not filter any rows
-// return all rows with OK status
-roaring->addRange(0, num_rows);
-break;
-} else if (st != Status::OK()) {
-return st;
-}
+segment_v2::InvertedIndexQueryType::EQUAL_QUERY, num_rows,
single_res));
*roaring |= *single_res;
}
diff --git
a/regression-test/data/inverted_index_p0/test_array_contains_with_inverted_index.out
b/regression-test/data/inverted_index_p0/test_array_contains_with_inverted_index.out
index c1c4ee1dc04..a93c7e2a2cf 100644
---
a/regression-test/data/inverted_index_p0/test_array_contains_with_inverted_index.out
+++
b/regression-test/data/inverted_index_p0/test_array_contains_with_inverted_index.out
@@ -242,6 +242,14 @@
2019-01-01 a648a447b8f71522f11632eba4b4adde["p", "q", "r", "s",
"t"]
-- !sql --
+2019-01-01 a648a447b8f71522f11632eba4b4adde["p", "q", "r", "s",
"t"]
+
+-- !sql --
+
+-- !sql --
+
+-- !sql --
+2019-01-01 a648a447b8f71522f11632eba4b4adde["p", "q", "r", "s",
"t"]
-- !sql --
2019-01-01 a648a447b8f71522f11632eba4b4adde["p", "q", "r", "s",
"t"]
@@ -260,6 +268,20 @@
2017-01-01 d93d942d985a8fb7547c72dada8d332d["d", "e", "f", "g",
"h", "i", "j", "k", "l"]
2019-01-01 a648a447b8f71522f11632eba4b4adde["p", "q", "r", "s",
"t"]
+-- !sql --
+2017-01-01 021603e7dcfe65d44af0efd0e5aee154["n"]
+2017-01-01 48a33ec3453a28bce84b8f96fe161956["m"]
+2017-01-01 6afef581285b6608bf80d5a4e46cf839["a", "b", "c"]
+2017-01-01 8fcb57ae675f0af4d613d9e6c0e8a2a3\N
+2017-01-01 8fcb57ae675f0af4d613d9e6c0e8a2a4\N
+2017-01-01 8fcb57ae675f0af4d613d9e6c0e8a2a5[]
+2017-01-01 8fcb57ae675f0af4d613d9e6c0e8a2a
(doris) branch master updated: [test](inverted index) fix test case for no need read data (#41564)
This is an automated email from the ASF dual-hosted git repository.
airborne pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 17422ff540b [test](inverted index) fix test case for no need read data
(#41564)
17422ff540b is described below
commit 17422ff540b374e1bd273148d954475101e0a176
Author: airborne12
AuthorDate: Wed Oct 9 09:36:31 2024 +0800
[test](inverted index) fix test case for no need read data (#41564)
## Proposed changes
fix regression case for no need read data opt.
---
regression-test/suites/inverted_index_p0/test_need_read_data.groovy | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/regression-test/suites/inverted_index_p0/test_need_read_data.groovy
b/regression-test/suites/inverted_index_p0/test_need_read_data.groovy
index e1251d207f8..3bff37e261d 100644
--- a/regression-test/suites/inverted_index_p0/test_need_read_data.groovy
+++ b/regression-test/suites/inverted_index_p0/test_need_read_data.groovy
@@ -136,6 +136,6 @@ suite("test_need_read_data", "p0"){
logger.info("show variales result: " + var_result )
sql "INSERT INTO ${indexTblName3} VALUES (1, 1),(1, -2),(1, -1);"
-qt_sql "SELECT /*+SET_VAR(enable_common_expr_pushdown=false) */ id FROM
${indexTblName3} WHERE value<0 and abs(value)>1;"
-qt_sql "SELECT /*+SET_VAR(enable_common_expr_pushdown=true) */ id FROM
${indexTblName3} WHERE value<0 and abs(value)>1;"
+qt_sql "SELECT
/*+SET_VAR(enable_common_expr_pushdown=false,inverted_index_skip_threshold=100)
*/ id FROM ${indexTblName3} WHERE value<0 and abs(value)>1;"
+qt_sql "SELECT
/*+SET_VAR(enable_common_expr_pushdown=true,inverted_index_skip_threshold=100)
*/ id FROM ${indexTblName3} WHERE value<0 and abs(value)>1;"
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [cherry-pick](branch2.1) fix hudi jni scanner [doris]
morningman merged PR #41566: URL: https://github.com/apache/doris/pull/41566 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](compatibility) make some agg function restrict [doris]
github-actions[bot] commented on PR #41459: URL: https://github.com/apache/doris/pull/41459#issuecomment-2401161821 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](wg&memtracker) using weak ptr to delete memtracker and query context automatically [doris]
yiguolei commented on PR #41549: URL: https://github.com/apache/doris/pull/41549#issuecomment-2401116105 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](inverted index) add DATEV2 and DATETIMEV2 for inverted index reader [doris]
github-actions[bot] commented on PR #41565: URL: https://github.com/apache/doris/pull/41565#issuecomment-2401165898 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (2255b186e88 -> 03aff6e2aef)
This is an automated email from the ASF dual-hosted git repository. airborne pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 2255b186e88 [chore](scan) remove useless enable_scan_node_run_serial (#41559) add 03aff6e2aef [Fix](inverted index) add DATEV2 and DATETIMEV2 for inverted index reader (#41565) No new revisions were added by this update. Summary of changes: be/src/olap/rowset/segment_v2/inverted_index_reader.h | 2 ++ 1 file changed, 2 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix]count tablet meta's static memory load from disk [doris]
wangbo commented on PR #41429: URL: https://github.com/apache/doris/pull/41429#issuecomment-2401167789 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix]count tablet meta's static memory load from disk [doris]
github-actions[bot] commented on code in PR #41429:
URL: https://github.com/apache/doris/pull/41429#discussion_r1792720206
##
be/src/olap/tablet_schema.h:
##
@@ -194,6 +196,9 @@ class TabletColumn {
return Status::OK();
}
+private:
+TabletColumn& operator=(const TabletColumn& ts) = default;
+
private:
Review Comment:
warning: redundant access specifier has the same accessibility as the
previous access specifier [readability-redundant-access-specifiers]
```suggestion
```
Additional context
**be/src/olap/tablet_schema.h:198:** previously declared here
```cpp
private:
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](jdbc catalog) Disable oracle scan null operator pushdown [doris]
github-actions[bot] commented on PR #41563: URL: https://github.com/apache/doris/pull/41563#issuecomment-2401174048 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](jdbc catalog) Disable oracle scan null operator pushdown [doris]
github-actions[bot] commented on PR #41563: URL: https://github.com/apache/doris/pull/41563#issuecomment-2401174070 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](mtmv) Support partition trace from multi join input when create parition materialized view [doris]
seawinde commented on PR #40942: URL: https://github.com/apache/doris/pull/40942#issuecomment-2401175806 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)remove old analyzer in OlapTableSink [doris]
starocean999 commented on PR #41298: URL: https://github.com/apache/doris/pull/41298#issuecomment-2401196478 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](catalog)Allow parallel running of insert overwrite on the external table [doris]
zddr commented on PR #41575: URL: https://github.com/apache/doris/pull/41575#issuecomment-2401195722 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
