morningman commented on code in PR #24154:
URL: https://github.com/apache/doris/pull/24154#discussion_r1322640794
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/ColumnStatistic.java:
##########
@@ -92,7 +91,7 @@ public class ColumnStatistic {
public final Histogram histogram;
@SerializedName("partitionIdToColStats")
- public final Map<Long, ColumnStatistic> partitionIdToColStats = new
HashMap<>();
+ public final Map<String, ColumnStatistic> partitionIdToColStats = new
HashMap<>();
Review Comment:
It is dangerous to modify a persitent field.
Why need to change to string?
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/ColumnStatistic.java:
##########
@@ -119,22 +118,26 @@ public ColumnStatistic(double count, double ndv,
ColumnStatistic original, doubl
}
public static ColumnStatistic fromResultRow(List<ResultRow> resultRows) {
- Map<Long, ColumnStatistic> partitionIdToColStats = new HashMap<>();
+ Map<String, ColumnStatistic> partitionIdToColStats = new HashMap<>();
ColumnStatistic columnStatistic = null;
try {
for (ResultRow resultRow : resultRows) {
String partId = resultRow.get(6);
if (partId == null) {
columnStatistic = fromResultRow(resultRow);
} else {
- partitionIdToColStats.put(Long.parseLong(partId),
fromResultRow(resultRow));
+ partitionIdToColStats.put(partId,
fromResultRow(resultRow));
}
}
} catch (Throwable t) {
LOG.debug("Failed to deserialize column stats", t);
return ColumnStatistic.UNKNOWN;
}
- Preconditions.checkState(columnStatistic != null, "Column stats is
null");
+ // columnStatistic may be null when there is only partition level
stats for this column.
+ if (columnStatistic == null) {
+ LOG.warn("Column stats is null");
Review Comment:
meaningless log, you'd either and more details in log, or just remove it.
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/HMSAnalysisTask.java:
##########
@@ -67,10 +70,8 @@ public class HMSAnalysisTask extends BaseAnalysisTask {
+ "NOW() "
+ "FROM `${catalogName}`.`${dbName}`.`${tblName}`";
- private static final String ANALYZE_SQL_PARTITION_TEMPLATE = "INSERT INTO "
- + "${internalDB}.${columnStatTbl}"
- + " SELECT "
- + "CONCAT(${tblId}, '-', ${idxId}, '-', '${colId}') AS id, "
+ private static final String ANALYZE_PARTITION_TEMPLATE = " SELECT "
+ + "CONCAT(${tblId}, '-', ${idxId}, '-', '${colId}', '-',
${partId}) AS id, "
Review Comment:
Why removing the `INSERT INTO`?
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/HMSAnalysisTask.java:
##########
@@ -67,10 +70,8 @@ public class HMSAnalysisTask extends BaseAnalysisTask {
+ "NOW() "
+ "FROM `${catalogName}`.`${dbName}`.`${tblName}`";
- private static final String ANALYZE_SQL_PARTITION_TEMPLATE = "INSERT INTO "
- + "${internalDB}.${columnStatTbl}"
- + " SELECT "
- + "CONCAT(${tblId}, '-', ${idxId}, '-', '${colId}') AS id, "
+ private static final String ANALYZE_PARTITION_TEMPLATE = " SELECT "
+ + "CONCAT(${tblId}, '-', ${idxId}, '-', '${colId}', '-',
${partId}) AS id, "
Review Comment:
And this makes the template not like `ANALYZE_TABLE_TEMPLATE`, which has
`insert into`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
