This is an automated email from the ASF dual-hosted git repository. englefly pushed a commit to branch pick-69209-69225-4.1 in repository https://gitbox.apache.org/repos/asf/doris.git
commit b75aadfbb267e09d7e7a17605ba8733525dc26cf Author: minghong <[email protected]> AuthorDate: Sat Oct 17 12:29:55 2026 +0800 branch-4.1 [improvement](statistics) Stop collecting hot value in the linear sample estimator (#69209) ### What problem does this PR solve? Related PR: #52048 Problem Summary: `LINEAR_ANALYZE_TEMPLATE` collected hot values in a third CTE: it grouped the sampled column by value, counted each group, and kept the top `hot_value_collect_count` groups. That CTE re-reads the whole sample, and it is the dominant cost of the template - the one sample analyze picks for a single distribution column / single unique key column. The same template also serves the full-scan fallback of a sample analyze (the requested sample rows exceed the table, or the picked tablets hold fewer rows than requested), where the extra aggregation is not bounded by the sample size at all. This PR removes that CTE. `hot_value` keeps its position in the result set and is reported as NULL, exactly like `FULL_ANALYZE_WITHOUT_HOT_VALUE_TEMPLATE` and `MERGE_PARTITION_TEMPLATE` already do, and every reader already treats NULL as "not collected" (`ColumnStatistic.hotValues == null`). Hot values are still collected by `DUJ1_ANALYZE_TEMPLATE`, which derives them from the grouped sample it already builds for the NDV estimate, and by a full analyze that asks for them explicitly (`ANALYZE TABLE ... WITH HOT VALUE`). The params that only served the removed CTE (`subStringColName`, `rowCount2` on the linear path, `hotValueCollectCount`) are now filled for the DUJ1 algorithm only. As a side effect, the linear branch of `PluginDrivenSampleAnalysisTask` no longer renders an unsubstituted `${subStringColName}` into its SQL: it never set that param (the legacy `HMSAnalysisTask` it was ported from did not either), so a bucketed plain-hive table that took that branch got invalid SQL before this change. Templates checked, and why they are left alone: | Template | Hot value | Action | |---|---|---| | `LINEAR_ANALYZE_TEMPLATE` | collected via an extra GROUP BY over the sample | **removed here** | | `DUJ1_ANALYZE_TEMPLATE` | collected from the grouped sample it already builds for NDV | kept | | `FULL_ANALYZE_TEMPLATE` | only when the user writes `WITH HOT VALUE` | kept (explicit opt-in, not a default cost) | | `FULL_ANALYZE_WITHOUT_HOT_VALUE_TEMPLATE`, `MERGE_PARTITION_TEMPLATE` | `null as hot_value` | unchanged | | `PARTITION_ANALYZE_TEMPLATE`, `ANALYZE_PARTITION_COLUMN_TEMPLATE` | no hot value | unchanged | Behaviour changes: - A sample analyze no longer collects hot values for columns estimated with the linear estimator: a single distribution column / single unique key column, and any sample analyze that falls back to a full table scan. - Planner consequences of the missing hot value, stated explicitly because they are the point of the trade-off: `ChildrenPropertiesRegulator` (`shouldBanOnePhaseAgg` → `skewOnShuffleExpr` requires hot values and now bails out) no longer picks the skew-aware aggregation shape for those columns, `ShuffleKeyPruneUtils` skips the shuffle-key prune, `SkewJoin` finds no hot value, and `DecomposeRepeatWithPreAggregation` treats the missing hot value as "balanced" (it calls `hasSignificantHotValues` with `strict=false`). These change plan *quality* only: every consumer null-checks "not collected", and none of them produces a wrong result. The behaviour is the same one a full analyze without `WITH HOT VALUE` already produces today. - `ANALYZE ... WITH SAMPLE ... WITH HOT VALUE` was already rejected before this change; its message changes from "Sample analyze always collects hot value" to "Sample analyze does not support WITH HOT VALUE", because the old wording is no longer true for linear-estimated columns. - For a column in the linear class an automatic analyze can no longer collect hot values at all: `collectHotValue` is still forced to true for every sample job, but only the DUJ1 estimator publishes them, and the automatic job never runs a full analyze. Getting them back requires a manual `ANALYZE TABLE ... WITH HOT VALUE`. - `ANALYZE ... WITH HOT VALUE` still does not publish hot values when partition-level analyze is enabled (`enable_partition_analyze=true` on a partitioned table), because that path merges per-partition statistics; pre-existing behaviour, noted here only so the release note is not read as unconditional. - Regression suites that pinned the old behaviour are updated in this PR: `test_hot_value` (both sample-request-larger-than-table blocks now expect no hot value, and the sampled linear path is pinned as well), `test_full_analyze_hot_value` (test 5 documents the fallback, test 7 compares a full analyze against a real DUJ1 sample on a uniform column), `hot_value_analyze_sync` (single bucket plus an enlarged table, so the analyze is guaranteed to sample and its column keeps collecting hot value through DUJ1), and `physical_agg_regulator` (its two `analyze` statements now use `WITH HOT VALUE`, because its expected plan shapes were built on collected hot values and the suite tests the aggregation strategy, not the analyze flavour). ### Release note Sample analyze no longer collects hot values for columns that use the linear NDV estimator (single distribution / single key column, and the full-scan fallback of a sample analyze), which removes a second aggregation over the sampled data. Hot values are still collected by sample analyze for columns estimated with DUJ1, and by `ANALYZE TABLE ... WITH HOT VALUE` in a full analyze - except when partition-level analyze is enabled, where the per-partition path has never produced them. For a column in the linear class an automatic analyze cannot collect hot values any more, so a manual `ANALYZE TABLE ... WITH HOT VALUE` is the way to get them back. branch-4.1 adaptation: - Applied against branch-4.1's flat org.apache.doris.statistics package (no statistics/analysis or statistics/model split); the unit test lives at statistics/OlapAnalysisTaskTest.java. - ANALYZE ... WITH HOT VALUE does not exist in branch-4.1, so the AnalyzeCommand rejection-message change and its test update are not applicable. - PluginDrivenSampleAnalysisTask (and its subStringColName fix) does not exist in branch-4.1; the legacy HMSAnalysisTask is left unchanged. - test_full_analyze_hot_value.groovy is master-only and was skipped. - physical_agg_regulator keeps its sample analyze statements, because branch-4.1 cannot request hot values explicitly; its expected plan is updated to the no-hot-value linear estimator behaviour. (cherry picked from commit a1a93fc43027331af371a172904342c056dbc81e) --- .../apache/doris/statistics/BaseAnalysisTask.java | 25 ++++++++++----------- .../apache/doris/statistics/ColumnStatistic.java | 3 ++- .../apache/doris/statistics/OlapAnalysisTask.java | 18 ++++++++------- .../doris/statistics/OlapAnalysisTaskTest.java | 19 +++++++--------- .../agg_strategy/physical_agg_regulator.out | 4 +--- .../stats/col_stats/hot_value_analyze_sync.groovy | 12 +++++++--- .../agg_strategy/physical_agg_regulator.groovy | 6 +++++ .../suites/statistics/test_hot_value.groovy | 26 +++++++++++----------- 8 files changed, 61 insertions(+), 52 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/statistics/BaseAnalysisTask.java b/fe/fe-core/src/main/java/org/apache/doris/statistics/BaseAnalysisTask.java index 2707d28ca6d..8918e7e5850 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/statistics/BaseAnalysisTask.java +++ b/fe/fe-core/src/main/java/org/apache/doris/statistics/BaseAnalysisTask.java @@ -107,6 +107,14 @@ public abstract class BaseAnalysisTask { + "FROM (SELECT `${colName}`${lengthAssert} " + "FROM `${catalogName}`.`${dbName}`.`${tblName}` ${index}) __lc_t"; + // Hot values are intentionally NOT collected by the linear estimator. The extra + // GROUP BY over the sampled column re-reads the whole sample and is the dominant cost + // of this template, which is used for every single distribution / single key column + // sample analyze. `hot_value` keeps its position in the result set and is reported as + // NULL, which every reader treats as "not collected" (see ColumnStatistic#hotValues), + // so the row layout stays identical to that of the other templates. Hot values are + // still collected by DUJ1_ANALYZE_TEMPLATE, which derives them from the grouped sample + // it already builds for the NDV estimate. protected static final String LINEAR_ANALYZE_TEMPLATE = "WITH cte1 AS (" + "SELECT `${colName}`${lengthAssert} " + "FROM `${catalogName}`.`${dbName}`.`${tblName}` ${index} ${sampleHints} ${limit} ${preAggHint}), " @@ -124,19 +132,10 @@ public abstract class BaseAnalysisTask { + "SUBSTRING(CAST(${min} AS STRING), 1, 1024) AS `min`, " + "SUBSTRING(CAST(${max} AS STRING), 1, 1024) AS `max`, " + "${dataSizeFunction} * ${scaleFactor} AS `data_size`, " - + "NOW() FROM cte1), " - + "cte3 AS (" - + "SELECT IFNULL(GROUP_CONCAT(CONCAT(" - + "REPLACE(REPLACE(t.`column_key`, \":\", \"\\\\:\"), \";\", \"\\\\;\"), " - + "\" :\", ROUND(t.`count` / ${rowCount2}, 2)), \" ;\"), '') " - + "as `hot_value` " - + "FROM (" - + "SELECT ${subStringColName} as `hash_value`, " - + "MAX(`${colName}`) as `column_key`, " - + "COUNT(1) AS `count` " - + "FROM cte1 WHERE `${colName}` IS NOT NULL " - + "GROUP BY `hash_value` ORDER BY `count` DESC LIMIT ${hotValueCollectCount}) t) " - + "SELECT * FROM cte2 CROSS JOIN cte3"; + + "NOW() AS `update_time`, " + + "null as `hot_value` " + + "FROM cte1) " + + "SELECT * FROM cte2"; protected static final String DUJ1_ANALYZE_TEMPLATE = "WITH cte1 AS (" + "SELECT MAX(t0.`col_value`) as `col_value`, COUNT(1) as `count`, SUM(`len`) as `column_length` " diff --git a/fe/fe-core/src/main/java/org/apache/doris/statistics/ColumnStatistic.java b/fe/fe-core/src/main/java/org/apache/doris/statistics/ColumnStatistic.java index 8dfd8b86f1f..3b887086bc4 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/statistics/ColumnStatistic.java +++ b/fe/fe-core/src/main/java/org/apache/doris/statistics/ColumnStatistic.java @@ -98,7 +98,8 @@ public class ColumnStatistic { public final String updatedTime; /** - * hotValues == null: not collected (e.g. full analyze). Do NOT use for optimization. + * hotValues == null: not collected (e.g. full analyze, or a sample analyze that used + * the linear estimator). Do NOT use for optimization. * hotValues != null && hotValues.isEmpty(): collected but no hot values (not skewed). * hotValues != null && !hotValues.isEmpty(): collected with hot values. */ diff --git a/fe/fe-core/src/main/java/org/apache/doris/statistics/OlapAnalysisTask.java b/fe/fe-core/src/main/java/org/apache/doris/statistics/OlapAnalysisTask.java index 0e856c9787e..2712ea4a245 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/statistics/OlapAnalysisTask.java +++ b/fe/fe-core/src/main/java/org/apache/doris/statistics/OlapAnalysisTask.java @@ -118,19 +118,20 @@ public class OlapAnalysisTask extends BaseAnalysisTask { Map<String, String> params = buildSqlParams(); params.put("min", StatisticsUtil.quote(min)); params.put("max", StatisticsUtil.quote(max)); - params.put("hotValueCollectCount", String.valueOf(SessionVariable.getHotValueCollectCount())); long tableRowCount = info.indexId == -1 ? tbl.getRowCount() : ((OlapTable) tbl).getRowCountForIndex(info.indexId, false); SampleCollectInfo collectInfo = getSampleCollectInfo(tableRowCount); getSampleParams(params, tableRowCount, collectInfo); - StringSubstitutor stringSubstitutor = new StringSubstitutor(params); - String sql; + String template = LINEAR_ANALYZE_TEMPLATE; if (collectInfo.algorithm == AnalyzeSampleAlgorithm.DUJ1) { - sql = stringSubstitutor.replace(DUJ1_ANALYZE_TEMPLATE); - } else { - sql = stringSubstitutor.replace(LINEAR_ANALYZE_TEMPLATE); + // DUJ1 is the only sample algorithm that collects hot values, and it derives them + // from the grouped sample it already builds for the NDV estimate, so only it needs + // the top-N count. + params.put("hotValueCollectCount", String.valueOf(SessionVariable.getHotValueCollectCount())); + template = DUJ1_ANALYZE_TEMPLATE; } + String sql = new StringSubstitutor(params).replace(template); LOG.info("Analyze param: algorithm {}, partitionColumnTooMany {}, keyColumnTooMany {}", collectInfo.algorithm, partitionColumnSampleTooManyRows, keyColumnSampleTooManyRows); LOG.debug(sql); @@ -306,7 +307,6 @@ public class OlapAnalysisTask extends BaseAnalysisTask { params.put("rowCount", String.valueOf(tableRowCount)); params.put("type", col.getType().toString()); params.put("limit", ""); - params.put("subStringColName", getStringTypeColName(col)); // For agg table and mor unique table, set PREAGGOPEN preAggHint. if (((OlapTable) tbl).getKeysType().equals(KeysType.AGG_KEYS) @@ -356,11 +356,13 @@ public class OlapAnalysisTask extends BaseAnalysisTask { protected void setSampleParamsByAlgorithm(Map<String, String> params, long tableRowCount, AnalyzeSampleAlgorithm algorithm) { if (algorithm == AnalyzeSampleAlgorithm.DUJ1) { + // DUJ1 is the only sample algorithm that collects hot values: it derives them from + // the grouped sample it already builds for the NDV estimate below. + params.put("subStringColName", getStringTypeColName(col)); params.put("ndvFunction", getNdvFunction(String.valueOf(tableRowCount))); params.put("dataSizeFunction", getDataSizeFunction(col, true)); params.put("rowCount2", "(SELECT SUM(`count`) FROM cte1 WHERE `col_value` IS NOT NULL)"); } else { - params.put("rowCount2", "(SELECT COUNT(1) FROM cte1 WHERE `${colName}` IS NOT NULL)"); // For single unique key, use count as ndv. if (isSingleUniqueKey()) { params.put("ndvFunction", String.valueOf(tableRowCount)); diff --git a/fe/fe-core/src/test/java/org/apache/doris/statistics/OlapAnalysisTaskTest.java b/fe/fe-core/src/test/java/org/apache/doris/statistics/OlapAnalysisTaskTest.java index b63816348b4..55d50597ff8 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/statistics/OlapAnalysisTaskTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/statistics/OlapAnalysisTaskTest.java @@ -159,13 +159,8 @@ public class OlapAnalysisTaskTest { + "ROUND(SUM(CASE WHEN `null` IS NULL THEN 1 ELSE 0 END) * ${scaleFactor}) AS `null_count`, " + "SUBSTRING(CAST('1' AS STRING), 1, 1024) AS `min`, " + "SUBSTRING(CAST('2' AS STRING), 1, 1024) AS `max`, " - + "COUNT(1) * 4 * ${scaleFactor} AS `data_size`, NOW() FROM cte1), " - + "cte3 AS (SELECT IFNULL(GROUP_CONCAT(CONCAT(REPLACE(REPLACE(t.`column_key`, " - + "\":\", \"\\\\:\"), \";\", \"\\\\;\"), \" :\", ROUND(t.`count` / ${rowCount2}, 2)), " - + "\" ;\"), '') as `hot_value` FROM (SELECT ${subStringColName} as `hash_value`, " - + "MAX(`null`) as `column_key`, COUNT(1) AS `count` FROM cte1 " - + "WHERE `null` IS NOT NULL GROUP BY `hash_value` ORDER BY `count` DESC LIMIT 10) t) " - + "SELECT * FROM cte2 CROSS JOIN cte3", sql); + + "COUNT(1) * 4 * ${scaleFactor} AS `data_size`, NOW() AS `update_time`, " + + "null as `hot_value` FROM cte1) SELECT * FROM cte2", sql); } }; @@ -384,8 +379,9 @@ public class OlapAnalysisTaskTest { new OlapAnalysisTask.SampleCollectInfo(AnalyzeSampleAlgorithm.FULL, null)); Assertions.assertEquals("1", params.get("scaleFactor")); Assertions.assertEquals("", params.get("sampleHints")); - Assertions.assertEquals("(SELECT COUNT(1) FROM cte1 WHERE `${colName}` IS NOT NULL)", - params.get("rowCount2")); + // The linear estimator does not collect hot values, so it fills no hot value param. + Assertions.assertNull(params.get("rowCount2")); + Assertions.assertNull(params.get("subStringColName")); Assertions.assertEquals("ROUND(NDV(`${colName}`) * ${scaleFactor})", params.get("ndvFunction")); Assertions.assertNull(params.get("preAggHint")); Assertions.assertEquals("COUNT(1)", params.get("rowCount")); @@ -396,8 +392,8 @@ public class OlapAnalysisTaskTest { new OlapAnalysisTask.SampleCollectInfo(AnalyzeSampleAlgorithm.LINEAR, Pair.of(Lists.newArrayList(1L, 2L), 100L))); Assertions.assertEquals("TABLET(1, 2)", params.get("sampleHints")); - Assertions.assertEquals("(SELECT COUNT(1) FROM cte1 WHERE `${colName}` IS NOT NULL)", - params.get("rowCount2")); + Assertions.assertNull(params.get("rowCount2")); + Assertions.assertNull(params.get("subStringColName")); Assertions.assertEquals("ROUND(NDV(`${colName}`) * ${scaleFactor})", params.get("ndvFunction")); params.clear(); @@ -410,6 +406,7 @@ public class OlapAnalysisTaskTest { Assertions.assertEquals("TABLET(1, 2)", params.get("sampleHints")); Assertions.assertEquals("(SELECT SUM(`count`) FROM cte1 WHERE `col_value` IS NOT NULL)", params.get("rowCount2")); + Assertions.assertEquals("`${colName}`", params.get("subStringColName")); Assertions.assertTrue(params.get("ndvFunction").contains("`t1`.`col_value`"), "DUJ1 ndvFunction must reference cte1 output, got: " + params.get("ndvFunction")); params.clear(); diff --git a/regression-test/data/nereids_rules_p0/agg_strategy/physical_agg_regulator.out b/regression-test/data/nereids_rules_p0/agg_strategy/physical_agg_regulator.out index a33a83f01e0..800cd8b556b 100644 --- a/regression-test/data/nereids_rules_p0/agg_strategy/physical_agg_regulator.out +++ b/regression-test/data/nereids_rules_p0/agg_strategy/physical_agg_regulator.out @@ -15,9 +15,7 @@ PhysicalCteAnchor ( cteId=CTEId#0 ) --------hashAgg[GLOBAL] ----------hashAgg[GLOBAL] ------------PhysicalDistribute[DistributionSpecHash] ---------------hashAgg[LOCAL] -----------------PhysicalDistribute[DistributionSpecExecutionAny] -------------------PhysicalCteConsumer ( cteId=CTEId#0 ) +--------------PhysicalCteConsumer ( cteId=CTEId#0 ) -- !not_skew -- PhysicalCteAnchor ( cteId=CTEId#0 ) diff --git a/regression-test/suites/nereids_p0/stats/col_stats/hot_value_analyze_sync.groovy b/regression-test/suites/nereids_p0/stats/col_stats/hot_value_analyze_sync.groovy index 39c43134ab7..8ebc437890e 100644 --- a/regression-test/suites/nereids_p0/stats/col_stats/hot_value_analyze_sync.groovy +++ b/regression-test/suites/nereids_p0/stats/col_stats/hot_value_analyze_sync.groovy @@ -49,10 +49,16 @@ suite('hot_value_analyze_sync') { String realDb = context.config.getDbNameByFile(context.file) sql """ drop table if exists t1025; - create table t1025(a_1 int, b_5 int, c_10 int, d_1025 int) distributed by hash(c_10) properties('replication_num'='1'); - insert into t1025 select 1, number%5 , number%10, number from numbers("number"="100"); + create table t1025(a_1 int, b_5 int, c_10 int, d_1025 int) distributed by hash(c_10) buckets 1 properties('replication_num'='1'); + insert into t1025 select 1, number%5 , number%10, number from numbers("number"="10000"); """ - wait_row_count_reported(realDb, "t1025", 0, 4, "100") + wait_row_count_reported(realDb, "t1025", 0, 4, "10000") + // One bucket, so sample analyze is guaranteed to sample instead of falling back to a full scan: + // getSampleCollectInfo falls back when the sample request exceeds the table, or when the picked + // tablets hold fewer rows than requested, and both fallbacks use the linear estimator for every + // column, which does not collect hot value (`null as hot_value`). d_1025 is not the distribution + // column, so a real sample analyzes it with the DUJ1 estimator, which does collect hot value - + // the property this suite exists to check. sql "analyze table t1025 with sample rows 4000 with sync;" def result = sql "show column cached stats t1025" log.info(result.toString()) diff --git a/regression-test/suites/nereids_rules_p0/agg_strategy/physical_agg_regulator.groovy b/regression-test/suites/nereids_rules_p0/agg_strategy/physical_agg_regulator.groovy index 7b9f2c04f9a..868eb044ce8 100644 --- a/regression-test/suites/nereids_rules_p0/agg_strategy/physical_agg_regulator.groovy +++ b/regression-test/suites/nereids_rules_p0/agg_strategy/physical_agg_regulator.groovy @@ -22,6 +22,11 @@ suite("physical_agg_regulator") { set enable_parallel_result_sink=false; set be_number_for_test=1; """ + // A sample request larger than the table falls back to a full scan with the linear estimator, + // which does not collect hot value. branch-4.1 has no `WITH HOT VALUE` clause to ask for hot + // values explicitly, and this table is distributed by a single column, so every sample analyze + // of it uses the linear estimator. The expected plan therefore reflects the no-hot-value + // behaviour of the linear estimator. multi_sql """ analyze table t1025_skew5000 with sample rows 7000 with sync; --drop cached stats t1025; @@ -31,6 +36,7 @@ suite("physical_agg_regulator") { select count(distinct a_1), count(distinct b_5) from t1025_skew5000 group by d_1025;""" + // Same reason as above: no hot values are collected on this path in branch-4.1. multi_sql """ analyze table t1025 with sample rows 4000 with sync; --drop cached stats t1025; diff --git a/regression-test/suites/statistics/test_hot_value.groovy b/regression-test/suites/statistics/test_hot_value.groovy index b2d6de2be52..0baa7bc37ac 100644 --- a/regression-test/suites/statistics/test_hot_value.groovy +++ b/regression-test/suites/statistics/test_hot_value.groovy @@ -109,6 +109,9 @@ suite("test_hot_value") { logger.info("result " + result) assertEquals(1, result.size()) assertEquals("10000.0", result[0][2]) + // key1 is the single distribution column, so a real sample analyzes it with the linear + // estimator, which does not collect hot_value. + assertEquals("null", String.valueOf(result[0][17])) result = sql """show column stats test1(value1)""" logger.info("1. result " + result) assertEquals(1, result.size()) @@ -128,29 +131,24 @@ suite("test_hot_value") { assertTrue(hotValues[1] == "'1':0.5" || hotValues[1] == "'0':0.5") sql """drop stats test1""" + // The requested sample rows are larger than the table, so sample analyze falls back to a + // full scan with the linear estimator, which does not collect hot_value. sql """analyze table test1 with sample rows 40000 with sync""" result = sql """show column stats test1(key1)""" logger.info("result " + result) assertEquals(1, result.size()) assertEquals("10000.0", result[0][2]) + assertEquals("null", String.valueOf(result[0][17])) result = sql """show column stats test1(value1)""" logger.info("2. result " + result) - logger.info("result.size()" + result.size()) assertEquals(1, result.size()) - logger.info("result[0][2] " + result[0][2]) assertEquals("10000.0", result[0][2]) - hotValues = result[0][17].split(";") - assertEquals(2, hotValues.length) - assertTrue(hotValues[0] == "'1':0.5" || hotValues[0] == "'0':0.5") - assertTrue(hotValues[1] == "'1':0.5" || hotValues[1] == "'0':0.5") + assertEquals("null", String.valueOf(result[0][17])) result = sql """show column cached stats test1(value1)""" logger.info("result " + result) assertEquals(1, result.size()) assertEquals("10000.0", result[0][2]) - hotValues = result[0][17].split(";") - assertEquals(2, hotValues.length) - assertTrue(hotValues[0] == "'1':0.5" || hotValues[0] == "'0':0.5") - assertTrue(hotValues[1] == "'1':0.5" || hotValues[1] == "'0':0.5") + assertEquals("null", String.valueOf(result[0][17])) sql """alter table test1 modify column value1 set stats ('row_count'='100.0', 'ndv'='5.0', 'num_nulls'='0.0', 'data_size'='34.0', 'min_value'='AFRICA', 'max_value'='MIDDLE EAST', 'hot_values'='aaa :0.8');""" result = sql """show column stats test1(value1)""" @@ -195,13 +193,15 @@ suite("test_hot_value") { assertEquals("' : ;a':1.0", result[0][17]) sql """drop stats test2""" + // As for test1 above, the requested sample rows are larger than the table, so sample analyze + // falls back to a full scan with the linear estimator, which does not collect hot_value. sql """analyze table test2 with sample rows 4000000 with sync""" result = sql """show column stats test2(value1)""" assertEquals(1, result.size()) - assertEquals("' : ;a':1.0", result[0][17]) - result = sql """show column cached stats test2(value1)""" + assertEquals("null", String.valueOf(result[0][17])) + result = sql """show column cached stats test2(value1)""" assertEquals(1, result.size()) - assertEquals("' : ;a':1.0", result[0][17]) + assertEquals("null", String.valueOf(result[0][17])) sql """drop database if exists test_hot_value""" } --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
