[GitHub] [doris] morrySnow commented on a diff in pull request #23978: [feat](nereids): Support analyze with sample automatically

[via GitHub]

morrySnow commented on code in PR #23978:
URL: https://github.com/apache/doris/pull/23978#discussion_r1319468714


##########
fe/fe-common/src/main/java/org/apache/doris/common/Config.java:
##########
@@ -1510,7 +1512,7 @@ public class Config extends ConfigBase {
      * the system automatically checks the time interval for statistics
      */
     @ConfField(mutable = true, masterOnly = true)
-    public static int auto_check_statistics_in_minutes = 1;
+    public static int auto_check_statistics_in_minutes = 10;

Review Comment:
   add config desc



##########
fe/fe-core/src/main/cup/sql_parser.cup:
##########
@@ -6048,6 +6048,13 @@ with_analysis_properties ::=
             put("period.cron", cron_expr);
         }};
     :}
+    | KW_FULL
+    {:
+        RESULT = new HashMap<String, String>() {{
+             put("forcefull", "true");

Review Comment:
   should use `AnalyzeProperties.PROPERTY_FORCE_FULL` not string literal



##########
fe/fe-common/src/main/java/org/apache/doris/common/Config.java:
##########
@@ -2157,4 +2159,17 @@ public class Config extends ConfigBase {
             "是否禁止LocalDeployManager删除节点",
             "Whether to disable LocalDeployManager drop node"})
     public static boolean disable_local_deploy_manager_drop_node = true;
+
+    @ConfField
+    public static long huge_table_auto_analyze_interval_in_millis = 
TimeUnit.HOURS.toMillis(12);
+
+    @ConfField
+    public static long huge_table_lower_bound_size_in_bytes = 5L * 1024 * 1024 
* 1024;
+
+    @ConfField
+    public static int huge_table_default_sample_rows = 20_0000;
+
+    @ConfField
+    public static boolean enable_auto_sample = false;

Review Comment:
   add config desc



##########
fe/fe-core/src/main/java/org/apache/doris/catalog/TableIf.java:
##########
@@ -244,5 +244,9 @@ default boolean isManagedTable() {
     default long getLastUpdateTime() {
         return -1L;
     }
+
+    default long getDataSize() {
+        return 0;
+    }

Review Comment:
   should not provide default implementation. all tableIf should impl it by 
themsleves



##########
fe/fe-core/src/main/java/org/apache/doris/statistics/TableStats.java:
##########
@@ -72,7 +75,13 @@ public TableStats(long tblId, long rowCount, AnalysisInfo 
analyzedJob) {
         analysisMethod = analyzedJob.analysisMethod;
         analysisType = analyzedJob.analysisType;
         updatedTime = System.currentTimeMillis();
-        columns = analyzedJob.colName;
+        String cols = analyzedJob.colName;
+        if (analyzedJob.colName.startsWith("[") && 
analyzedJob.colName.endsWith("]")) {
+            cols = cols.substring(1, cols.length() - 1);
+        }
+        for (String col : cols.split(",")) {
+            colLastUpdatedTime.put(col, updatedTime);
+        }

Review Comment:
   add comment to explain this code block



##########
fe/fe-core/src/main/java/org/apache/doris/analysis/AnalyzeProperties.java:
##########
@@ -42,6 +43,8 @@ public class AnalyzeProperties {
     public static final String PROPERTY_ANALYSIS_TYPE = "analysis.type";
     public static final String PROPERTY_PERIOD_SECONDS = "period.seconds";
 
+    public static final String PROPERTY_FORCE_FULL = "forcefull";

Review Comment:
   ```suggestion
       public static final String PROPERTY_FORCE_FULL = "force.full";
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org