akshayrai commented on a change in pull request #5740:
URL: https://github.com/apache/incubator-pinot/pull/5740#discussion_r459665554
##########
File path: thirdeye/thirdeye-frontend/app/utils/yaml-tools.js
##########
@@ -29,6 +28,20 @@ rules:
type: DATA_SLA # Alert if data is missing.
params:
sla: 3_DAYS # Data is missing for 3 days since last
availability
+
+# You can mention a cron to allow this detection to be run periodically
+# For example, the cron below would execute every 5 minutes.
+# cron: "0 0/5 * 1/1 * ? *"
+
+# Backfill detections
+# You can use the backfillStart parameter to denote how far back you want the
anomalies
+# to be detected.
+# A valid entry is a timestamp value in millis.
+#
+# - By default, (if not mentioned), the lookback is 30 days
+# - A value of 0 (zero) implies complete backfill till the start of data. As
shown below.
+# backfillStart: 0
Review comment:
Can you shed some light on the need for a custom backfill configuration?
Since this is a one time setting, I am not very confident of putting it in the
detection config.
##########
File path:
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/translator/DetectionConfigTranslator.java
##########
@@ -189,6 +190,7 @@ private DetectionConfigDTO
generateDetectionConfig(Map<String, Object> yamlConfi
config.setCron(cron);
config.setActive(MapUtils.getBooleanValue(yamlConfigMap, PROP_ACTIVE,
true));
config.setYaml(yamlConfig);
+
config.setBackfillStart(parseTimeStampLong(yamlConfigMap.get(PROP_BACKFILL_START)));
Review comment:
I think we already have a utility for this. Can you try MapUtils or
ConfigUtils?
##########
File path: thirdeye/thirdeye-frontend/app/utils/yaml-tools.js
##########
@@ -29,6 +28,20 @@ rules:
type: DATA_SLA # Alert if data is missing.
params:
sla: 3_DAYS # Data is missing for 3 days since last
availability
+
+# You can mention a cron to allow this detection to be run periodically
+# For example, the cron below would execute every 5 minutes.
+# cron: "0 0/5 * 1/1 * ? *"
+
+# Backfill detections
+# You can use the backfillStart parameter to denote how far back you want the
anomalies
+# to be detected.
+# A valid entry is a timestamp value in millis.
+#
+# - By default, (if not mentioned), the lookback is 30 days
+# - A value of 0 (zero) implies complete backfill till the start of data. As
shown below.
+# backfillStart: 0
+
Review comment:
I'd prefer keeping the template simple and put all the additional
configurations in the user guide. Regular users need not know about these
parameters.
##########
File path: thirdeye/thirdeye-frontend/app/utils/yaml-tools.js
##########
@@ -29,6 +28,20 @@ rules:
type: DATA_SLA # Alert if data is missing.
params:
sla: 3_DAYS # Data is missing for 3 days since last
availability
+
+# You can mention a cron to allow this detection to be run periodically
+# For example, the cron below would execute every 5 minutes.
+# cron: "0 0/5 * 1/1 * ? *"
+
+# Backfill detections
Review comment:
The term backfill is a bit overloaded. It could also mean rerunning the
detection which is not what is intended.
##########
File path: thirdeye/thirdeye-frontend/app/utils/yaml-tools.js
##########
@@ -12,12 +12,11 @@ dataset: dataset_to_which_this_metric_belongs
rules:
- detection:
- name: detection_rule_1
- type: ALGORITHM # Configure the detection type here. See doc
for more details.
+ type: PERCENTAGE_RULE # Configure the detection type here. See doc
for more details.
Review comment:
+ @harleyjj
Can we customize and configure the default template? At LinkedIn, we are
relying on ALGORITHM as the default detection. If we change this, we need to
have an alternative to plug a different default setting.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
