depatel1010 opened a new issue #6195:
URL: https://github.com/apache/incubator-pinot/issues/6195
An SQLException is thrown by the backend for `EVALUATION_INDEX_INDEX_C ON
PUBLIC.EVALUATION_INDEX(DETECTION_CONFIG_ID, START_TIME, END_TIME)` key. The
issue occurs when `dimensionExploration` is turned on and multiple anomalies
for the same time window are detected in a single run.
PFA the logs and Detection configurations used to recreate the issue with
the given default h2db and default `data-source-configs.yml` file.
Dataset used: "H2.H2.total_pageviews" (uses pageviews.csv)
dimension explored during anomaly detection: "device"
Additional Changes to make: mark `worker`, `detectionPipeline` and
`detectionAlert` flags as `true` in `detector.yml`
Error-Log:
``` log
11:52:02.155 [task-executor-2] INFO
org.apache.pinot.thirdeye.detection.DefaultDataProvider - Fetched 2 anomalies
for slice AnomalySlice{detectionId='15', startTime='1603771003484',
endTime='1603786920012', filters='{}', isTaggedAsChild='false',
detectionComponentNames='[]'}
11:52:02.157 [task-executor-2] INFO
org.apache.pinot.thirdeye.detection.DefaultDataProvider - Fetched 2 anomalies
for slice AnomalySlice{detectionId='15', startTime='1603771003484',
endTime='1603786920012', filters='{}', isTaggedAsChild='false',
detectionComponentNames='[]'}
11:52:02.163 [task-executor-2] ERROR
org.apache.pinot.thirdeye.datalayer.dao.GenericPojoDao - Exception while
executing query task
org.h2.jdbc.JdbcSQLException: Unique index or primary key violation:
"EVALUATION_INDEX_INDEX_C ON PUBLIC.EVALUATION_INDEX(DETECTION_CONFIG_ID,
START_TIME, END_TIME) VALUES (15, 1603778203485, 1603779720011, 3)"; SQL
statement:
INSERT INTO
evaluation_index(base_id,detection_config_id,start_time,end_time,detectorname,mape,create_time,update_time,version)
VALUES(?,?,?,?,?,?,?,?,?) [23505-192]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
at org.h2.message.DbException.get(DbException.java:179)
at org.h2.message.DbException.get(DbException.java:155)
at org.h2.index.BaseIndex.getDuplicateKeyException(BaseIndex.java:103)
at
org.h2.mvstore.db.MVSecondaryIndex.checkUnique(MVSecondaryIndex.java:231)
at org.h2.mvstore.db.MVSecondaryIndex.add(MVSecondaryIndex.java:190)
at org.h2.mvstore.db.MVTable.addRow(MVTable.java:704)
at org.h2.command.dml.Insert.insertRows(Insert.java:156)
at org.h2.command.dml.Insert.update(Insert.java:114)
at org.h2.command.CommandContainer.update(CommandContainer.java:98)
at org.h2.command.Command.executeUpdate(Command.java:258)
at
org.h2.jdbc.JdbcPreparedStatement.executeUpdateInternal(JdbcPreparedStatement.java:160)
at
org.h2.jdbc.JdbcPreparedStatement.executeUpdate(JdbcPreparedStatement.java:146)
at sun.reflect.GeneratedMethodAccessor133.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.tomcat.jdbc.pool.StatementFacade$StatementProxy.invoke(StatementFacade.java:114)
at com.sun.proxy.$Proxy78.executeUpdate(Unknown Source)
at
org.apache.pinot.thirdeye.datalayer.dao.GenericPojoDao$1.handle(GenericPojoDao.java:261)
at
org.apache.pinot.thirdeye.datalayer.dao.GenericPojoDao$1.handle(GenericPojoDao.java:229)
at
org.apache.pinot.thirdeye.datalayer.dao.GenericPojoDao.runTask(GenericPojoDao.java:890)
at
org.apache.pinot.thirdeye.datalayer.dao.GenericPojoDao.put(GenericPojoDao.java:229)
at
org.apache.pinot.thirdeye.datalayer.bao.jdbc.AbstractManagerImpl.save(AbstractManagerImpl.java:66)
at
org.apache.pinot.thirdeye.detection.DetectionPipelineTaskRunner.execute(DetectionPipelineTaskRunner.java:159)
at
org.apache.pinot.thirdeye.anomaly.task.TaskDriver$1$1.call(TaskDriver.java:128)
at
org.apache.pinot.thirdeye.anomaly.task.TaskDriver$1$1.call(TaskDriver.java:125)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
Detection Configurations:
``` yaml
detectionName: test
description: If this alert fires then it means so-and-so and check so-and-so
for irregularities
metric: pageviews
dataset: H2.H2.total_pageviews
cron: "0 */2 * * * ? *"
dimensionExploration:
dimensions:
- device
rules:
- detection:
- name: detection_rule_1
type: PERCENTAGE_RULE
params:
offset: wo1w
percentageChange: 0.01
```
Slack Chat Reference:
https://apache-thirdeye.slack.com/archives/C014CS60A22/p1598938054008700
@suvodeep-pyne please have a look. 😀☝️
Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
