This is an automated email from the ASF dual-hosted git repository. jfim pushed a change to branch handle-invalid-durations-in-duration-dsl in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard cbe5281 Add logging message discard e9f7074 Handle invalid durations in the duration DSL add 0d92355 Get quickstart-realtime working again (#3374) add 8f2dba8 Add abstract class PeriodicTask to unify duplicate code (#3264) add 1eccd34 Backward compatibility for StreamConfig (#3380) add a3c499c Disable LLC validation when LLC is not enabled (#3381) add bd80360 Reduce the ZK access for segment metadata fetching (#3356) add 57014c8 [TE] dashboard - support expected anomaly classification (#3382) add ee5fa0d [TE] detection - pipeline/preview UI bug fix (#3383) add 526ba5d Adding way to construct old segment upload endpoint (#3343) add 86f65e0 Add a flag numGroupsLimitReached to mark responses that hit the number of groups limit (#3378) add 7d347e9 Cleaning up crypter initialization (#3369) add 83d61c9 Fix int overflow in FixedByteValueReaderWriter (#3385) add cd6e32e [TE] rootcause - request scheduler (#3366) add fcab1ba Supporting delete api for different storage options (#3370) add b7ea9ee Support .tar.gz upload for pluggable storage (#3377) add 9c6a0c6 [TE] Iris Alerts - Phase 2 : Refactoring | Cleanup DetectionEmailAlerter (#3375) add 5071a74 TE rca anomaly label text shorten (#3384) add f26b818 Update the TimeboundaryInfo in Pinot Broker when offline segments are refreshed. (#3354) add 0a4712f Reduce FixedByteValueReaderWriter.writeUnpaddedString() to one write to data buffer (#3392) add 9138751 i`[TE] frontend - aaronucsd/Added new share custom template header (#3391) add 673af86 [TE] metric onboarding - support custom fabric groups (#3394) add b763b65 Remove the columnName argument in FunctionExpressionEvaluator constructor (#3397) add 0c31ba5 Instance level semaphore for realtime parallel segment builds (#3393) add 390b4f9 [TE] build - cut down CI build time (#3401) add 309953f [TE] Introduce configurations for alert schemes (#3395) add 0045436 [TE] Change default alert cron for bulk onboard (#3390) add b59b03b [TE] detection - moving monitoring window aligned to data set granularity (#3396) add 07aa3ea Editing Reading of CSVReaderConfig in Segment Creation Job (#3372) add 4de0b3f [TE] Read alerter configurations as a Map for easy extensibility (#3389) add 4f34f83 Introduce OffsetCriteria class to handle different offset types and operations (#3402) add a9550e3 [TE] Move emailWhitelist under alerterConfiguration (#3403) add cfb97d5 [TE] rootcause - support query params for external setup of investigation (#3388) add dab4494 [TE] Group together to, cc and bcc under recipients add 75aa028 Adding overwrite parameter to move in PinotFS (#3386) add 870c5d7 fix schedulerName (#3421) add 2d404ac Use maven enforcer plugin to clean up dependencies (#2998) add e111b8c [TE] frontend - aaronucsd/update Travis to use Chrome (#3400) add 27ae042 Remove references to kafka from generic classes (#3423) add f70c9c3 Add stream agnostic metrics for realtime (#3426) add 4ce685c [TE] frontend - Alert Filter Component (#3419) add 75e718d Move group id to StreamConfig level (#3425) add 0a690ad [TE] frontend - aarouncsd/remove rca2 poc route for now (#3428) add 4f2696d [TE] database - update alias field size (#3429) add 2023bd1 [TE] frontend - random linter fixes Vol 1 (#3422) add 18640fe Add segment merge command (#3427) add f73a365 [TE] yaml - detection annotation (#3399) add eccdd7d [TE] frontend - aarouncsd/update RCA title (#3430) add 668aaf4 [TE] alerts - fix null array exception (#3431) add 5ee6df6 [TE] Fix for zero anomalies email bug and added some additional checks (#3432) add 5c296de Add RecordTransformer to perform transformation to incoming record (GenericRow) (#3424) add 0c3f416 PINOT-7296: Reduce lock contention by not sharing DecimalFormat instances (#3433) add 2e8e20e [TE] user dashboard - hide anomalies if alert is disabled (#3435) add 0acea8f [TE] Introduce ability to suppress alerts (#3437) add bbb300c Adding RetentionManager Support for Pluggable Storage (#3371) add 65ad8a4 Replace ENABLE_SEGMENT_LEVEL_VALIDATION config with SEGMENT_LEVEL_VALIDATION_INTERVAL_IN_SECONDS (#3441) add e692b89 Add the support for dynamic routing table (#3434) add ba9f2dd Add stream related javadoc (#3436) add b3c2e08 [TE] frontend - Anomaly resolution change label fix (#3444) add 16559dc Limit the ByteArray wrapper logic inside segment creator (#3439) add a5291e1 Allow generating new star-tree using Hadoop job (#3442) add e41cd17 [TE] rootcause - inconsistent data warning (#3449) add 283c70a Add max length limit to string values (#3445) add b7a8cc1 [TE] anomalies - backend label translation fix (#3452) add ac2ab14 Fix the bug for transferring more than 2G bytes (#3453) add b70f2d5 Fixed the naming bug for merge segment admin command (#3446) add d4104d4 [TE] frontend - aaronucsd/Added basic YAML editor (#3447) add 6b790e1 [TE] onboarding - support bulk onboarding by dataset (#3455) add 7341670 [TE] datasource - change epoch time column to aggregation granularity (#3458) add f9317ec [TE] new detection pipeline - components (#3462) add b4496b6 [TE] frontend - aaronucsd/fixed some styling issues for share template (THIRDEYE-2432) (#3457) add 4e73ba3 [TE] detection - Input data fetcher interface (#3468) add 2f7ddf8 Relocation manager should relocate segments from any servers (irrespective of the tag) to completed servers (#3466) add a1cd143 Misc Cleanup: Removed unused member variable from (#3472) add 506799f [TE] Percentage change and absolute change rule detector (#3467) add eeaf9db [TE] Reset default lookback for merging anomalies (#3450) add 3bd8fec [TE] datasource - update dataset if underline pinot table time field changed (#3456) add ec9de38 [TE] rootcause - adjustment anomaly baseline rather than other numbers (#3461) add 0b4fd51 [TE] Implement Threshold based Time Window Suppressor (#3463) add a7df756 [TE] Ability to configure reference links like oncall run-book in the email alert (#3473) add 1ffadef Enhance StarTreeV2ClusterIntegrationTest to cover multi-trees case (#3448) add d1fb928 [TE] frontend - aaronucsd/home - changes to sort the application list (#3465) add e7e9954 Fix typo in BaseRoutingTableBuilder (#3479) add 19286f0 [TE] migrate minute-level and hourly properties (#3480) add e0b0f3a [TE] detection - site-wide impact rule filter (#3482) add c9a4e60 [TE] frontend - aarouncsd/Create Alert - Yaml quick fix (#3483) add e358d7a [TE] frontend - harleyjj/rootcause - integration test for inconsistent data warning in rootcause-anomaly compoment (#3464) add 3fee99d [TE] frontend harleyjj/manage remove WoW and change buildMetricDataUrl to take user inputs (#3485) add 2d1bc7a [TE] frontend harleyjj/rootcause patch integration test for inconsistent data warning (#3488) add e2dface [TE] dataframe - tolerate complex column names (#3491) add e8516cb [TE] alert filter UX improvements (#3489) add 6711f67 [TE] site wide impact filter - config global metric from yaml (#3490) add 1beedd5 Enhance controller periodic task and scheduler (#3475) add 285649f [PINOT-7328] Reduce lock contention in physical planning phase by reducing the total number of tasks (#3470) add f6b5dfb [TE] config - enable rca holiday ranking with dimensions by default (#3493) add fa54c19 [TE] config - fix broken defaults (#3492) add 402cf3b Change build status to point apache incubator repository (#3498) add f21519e Add some documentation for readthedocs.io (#3481) add f64625b [TE] frontend - aaronucsd/share - added % (+/-) for offsets against current like predicted. (#3509) add 27cbc56 [TE] New detection alerter to support alerting of legacy anomalies (#3495) add ac915e4 Let PinotDataBuffer.newIndexFor(...) takes long value (#3512) add 60a72e4 [TE] detection - add two side pattern in rule filters (#3510) add 0a43bac [TE] Percentage and absolute change rule filter (#3514) add bcd05ed [TE] alert performnace - resilience to missing alert (#3497) add 34d0e12 [TE] rootcause - request chunking for aggregates and scores (#3496) add 8c845ed [TE] rootcause - fix chunking offset cache (#3518) add 4323070 [TE] rootcause - prevent add-to-chart popup by default (#3517) add 6581245 [TE] rootcause - prevent line-break of exclude/hide links in filter selector (#3519) add 7fe991f Deleting extra check for deletedDir during retention (#3516) add d77982d [TE] Organize the Holiday list in the alert email (#3520) add d5ba9b0 [TE] Some minor bug fixes (#3521) add 3774532 [TE] rootcause - fix filter processing for predicted baseline (#3523) add ce092c8 Fix the bug where time conversion is skipped when incoming and outgoing time column name are the same (#3484) add fdb9357 Added a log message on pinot-server when controller leadership change is read from zookeeper. (#3522) add 3fcf8f2 [TE] rootcause - support metric selection from chart labels (#3526) add bcd91e3 [TE] Attempt to organize and cleanup the email template (#3527) add 8fb5f28 Ignore generated documentation files while checking for license headers (#3524) add 77e394d [TE] datasource - aggregate to one minute level for epoch timestamp (#3530) add 4741882 [TE] rootcause - primary metric selection from legend label (#3529) add ebe5060 [TE] rootcause - support remove metric from chart label (#3531) add 5b1e6ec [TE] Some more minor changes to the look of the template (#3534) add 0389a6c [TE] roocause - fix dimension table metric select/unselect (#3533) add 2ce0bca [TE] frontend - harleyjj/rootcause - fix data inconsistency warning t… (#3532) add 4162fb3 [TE] Anomaly function to new pipeline YAML converter (#3537) add 79f83e9 [TE] rootcause - prevent metric selection change on dimension table tab change (#3536) add ce6a53f [TE] detection - add more logs for merger for debugging (#3540) add d48499b [TE] Pipeline migration - fill in metric timezone (#3542) add 234abc6 [TE] Display Holidays only from Top Countries (#3544) add f915200 [TE] rootcause - remove tooltipEntities property (#3546) add 16c5926 [TE] rootcause - redo route hooks (#3547) add 81589c7 [TE] frontend - harleyjj/rootcause - warning value uses aggregate multiplier and can be turned off if aggregates change without slider being adjusted (#3548) add 0bea669 First cut documentation. (#3545) add 9607ca4 Adding execution stats for numSegmentsQueried/Processed/Matched (#3525) add 37ad801 [TE] rootcause - default millisecond datasets to 15min granularity (#3550) add 8bf0050 Optimize all filter predicates by adding isAlwaysTrue() (#3535) add e23f953 [TE] Alerts created by YAML to show up in the alert list (#3551) add 8e2076f Fix the wrong number reported by numSegmentsQueried (#3552) add 7d4ab9e Cleaning data dir so URI construction works as expected (#3549) add ee4784d [TE] rootcause - preselect metric name in metric dropdown (#3553) add 4f5b2dd Fixing TODOs in new documentation (#3555) add 61b4814 [TE] Make new alerter tag old anomalies as notified (#3554) add ac74d13 [TE] frontend - harleyjj/manage - pass correct granularity to buildMetricDataUrl (#3557) add a834a49 [TE] pom - fix javassist dependency (#3559) add 232360f Fixes to doc (#3558) add d9ad5c3 Removing warnings in docs (#3560) add 1e6fc74 Adding pluggable storage support for realtime upload (#3541) add 44c2859 [TE] detection - copy nested property in wrapper (#3562) add 45063b0 Re-org documentation (#3563) add b62c611 [TE] frontend - harleyjj/create-alert - make minor improvements to YAML alert creator (#3564) add eceaf76 [TE] Legacy Alert Filter should pick recipients from new alerter configs (#3565) add 5a610be Add metric gauge to track the number of llc simultaneous segment builds (#3566) add 9dfb949 [TE] rootcause - rich metric selector (#3567) add 8b2efec [TE] detection - filter out child anomaly in anomaly fetching (#3570) add f1559a7 Docs: Fix formatting for UDF docs. (#3569) add 88ba23a [TE] rootcause - change frontend hack for baseline for new pipeline (#3571) add 213ff9c [TE] Configure Filters per Alert in Legacy Alerter (#3572) add f1512e6 [TE] rootcause - hide heatmap tooltip on click (#3574) add c752a4f Improve the logging for segment merge command (#3573) add 75a4f85 Adding support for bloom filter (#3528) add 1d4bf61 [TE] frontend - harleyjj/rootcause - use humanizeFloat to change formatting of y-axis and prevent apparent duplicates (#3575) add ef42e58 [TE] Minor fix in legacy alert filter (#3576) add a4505af Applied humanizeFloat to anomaly-graph component (#3577) add dac899f Changing segmentCommitEnd to support deep storage (#3578) add acff279 [TE] frontend - default redirect to home dashboard (#3579) add 5067baf [TE] Support sendOnce in LegacyAlertFilter (#3580) add 3219189 Segment merge lineage data structure (#3543) add 15ff8d8 [TE] rootcause - Round end time after adjusting with the maxDateTime (#3581) add 366bb39 [TE] frontend - harleyjj/manage - align startDate at startOf() instead of endOf() (#3582) add b21805c Change format of the ApplyConfig computed config (#3500) add ed1e0c0 Fix NPE while unparsing single key DSL output (#3501) add d99d8d8 Add new config API Java-properties format output (#3502) add 5900ff1 Fix NPE on configs with per-profile table types (#3503) add 5c84702 Change profile separator to triple underscores (#3504) add ed88fb4 Config loader includes also look in profiles (#3507) add e2916ed Minor deserialization fixes (#3508) add de218b2 Handle invalid durations in the duration DSL This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (cbe5281) \ N -- N -- N refs/heads/handle-invalid-durations-in-duration-dsl (de218b2) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .gitignore | 1 + .travis.yml | 9 +- .travis_install.sh | 22 +- .travis_test.sh | 2 +- README.md | 2 +- docs/High-level-stream.png | Bin 0 -> 39800 bytes docs/Low-level-stream.png | Bin 0 -> 37306 bytes docs/Makefile | 230 + docs/Pinot - Hybrid flow.png | Bin 0 -> 15756 bytes docs/Pinot-Offline-only-flow.png | Bin 0 -> 8179 bytes docs/PlanNode.png | Bin 0 -> 41056 bytes docs/ServerSegmentCompletion.dot.png | Bin 0 -> 72968 bytes docs/architecture.rst | 142 + docs/client_api.rst | 229 + docs/commit-happy-path-1.png | Bin 0 -> 22933 bytes docs/commit-happy-path-2.png | Bin 0 -> 22939 bytes docs/committer-failed.png | Bin 0 -> 15412 bytes docs/conf.py | 308 + docs/controller-failed.png | Bin 0 -> 23599 bytes docs/controller-segment-completion.png | Bin 0 -> 85092 bytes docs/delayed-server.png | Bin 0 -> 21690 bytes docs/docutils.conf | 2 + docs/expressionTree.jpg | Bin 0 -> 12881 bytes docs/expressions_udf.rst | 120 + docs/hlc-arch.png | Bin 0 -> 47924 bytes docs/image2014-11-12-1-54-56.png | Bin 0 -> 112436 bytes docs/image2014-11-12-19-54-12.png | Bin 0 -> 54744 bytes docs/image2014-11-12-20-59-50.png | Bin 0 -> 36925 bytes docs/image2014-11-13-11-21-7.png | Bin 0 -> 99075 bytes docs/image2014-11-13-11-22-57.png | Bin 0 -> 27562 bytes docs/image2015-4-14-194451.png | Bin 0 -> 83171 bytes docs/image2015-5-12-7-24-58.png | Bin 0 -> 38969 bytes docs/image2015-5-13 15-2-9.png | Bin 0 -> 41882 bytes docs/image2015-5-17 13-32-28.png | Bin 0 -> 89700 bytes docs/image2015-5-17 17-59-10.png | Bin 0 -> 82542 bytes docs/image2015-5-19 0-29-34.png | Bin 0 -> 39085 bytes docs/image2015-5-19 0-58-54.png | Bin 0 -> 54859 bytes docs/image2015-5-19 1-28-35.png | Bin 0 -> 34374 bytes docs/image2015-5-19 1-47-44.png | Bin 0 -> 34374 bytes docs/image2015-5-19 1-59-51.png | Bin 0 -> 37098 bytes docs/in_production.rst | 48 + docs/index.rst | 62 + docs/intro.rst | 47 + docs/llc-arch.png | Bin 0 -> 39899 bytes docs/llc.rst | 164 + docs/make.bat | 297 + docs/management_api.rst | 8 + docs/multiple-server-failure.png | Bin 0 -> 18036 bytes docs/multitenancy.rst | 340 + docs/parseTree.png | Bin 0 -> 55136 bytes docs/partition_aware_routing.rst | 141 + docs/pinot-architecture.png | Bin 0 -> 58474 bytes docs/pinot_hadoop.rst | 177 + docs/pluggable_streams.png | Bin 0 -> 136984 bytes docs/pluggable_streams.rst | 124 + docs/pql_examples.rst | 260 + docs/reference.rst | 10 + docs/schema_timespec.rst | 111 + docs/segment-consumer-fsm.png | Bin 0 -> 30109 bytes docs/segment-creation.png | Bin 0 -> 9830 bytes docs/segment-helix-fsm.png | Bin 0 -> 7494 bytes docs/segment_fetcher.rst | 77 + docs/trying_pinot.rst | 84 + docs/zk-setup.png | Bin 0 -> 39246 bytes docs/znode_layout.png | Bin 0 -> 43271 bytes pinot-api/pom.xml | 4 + .../linkedin/pinot/filesystem/AzurePinotFS.java | 51 +- pinot-broker/pom.xml | 4 + .../broker/helix/DefaultHelixBrokerConfig.java | 1 - .../broker/broker/helix/HelixBrokerStarter.java | 14 + .../TimeboundaryRefreshMessageHandlerFactory.java | 141 + .../requesthandler/BaseBrokerRequestHandler.java | 12 +- .../routing/HelixExternalViewBasedRouting.java | 6 +- .../broker/routing/RoutingTableBuilderFactory.java | 16 +- .../builder/BalancedRandomRoutingTableBuilder.java | 44 +- .../BasePartitionAwareRoutingTableBuilder.java | 24 +- .../routing/builder/BaseRoutingTableBuilder.java | 134 +- .../builder/DefaultOfflineRoutingTableBuilder.java | 8 +- .../DefaultRealtimeRoutingTableBuilder.java | 12 +- .../builder/GeneratorBasedRoutingTableBuilder.java | 221 +- ...HighLevelConsumerBasedRoutingTableBuilder.java} | 59 +- .../KafkaLowLevelConsumerRoutingTableBuilder.java | 160 - .../builder/LargeClusterRoutingTableBuilder.java | 52 +- .../LowLevelConsumerRoutingTableBuilder.java | 151 + ...l.java => LowLevelRoutingTableBuilderUtil.java} | 20 +- .../PartitionAwareOfflineRoutingTableBuilder.java | 10 +- .../PartitionAwareRealtimeRoutingTableBuilder.java | 26 +- .../routing/builder/RoutingTableBuilder.java | 6 +- .../broker/broker/HelixBrokerStarterTest.java | 99 +- .../pinot/broker/routing/RoutingTableTest.java | 4 +- .../BalancedRandomRoutingTableBuilderTest.java | 79 +- ... HighLevelConsumerRoutingTableBuilderTest.java} | 8 +- .../LargeClusterRoutingTableBuilderTest.java | 6 +- ...> LowLevelConsumerRoutingTableBuilderTest.java} | 51 +- ...rtitionAwareOfflineRoutingTableBuilderTest.java | 22 +- ...titionAwareRealtimeRoutingTableBuilderTest.java | 4 +- .../builder/RoutingTableBuilderTestUtil.java | 43 + pinot-common/pom.xml | 14 + .../config/AdjustTableNameChildKeyTransformer.java | 4 +- .../pinot/common/config/CombinedConfigLoader.java | 19 +- ...CombinedConfigSeparatorChildKeyTransformer.java | 8 + .../linkedin/pinot/common/config/Deserializer.java | 10 +- .../pinot/common/config/IndexingConfig.java | 48 +- .../pinot/common/config/RoutingConfig.java | 7 +- .../SegmentsValidationAndRetentionConfig.java | 6 +- .../linkedin/pinot/common/config/Serializer.java | 18 +- .../pinot/common/config/StarTreeIndexConfig.java | 68 + .../linkedin/pinot/common/config/TableConfig.java | 7 + .../com/linkedin/pinot/common/data/FieldSpec.java | 75 +- .../pinot/common/lineage/SegmentGroup.java | 72 + .../pinot/common/lineage/SegmentMergeLineage.java | 348 + .../lineage/SegmentMergeLineageAccessHelper.java | 85 + .../messages/TimeboundaryRefreshMessage.java | 42 + .../pinot/common/metadata/ZKMetadataProvider.java | 60 +- .../pinot/common/metrics/AbstractMetrics.java | 29 +- .../linkedin/pinot/common/metrics/BrokerMeter.java | 2 + .../pinot/common/metrics/ControllerMeter.java | 3 + .../linkedin/pinot/common/metrics/ServerGauge.java | 9 +- .../linkedin/pinot/common/metrics/ServerMeter.java | 12 +- .../pinot/common/metrics/ValidationMetrics.java | 2 +- .../protocols/SegmentCompletionProtocol.java | 8 +- .../pinot/common/response/BrokerResponse.java | 20 + .../response/broker/BrokerResponseNative.java | 64 +- .../pinot/common/segment/SegmentMetadata.java | 2 + .../pinot/common/utils/CommonConstants.java | 19 +- .../com/linkedin/pinot/common/utils/DataTable.java | 4 + .../common/utils/FileUploadDownloadClient.java | 19 + .../com/linkedin/pinot/common/utils/FileUtils.java | 28 +- .../linkedin/pinot/common/utils/StringUtil.java | 40 +- .../common/utils/time/DefaultTimeConverter.java | 83 - .../pinot/common/utils/time/TimeConverter.java | 54 +- .../common/lineage/SegmentMergeLineageTest.java | 142 + .../pinot/common/utils/StringUtilTest.java | 23 +- .../utils/time/DefaultTimeConverterTest.java | 174 - .../pinot/common/utils/time/TimeConverterTest.java | 97 + pinot-controller/pom.xml | 4 + .../linkedin/pinot/controller/ControllerConf.java | 47 +- .../pinot/controller/ControllerStarter.java | 67 +- .../api/resources/FileUploadPathProvider.java | 19 +- .../resources/LLCSegmentCompletionHandlers.java | 77 +- .../PinotSegmentUploadRestletResource.java | 33 +- .../resources/PinotTableConfigRestletResource.java | 32 +- .../controller/helix/SegmentStatusChecker.java | 127 +- .../helix/core/PinotHelixResourceManager.java | 105 +- .../helix/core/PinotTableIdealStateBuilder.java | 25 +- .../helix/core/SegmentDeletionManager.java | 130 +- .../helix/core/minion/PinotTaskManager.java | 64 +- .../core/periodictask/ControllerPeriodicTask.java | 112 + .../realtime/PinotLLCRealtimeSegmentManager.java | 209 +- .../core/realtime/PinotRealtimeSegmentManager.java | 2 +- .../core/realtime/SegmentCompletionManager.java | 2 +- .../core/relocation/RealtimeSegmentRelocator.java | 114 +- .../helix/core/retention/RetentionManager.java | 62 +- .../helix/core/util/HelixSetupUtils.java | 4 +- .../controller/validation/ValidationManager.java | 81 +- .../controller/api/resources/TableViewsTest.java | 4 +- .../controller/helix/SegmentStatusCheckerTest.java | 98 +- .../periodictask/ControllerPeriodicTaskTest.java | 106 + .../PinotLLCRealtimeSegmentManagerTest.java | 103 +- .../segment/FlushThresholdUpdaterTest.java | 4 +- .../rebalance/DefaultRebalanceStrategyTest.java | 4 +- .../relocation/RealtimeSegmentRelocatorTest.java | 58 +- .../helix/core/retention/RetentionManagerTest.java | 12 +- .../core/util/SegmentDeletionManagerTest.java | 19 +- .../controller/utils/SegmentMetadataMockUtils.java | 21 + pinot-core/pom.xml | 4 + .../com/linkedin/pinot/core/bloom/BloomFilter.java | 69 + .../linkedin/pinot/core/bloom/BloomFilterType.java | 50 + .../linkedin/pinot/core/bloom/BloomFilterUtil.java | 54 + .../pinot/core/bloom/GuavaOnHeapBloomFilter.java | 74 + .../core/bloom/SegmentBloomFilterFactory.java | 48 + .../com/linkedin/pinot/core/common/DataSource.java | 3 + .../pinot/core/crypt/PinotCrypterFactory.java | 55 +- .../core/data/extractors/PlainFieldExtractor.java | 294 - .../data/function/FunctionExpressionEvaluator.java | 89 +- .../manager/config/TableDataManagerConfig.java | 7 - .../manager/offline/TableDataManagerProvider.java | 13 +- .../realtime/HLRealtimeSegmentDataManager.java | 71 +- .../realtime/LLRealtimeSegmentDataManager.java | 103 +- .../manager/realtime/RealtimeTableDataManager.java | 18 +- .../core/data/readers/GenericRowRecordReader.java | 8 +- .../recordtransformer/CompoundTransformer.java | 74 + .../recordtransformer/DataTypeTransformer.java | 112 + .../recordtransformer/ExpressionTransformer.java | 65 + .../PinotDataType.java | 77 +- .../RecordTransformer.java} | 26 +- .../recordtransformer/SanitationTransformer.java | 74 + .../data/recordtransformer/TimeTransformer.java | 92 + .../generator/SegmentGeneratorConfig.java | 18 +- .../indexsegment/mutable/MutableSegmentImpl.java | 5 +- ...ixedByteSingleColumnMultiValueReaderWriter.java | 13 +- ...xedByteSingleColumnSingleValueReaderWriter.java | 2 +- ...ixedByteSingleValueMultiColumnReaderWriter.java | 5 +- .../io/util/DictionaryDelegatingValueReader.java | 6 +- .../core/io/util/FixedByteValueReaderWriter.java | 59 +- .../linkedin/pinot/core/io/util/ValueReader.java | 2 +- .../core/io/writer/impl/DirectMemoryManager.java | 3 +- .../impl/FixedByteSingleValueMultiColWriter.java | 10 +- .../impl/v1/FixedByteChunkSingleValueWriter.java | 6 +- .../impl/v1/VarByteChunkSingleValueWriter.java | 5 +- .../pinot/core/minion/BackfillDateTimeColumn.java | 4 +- .../core/operator/CombineGroupByOperator.java | 6 + .../pinot/core/operator/CombineOperator.java | 2 + .../pinot/core/operator/ExecutionStatistics.java | 20 +- .../operator/blocks/IntermediateResultsBlock.java | 29 + .../docidsets/BitmapBasedBlockDocIdSet.java | 41 - .../core/operator/filter/AndFilterOperator.java | 25 +- .../core/operator/filter/BaseFilterOperator.java | 13 +- .../operator/filter/BitmapBasedFilterOperator.java | 15 +- .../core/operator/filter/EmptyFilterOperator.java | 8 +- .../core/operator/filter/FilterOperatorUtils.java | 65 +- .../operator/filter/MatchAllFilterOperator.java | 8 +- .../core/operator/filter/OrFilterOperator.java | 25 +- .../operator/filter/ScanBasedFilterOperator.java | 16 +- .../SortedInvertedIndexBasedFilterOperator.java | 15 +- .../BaseDictionaryBasedPredicateEvaluator.java | 12 + .../BaseRawValueBasedPredicateEvaluator.java | 15 +- .../predicate/EqualsPredicateEvaluatorFactory.java | 9 +- .../predicate/InPredicateEvaluatorFactory.java | 14 +- .../NotEqualsPredicateEvaluatorFactory.java | 9 +- .../predicate/NotInPredicateEvaluatorFactory.java | 16 +- .../filter/predicate/PredicateEvaluator.java | 5 + .../predicate/RangePredicateEvaluatorFactory.java | 33 +- .../RegexpLikePredicateEvaluatorFactory.java | 5 - .../pinot/core/periodictask/BasePeriodicTask.java | 52 + .../PeriodicTask.java} | 55 +- .../core/periodictask/PeriodicTaskScheduler.java | 80 + .../linkedin/pinot/core/plan/CombinePlanNode.java | 28 +- .../linkedin/pinot/core/plan/FilterPlanNode.java | 37 +- .../core/plan/maker/InstancePlanMakerImplV2.java | 6 + .../query/executor/ServerQueryExecutorV1Impl.java | 12 +- .../core/query/pruner/AbstractSegmentPruner.java | 17 +- .../query/pruner/ColumnValueSegmentPruner.java | 44 +- .../core/query/pruner/PartitionSegmentPruner.java | 10 +- .../core/query/reduce/BrokerReduceService.java | 23 + .../core/query/request/ServerQueryRequest.java | 8 - .../pinot/core/query/scheduler/QueryScheduler.java | 23 +- .../query/scheduler/QuerySchedulerFactory.java | 2 +- .../query/selection/SelectionOperatorUtils.java | 61 +- .../converter/RealtimeSegmentConverter.java | 10 +- .../dictionary/BytesOffHeapMutableDictionary.java | 66 +- .../dictionary/BytesOnHeapMutableDictionary.java | 51 +- ...sumerFactory.java => KafkaConsumerFactory.java} | 6 +- .../impl/kafka/KafkaJSONMessageDecoder.java | 4 +- .../impl/kafka/KafkaStreamConfigProperties.java | 1 - .../impl/kafka/KafkaStreamLevelConsumer.java | 20 - .../impl/kafka/KafkaStreamMetadataProvider.java | 16 +- .../realtime/impl/kafka/SimpleConsumerFactory.java | 64 +- .../pinot/core/realtime/stream/MessageBatch.java | 10 +- .../pinot/core/realtime/stream/OffsetCriteria.java | 210 + .../realtime/stream/PartitionLevelConsumer.java | 2 +- .../realtime/stream/PartitionOffsetFetcher.java | 4 +- .../pinot/core/realtime/stream/StreamConfig.java | 58 +- .../realtime/stream/StreamConfigProperties.java | 8 +- .../realtime/stream/StreamConsumerFactory.java | 16 +- .../core/realtime/stream/StreamLevelConsumer.java | 41 +- .../core/realtime/stream/StreamMessageDecoder.java | 10 +- .../realtime/stream/StreamMetadataProvider.java | 6 +- .../RecordReaderSegmentCreationDataSource.java | 18 +- .../segment/creator/SegmentIndexCreationInfo.java | 36 - .../creator/impl/SegmentColumnarIndexCreator.java | 12 - .../creator/impl/SegmentDictionaryCreator.java | 12 +- .../impl/SegmentIndexCreationDriverImpl.java | 88 +- .../core/segment/creator/impl/V1Constants.java | 1 + .../creator/impl/bloom/BloomFilterCreator.java | 66 + .../fwd/SingleValueVarByteRawIndexCreator.java | 3 - .../stats/BytesColumnPredIndexStatsCollector.java | 82 +- .../core/segment/index/SegmentMetadataImpl.java | 5 + .../segment/index/column/ColumnIndexContainer.java | 3 + .../index/column/PhysicalColumnIndexContainer.java | 28 +- .../converter/SegmentV1V2ToV3FormatConverter.java | 2 +- .../index/data/source/ColumnDataSource.java | 17 +- .../segment/index/loader/IndexLoadingConfig.java | 17 + .../core/segment/index/loader/LoaderUtils.java | 2 +- .../segment/index/loader/SegmentPreProcessor.java | 7 +- .../loader/bloomfilter/BloomFilterHandler.java | 160 + .../segment/index/readers/BloomFilterReader.java | 54 + .../segment/index/readers/BytesDictionary.java | 4 +- .../{ => name}/DefaultSegmentNameGenerator.java | 2 +- .../name/NormalizedDateSegmentNameGenerator.java | 160 + .../segment/{ => name}/SegmentNameGenerator.java | 4 +- .../core/segment/store/ColumnIndexDirectory.java | 24 +- .../pinot/core/segment/store/ColumnIndexType.java | 3 +- .../core/segment/store/FilePerIndexDirectory.java | 27 +- .../pinot/core/segment/store/SegmentDirectory.java | 2 +- .../segment/store/SegmentLocalFSDirectory.java | 14 +- .../segment/store/SingleFileIndexDirectory.java | 20 +- .../virtualcolumn/VirtualColumnIndexContainer.java | 6 + .../core/startree/OffHeapStarTreeBuilder.java | 6 +- .../startree/operator/StarTreeFilterOperator.java | 58 +- .../startree/v2/builder/StarTreeIndexCombiner.java | 5 +- .../v2/builder/StarTreeV2BuilderConfig.java | 23 +- .../v2/store/StarTreeDimensionDataSource.java | 6 + .../v2/store/StarTreeMetricDataSource.java | 6 + .../com/linkedin/pinot/core/util/AvroUtils.java | 1 - .../server/realtime/ControllerLeaderLocator.java | 2 + .../core/common/RealtimeNoDictionaryTest.java | 8 +- .../pinot/core/crypt/PinotCrypterFactoryTest.java | 38 +- .../data/extractors/PlainFieldExtractorTest.java | 245 - .../function/FunctionExpressionEvaluatorTest.java | 34 +- .../realtime/LLRealtimeSegmentDataManagerTest.java | 4 +- .../data/recordtransformer/PinotDataTypeTest.java | 144 + .../recordtransformer/RecordTransformerTest.java | 133 + .../recordtransformer/TimeTransformerTest.java | 146 + .../operator/filter/AndFilterOperatorTest.java | 27 +- .../operator/filter/FilterOperatorUtilsTest.java | 98 + .../{ => core}/operator/filter/IntRangesTest.java | 3 +- .../operator/filter/OrFilterOperatorTest.java | 20 +- .../core/operator/filter/TestFilterOperator.java | 78 + .../periodictask/PeriodicTaskSchedulerTest.java | 85 + .../pinot/core/plan/CombinePlanNodeTest.java | 36 + ...ngeOfflineDictionaryPredicateEvaluatorTest.java | 44 +- .../impl/dictionary/MutableDictionaryTest.java | 23 +- .../kafka/KafkaPartitionLevelConsumerTest.java | 4 +- .../core/realtime/stream/OffsetCriteriaTest.java | 96 + .../core/realtime/stream/StreamConfigTest.java | 29 +- .../index/creator/BloomFilterCreatorTest.java | 150 + .../segment/index/creator/RawIndexCreatorTest.java | 31 +- .../readers/ImmutableDictionaryReaderTest.java | 115 +- .../DefaultSegmentNameGeneratorTest.java | 2 +- .../NormalizedDateSegmentNameGeneratorTest.java | 140 + .../store/ColumnIndexDirectoryTestHelper.java | 15 + .../operator/filter/FilterOperatorTestUtils.java | 85 - .../linkedin/pinot/queries/BaseQueriesTest.java | 19 +- ...nerSegmentAggregationMultiValueQueriesTest.java | 12 +- ...InnerSegmentSelectionMultiValueQueriesTest.java | 6 +- ...terSegmentAggregationMultiValueQueriesTest.java | 46 +- ...erSegmentAggregationSingleValueQueriesTest.java | 14 + .../query/pruner/ColumnValueSegmentPrunerTest.java | 4 +- pinot-distribution/pom.xml | 4 + pinot-filesystem/pom.xml | 8 + .../linkedin/pinot/filesystem/LocalPinotFS.java | 75 +- .../com/linkedin/pinot/filesystem/PinotFS.java | 42 +- .../pinot/filesystem/LocalPinotFSTest.java | 46 +- .../pinot/filesystem/PinotFSFactoryTest.java | 13 +- pinot-hadoop-filesystem/pom.xml | 8 + .../linkedin/pinot/filesystem/HadoopPinotFS.java | 29 +- pinot-hadoop/pom.xml | 6 +- .../mapper/HadoopSegmentCreationMapReduceJob.java | 26 +- pinot-integration-tests/pom.xml | 12 + .../tests/BaseClusterIntegrationTest.java | 10 +- .../tests/BaseClusterIntegrationTestSet.java | 23 +- .../pinot/integration/tests/ClusterTest.java | 28 +- ...lakyConsumerRealtimeClusterIntegrationTest.java | 20 - ...mentBuildPushOfflineClusterIntegrationTest.java | 2 +- .../tests/HybridClusterIntegrationTest.java | 10 +- ...ridClusterIntegrationTestCommandLineRunner.java | 2 +- ...onaryAggregationPlanClusterIntegrationTest.java | 31 +- .../tests/OfflineClusterIntegrationTest.java | 43 +- .../tests/RealtimeClusterIntegrationTest.java | 4 +- .../tests/SegmentCompletionIntegrationTests.java | 5 +- .../tests/SimpleMinionClusterIntegrationTest.java | 6 +- .../tests/StarTreeClusterIntegrationTest.java | 9 +- .../tests/StarTreeV2ClusterIntegrationTest.java | 62 +- pinot-minion/pom.xml | 4 + .../com/linkedin/pinot/minion/MinionStarter.java | 6 + pinot-perf/pom.xml | 4 + .../linkedin/pinot/perf/PerfBenchmarkRunner.java | 71 - .../com/linkedin/pinot/perf/PerfBenchmarkTest.java | 77 - .../com/linkedin/pinot/perf/RawIndexBenchmark.java | 7 +- pinot-server/pom.xml | 47 +- .../starter/helix/HelixInstanceDataManager.java | 3 + .../starter/helix/SegmentFetcherAndLoader.java | 2 + pinot-tools/pom.xml | 4 + .../pinot/tools/admin/PinotAdministrator.java | 11 +- .../admin/command/ApplyTableConfigCommand.java | 3 +- .../pinot/tools/perf/PerfBenchmarkDriver.java | 6 +- .../pinot/tools/perf/PerfBenchmarkRunner.java | 16 +- .../converter/ColumnarToStarTreeConverter.java | 4 +- .../segment/converter/SegmentMergeCommand.java | 289 + .../meetupRsvp_realtime_table_config.json | 1 + pinot-transport/pom.xml | 4 + pom.xml | 69 +- thirdeye/pom.xml | 7 + thirdeye/thirdeye-frontend/.gitignore | 1 + thirdeye/thirdeye-frontend/.travis.yml | 3 - .../thirdeye-frontend/app/adapters/application.js | 2 +- thirdeye/thirdeye-frontend/app/adapters/base.js | 16 +- .../thirdeye-frontend/app/adapters/share-config.js | 13 + thirdeye/thirdeye-frontend/app/mirage/config.js | 8 + .../app/mirage/endpoints/rootcause.js | 158 +- .../app/mirage/endpoints/selfserve.js | 24 +- .../thirdeye-frontend/app/mocks/alertConfig.js | 6 +- thirdeye/thirdeye-frontend/app/mocks/anomalies.js | 2 +- .../app/mocks/anomalyPerformance.js | 2 +- .../app/mocks/metricPeripherals.js | 2 +- .../app/mocks/timeseriesCompare.js | 2 +- thirdeye/thirdeye-frontend/app/models/.gitkeep | 0 .../thirdeye-frontend/app/models/share-config.js | 8 + .../app/pods/application/route.js | 10 +- .../app/pods/components/anomaly-graph/component.js | 35 +- .../app/pods/components/entity-filter/component.js | 117 +- .../app/pods/components/entity-filter/template.hbs | 81 +- .../app/pods/components/filter-select/component.js | 14 +- .../app/pods/components/heatmap-chart/component.js | 3 + .../pods/components/rootcause-anomaly/component.js | 89 +- .../pods/components/rootcause-anomaly/template.hbs | 89 +- .../pods/components/rootcause-chart/component.js | 52 +- .../pods/components/rootcause-chart/template.hbs | 11 +- .../rootcause-dimensions-table/component.js | 18 +- .../pods/components/rootcause-header/template.hbs | 7 +- .../pods/components/rootcause-legend/component.js | 42 +- .../pods/components/rootcause-legend/template.hbs | 22 +- .../pods/components/rootcause-metrics/component.js | 6 - .../pods/components/rootcause-metrics/template.hbs | 9 - .../rootcause-select-metric-dimension/component.js | 9 +- .../rootcause-select-metric-dimension/template.hbs | 20 +- .../rootcause-select-metric/component.js | 43 +- .../rootcause-select-metric/template.hbs | 6 +- .../pods/components/self-serve-graph/template.hbs | 2 +- .../components/share-custom-template/component.js | 125 + .../components/share-custom-template/template.hbs | 39 + .../custom/anomalies-table/resolution/component.js | 14 +- .../custom/anomalies-table/resolution/template.hbs | 15 +- .../pods/custom/metrics-table-metric/template.hbs | 2 +- .../app/pods/home/index/controller.js | 13 + .../app/pods/home/index/template.hbs | 11 +- .../app/pods/home/share-dashboard/controller.js | 54 +- .../app/pods/home/share-dashboard/route.js | 6 + .../app/pods/home/share-dashboard/template.hbs | 130 +- .../app/pods/manage/alert/explore/controller.js | 18 +- .../app/pods/manage/alert/explore/route.js | 5 +- .../app/pods/manage/alert/explore/template.hbs | 13 +- .../app/pods/manage/alerts/index/controller.js | 428 +- .../app/pods/manage/alerts/index/route.js | 212 +- .../app/pods/manage/alerts/index/template.hbs | 124 +- .../app/pods/manage/alerts/performance/route.js | 15 +- .../pods/partials/rootcause/metrics/template.hbs | 10 + .../rootcause/select-metric-label/template.hbs | 6 + .../app/pods/preview/template.hbs | 28 - .../app/pods/rootcause/controller.js | 151 +- .../thirdeye-frontend/app/pods/rootcause/route.js | 151 +- .../app/pods/rootcause/template.hbs | 9 +- .../app/pods/rootcause2/controller.js | 1216 -- .../thirdeye-frontend/app/pods/rootcause2/route.js | 391 - .../app/pods/rootcause2/template.hbs | 290 - .../app/pods/self-serve/create-alert/controller.js | 67 +- .../app/pods/self-serve/create-alert/route.js | 1 - .../app/pods/self-serve/create-alert/template.hbs | 781 +- .../pods/self-serve/import-metric/controller.js | 25 +- .../app/pods/self-serve/import-metric/template.hbs | 12 + .../app/pods/services/api/anomalies/service.js | 37 +- .../app/pods/services/api/dimensions/service.js | 2 +- .../pods/services/api/share-dashboard/service.js | 9 +- .../services/api/share-template-config/service.js | 46 + .../app/pods/services/duration/service.js | 11 +- .../app/pods/services/query-cache/service.js | 30 +- .../services/rootcause-aggregates-cache/service.js | 101 +- .../rootcause-anomalyfunction-cache/service.js | 3 +- .../services/rootcause-breakdowns-cache/service.js | 20 +- .../services/rootcause-entities-cache/service.js | 1 - .../app/pods/services/rootcause-fetcher/service.js | 232 + .../services/rootcause-scores-cache/service.js | 33 +- .../services/rootcause-timeseries-cache/service.js | 20 +- thirdeye/thirdeye-frontend/app/router.js | 1 - thirdeye/thirdeye-frontend/app/serializers/base.js | 12 +- .../app/serializers/performance.js | 16 +- .../app/serializers/share-config.js | 19 + .../thirdeye-frontend/app/serializers/share.js | 16 +- .../app/shared/anomaliesTableColumns.js | 2 +- .../app/shared/dimensionAnalysisTableConfig.js | 2 +- .../app/shared/filterBarConfig.js | 2 +- thirdeye/thirdeye-frontend/app/styles/app.scss | 2 +- .../app/styles/components/entity-filter.scss | 57 +- .../app/styles/components/rootcause-anomaly.scss | 10 + .../app/styles/components/rootcause-chart.scss | 25 + .../app/styles/components/rootcause-legend.scss | 4 + .../styles/components/rootcause-select-metric.scss | 28 +- .../app/styles/components/te-anomaly-table.scss | 17 +- .../app/styles/components/te-radio.scss | 1 + .../pods/home/share-dashboard/share-dashboard.scss | 22 - .../app/styles/pods/manage/alerts.scss | 19 +- .../app/styles/shared/_placeholders.scss | 10 + .../app/styles/shared/_styles.scss | 43 +- .../app/styles/shared/_variable.scss | 3 + thirdeye/thirdeye-frontend/app/utils/anomaly.js | 2 +- .../thirdeye-frontend/app/utils/api/self-serve.js | 2 +- thirdeye/thirdeye-frontend/app/utils/constants.js | 33 + .../app/utils/manage-alert-utils.js | 68 +- thirdeye/thirdeye-frontend/app/utils/rca-utils.js | 2 +- thirdeye/thirdeye-frontend/app/utils/utils.js | 9 +- thirdeye/thirdeye-frontend/ember-cli-build.js | 7 + thirdeye/thirdeye-frontend/package-lock.json | 18991 ------------------- thirdeye/thirdeye-frontend/package.json | 7 +- thirdeye/thirdeye-frontend/pom.xml | 14 +- thirdeye/thirdeye-frontend/testem.js | 7 +- .../tests/acceptance/rootcause-test.js | 25 +- .../acceptance/self-serve-alert-tuning-test.js | 4 +- .../acceptance/self-serve-import-metric-test.js | 4 +- .../tests/acceptance/self-serve-onboarding-test.js | 4 +- .../components/entity-filter/component-test.js | 130 + .../components/rootcause-anomaly/component-test.js | 161 + .../component-test.js | 12 +- .../share-custom-template/component-test.js | 70 + thirdeye/thirdeye-frontend/yarn.lock | 4322 ++--- thirdeye/thirdeye-pinot/config/dashboard.yml | 7 +- thirdeye/thirdeye-pinot/config/detector.yml | 9 +- thirdeye/thirdeye-pinot/config/rca.yml | 6 +- thirdeye/thirdeye-pinot/pom.xml | 47 +- .../alert/content/BaseEmailContentFormatter.java | 19 +- .../EmailContentFormatterConfiguration.java | 16 +- ...HierarchicalAnomaliesEmailContentFormatter.java | 7 +- .../MultipleAnomaliesEmailContentFormatter.java | 11 +- .../thirdeye/anomaly/SmtpConfiguration.java | 25 +- .../anomaly/ThirdEyeAnomalyApplication.java | 4 + .../anomaly/ThirdEyeAnomalyConfiguration.java | 20 +- .../anomaly/alert/util/AnomalyReportGenerator.java | 8 +- .../thirdeye/anomaly/alert/util/EmailHelper.java | 11 +- .../anomaly/alert/util/EmailScreenshotHelper.java | 8 +- .../anomaly/alert/v2/AlertTaskRunnerV2.java | 36 +- .../detection/DetectionJobSchedulerUtils.java | 4 +- .../thirdeye/anomaly/events/EventFilter.java | 14 +- .../anomaly/events/HolidayEventProvider.java | 20 +- .../thirdeye/anomaly/merge/AnomalyMergeConfig.java | 5 +- .../anomaly/merge/AnomalyTimeBasedSummarizer.java | 10 +- .../anomaly/merge/TimeBasedAnomalyMerger.java | 5 +- .../thirdeye/anomaly/onboard/ReplayTaskRunner.java | 5 +- .../tasks/FunctionCreationOnboardingTask.java | 10 +- .../onboard/AutoOnboardPinotMetadataSource.java | 16 +- .../thirdeye/auto/onboard/ConfigGenerator.java | 20 +- .../thirdeye/common/ThirdEyeConfiguration.java | 21 +- .../dashboard/ThirdEyeDashboardApplication.java | 10 +- .../dashboard/ThirdEyeDashboardConfiguration.java | 1 + .../dashboard/resources/EmailResource.java | 18 +- .../dashboard/resources/OnboardResource.java | 87 +- .../dashboard/resources/v2/ResourceUtils.java | 2 +- .../resources/v2/RootCauseMetricResource.java | 97 +- .../dashboard/resources/v2/TimeSeriesResource.java | 5 +- .../resources/v2/UserDashboardResource.java | 4 +- .../resources/v2/pojo/AnomalyDetails.java | 18 +- .../v2/rootcause/AnomalyEventFormatter.java | 4 - .../com/linkedin/thirdeye/dataframe/DataFrame.java | 19 +- .../thirdeye/dataframe/util/DataFrameUtils.java | 4 - .../thirdeye/dataframe/util/MetricSlice.java | 3 +- .../linkedin/thirdeye/datalayer/ScriptRunner.java | 4 +- .../thirdeye/datalayer/dto/DetectionConfigDTO.java | 11 + .../thirdeye/datalayer/pojo/AlertConfigBean.java | 9 + .../datalayer/pojo/DetectionAlertConfigBean.java | 34 +- .../datalayer/pojo/DetectionConfigBean.java | 9 + .../thirdeye/datasource/pinot/PqlUtils.java | 24 +- .../linkedin/thirdeye/detection/DataProvider.java | 29 +- .../thirdeye/detection/DefaultDataProvider.java | 77 +- .../detection/DefaultInputDataFetcher.java | 87 + .../detection/DetectionMigrationResource.java | 178 +- .../thirdeye/detection/DetectionPipeline.java | 59 +- .../thirdeye/detection/DetectionResource.java | 44 +- ...{DetectionPipeline.java => DetectionUtils.java} | 153 +- .../com/linkedin/thirdeye/detection/InputData.java | 66 - .../thirdeye/detection/InputDataFetcher.java | 18 +- .../linkedin/thirdeye/detection/InputDataSpec.java | 98 - .../Pattern.java} | 16 +- .../detection/StaticDetectionPipeline.java | 12 +- .../thirdeye/detection/alert/AlertUtils.java | 58 + .../detection/alert/DetectionAlertTaskFactory.java | 63 +- .../detection/alert/DetectionAlertTaskRunner.java | 85 +- .../alert/StatefulDetectionAlertFilter.java | 16 +- .../filter/DimensionDetectionAlertFilter.java | 20 +- .../detection/alert/filter/LegacyAlertFilter.java | 98 +- .../ToAllRecipientsDetectionAlertFilter.java | 14 +- .../alert/scheme/DetectionEmailAlerter.java | 219 +- .../alert/suppress/DetectionAlertSuppressor.java | 22 + .../DetectionAlertTimeWindowSuppressor.java | 127 + .../detection/algorithm/BaselineAlgorithm.java | 4 +- .../detection/algorithm/DimensionWrapper.java | 64 +- .../algorithm/LegacyAnomalyFunctionAlgorithm.java | 6 +- .../detection/algorithm/LegacyMergeWrapper.java | 5 +- .../thirdeye/detection/algorithm/MergeWrapper.java | 42 +- .../detection/algorithm/MovingWindowAlgorithm.java | 7 +- .../detection/algorithm/ThresholdAlgorithm.java | 4 +- .../stage/AnomalyDetectionStageWrapper.java | 167 +- .../algorithm/stage/AnomalyFilterStageWrapper.java | 1 - .../stage/BaselineRuleDetectionStage.java | 25 +- .../algorithm/stage/BaselineRuleFilterStage.java | 11 +- .../detection/algorithm/stage/StageUtils.java | 50 - .../stage/StaticAnomalyDetectionStage.java | 62 +- .../algorithm/stage/StaticAnomalyFilterStage.java | 19 +- .../algorithm/stage/StaticGrouperStage.java | 50 - .../stage/ThresholdRuleDetectionStage.java | 23 +- .../algorithm/stage/ThresholdRuleFilterStage.java | 17 +- .../thirdeye/detection/annotation/Components.java | 49 + .../annotation/DetectionConfigurationResource.java | 37 + .../detection/annotation/DetectionRegistry.java | 145 + .../annotation/DetectionTag.java} | 14 +- .../thirdeye/detection/annotation/Param.java | 41 + .../detection/annotation/PresentationOption.java | 25 +- .../thirdeye/detection/annotation/Tune.java | 36 +- .../thirdeye/detection/annotation/Yaml.java | 29 +- .../detection/baseline/BaselineProvider.java | 51 - .../detection/baseline/BaselineProviderLoader.java | 34 - .../detection/baseline/RuleBaselineProvider.java | 97 - .../detection/baseline/StaticBaselineProvider.java | 105 - .../AbsoluteChangeRuleAnomalyFilter.java | 97 + .../components/AbsoluteChangeRuleDetector.java | 112 + .../PercentageChangeRuleAnomalyFilter.java | 96 + .../components/PercentageChangeRuleDetector.java | 121 + .../detection/components/RuleBaselineProvider.java | 68 + .../SitewideImpactRuleAnomalyFilter.java | 126 + .../components/ThresholdRuleAnomalyFilter.java | 77 + .../components/ThresholdRuleDetector.java | 92 + .../finetune/GridSearchTuningAlgorithm.java | 6 +- .../detection/finetune/TuningAlgorithm.java | 4 +- .../spec/AbsoluteChangeRuleAnomalyFilterSpec.java | 56 + .../spec/AbsoluteChangeRuleDetectorSpec.java | 56 + .../thirdeye/detection/spec/AbstractSpec.java | 24 +- .../PercentageChangeRuleAnomalyFilterSpec.java | 56 + .../spec/PercentageChangeRuleDetectorSpec.java | 56 + .../detection/spec/RuleBaselineProviderSpec.java | 50 + .../spec/SitewideImpactRuleAnomalyFilterSpec.java | 72 + .../detection/spec/ThresholdRuleDetectorSpec.java | 33 +- .../detection/spec/ThresholdRuleFilterSpec.java | 33 +- .../detection/spi/components/AnomalyDetector.java | 57 +- .../spi/components/AnomalyFilter.java} | 15 +- .../spi/components/BaseComponent.java} | 11 +- .../detection/spi/components/BaselineProvider.java | 44 + .../thirdeye/detection/spi/components/Grouper.java | 23 +- .../thirdeye/detection/spi/components/Tunable.java | 36 + .../detection/{ => spi/model}/AnomalySlice.java | 29 +- .../detection/{ => spi/model}/EventSlice.java | 4 +- .../thirdeye/detection/spi/model/InputData.java | 118 + .../detection/spi/model/InputDataSpec.java | 177 + .../thirdeye/detection/spi/model/TimeSeries.java | 64 + .../detection/wrapper/AnomalyDetectorWrapper.java | 214 + .../AnomalyFilterWrapper.java} | 50 +- .../BaselineFillingMergeWrapper.java | 69 +- .../ChildKeepingMergeWrapper.java | 18 +- .../yaml/CompositePipelineConfigTranslator.java | 324 +- .../thirdeye/detection/yaml/DetectionRegistry.java | 75 - .../yaml/YamlDetectionAlertConfigTranslator.java | 2 + .../yaml/YamlDetectionConfigTranslator.java | 67 +- .../yaml/YamlDetectionTranslatorLoader.java | 18 +- .../thirdeye/detection/yaml/YamlResource.java | 106 +- .../detection/yaml/YamlTranslationResult.java | 84 + .../detector/email/filter/BaseAlertFilter.java | 2 +- .../thirdeye/rootcause/impl/MetricEntity.java | 1 - .../rootcause/impl/RCAFrameworkRunner.java | 3 - .../dashboard/views/admin/thirdeye-admin.ftl | 4 +- .../thirdeye/detector/holiday-anomaly-report.ftl | 144 +- .../com/linkedin/thirdeye/detector/lib/utils.ftl | 12 + .../src/main/resources/schema/create-schema.sql | 2 +- .../com/linkedin/thirdeye/TestDBResources.java | 134 - ...HierarchicalAnomaliesEmailContentFormatter.java | 10 + ...TestMultipleAnomaliesEmailContentFormatter.java | 11 + ...TestOnboardingNotificationContentFormatter.java | 10 + .../AutoOnboardPinotMetricsServiceTest.java | 14 + .../dashboard/resource/OnboardResourceTest.java | 8 +- .../linkedin/thirdeye/dataframe/DataFrameTest.java | 14 + .../thirdeye/datalayer/bao/DAOTestBase.java | 9 +- .../thirdeye/detection/DataProviderTest.java | 16 +- .../detection/DefaultInputDataFetcherTest.java | 54 + .../thirdeye/detection/DetectionTestUtils.java | 11 +- .../thirdeye/detection/MockDataProvider.java | 37 +- .../alert/DetectionAlertTaskFactoryTest.java | 40 +- .../thirdeye/detection/alert/SendAlertTest.java | 23 +- .../filter/DimensionDetectionAlertFilterTest.java | 44 +- .../alert/filter/LegacyAlertFilterTest.java | 55 +- .../ToAllRecipientsDetectionAlertFilterTest.java | 47 +- .../alert/scheme/AnotherRandomAlerter.java | 5 +- .../detection/alert/scheme/RandomAlerter.java | 5 +- .../DetectionTimeWindowSuppressorTest.java | 157 + .../detection/algorithm/BaselineAlgorithmTest.java | 7 +- .../LegacyAnomalyFunctionAlgorithmTest.java | 1 - .../algorithm/MovingWindowAlgorithmTest.java | 3 +- .../stage/AnomalyDetectionStageWrapperTest.java | 87 + .../stage/BaselineRuleDetectionStageTest.java | 1 - .../detection/baseline/MockBaselineProvider.java | 50 - .../AbsoluteChangeRuleAnomalyFilterTest.java | 90 + .../components/AbsoluteChangeRuleDetectorTest.java | 94 + .../detection/components/MockBaselineProvider.java | 45 + .../PercentageChangeRuleAnomalyFilterTest.java | 89 + .../PercentageChangeRuleDetectorTest.java} | 117 +- .../RuleBaselineProviderTest.java | 21 +- .../SitewideImpactRuleAnomalyFilterTest.java | 105 + .../components/ThresholdRuleAnomalyFilterTest.java | 170 + .../ThresholdRuleDetectorTest.java} | 48 +- .../finetune/GridSearchTuningAlgorithmTest.java | 4 +- .../MergeDimensionThresholdIntegrationTest.java | 6 +- .../thirdeye/detection/spec/AbstractSpecTest.java | 69 + .../detection/spec/MockBaselineProviderSpec.java | 43 + .../linkedin/thirdeye/detection/spec/TestSpec.java | 70 + .../wrapper/AnomalyDetectorWrapperTest.java | 92 + .../BaselineFillingMergeWrapperTest.java | 20 +- .../ChildKeepingMergeWrapperTest.java | 15 +- .../CompositePipelineConfigTranslatorTest.java | 183 +- .../yaml/MockYamlDetectionConfigTranslator.java | 8 +- .../YamlDetectionAlertConfigTranslatorTest.java | 6 +- .../yaml/YamlDetectionConfigTranslatorTest.java | 7 +- .../eventprovider/TestHolidayEventProvider.java | 1 + .../tools/anomaly/report/AnomalyReportDriver.java | 7 +- .../anomaly/report/GenerateAnomalyReport.java | 6 +- .../compositePipelineTranslatorTestResult-1.json | 76 + .../compositePipelineTranslatorTestResult-2.json | 31 + .../thirdeye/detection/yaml/pipeline-config-1.yaml | 42 + .../thirdeye/detection/yaml/pipeline-config-2.yaml | 22 + .../thirdeye/detection/yaml/pipeline-config.yaml | 29 - ...multiple-anomalies-email-content-formatter.html | 68 +- 695 files changed, 23153 insertions(+), 30965 deletions(-) create mode 100644 docs/High-level-stream.png create mode 100644 docs/Low-level-stream.png create mode 100644 docs/Makefile create mode 100644 docs/Pinot - Hybrid flow.png create mode 100644 docs/Pinot-Offline-only-flow.png create mode 100644 docs/PlanNode.png create mode 100644 docs/ServerSegmentCompletion.dot.png create mode 100644 docs/architecture.rst create mode 100644 docs/client_api.rst create mode 100644 docs/commit-happy-path-1.png create mode 100644 docs/commit-happy-path-2.png create mode 100644 docs/committer-failed.png create mode 100644 docs/conf.py create mode 100644 docs/controller-failed.png create mode 100644 docs/controller-segment-completion.png create mode 100644 docs/delayed-server.png create mode 100644 docs/docutils.conf create mode 100644 docs/expressionTree.jpg create mode 100644 docs/expressions_udf.rst create mode 100644 docs/hlc-arch.png create mode 100644 docs/image2014-11-12-1-54-56.png create mode 100644 docs/image2014-11-12-19-54-12.png create mode 100644 docs/image2014-11-12-20-59-50.png create mode 100644 docs/image2014-11-13-11-21-7.png create mode 100644 docs/image2014-11-13-11-22-57.png create mode 100644 docs/image2015-4-14-194451.png create mode 100644 docs/image2015-5-12-7-24-58.png create mode 100644 docs/image2015-5-13 15-2-9.png create mode 100644 docs/image2015-5-17 13-32-28.png create mode 100644 docs/image2015-5-17 17-59-10.png create mode 100644 docs/image2015-5-19 0-29-34.png create mode 100644 docs/image2015-5-19 0-58-54.png create mode 100644 docs/image2015-5-19 1-28-35.png create mode 100644 docs/image2015-5-19 1-47-44.png create mode 100644 docs/image2015-5-19 1-59-51.png create mode 100644 docs/in_production.rst create mode 100644 docs/index.rst create mode 100644 docs/intro.rst create mode 100644 docs/llc-arch.png create mode 100644 docs/llc.rst create mode 100644 docs/make.bat create mode 100644 docs/management_api.rst create mode 100644 docs/multiple-server-failure.png create mode 100644 docs/multitenancy.rst create mode 100644 docs/parseTree.png create mode 100644 docs/partition_aware_routing.rst create mode 100644 docs/pinot-architecture.png create mode 100644 docs/pinot_hadoop.rst create mode 100644 docs/pluggable_streams.png create mode 100644 docs/pluggable_streams.rst create mode 100644 docs/pql_examples.rst create mode 100644 docs/reference.rst create mode 100644 docs/schema_timespec.rst create mode 100644 docs/segment-consumer-fsm.png create mode 100644 docs/segment-creation.png create mode 100644 docs/segment-helix-fsm.png create mode 100644 docs/segment_fetcher.rst create mode 100644 docs/trying_pinot.rst create mode 100644 docs/zk-setup.png create mode 100644 docs/znode_layout.png create mode 100644 pinot-broker/src/main/java/com/linkedin/pinot/broker/broker/helix/TimeboundaryRefreshMessageHandlerFactory.java rename pinot-broker/src/main/java/com/linkedin/pinot/broker/routing/builder/{KafkaHighLevelConsumerBasedRoutingTableBuilder.java => HighLevelConsumerBasedRoutingTableBuilder.java} (52%) delete mode 100644 pinot-broker/src/main/java/com/linkedin/pinot/broker/routing/builder/KafkaLowLevelConsumerRoutingTableBuilder.java create mode 100644 pinot-broker/src/main/java/com/linkedin/pinot/broker/routing/builder/LowLevelConsumerRoutingTableBuilder.java rename pinot-broker/src/main/java/com/linkedin/pinot/broker/routing/builder/{KafkaLowLevelRoutingTableBuilderUtil.java => LowLevelRoutingTableBuilderUtil.java} (78%) rename pinot-broker/src/test/java/com/linkedin/pinot/broker/routing/builder/{KafkaHighLevelConsumerRoutingTableBuilderTest.java => HighLevelConsumerRoutingTableBuilderTest.java} (93%) rename pinot-broker/src/test/java/com/linkedin/pinot/broker/routing/builder/{KafkaLowLevelConsumerRoutingTableBuilderTest.java => LowLevelConsumerRoutingTableBuilderTest.java} (77%) create mode 100644 pinot-broker/src/test/java/com/linkedin/pinot/broker/routing/builder/RoutingTableBuilderTestUtil.java create mode 100644 pinot-common/src/main/java/com/linkedin/pinot/common/config/StarTreeIndexConfig.java create mode 100644 pinot-common/src/main/java/com/linkedin/pinot/common/lineage/SegmentGroup.java create mode 100644 pinot-common/src/main/java/com/linkedin/pinot/common/lineage/SegmentMergeLineage.java create mode 100644 pinot-common/src/main/java/com/linkedin/pinot/common/lineage/SegmentMergeLineageAccessHelper.java create mode 100644 pinot-common/src/main/java/com/linkedin/pinot/common/messages/TimeboundaryRefreshMessage.java delete mode 100644 pinot-common/src/main/java/com/linkedin/pinot/common/utils/time/DefaultTimeConverter.java create mode 100644 pinot-common/src/test/java/com/linkedin/pinot/common/lineage/SegmentMergeLineageTest.java delete mode 100644 pinot-common/src/test/java/com/linkedin/pinot/common/utils/time/DefaultTimeConverterTest.java create mode 100644 pinot-common/src/test/java/com/linkedin/pinot/common/utils/time/TimeConverterTest.java create mode 100644 pinot-controller/src/main/java/com/linkedin/pinot/controller/helix/core/periodictask/ControllerPeriodicTask.java create mode 100644 pinot-controller/src/test/java/com/linkedin/pinot/controller/helix/core/periodictask/ControllerPeriodicTaskTest.java create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/bloom/BloomFilter.java create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/bloom/BloomFilterType.java create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/bloom/BloomFilterUtil.java create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/bloom/GuavaOnHeapBloomFilter.java create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/bloom/SegmentBloomFilterFactory.java delete mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/data/extractors/PlainFieldExtractor.java create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/data/recordtransformer/CompoundTransformer.java create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/data/recordtransformer/DataTypeTransformer.java create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/data/recordtransformer/ExpressionTransformer.java rename pinot-core/src/main/java/com/linkedin/pinot/core/data/{extractors => recordtransformer}/PinotDataType.java (83%) rename pinot-core/src/main/java/com/linkedin/pinot/core/data/{extractors/FieldExtractor.java => recordtransformer/RecordTransformer.java} (58%) create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/data/recordtransformer/SanitationTransformer.java create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/data/recordtransformer/TimeTransformer.java delete mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/operator/docidsets/BitmapBasedBlockDocIdSet.java create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/periodictask/BasePeriodicTask.java copy pinot-core/src/main/java/com/linkedin/pinot/core/{realtime/stream/StreamLevelConsumer.java => periodictask/PeriodicTask.java} (50%) create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/periodictask/PeriodicTaskScheduler.java copy pinot-core/src/main/java/com/linkedin/pinot/core/realtime/impl/kafka/{SimpleConsumerFactory.java => KafkaConsumerFactory.java} (87%) create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/realtime/stream/OffsetCriteria.java create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/segment/creator/impl/bloom/BloomFilterCreator.java create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/segment/index/loader/bloomfilter/BloomFilterHandler.java create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/segment/index/readers/BloomFilterReader.java rename pinot-core/src/main/java/com/linkedin/pinot/core/segment/{ => name}/DefaultSegmentNameGenerator.java (99%) create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/segment/name/NormalizedDateSegmentNameGenerator.java copy pinot-core/src/main/java/com/linkedin/pinot/core/segment/{ => name}/SegmentNameGenerator.java (86%) delete mode 100644 pinot-core/src/test/java/com/linkedin/pinot/core/data/extractors/PlainFieldExtractorTest.java create mode 100644 pinot-core/src/test/java/com/linkedin/pinot/core/data/recordtransformer/PinotDataTypeTest.java create mode 100644 pinot-core/src/test/java/com/linkedin/pinot/core/data/recordtransformer/RecordTransformerTest.java create mode 100644 pinot-core/src/test/java/com/linkedin/pinot/core/data/recordtransformer/TimeTransformerTest.java rename pinot-core/src/test/java/com/linkedin/pinot/{ => core}/operator/filter/AndFilterOperatorTest.java (77%) create mode 100644 pinot-core/src/test/java/com/linkedin/pinot/core/operator/filter/FilterOperatorUtilsTest.java rename pinot-core/src/test/java/com/linkedin/pinot/{ => core}/operator/filter/IntRangesTest.java (96%) rename pinot-core/src/test/java/com/linkedin/pinot/{ => core}/operator/filter/OrFilterOperatorTest.java (82%) create mode 100644 pinot-core/src/test/java/com/linkedin/pinot/core/operator/filter/TestFilterOperator.java create mode 100644 pinot-core/src/test/java/com/linkedin/pinot/core/periodictask/PeriodicTaskSchedulerTest.java create mode 100644 pinot-core/src/test/java/com/linkedin/pinot/core/realtime/stream/OffsetCriteriaTest.java create mode 100644 pinot-core/src/test/java/com/linkedin/pinot/core/segment/index/creator/BloomFilterCreatorTest.java rename pinot-core/src/test/java/com/linkedin/pinot/core/segment/{ => name}/DefaultSegmentNameGeneratorTest.java (99%) create mode 100644 pinot-core/src/test/java/com/linkedin/pinot/core/segment/name/NormalizedDateSegmentNameGeneratorTest.java delete mode 100644 pinot-core/src/test/java/com/linkedin/pinot/operator/filter/FilterOperatorTestUtils.java delete mode 100644 pinot-perf/src/main/java/com/linkedin/pinot/perf/PerfBenchmarkRunner.java delete mode 100644 pinot-perf/src/main/java/com/linkedin/pinot/perf/PerfBenchmarkTest.java create mode 100644 pinot-tools/src/main/java/com/linkedin/pinot/tools/segment/converter/SegmentMergeCommand.java create mode 100644 thirdeye/thirdeye-frontend/app/adapters/share-config.js delete mode 100644 thirdeye/thirdeye-frontend/app/models/.gitkeep create mode 100644 thirdeye/thirdeye-frontend/app/models/share-config.js create mode 100644 thirdeye/thirdeye-frontend/app/pods/components/share-custom-template/component.js create mode 100644 thirdeye/thirdeye-frontend/app/pods/components/share-custom-template/template.hbs create mode 100644 thirdeye/thirdeye-frontend/app/pods/partials/rootcause/select-metric-label/template.hbs delete mode 100644 thirdeye/thirdeye-frontend/app/pods/rootcause2/controller.js delete mode 100644 thirdeye/thirdeye-frontend/app/pods/rootcause2/route.js delete mode 100644 thirdeye/thirdeye-frontend/app/pods/rootcause2/template.hbs create mode 100644 thirdeye/thirdeye-frontend/app/pods/services/api/share-template-config/service.js create mode 100644 thirdeye/thirdeye-frontend/app/pods/services/rootcause-fetcher/service.js create mode 100644 thirdeye/thirdeye-frontend/app/serializers/share-config.js delete mode 100644 thirdeye/thirdeye-frontend/app/styles/pods/home/share-dashboard/share-dashboard.scss create mode 100644 thirdeye/thirdeye-frontend/app/styles/shared/_placeholders.scss delete mode 100644 thirdeye/thirdeye-frontend/package-lock.json create mode 100644 thirdeye/thirdeye-frontend/tests/integration/pods/components/entity-filter/component-test.js create mode 100644 thirdeye/thirdeye-frontend/tests/integration/pods/components/rootcause-anomaly/component-test.js create mode 100644 thirdeye/thirdeye-frontend/tests/integration/pods/components/share-custom-template/component-test.js create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/DefaultInputDataFetcher.java copy thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/{DetectionPipeline.java => DetectionUtils.java} (54%) delete mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/InputData.java rename pinot-core/src/main/java/com/linkedin/pinot/core/segment/SegmentNameGenerator.java => thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/InputDataFetcher.java (62%) delete mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/InputDataSpec.java copy thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/{datalayer/dto/DetectionConfigDTO.java => detection/Pattern.java} (78%) create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/alert/suppress/DetectionAlertSuppressor.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/alert/suppress/DetectionAlertTimeWindowSuppressor.java delete mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/algorithm/stage/StageUtils.java delete mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/algorithm/stage/StaticGrouperStage.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/annotation/Components.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/annotation/DetectionConfigurationResource.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/annotation/DetectionRegistry.java copy thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/{datalayer/dto/DetectionConfigDTO.java => detection/annotation/DetectionTag.java} (78%) create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/annotation/Param.java copy pinot-core/src/main/java/com/linkedin/pinot/core/common/DataSource.java => thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/annotation/PresentationOption.java (53%) copy pinot-core/src/main/java/com/linkedin/pinot/core/io/util/ValueReader.java => thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/annotation/Tune.java (50%) rename pinot-core/src/main/java/com/linkedin/pinot/core/data/extractors/FieldExtractorFactory.java => thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/annotation/Yaml.java (54%) delete mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/baseline/BaselineProvider.java delete mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/baseline/BaselineProviderLoader.java delete mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/baseline/RuleBaselineProvider.java delete mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/baseline/StaticBaselineProvider.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/components/AbsoluteChangeRuleAnomalyFilter.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/components/AbsoluteChangeRuleDetector.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/components/PercentageChangeRuleAnomalyFilter.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/components/PercentageChangeRuleDetector.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/components/RuleBaselineProvider.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/components/SitewideImpactRuleAnomalyFilter.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/components/ThresholdRuleAnomalyFilter.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/components/ThresholdRuleDetector.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spec/AbsoluteChangeRuleAnomalyFilterSpec.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spec/AbsoluteChangeRuleDetectorSpec.java rename pinot-common/src/main/java/com/linkedin/pinot/common/utils/time/TimeConverterProvider.java => thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spec/AbstractSpec.java (56%) create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spec/PercentageChangeRuleAnomalyFilterSpec.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spec/PercentageChangeRuleDetectorSpec.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spec/RuleBaselineProviderSpec.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spec/SitewideImpactRuleAnomalyFilterSpec.java copy pinot-core/src/test/java/com/linkedin/pinot/core/crypt/PinotCrypterFactoryTest.java => thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spec/ThresholdRuleDetectorSpec.java (56%) copy pinot-core/src/test/java/com/linkedin/pinot/core/crypt/PinotCrypterFactoryTest.java => thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spec/ThresholdRuleFilterSpec.java (56%) copy pinot-core/src/main/java/com/linkedin/pinot/core/realtime/stream/StreamLevelConsumer.java => thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spi/components/AnomalyDetector.java (50%) copy thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/{datalayer/dto/DetectionConfigDTO.java => detection/spi/components/AnomalyFilter.java} (57%) copy thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/{datalayer/dto/DetectionConfigDTO.java => detection/spi/components/BaseComponent.java} (69%) create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spi/components/BaselineProvider.java copy pinot-core/src/main/java/com/linkedin/pinot/core/operator/filter/BaseFilterOperator.java => thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spi/components/Grouper.java (53%) create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spi/components/Tunable.java rename thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/{ => spi/model}/AnomalySlice.java (69%) rename thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/{ => spi/model}/EventSlice.java (97%) create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spi/model/InputData.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spi/model/InputDataSpec.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/spi/model/TimeSeries.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/wrapper/AnomalyDetectorWrapper.java copy thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/{algorithm/stage/AnomalyFilterStageWrapper.java => wrapper/AnomalyFilterWrapper.java} (64%) rename thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/{algorithm => wrapper}/BaselineFillingMergeWrapper.java (54%) rename thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/{algorithm => wrapper}/ChildKeepingMergeWrapper.java (84%) delete mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/yaml/DetectionRegistry.java create mode 100644 thirdeye/thirdeye-pinot/src/main/java/com/linkedin/thirdeye/detection/yaml/YamlTranslationResult.java create mode 100644 thirdeye/thirdeye-pinot/src/main/resources/com/linkedin/thirdeye/detector/lib/utils.ftl delete mode 100644 thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/TestDBResources.java create mode 100644 thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/DefaultInputDataFetcherTest.java create mode 100644 thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/alert/suppress/DetectionTimeWindowSuppressorTest.java create mode 100644 thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/algorithm/stage/AnomalyDetectionStageWrapperTest.java delete mode 100644 thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/baseline/MockBaselineProvider.java create mode 100644 thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/components/AbsoluteChangeRuleAnomalyFilterTest.java create mode 100644 thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/components/AbsoluteChangeRuleDetectorTest.java create mode 100644 thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/components/MockBaselineProvider.java create mode 100644 thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/components/PercentageChangeRuleAnomalyFilterTest.java copy thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/{algorithm/stage/BaselineRuleDetectionStageTest.java => components/PercentageChangeRuleDetectorTest.java} (50%) rename thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/{baseline => components}/RuleBaselineProviderTest.java (77%) create mode 100644 thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/components/SitewideImpactRuleAnomalyFilterTest.java create mode 100644 thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/components/ThresholdRuleAnomalyFilterTest.java copy thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/{algorithm/LegacyAnomalyFunctionAlgorithmTest.java => components/ThresholdRuleDetectorTest.java} (68%) create mode 100644 thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/spec/AbstractSpecTest.java create mode 100644 thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/spec/MockBaselineProviderSpec.java create mode 100644 thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/spec/TestSpec.java create mode 100644 thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/wrapper/AnomalyDetectorWrapperTest.java rename thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/{algorithm => wrapper}/BaselineFillingMergeWrapperTest.java (82%) rename thirdeye/thirdeye-pinot/src/test/java/com/linkedin/thirdeye/detection/{algorithm => wrapper}/ChildKeepingMergeWrapperTest.java (96%) create mode 100644 thirdeye/thirdeye-pinot/src/test/resources/com/linkedin/thirdeye/detection/yaml/compositePipelineTranslatorTestResult-1.json create mode 100644 thirdeye/thirdeye-pinot/src/test/resources/com/linkedin/thirdeye/detection/yaml/compositePipelineTranslatorTestResult-2.json create mode 100644 thirdeye/thirdeye-pinot/src/test/resources/com/linkedin/thirdeye/detection/yaml/pipeline-config-1.yaml create mode 100644 thirdeye/thirdeye-pinot/src/test/resources/com/linkedin/thirdeye/detection/yaml/pipeline-config-2.yaml delete mode 100644 thirdeye/thirdeye-pinot/src/test/resources/com/linkedin/thirdeye/detection/yaml/pipeline-config.yaml --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org