This is an automated email from the ASF dual-hosted git repository. kishoreg pushed a change to branch h3-index in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard fe0e286 Initial commit for H3 based geospatial indexing add 0dee4d1 [TE] fix changing createdTime of anomalies (#6269) add 3eb0f9c Use sorted index based filtering only for dictionary encoded column (#6288) add b31b82e [TE] frontend - harleyjj/alert-details - change alert charting logic to be agnostic of whether anomaly timestamps are included in data timestamps (#6260) add d6862a2 Adding custom metadata props into both segment metadata properties file and zk metadata record (#6299) add 5b0d1df [TE] frontend harleyjj/rca - double encode filter value to protect against special characters (#6281) add 0565f86 Make plugins to be configurable in environment variables (#6294) add 66e9394 Improve performance for distinct queries (#6285) add 3b7bfd2 [TE] frontend - rca/harleyjj - Enable forecast baseline in RCA UI (#6304) add 4be939a some geo function improvements (#6306) add 33e7ec8 [TE](feat): add formatting pipeline for js, hbs code (#6245) add 47a30ba [TE]frontend - Build new subroutes for single-metric-anomalies and composite-anomalies (#6263) add e8ceb2e [TE] rest-api - harleyjj/rca - add forecast as baseline option for metric/timeseries endpoint (#6265) add f898c18 support to add offline and realtime tables, individually able to add schema and schema listing in UI (#6296) add 9ce5d78 Include exception message in the ControllerApplicationException for /validate (#6312) add 3cf3154 [TE]frontend - Build the tree parser for composite anomalies (#6290) add 9a5cc4b Use StringUtils.replace to avoid regex for setting literal expression (#6314) add 41a7722 Fix: Close HelixAdmin appropriately, when setting up Helix Cluster. (#6315) add 8ecfd4a Update pinot_tests.yml (#6316) add 41a3fc4 API to get status of consumption of a table (#6322) add 70be687 Enhance JSONRecordReader to handle GZIP compressed JSON files. (#6321) add 682c95f Decimal percentile support. (#6323) add 22d25ff [TE]frontend - Refactor to integrate the Performance Stats API into the front-end (#6310) add fe2a63c use RoaringBitmapWriter and direct to ByteBuffer serialization in BitmapInvertedIndexCreators (#6320) add ed9f122 fixing bugs in bootstrap table tool (#6335) add a6446e9 [Controller UI] show cluster name (#6338) add c124334 Adding Pinot Minion client (#6339) add d6484f6 Added proper tooltips, ability to enable-disable table state and fixed page crash on reload status (#6327) add 7c0e22d Update .travis.yml (#6342) add 4ba7204 Fix table cache in pinot-broker (#6329) add 64d1054 Adding offline dim table creation and assignment (#6286) add 75f9fd3 Add a Controller endpoint to return table creation time (#6331) add f2c37d5 Creating a pluggable interface for Table config tuner (#6255) add e691a38 Fix the logging bug in star-tree builder (#6348) add 2796b83 Adding json path functions to extract values from json object (#6347) add 4183ffe simplify batch config and corresponding utils (#6332) add bdeec8d Fixing the issue with result schema (#6353) add 9644350 add redis module dependency (#6357) add 946ff55 [TE] fix javassist version (#6356) add fb42e72 [TE]frontend - Add breadcrumb component (#6350) add a76e766 Broker time range pruning(#6189) (#6259) add 5432099 [TE]frontend - Add a lightweight PubSub system (#6358) add 8e41708 6355 don't lose authority portion of inputDirURI (#6359) add 2ea0185 Enhance task schedule api for single type/table support (#6352) add 79b12ed Fix empty data table for distinct query (#6363) add 0522acc pinot-controller unit test suite. (#6326) add 003442a [TE]frontend - Fix the display for "NaN" values in performance-stats (#6365) add 39a24cf [TE]frontend - THIRDEYE-3772 - create a new 'composite-anomalies' component plus very simple unit integration test (#6369) add 75b0792 Fix mvn profile name for github-actions. (#6370) add 2a7d506 Improve and bug fix on json record extraction logic (#6372) new 4bb6677 Initial commit for H3 based geospatial indexing new 5b13496 Wiring H3 Index for query processing new e2ea4d6 Adding a quickstart This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (fe0e286) \ N -- N -- N refs/heads/h3-index (e2ea4d6) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .github/workflows/pinot_tests.yml | 6 + .github/workflows/scripts/.pinot_test.sh | 4 +- .gitignore | 2 + .travis.yml | 14 + docker/images/pinot-superset/requirements-db.txt | 1 + .../requesthandler/BaseBrokerRequestHandler.java | 2 +- .../pinot/broker/routing/RoutingManager.java | 4 +- .../instanceselector/BaseInstanceSelector.java | 2 - .../segmentpruner/PartitionSegmentPruner.java | 7 +- .../routing/segmentpruner/SegmentPruner.java | 3 +- .../segmentpruner/SegmentPrunerFactory.java | 50 +- .../routing/segmentpruner/TimeSegmentPruner.java | 346 ++ .../routing/segmentpruner/interval/Interval.java | 104 + .../segmentpruner/interval/IntervalTree.java | 201 + .../segmentselector/OfflineSegmentSelector.java | 8 +- .../segmentselector/RealtimeSegmentSelector.java | 31 +- .../routing/segmentselector/SegmentSelector.java | 3 +- .../broker/routing/IntervalST/IntervalTest.java | 57 + .../routing/IntervalST/IntervalTreeTest.java | 89 + .../routing/segmentpruner/SegmentPrunerTest.java | 285 +- pinot-common/pom.xml | 4 + .../config/tuner/NoOpTableTableConfigTuner.java | 22 +- .../config/tuner/RealTimeAutoIndexTuner.java | 51 + .../config/tuner/TableConfigTunerRegistry.java | 77 + .../common/function/scalar/JsonFunctions.java | 88 + .../apache/pinot/common/minion/MinionClient.java | 122 + .../common/minion/MinionRequestURLBuilder.java | 93 + .../restlet/resources/SegmentConsumerInfo.java | 61 + .../common/utils/config/TableConfigUtils.java | 15 +- .../pinot/common/utils/request/RequestUtils.java | 13 +- .../pinot/pql/parsers/pql2/ast/SelectAstNode.java | 4 +- .../config/tuner/RealTimeAutoIndexTunerTest.java | 76 + .../common/config/tuner/TunerRegistryTest.java | 55 + .../pinot/common/function/JsonFunctionsTest.java | 68 + .../common/function/TransformFunctionTypeTest.java | 7 + .../common/metadata/SegmentZKMetadataTest.java | 3 + .../pinot/common/minion/MinionClientTest.java | 91 + .../common/utils/config/TableConfigSerDeTest.java | 22 + pinot-controller/pom.xml | 16 + .../api/resources/PinotSegmentRestletResource.java | 29 + .../api/resources/PinotTableRestletResource.java | 68 +- .../api/resources/PinotTaskRestletResource.java | 16 +- .../helix/core/PinotHelixResourceManager.java | 31 + .../segment/OfflineDimTableSegmentAssignment.java | 93 + .../segment/SegmentAssignmentFactory.java | 2 +- .../assignment/segment/SegmentAssignmentUtils.java | 1 - .../helix/core/minion/PinotTaskManager.java | 169 +- .../helix/core/util/HelixSetupUtils.java | 39 +- .../helix/core/util/ZKMetadataUtils.java | 5 + .../util/ConsumingSegmentInfoReader.java | 167 + pinot-controller/src/main/resources/app/App.tsx | 4 +- .../main/resources/app/components/AppLoader.tsx | 2 +- .../main/resources/app/components/Breadcrumbs.tsx | 15 +- .../src/main/resources/app/components/Confirm.tsx | 2 +- .../main/resources/app/components/CustomDialog.tsx | 2 +- .../src/main/resources/app/components/Header.tsx | 63 +- .../Homepage/Operations/AddDeleteComponent.tsx | 177 + .../Homepage/Operations/AddIndexingComponent.tsx | 162 + .../Homepage/Operations/AddIngestionComponent.tsx | 173 + ...{AddTableSchemaOp.tsx => AddOfflineTableOp.tsx} | 236 +- .../Operations/AddOfflineTenantComponent.tsx | 151 + .../Homepage/Operations/AddPartionComponent.tsx | 263 + .../Homepage/Operations/AddQueryComponent.tsx | 91 + .../Operations/AddRealTimeIngestionComponent.tsx | 172 + .../Operations/AddRealTimePartionComponent.tsx | 234 + .../{AddTableOp.tsx => AddRealtimeTableOp.tsx} | 162 +- .../components/Homepage/Operations/AddSchemaOp.tsx | 202 + .../Homepage/Operations/AddStorageComponent.tsx | 118 + .../Homepage/Operations/AddTableComponent.tsx | 15 +- .../Homepage/Operations/AddTenantComponent.tsx | 175 + .../Homepage/Operations/MultiIndexingComponent.tsx | 350 ++ .../Homepage/Operations/MultiMetricComponent.tsx | 182 + .../Operations/MultipleSelectComponent.tsx | 188 + .../Homepage/Operations/RebalanceServerTableOp.tsx | 5 +- .../Homepage/Operations/ReloadStatusOp.tsx | 86 +- .../Homepage/Operations/SchemaComponent.tsx | 44 +- .../Homepage/Operations/SchemaNameComponent.tsx | 104 + .../resources/app/components/SimpleAccordion.tsx | 12 +- .../src/main/resources/app/components/Table.tsx | 39 +- .../src/main/resources/app/interfaces/types.d.ts | 8 + .../main/resources/app/pages/InstanceDetails.tsx | 9 +- .../{SegmentDetails.tsx => SchemaPageDetails.tsx} | 234 +- .../main/resources/app/pages/SegmentDetails.tsx | 2 +- .../main/resources/app/pages/TablesListingPage.tsx | 74 +- .../src/main/resources/app/pages/TenantDetails.tsx | 47 +- .../src/main/resources/app/pages/Tenants.tsx | 17 +- .../src/main/resources/app/requests/index.ts | 6 + pinot-controller/src/main/resources/app/router.tsx | 3 + .../src/main/resources/app/styles/styles.css | 37 + .../main/resources/app/utils/PinotMethodUtils.ts | 50 +- .../pinot/controller/ControllerTestSetup.java | 51 + ...ontrollerTest.java => ControllerTestUtils.java} | 428 +- .../controller/LeadControllerManagerTest.java | 2 +- .../pinot/controller/api/AccessControlTest.java | 32 +- .../api/ConsumingSegmentInfoReaderTest.java | 319 + ...> PinotBrokerRestletResourceStatelessTest.java} | 2 +- .../pinot/controller/api/PinotFileUploadTest.java | 30 +- ...PinotInstanceAssignmentRestletResourceTest.java | 157 +- .../api/PinotInstanceRestletResourceTest.java | 95 +- .../api/PinotSchemaRestletResourceTest.java | 44 +- .../api/PinotSegmentRestletResourceTest.java | 41 +- .../api/PinotTableRestletResourceTest.java | 194 +- .../api/PinotTenantRestletResourceTest.java | 46 +- .../pinot/controller/api/TableViewsTest.java | 77 +- .../controller/api/upload/ZKOperatorTest.java | 64 +- .../helix/ControllerInstanceToggleTest.java | 79 +- ...ontrollerPeriodicTaskStarterStatelessTest.java} | 2 +- .../controller/helix/ControllerSentinelTestV2.java | 76 +- ...est.java => ControllerTenantStatelessTest.java} | 2 +- .../pinot/controller/helix/ControllerTest.java | 98 +- .../pinot/controller/helix/HelixHelperTest.java | 19 +- ....java => PinotControllerModeStatelessTest.java} | 14 +- .../controller/helix/PinotResourceManagerTest.java | 76 +- .../pinot/controller/helix/TableCacheTest.java | 119 +- .../PinotHelixResourceManagerStatelessTest.java | 227 + .../helix/core/PinotHelixResourceManagerTest.java | 403 +- .../OfflineDimTableSegmentAssignmentTest.java | 108 + ...fflineNonReplicaGroupSegmentAssignmentTest.java | 22 +- .../PinotLLCRealtimeSegmentManagerTest.java | 47 +- .../helix/core/realtime/SegmentCompletionTest.java | 502 +- ...va => TableRebalancerClusterStatelessTest.java} | 2 +- .../core/retention/SegmentLineageCleanupTest.java | 77 +- .../validation/ValidationManagerStatelessTest.java | 120 + .../validation/ValidationManagerTest.java | 105 +- pinot-controller/testng-statefull.xml | 84 + pinot-controller/testng-stateless.xml | 42 + .../org/apache/pinot/core/common/DataSource.java | 9 + .../apache/pinot/core/common/ObjectSerDeUtils.java | 6 +- .../core/common/datatable/DataTableUtils.java | 95 +- .../manager/config/TableDataManagerConfig.java | 8 +- .../realtime/HLRealtimeSegmentDataManager.java | 16 + .../realtime/LLRealtimeSegmentDataManager.java | 17 + .../realtime/RealtimeSegmentDataManager.java | 21 + .../recordtransformer/DataTypeTransformer.java | 12 +- .../pinot/core/geospatial/GeometryUtils.java | 16 + .../transform/function/ScalarFunctions.java | 18 + .../transform/function/StContainsFunction.java | 12 +- .../transform/function/StDistanceFunction.java | 11 +- .../transform/function/StPointFunction.java | 18 +- .../indexsegment/mutable/MutableSegmentImpl.java | 12 +- .../operator/combine/DistinctCombineOperator.java | 84 + .../core/operator/filter/FilterOperatorUtils.java | 10 +- .../operator/filter/H3IndexFilterOperator.java | 65 + .../core/operator/query/DistinctOperator.java | 84 + .../function/LiteralTransformFunction.java | 8 +- .../core/periodictask/PeriodicTaskScheduler.java | 12 + .../apache/pinot/core/plan/CombinePlanNode.java | 6 +- .../apache/pinot/core/plan/DistinctPlanNode.java | 52 + .../org/apache/pinot/core/plan/FilterPlanNode.java | 17 +- .../core/plan/maker/InstancePlanMakerImplV2.java | 12 +- .../function/AggregationFunctionFactory.java | 25 +- .../function/DistinctAggregationFunction.java | 117 +- .../function/PercentileAggregationFunction.java | 19 +- .../function/PercentileEstAggregationFunction.java | 19 +- .../PercentileEstMVAggregationFunction.java | 12 +- .../function/PercentileMVAggregationFunction.java | 11 +- .../PercentileTDigestAggregationFunction.java | 20 +- .../PercentileTDigestMVAggregationFunction.java | 12 +- .../DistinctExecutor.java} | 26 +- .../query/distinct/DistinctExecutorFactory.java | 190 + .../customobject => distinct}/DistinctTable.java | 254 +- ...DictionaryBasedMultiColumnDistinctExecutor.java | 98 + ...ictionaryBasedSingleColumnDistinctExecutor.java | 72 + ...ionaryBasedMultiColumnDistinctOnlyExecutor.java | 59 + ...aryBasedMultiColumnDistinctOrderByExecutor.java | 95 + ...onaryBasedSingleColumnDistinctOnlyExecutor.java | 51 + ...ryBasedSingleColumnDistinctOrderByExecutor.java | 71 + .../BaseRawBytesSingleColumnDistinctExecutor.java | 60 + .../BaseRawDoubleSingleColumnDistinctExecutor.java | 61 + .../BaseRawFloatSingleColumnDistinctExecutor.java | 61 + .../BaseRawIntSingleColumnDistinctExecutor.java | 61 + .../BaseRawLongSingleColumnDistinctExecutor.java | 61 + .../BaseRawStringSingleColumnDistinctExecutor.java | 59 + .../RawBytesSingleColumnDistinctOnlyExecutor.java | 50 + ...awBytesSingleColumnDistinctOrderByExecutor.java | 71 + .../RawDoubleSingleColumnDistinctOnlyExecutor.java | 49 + ...wDoubleSingleColumnDistinctOrderByExecutor.java | 70 + .../RawFloatSingleColumnDistinctOnlyExecutor.java | 49 + ...awFloatSingleColumnDistinctOrderByExecutor.java | 70 + .../RawIntSingleColumnDistinctOnlyExecutor.java | 49 + .../RawIntSingleColumnDistinctOrderByExecutor.java | 70 + .../RawLongSingleColumnDistinctOnlyExecutor.java | 49 + ...RawLongSingleColumnDistinctOrderByExecutor.java | 70 + .../raw/RawMultiColumnDistinctExecutor.java | 86 + .../RawStringSingleColumnDistinctOnlyExecutor.java | 49 + ...wStringSingleColumnDistinctOrderByExecutor.java | 70 + .../query/pruner/SelectionQuerySegmentPruner.java | 4 +- .../query/reduce/DistinctDataTableReducer.java | 4 +- .../request/context/predicate/GeoPredicate.java | 22 + .../request/context/utils/QueryContextUtils.java | 23 +- .../query/selection/SelectionOperatorUtils.java | 2 +- .../core/segment/creator/impl/V1Constants.java | 3 + .../creator/impl/geospatial/H3IndexCreator.java | 4 +- .../inv/OffHeapBitmapInvertedIndexCreator.java | 59 +- .../impl/inv/OnHeapBitmapInvertedIndexCreator.java | 63 +- .../segment/index/column/ColumnIndexContainer.java | 6 + .../index/column/PhysicalColumnIndexContainer.java | 15 + .../segment/index/datasource/BaseDataSource.java | 11 +- .../index/datasource/ImmutableDataSource.java | 2 +- .../index/datasource/MutableDataSource.java | 7 +- .../segment/index/loader/IndexLoadingConfig.java | 10 + .../segment/index/loader/SegmentPreProcessor.java | 6 + .../index/loader/invertedindex/H3IndexHandler.java | 205 + .../segment/index/metadata/SegmentMetadata.java | 3 + .../index/metadata/SegmentMetadataImpl.java | 28 + .../index/readers/geospatial/H3IndexReader.java | 5 +- .../pinot/core/segment/store/ColumnIndexType.java | 3 +- .../virtualcolumn/VirtualColumnIndexContainer.java | 6 + .../startree/v2/builder/BaseSingleTreeBuilder.java | 2 +- .../core/startree/v2/store/StarTreeDataSource.java | 3 +- .../apache/pinot/core/util/TableConfigUtils.java | 3 +- .../core/common/datatable/DataTableUtilsTest.java | 28 +- .../transform/StDistanceFunctionTest.java | 8 +- .../geospatial/transform/StPointFunctionTest.java | 27 +- .../function/AggregationFunctionFactoryTest.java | 76 +- .../BrokerRequestToQueryContextConverterTest.java | 30 +- .../segment/index/SegmentMetadataImplTest.java | 4 + .../pinot/core/util/TableConfigUtilsTest.java | 19 +- .../apache/pinot/queries/DistinctQueriesTest.java | 1089 ++-- ...erSegmentAggregationSingleValueQueriesTest.java | 48 +- ...terSegmentResultTableMultiValueQueriesTest.java | 39 +- ...erSegmentResultTableSingleValueQueriesTest.java | 11 +- .../queries/PercentileTDigestMVQueriesTest.java | 14 +- .../queries/PercentileTDigestQueriesTest.java | 73 +- .../RangePredicateWithSortedInvertedIndexTest.java | 32 +- ...vertToRawIndexMinionClusterIntegrationTest.java | 6 +- .../tests/HybridClusterIntegrationTest.java | 2 +- .../tests/OfflineClusterIntegrationTest.java | 18 +- ...fflineSegmentsMinionClusterIntegrationTest.java | 6 +- .../tests/SimpleMinionClusterIntegrationTest.java | 17 +- ...BenchmarkOffheapBitmapInvertedIndexCreator.java | 105 + .../batch/common/SegmentGenerationTaskRunner.java | 1 + .../batch/common/SegmentGenerationUtilsTest.java | 16 + .../batch/hadoop/HadoopSegmentCreationMapper.java | 2 + .../spark/SparkSegmentGenerationJobRunner.java | 2 + .../standalone/SegmentGenerationJobRunner.java | 12 +- .../standalone/SegmentGenerationJobRunnerTest.java | 54 + .../src/test/resources/log4j2.xml | 0 .../inputformat/avro/AvroRecordExtractor.java | 6 +- .../plugin/inputformat/csv/CSVRecordExtractor.java | 3 +- .../inputformat/json/JSONRecordExtractor.java | 6 +- .../plugin/inputformat/json/JSONRecordReader.java | 11 +- .../protobuf/ProtoBufRecordExtractor.java | 6 +- .../inputformat/thrift/ThriftRecordExtractor.java | 6 +- .../pinot/server/api/resources/TablesResource.java | 41 + pinot-spi/pom.xml | 4 + .../pinot/spi/config/table/IndexingConfig.java | 9 + .../pinot/spi/config/table/RoutingConfig.java | 1 + .../apache/pinot/spi/config/table/TableConfig.java | 24 +- .../apache/pinot/spi/config/table/TableStats.java | 29 +- .../apache/pinot/spi/config/table/TunerConfig.java | 61 + .../spi/config/table/tuner/TableConfigTuner.java | 24 +- .../apache/pinot/spi/config/table/tuner/Tuner.java | 29 +- .../pinot/spi/filesystem/PinotFSFactory.java | 2 +- .../pinot/spi/ingestion/batch/BatchConfig.java | 83 +- .../spi/ingestion/batch/BatchConfigProperties.java | 48 +- .../batch/spec/SegmentGenerationTaskSpec.java | 26 + .../pinot/spi/utils/IngestionConfigUtils.java | 42 +- .../spi/utils/builder/TableConfigBuilder.java | 15 +- .../pinot/spi/ingestion/batch/BatchConfigTest.java | 133 +- .../org/apache/pinot/tools/BootstrapTableTool.java | 3 + .../org/apache/pinot/tools/GenericQuickstart.java | 145 + .../tools/admin/command/BootstrapTableCommand.java | 8 +- .../src/main/resources/appAssemblerScriptTemplate | 16 +- .../batch/starbucksStores/ingestionJobSpec.yaml | 139 + .../batch/starbucksStores/rawdata/data.csv | 6444 ++++++++++++++++++++ .../starbucksStores_offline_table_config.json | 27 + .../starbucksStores/starbucksStores_schema.json | 29 + .../meetupRsvp_realtime_table_config.json | 4 +- pom.xml | 4 +- thirdeye/pom.xml | 4 +- .../dashboard/ThirdEyeDashboardModule.java | 12 +- .../resources/v2/RootCauseMetricResource.java | 52 +- .../dashboard/resources/SummaryResourceTest.java | 8 + .../detection/algorithm/MergeWrapperTest.java | 19 +- thirdeye/thirdeye-frontend/.eslintrc.js | 34 +- thirdeye/thirdeye-frontend/.prettierignore | 7 + thirdeye/thirdeye-frontend/.prettierrc | 8 + .../app/mocks/compositeAnomalies.js | 256 + .../app/pods/components/alert-details/component.js | 1203 ++-- .../app/pods/components/alert-details/template.hbs | 35 +- .../components/alert-report-modal/template.hbs | 1 + .../pods/components/anomaly-summary/component.js | 259 +- .../pods/components/breadcrumb-list/component.js | 56 + .../pods/components/breadcrumb-list/template.hbs | 10 + .../components/composite-anomalies/component.js | 87 + .../parent-anomalies/component.js | 117 + .../parent-anomalies/template.hbs | 20 + .../components/composite-anomalies/template.hbs | 8 + .../rootcause-chart-toolbar/component.js | 49 +- .../rootcause-chart-toolbar/template.hbs | 1 + .../rootcause-select-comparison-range/component.js | 10 +- .../self-serve-alert-yaml-details/template.hbs | 3 +- .../app/pods/components/stats-cards/component.js | 110 +- .../app/pods/components/stats-cards/template.hbs | 43 +- .../anomalies-list/template.hbs | 3 + .../resolution/component.js | 15 + .../resolution/template.hbs | 11 + .../start-duration/template.hbs | 8 + .../app/pods/home/index/controller.js | 263 +- .../explore/composite-anomalies/controller.js | 139 + .../manage/explore/composite-anomalies/route.js | 28 + .../explore/composite-anomalies/template.hbs | 44 + .../app/pods/manage/explore/route.js | 15 + .../explore/single-metric-anomalies/controller.js | 4 + .../explore/single-metric-anomalies/route.js | 16 + .../explore/single-metric-anomalies/template.hbs | 20 + .../app/pods/manage/explore/template.hbs | 30 +- thirdeye/thirdeye-frontend/app/router.js | 5 +- thirdeye/thirdeye-frontend/app/styles/app.scss | 2 + .../app/styles/components/breadcrumb-list.scss | 19 + .../styles/pods/custom/parent-anomalies-table.scss | 21 + .../app/styles/shared/_styles.scss | 58 +- .../app/utils/anomalies-tree-parser.js | 541 ++ thirdeye/thirdeye-frontend/app/utils/anomaly.js | 14 + .../thirdeye-frontend/app/utils/api/anomaly.js | 12 + thirdeye/thirdeye-frontend/app/utils/constants.js | 5 +- .../app/utils/date-picker-utils.js | 66 + thirdeye/thirdeye-frontend/app/utils/pub-sub.js | 72 + thirdeye/thirdeye-frontend/app/utils/rca-utils.js | 8 +- thirdeye/thirdeye-frontend/package.json | 19 + .../components/breadcrumb-list/component-test.js | 33 + .../composite-anomalies/component-test.js | 20 + .../parent-anomalies/component-test.js | 69 + .../pods/components/stats-cards/component-test.js | 94 +- .../explore/composite-anomalies/route-test.js | 11 + .../explore/single-metric-anomalies/route-test.js | 11 + .../tests/unit/utils/anomalies-tree-parser-test.js | 255 + .../tests/unit/utils/date-picker-utils-test.js | 36 + .../tests/unit/utils/pub-sub-test.js | 19 + thirdeye/thirdeye-frontend/yarn.lock | 1698 +++++- thirdeye/thirdeye-pinot/pom.xml | 2 +- .../pinot/resources/PinotDataSourceResource.java | 10 +- .../thirdeye/detection/algorithm/MergeWrapper.java | 42 +- 334 files changed, 25041 insertions(+), 4161 deletions(-) create mode 100644 pinot-broker/src/main/java/org/apache/pinot/broker/routing/segmentpruner/TimeSegmentPruner.java create mode 100644 pinot-broker/src/main/java/org/apache/pinot/broker/routing/segmentpruner/interval/Interval.java create mode 100644 pinot-broker/src/main/java/org/apache/pinot/broker/routing/segmentpruner/interval/IntervalTree.java create mode 100644 pinot-broker/src/test/java/org/apache/pinot/broker/routing/IntervalST/IntervalTest.java create mode 100644 pinot-broker/src/test/java/org/apache/pinot/broker/routing/IntervalST/IntervalTreeTest.java copy pinot-core/src/main/java/org/apache/pinot/core/query/request/context/utils/QueryContextUtils.java => pinot-common/src/main/java/org/apache/pinot/common/config/tuner/NoOpTableTableConfigTuner.java (58%) create mode 100644 pinot-common/src/main/java/org/apache/pinot/common/config/tuner/RealTimeAutoIndexTuner.java create mode 100644 pinot-common/src/main/java/org/apache/pinot/common/config/tuner/TableConfigTunerRegistry.java create mode 100644 pinot-common/src/main/java/org/apache/pinot/common/minion/MinionClient.java create mode 100644 pinot-common/src/main/java/org/apache/pinot/common/minion/MinionRequestURLBuilder.java create mode 100644 pinot-common/src/main/java/org/apache/pinot/common/restlet/resources/SegmentConsumerInfo.java create mode 100644 pinot-common/src/test/java/org/apache/pinot/common/config/tuner/RealTimeAutoIndexTunerTest.java create mode 100644 pinot-common/src/test/java/org/apache/pinot/common/config/tuner/TunerRegistryTest.java create mode 100644 pinot-common/src/test/java/org/apache/pinot/common/function/JsonFunctionsTest.java create mode 100644 pinot-common/src/test/java/org/apache/pinot/common/minion/MinionClientTest.java create mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/OfflineDimTableSegmentAssignment.java create mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/util/ConsumingSegmentInfoReader.java create mode 100644 pinot-controller/src/main/resources/app/components/Homepage/Operations/AddDeleteComponent.tsx create mode 100644 pinot-controller/src/main/resources/app/components/Homepage/Operations/AddIndexingComponent.tsx create mode 100644 pinot-controller/src/main/resources/app/components/Homepage/Operations/AddIngestionComponent.tsx rename pinot-controller/src/main/resources/app/components/Homepage/Operations/{AddTableSchemaOp.tsx => AddOfflineTableOp.tsx} (53%) create mode 100644 pinot-controller/src/main/resources/app/components/Homepage/Operations/AddOfflineTenantComponent.tsx create mode 100644 pinot-controller/src/main/resources/app/components/Homepage/Operations/AddPartionComponent.tsx create mode 100644 pinot-controller/src/main/resources/app/components/Homepage/Operations/AddQueryComponent.tsx create mode 100644 pinot-controller/src/main/resources/app/components/Homepage/Operations/AddRealTimeIngestionComponent.tsx create mode 100644 pinot-controller/src/main/resources/app/components/Homepage/Operations/AddRealTimePartionComponent.tsx rename pinot-controller/src/main/resources/app/components/Homepage/Operations/{AddTableOp.tsx => AddRealtimeTableOp.tsx} (60%) create mode 100644 pinot-controller/src/main/resources/app/components/Homepage/Operations/AddSchemaOp.tsx create mode 100644 pinot-controller/src/main/resources/app/components/Homepage/Operations/AddStorageComponent.tsx create mode 100644 pinot-controller/src/main/resources/app/components/Homepage/Operations/AddTenantComponent.tsx create mode 100644 pinot-controller/src/main/resources/app/components/Homepage/Operations/MultiIndexingComponent.tsx create mode 100644 pinot-controller/src/main/resources/app/components/Homepage/Operations/MultiMetricComponent.tsx create mode 100644 pinot-controller/src/main/resources/app/components/Homepage/Operations/MultipleSelectComponent.tsx create mode 100644 pinot-controller/src/main/resources/app/components/Homepage/Operations/SchemaNameComponent.tsx copy pinot-controller/src/main/resources/app/pages/{SegmentDetails.tsx => SchemaPageDetails.tsx} (50%) create mode 100644 pinot-controller/src/test/java/org/apache/pinot/controller/ControllerTestSetup.java copy pinot-controller/src/test/java/org/apache/pinot/controller/{helix/ControllerTest.java => ControllerTestUtils.java} (62%) create mode 100644 pinot-controller/src/test/java/org/apache/pinot/controller/api/ConsumingSegmentInfoReaderTest.java rename pinot-controller/src/test/java/org/apache/pinot/controller/api/{PinotBrokerRestletResourceTest.java => PinotBrokerRestletResourceStatelessTest.java} (98%) rename pinot-controller/src/test/java/org/apache/pinot/controller/helix/{ControllerPeriodicTaskStarterTest.java => ControllerPeriodicTaskStarterStatelessTest.java} (97%) rename pinot-controller/src/test/java/org/apache/pinot/controller/helix/{ControllerTenantTest.java => ControllerTenantStatelessTest.java} (99%) rename pinot-controller/src/test/java/org/apache/pinot/controller/helix/{PinotControllerModeTest.java => PinotControllerModeStatelessTest.java} (99%) create mode 100644 pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManagerStatelessTest.java create mode 100644 pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/assignment/segment/OfflineDimTableSegmentAssignmentTest.java rename pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/rebalance/{TableRebalancerClusterTest.java => TableRebalancerClusterStatelessTest.java} (99%) create mode 100644 pinot-controller/src/test/java/org/apache/pinot/controller/validation/ValidationManagerStatelessTest.java create mode 100644 pinot-controller/testng-statefull.xml create mode 100644 pinot-controller/testng-stateless.xml create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/operator/combine/DistinctCombineOperator.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/operator/filter/H3IndexFilterOperator.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/operator/query/DistinctOperator.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/plan/DistinctPlanNode.java copy pinot-core/src/main/java/org/apache/pinot/core/query/{request/context/utils/QueryContextUtils.java => distinct/DistinctExecutor.java} (51%) create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/DistinctExecutorFactory.java rename pinot-core/src/main/java/org/apache/pinot/core/query/{aggregation/function/customobject => distinct}/DistinctTable.java (52%) create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/dictionary/BaseDictionaryBasedMultiColumnDistinctExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/dictionary/BaseDictionaryBasedSingleColumnDistinctExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/dictionary/DictionaryBasedMultiColumnDistinctOnlyExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/dictionary/DictionaryBasedMultiColumnDistinctOrderByExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/dictionary/DictionaryBasedSingleColumnDistinctOnlyExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/dictionary/DictionaryBasedSingleColumnDistinctOrderByExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/BaseRawBytesSingleColumnDistinctExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/BaseRawDoubleSingleColumnDistinctExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/BaseRawFloatSingleColumnDistinctExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/BaseRawIntSingleColumnDistinctExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/BaseRawLongSingleColumnDistinctExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/BaseRawStringSingleColumnDistinctExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/RawBytesSingleColumnDistinctOnlyExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/RawBytesSingleColumnDistinctOrderByExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/RawDoubleSingleColumnDistinctOnlyExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/RawDoubleSingleColumnDistinctOrderByExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/RawFloatSingleColumnDistinctOnlyExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/RawFloatSingleColumnDistinctOrderByExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/RawIntSingleColumnDistinctOnlyExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/RawIntSingleColumnDistinctOrderByExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/RawLongSingleColumnDistinctOnlyExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/RawLongSingleColumnDistinctOrderByExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/RawMultiColumnDistinctExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/RawStringSingleColumnDistinctOnlyExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/distinct/raw/RawStringSingleColumnDistinctOrderByExecutor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/GeoPredicate.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/segment/index/loader/invertedindex/H3IndexHandler.java create mode 100644 pinot-perf/src/main/java/org/apache/pinot/perf/BenchmarkOffheapBitmapInvertedIndexCreator.java create mode 100644 pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-standalone/src/test/java/org/apache/pinot/plugin/ingestion/batch/standalone/SegmentGenerationJobRunnerTest.java copy {pinot-tools => pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-standalone}/src/test/resources/log4j2.xml (100%) copy pinot-core/src/main/java/org/apache/pinot/core/query/request/context/utils/QueryContextUtils.java => pinot-spi/src/main/java/org/apache/pinot/spi/config/table/TableStats.java (59%) create mode 100644 pinot-spi/src/main/java/org/apache/pinot/spi/config/table/TunerConfig.java copy pinot-core/src/main/java/org/apache/pinot/core/query/request/context/utils/QueryContextUtils.java => pinot-spi/src/main/java/org/apache/pinot/spi/config/table/tuner/TableConfigTuner.java (59%) copy pinot-core/src/main/java/org/apache/pinot/core/query/request/context/utils/QueryContextUtils.java => pinot-spi/src/main/java/org/apache/pinot/spi/config/table/tuner/Tuner.java (56%) create mode 100644 pinot-tools/src/main/java/org/apache/pinot/tools/GenericQuickstart.java create mode 100644 pinot-tools/src/main/resources/examples/batch/starbucksStores/ingestionJobSpec.yaml create mode 100644 pinot-tools/src/main/resources/examples/batch/starbucksStores/rawdata/data.csv create mode 100644 pinot-tools/src/main/resources/examples/batch/starbucksStores/starbucksStores_offline_table_config.json create mode 100644 pinot-tools/src/main/resources/examples/batch/starbucksStores/starbucksStores_schema.json create mode 100644 thirdeye/thirdeye-frontend/.prettierignore create mode 100644 thirdeye/thirdeye-frontend/.prettierrc create mode 100644 thirdeye/thirdeye-frontend/app/mocks/compositeAnomalies.js create mode 100644 thirdeye/thirdeye-frontend/app/pods/components/breadcrumb-list/component.js create mode 100644 thirdeye/thirdeye-frontend/app/pods/components/breadcrumb-list/template.hbs create mode 100644 thirdeye/thirdeye-frontend/app/pods/components/composite-anomalies/component.js create mode 100644 thirdeye/thirdeye-frontend/app/pods/components/composite-anomalies/parent-anomalies/component.js create mode 100644 thirdeye/thirdeye-frontend/app/pods/components/composite-anomalies/parent-anomalies/template.hbs create mode 100644 thirdeye/thirdeye-frontend/app/pods/components/composite-anomalies/template.hbs create mode 100644 thirdeye/thirdeye-frontend/app/pods/custom/composite-anomalies-table/anomalies-list/template.hbs create mode 100644 thirdeye/thirdeye-frontend/app/pods/custom/composite-anomalies-table/resolution/component.js create mode 100644 thirdeye/thirdeye-frontend/app/pods/custom/composite-anomalies-table/resolution/template.hbs create mode 100644 thirdeye/thirdeye-frontend/app/pods/custom/composite-anomalies-table/start-duration/template.hbs create mode 100644 thirdeye/thirdeye-frontend/app/pods/manage/explore/composite-anomalies/controller.js create mode 100644 thirdeye/thirdeye-frontend/app/pods/manage/explore/composite-anomalies/route.js create mode 100644 thirdeye/thirdeye-frontend/app/pods/manage/explore/composite-anomalies/template.hbs create mode 100644 thirdeye/thirdeye-frontend/app/pods/manage/explore/single-metric-anomalies/controller.js create mode 100644 thirdeye/thirdeye-frontend/app/pods/manage/explore/single-metric-anomalies/route.js create mode 100644 thirdeye/thirdeye-frontend/app/pods/manage/explore/single-metric-anomalies/template.hbs create mode 100644 thirdeye/thirdeye-frontend/app/styles/components/breadcrumb-list.scss create mode 100644 thirdeye/thirdeye-frontend/app/styles/pods/custom/parent-anomalies-table.scss create mode 100644 thirdeye/thirdeye-frontend/app/utils/anomalies-tree-parser.js create mode 100644 thirdeye/thirdeye-frontend/app/utils/date-picker-utils.js create mode 100644 thirdeye/thirdeye-frontend/app/utils/pub-sub.js create mode 100644 thirdeye/thirdeye-frontend/tests/integration/pods/components/breadcrumb-list/component-test.js create mode 100644 thirdeye/thirdeye-frontend/tests/integration/pods/components/composite-anomalies/component-test.js create mode 100644 thirdeye/thirdeye-frontend/tests/integration/pods/components/composite-anomalies/parent-anomalies/component-test.js create mode 100644 thirdeye/thirdeye-frontend/tests/unit/pods/manage/explore/composite-anomalies/route-test.js create mode 100644 thirdeye/thirdeye-frontend/tests/unit/pods/manage/explore/single-metric-anomalies/route-test.js create mode 100644 thirdeye/thirdeye-frontend/tests/unit/utils/anomalies-tree-parser-test.js create mode 100644 thirdeye/thirdeye-frontend/tests/unit/utils/date-picker-utils-test.js create mode 100644 thirdeye/thirdeye-frontend/tests/unit/utils/pub-sub-test.js --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org