This is an automated email from the ASF dual-hosted git repository.
kishoreg pushed a change to branch exact-distinct-count
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard dc1b19e Deleting file unrelated to this PR
discard a7ab0fd Support for exact distinct count for non int data types
add 6085578 Enable segment decryption for encrypted segments in Minion
(#5821)
add fd0130b Enhance sql parser for having and post-aggregation (#5825)
add 8a3cecb [TE] frontend - harleyjj/forecast - show tool tip for fit and
forecast (#5812)
add ce32362 Addressed issues in code review: (#5774)
add cc7a344 [TE] UI changes for anomalies page v3(#5824)
add 7f8e65c [TE] Fix template rendering errors (#5839)
add debadaa [TE] enchance anomaly api to propagate feedback (#5823)
add 63a4fd4 Clarifications in realtime provisioning helper (#5838)
add de14078 Support post-aggregation in QueryContext (#5827)
add 9f23e18 Tiered storage (#5793)
add 57d4b71 [TE] Presto JDBC lib upgrade (#5841)
add 3785aa7 Improving retention manager to handle segment lineage
clean-up (#5828)
add c04b8b3 Move quickstart tests to use Java 14 release (#5844)
add d8264c1 Fix data ingestion from Amazon S3 bucket (#5836)
add 9551062 update Swagger (OpenAPI) configuration for HTTP+HTTPS (#5817)
add b268012 [TE] add anomaly detection as a service - Phase 1 (#5769)
add 1c754f8 Added set-diff operators and changed distinctCountThetaSketch
syntax (#5832)
add 47323de [TE] Dashboard Resource Refactor (#5808)
add db48107 Adding controller healthcheck endpoint: /health (#5846)
add f4949e9 [TE] Added reset application API +refactor (#5847)
add bb8b19e DataGenerator to tolerate DATE_TIME and COMPLEX fields (#5848)
add d28c5cf Add pinot-spark-connector (#5787)
add 8ab032f Makes Pinot work on Alpine Linux or Distroless + BusyBox
(#5818)
add eb0f713 Fix encrypted file path in Segment Fetcher (#5854)
add 0b6ef98 Support multi-value non-dictionary group by (#5851)
add 2cfaed3 Support type conversion for all scalar functions (#5849)
add 09e9804 [TE] Fix wrong task pickup logic (#5855)
add 5469a84 add timeColumnName to tableConfig to enable TE auto-detection
(#5860)
add 449bf94 Fix NPE for aggregate metrics (#5862)
add 45d5d29 Add additional datetime functionality (#5438)
add 6dd54f8 Fix the variable names for off-heap alloc configs (#5852)
add 2b58bfb [TE] clean up legacy code (#5842)
add ee6e541 Support for exact distinct count for non int data types
add 2808c58 Deleting file unrelated to this PR
add ef30e07 Fixing serde for bytesset
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (dc1b19e)
\
N -- N -- N refs/heads/exact-distinct-count (ef30e07)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/pinot_tests.yml | 2 +-
LICENSE-binary | 2 +-
.../broker/broker/BrokerAdminApiApplication.java | 4 +-
.../common/assignment/InstancePartitionsUtils.java | 23 +-
.../pinot/common/function/DateTimeFunctions.java | 262 -------
.../common/function/DateTimePatternHandler.java | 4 +-
.../apache/pinot/common/function/FunctionInfo.java | 44 +-
.../pinot/common/function/FunctionInvoker.java | 137 ++--
.../pinot/common/function/FunctionRegistry.java | 29 +-
.../pinot/common/function/FunctionUtils.java | 118 ++++
.../common/function/TransformFunctionType.java | 2 +-
.../function/annotations/ScalarFunction.java | 23 +-
.../function/scalar/ArithmeticFunctions.java | 95 +++
.../common/function/scalar/DateTimeFunctions.java | 555 +++++++++++++++
.../function/{ => scalar}/JsonFunctions.java | 9 +-
.../function/{ => scalar}/StringFunctions.java | 40 +-
.../pinot/common/metrics/ControllerMeter.java | 2 +
.../pinot/common/tier/PinotServerTierStorage.java | 31 +-
.../java/org/apache/pinot/common/tier/Tier.java | 50 ++
.../org/apache/pinot/common/tier/TierFactory.java | 59 ++
.../pinot/common/tier/TierSegmentSelector.java | 26 +-
.../org/apache/pinot/common/tier/TierStorage.java | 13 +-
.../common/tier/TimeBasedTierSegmentSelector.java | 83 +++
.../apache/pinot/common/utils}/PinotDataType.java | 32 +-
.../common/utils/config/TableConfigUtils.java | 14 +-
.../pinot/common/utils/config/TierConfigUtils.java | 85 +++
.../utils/fetcher/SegmentFetcherFactory.java | 27 +-
.../parsers/PinotQuery2BrokerRequestConverter.java | 10 +-
.../apache/pinot/sql/parsers/CalciteSqlParser.java | 319 +++++----
.../pinot/common/tier/TierConfigUtilsTest.java | 135 ++++
.../pinot/common/tier/TierSegmentSelectorTest.java | 117 ++++
.../pinot/common/utils}/PinotDataTypeTest.java | 4 +-
.../common/utils/config/TableConfigSerDeTest.java | 44 +-
.../utils/fetcher/SegmentFetcherFactoryTest.java | 57 +-
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 383 +++++++++--
pinot-connectors/pinot-spark-connector/README.md | 69 ++
...pinot-connector-executor-server-interaction.jpg | Bin 0 -> 100316 bytes
.../documentation/read_model.md | 142 ++++
pinot-connectors/pinot-spark-connector/pom.xml | 259 +++++++
...org.apache.spark.sql.sources.DataSourceRegister | 1 +
.../connector/spark/connector/FilterPushDown.scala | 123 ++++
.../spark/connector/PinotClusterClient.scala | 203 ++++++
.../spark/connector/PinotServerDataFetcher.scala | 135 ++++
.../connector/spark/connector/PinotSplitter.scala | 106 +++
.../connector/spark/connector/PinotUtils.scala | 121 ++++
.../spark/connector/query/GeneratedSQLs.scala | 18 +-
.../query/SQLSelectionQueryGenerator.scala | 94 +++
.../datasource/PinotDataSourceReadOptions.scala | 99 +++
.../spark/datasource/PinotDataSourceReader.scala | 111 +++
.../spark/datasource/PinotDataSourceV2.scala | 36 +-
.../spark/datasource/PinotInputPartition.scala | 25 +-
.../datasource/PinotInputPartitionReader.scala | 59 ++
.../connector/spark/exceptions/exceptions.scala | 16 +-
.../org/apache/pinot/connector/spark/package.scala | 24 +-
.../pinot/connector/spark/utils/HttpUtils.scala | 74 ++
.../pinot/connector/spark/utils/Logging.scala | 77 +++
.../src/test/resources/log4j2.xml | 0
.../src/test/resources/schema/pinot-schema.json | 60 ++
.../src/test/resources/schema/spark-schema.json | 86 +++
.../apache/pinot/connector/spark/BaseTest.scala | 13 +-
.../spark/ExampleSparkPinotConnectorTest.scala | 155 +++++
.../spark/connector/FilterPushDownTest.scala | 75 ++
.../spark/connector/PinotSplitterTest.scala | 99 +++
.../connector/spark/connector/PinotUtilsTest.scala | 152 ++++
.../query/SQLSelectionQueryGeneratorTest.scala | 92 +++
.../PinotDataSourceReadOptionsTest.scala | 87 +++
pinot-connectors/pom.xml | 47 ++
.../apache/pinot/controller/ControllerConf.java | 95 ++-
.../apache/pinot/controller/ControllerStarter.java | 26 +-
.../api/ControllerAdminApiApplication.java | 14 +-
.../api/resources/PinotControllerHealthCheck.java | 26 +-
.../api/resources/PinotTableRestletResource.java | 13 +
.../segment/OfflineSegmentAssignment.java | 78 ++-
.../segment/RealtimeSegmentAssignment.java | 147 ++--
.../core/assignment/segment/SegmentAssignment.java | 8 +-
.../assignment/segment/SegmentAssignmentUtils.java | 66 ++
.../helix/core/rebalance/TableRebalancer.java | 55 +-
...SegmentRelocator.java => SegmentRelocator.java} | 56 +-
.../helix/core/retention/RetentionManager.java | 79 +++
.../controller/recommender/RecommenderDriver.java | 80 +++
.../exceptions/InvalidInputException.java | 13 +-
.../controller/recommender/io/ConfigManager.java | 63 ++
.../controller/recommender/io/InputManager.java | 682 ++++++++++++++++++
.../recommender/io/metadata/FieldMetadata.java | 71 ++
.../io/metadata/SchemaWithMetaData.java | 69 ++
.../io/metadata/TimeFieldSpecMetadata.java | 78 +++
.../controller/recommender/rules/AbstractRule.java | 27 +-
.../recommender/rules/RulesToExecute.java | 148 ++++
.../recommender/rules/impl/BloomFilterRule.java | 127 ++++
.../recommender/rules/impl/FlagQueryRule.java | 75 ++
.../rules/impl/InvertedSortedIndexJointRule.java | 267 ++++++++
.../recommender/rules/impl/KafkaPartitionRule.java | 65 ++
.../NoDictionaryOnHeapDictionaryJointRule.java | 241 +++++++
.../rules/impl/PinotTablePartitionRule.java | 240 +++++++
.../rules/impl/VariedLengthDictionaryRule.java | 50 ++
.../recommender/rules/io/FlaggedQueries.java | 54 ++
.../recommender/rules/io/configs/IndexConfig.java | 120 ++++
.../rules/io/configs/PartitionConfig.java | 78 +++
.../rules/io/params/BloomFilterRuleParams.java | 52 ++
.../rules/io/params/FlagQueryRuleParams.java | 41 ++
.../params/InvertedSortedIndexJointRuleParams.java | 153 +++++
...oDictionaryOnHeapDictionaryJointRuleParams.java | 119 ++++
.../rules/io/params/PartitionRuleParams.java | 108 +++
.../rules/io/params/RecommenderConstants.java | 109 +++
.../recommender/rules/utils/FixedLenBitset.java | 174 +++++
.../rules/utils/PredicateParseResult.java | 153 +++++
.../utils/QueryInvertedSortedIndexRecommender.java | 761 +++++++++++++++++++++
.../pinot/controller/util/ListenerConfigUtil.java | 17 -
...fflineNonReplicaGroupSegmentAssignmentTest.java | 5 +-
...NonReplicaGroupTieredSegmentAssignmentTest.java | 297 ++++++++
.../OfflineReplicaGroupSegmentAssignmentTest.java | 8 +-
...altimeNonReplicaGroupSegmentAssignmentTest.java | 23 +-
...NonReplicaGroupTieredSegmentAssignmentTest.java | 366 ++++++++++
.../RealtimeReplicaGroupSegmentAssignmentTest.java | 23 +-
.../core/rebalance/TableRebalancerClusterTest.java | 101 +++
.../core/retention/SegmentLineageCleanupTest.java | 182 +++++
.../controller/recommender/TestConfigEngine.java | 344 ++++++++++
.../controller/util/ListenerConfigUtilTest.java | 6 -
pinot-controller/src/test/resources/log4j2.xml | 5 -
.../recommenderInput/BloomFilterInput.json | 135 ++++
.../recommenderInput/DataSizeCalculationInput.json | 69 ++
.../resources/recommenderInput/FlagQueryInput.json | 130 ++++
.../resources/recommenderInput/InvalidInput1.json | 81 +++
.../resources/recommenderInput/InvalidInput2.json | 82 +++
.../recommenderInput/KafkaPartitionRuleInput.json | 126 ++++
.../recommenderInput/KafkaPartitionRuleInput2.json | 126 ++++
...NoDictionaryOnHeapDictionaryJointRuleInput.json | 147 ++++
.../PinotTablePartitionRuleInput.json | 146 ++++
.../recommenderInput/SortedInvertedIndexInput.json | 157 +++++
.../VariedLengthDictionaryInput.json | 82 +++
.../apache/pinot/core/common/ObjectSerDeUtils.java | 25 +-
.../data/function/FunctionEvaluatorFactory.java | 15 +-
.../data/function/InbuiltFunctionEvaluator.java | 78 +--
.../manager/config/InstanceDataManagerConfig.java | 2 +-
.../realtime/HLRealtimeSegmentDataManager.java | 6 +-
.../realtime/LLRealtimeSegmentDataManager.java | 4 +-
.../recordtransformer/DataTypeTransformer.java | 1 +
.../function/ScalarTransformFunctionWrapper.java | 331 ++++-----
.../function/TransformFunctionFactory.java | 25 +-
.../function/AggregationFunctionUtils.java | 33 -
...istinctCountThetaSketchAggregationFunction.java | 196 ++++--
.../NoDictionaryMultiColumnGroupKeyGenerator.java | 157 ++++-
.../postaggregation/PostAggregationFunction.java | 81 +++
.../core/query/request/context/QueryContext.java | 140 +++-
.../request/context/utils/QueryContextUtils.java | 21 +-
.../creator/impl/SegmentColumnarIndexCreator.java | 25 +-
.../segment/index/loader/IndexLoadingConfig.java | 16 +-
.../apache/pinot/core/util/TableConfigUtils.java | 74 +-
.../function/InbuiltFunctionEvaluatorTest.java | 91 +--
.../core/data/function/InbuiltFunctionsTest.java | 160 ++++-
.../ScalarTransformFunctionWrapperTest.java | 8 +-
.../PostAggregationFunctionTest.java | 62 ++
.../BrokerRequestToQueryContextConverterTest.java | 165 ++++-
.../pinot/core/util/TableConfigUtilsTest.java | 153 ++++-
.../pinot/queries/BaseMultiValueQueriesTest.java | 4 +-
.../queries/DistinctCountThetaSketchTest.java | 61 +-
...terSegmentAggregationMultiValueQueriesTest.java | 36 +-
.../AggregateMetricsClusterIntegrationTest.java | 130 ++++
.../ControllerPeriodicTasksIntegrationTest.java | 4 +-
.../tests/ThetaSketchIntegrationTest.java | 12 +-
.../BaseMultipleSegmentsConversionExecutor.java | 3 +-
.../BaseSingleSegmentConversionExecutor.java | 4 +-
.../pinot/minion/executor/BaseTaskExecutor.java | 10 +
.../minion/executor/MergeRollupTaskExecutor.java | 3 +-
.../pinot/minion/executor/PurgeTaskExecutor.java | 6 +-
.../standalone/SegmentGenerationJobRunner.java | 2 +-
.../apache/pinot/plugin/filesystem/S3PinotFS.java | 13 +-
.../pinot/plugin/filesystem/S3PinotFSTest.java | 13 +-
.../server/starter/helix/AdminApiApplication.java | 4 +-
.../helix/HelixInstanceDataManagerConfig.java | 4 +-
.../apache/pinot/spi/config/table/TableConfig.java | 18 +-
.../apache/pinot/spi/config/table/TierConfig.java | 84 +++
.../java/org/apache/pinot/spi/utils/TimeUtils.java | 12 +
.../spi/utils/builder/TableConfigBuilder.java | 9 +-
.../pinot/tools/admin/PinotAdministrator.java | 6 -
.../tools/admin/command/GenerateDataCommand.java | 5 +
.../command/RealtimeProvisioningHelperCommand.java | 7 +-
.../realtime/provisioning/MemoryEstimator.java | 2 +-
.../PinotServiceManagerAdminApiApplication.java | 4 +-
.../tools/tuner/CollectMetadataForIndexTuning.java | 97 ---
.../tools/tuner/EntriesScannedQuantileReport.java | 89 ---
.../pinot/tools/tuner/IndexTunerCommand.java | 197 ------
.../pinot/tools/tuner/driver/TunerDriver.java | 209 ------
.../meta/manager/JsonFileMetaManagerImpl.java | 247 -------
.../tools/tuner/meta/manager/MetaManager.java | 59 --
.../manager/collector/ColStatsAccumulatorObj.java | 156 -----
.../manager/collector/CompressedFilePathIter.java | 101 ---
.../tuner/meta/manager/collector/PathWrapper.java | 74 --
.../collector/SegmentMetadataCollector.java | 272 --------
.../tuner/query/src/LogInputIteratorImpl.java | 165 -----
.../query/src/parser/BrokerLogParserImpl.java | 76 --
.../stats/wrapper/IndexSuggestQueryStatsImpl.java | 117 ----
.../tools/tuner/strategy/AbstractAccumulator.java | 64 --
.../tools/tuner/strategy/FrequencyAccumulator.java | 54 --
.../pinot/tools/tuner/strategy/FrequencyImpl.java | 196 ------
.../tuner/strategy/ParseBasedAccumulator.java | 62 --
.../tools/tuner/strategy/ParserBasedImpl.java | 444 ------------
.../strategy/QuantileAnalysisAccumulator.java | 89 ---
.../tools/tuner/strategy/QuantileAnalysisImpl.java | 365 ----------
.../pinot/tools/tuner/strategy/TuningStrategy.java | 60 --
.../src/main/resources/appAssemblerScriptTemplate | 47 +-
.../resources/generator/complexWebsite_config.json | 3 +-
.../resources/generator/simpleWebsite_config.json | 3 +-
pom.xml | 6 +-
.../app/pods/anomalies/controller.js | 446 +++---------
.../thirdeye-frontend/app/pods/anomalies/route.js | 144 +---
.../app/pods/anomalies/template.hbs | 2 +-
.../app/pods/components/alert-details/component.js | 48 +-
.../app/pods/components/alert-details/template.hbs | 33 +
.../pods/components/anomaly-summary/component.js | 41 +-
.../pods/components/anomaly-summary/template.hbs | 14 +-
.../app/pods/components/entity-filter/component.js | 9 +-
.../app/styles/shared/_styles.scss | 51 ++
thirdeye/thirdeye-frontend/app/utils/anomaly.js | 54 +-
thirdeye/thirdeye-pinot/pom.xml | 4 +-
.../anomaly/ThirdEyeAnomalyApplication.java | 8 +-
.../anomaly/ThirdEyeAnomalyConfiguration.java | 9 +
.../anomaly/alert/util/DataReportHelper.java | 276 --------
.../thirdeye/anomaly/alert/util/EmailHelper.java | 130 ----
.../pinot/thirdeye/anomaly/task/TaskConstants.java | 3 +-
.../pinot/thirdeye/anomaly/task/TaskDriver.java | 31 +-
.../thirdeye/anomaly/task/TaskInfoFactory.java | 5 +-
.../thirdeye/anomaly/task/TaskRunnerFactory.java | 3 +
.../anomaly/utils/ThirdeyeMetricsUtil.java | 6 +
.../api/application/ApplicationResource.java | 135 ++--
.../api/detection/AnomalyDetectionResource.java | 745 ++++++++++++++++++++
.../api/user/dashboard/UserDashboardResource.java | 5 +-
.../dashboard/ThirdEyeDashboardApplication.java | 58 +-
.../dashboard/resources/AdminResource.java | 33 +-
.../resources/AnomalyFlattenResource.java | 3 +-
.../dashboard/resources/AnomalyResource.java | 39 +-
.../dashboard/resources/AutoOnboardResource.java | 3 +-
.../dashboard/resources/CacheResource.java | 3 +-
.../resources/CustomizedEventResource.java | 3 +-
.../dashboard/resources/DashboardResource.java | 71 +-
.../dashboard/resources/DatasetConfigResource.java | 3 +-
.../dashboard/resources/EntityManagerResource.java | 3 +-
.../dashboard/resources/EntityMappingResource.java | 3 +-
.../dashboard/resources/MetricConfigResource.java | 3 +-
.../resources/OnboardDatasetMetricResource.java | 3 +-
.../thirdeye/dashboard/resources/RootResource.java | 216 ++++++
.../dashboard/resources/SummaryResource.java | 33 +-
.../dashboard/resources/ThirdEyeResource.java | 25 +-
.../dashboard/resources/v2/AnomaliesResource.java | 3 +-
.../dashboard/resources/v2/ConfigResource.java | 3 +-
.../dashboard/resources/v2/DataResource.java | 3 +-
.../resources/v2/DetectionAlertResource.java | 3 +-
.../resources/v2/alerts/AlertResource.java | 4 +-
.../v2/anomalies/AnomalySearchResource.java | 4 +-
.../views/contributor/ContributionCell.java | 183 -----
.../contributor/ContributionViewTableBuilder.java | 142 ----
.../views/contributor/ContributorViewHandler.java | 331 ---------
.../views/contributor/ContributorViewResponse.java | 109 ---
.../datalayer/bao/MergedAnomalyResultManager.java | 2 +
.../pinot/thirdeye/datalayer/bao/TaskManager.java | 10 +
.../bao/jdbc/MergedAnomalyResultManagerImpl.java | 18 +
.../datalayer/bao/jdbc/TaskManagerImpl.java | 60 ++
.../thirdeye/datalayer/pojo/MetricConfigBean.java | 10 +
.../dataset/DatasetAutoOnboardResource.java | 3 +-
.../online/OnlineThirdEyeDataSource.java | 199 ++++++
.../pinot/resultset/ThirdEyeResultSetUtils.java | 1 +
.../sql/resources/SqlDataSourceResource.java | 3 +-
.../detection/DetectionConfigurationResource.java | 3 +-
.../detection/DetectionPipelineTaskInfo.java | 20 +-
.../detection/DetectionPipelineTaskRunner.java | 25 +-
.../thirdeye/detection/DetectionResource.java | 16 +-
.../thirdeye/detection/yaml/YamlResource.java | 11 +-
.../content/BaseNotificationContent.java | 2 -
.../content/templates/MetricAnomaliesContent.java | 4 +-
.../main/resources/detection-config-template.yml | 24 +
.../pinot/thirdeye/detector/anomaly-report-v2.ftl | 136 ----
.../pinot/thirdeye/detector/anomaly-report.ftl | 150 ----
.../thirdeye/detector/custom-anomaly-report.ftl | 96 ---
.../detector/data-report-by-metric-dimension.ftl | 105 ---
.../detector/metric-anomalies-template.ftl | 9 +-
.../detection/AnomalyDetectionResourceTest.java | 232 +++++++
.../dashboard/handler/ContributorTest.java | 70 --
.../bao/TestMergedAnomalyResultManager.java | 38 +
.../templates/TestMetricAnomaliesContent.java | 26 +-
.../thirdeye/api/detection/payload-bad-custom.json | 4 +
.../pinot/thirdeye/api/detection/payload-bad.json | 3 +
.../api/detection/payload-good-custom.json | 4 +
.../pinot/thirdeye/api/detection/payload-good.json | 4 +
283 files changed, 16163 insertions(+), 7228 deletions(-)
delete mode 100644
pinot-common/src/main/java/org/apache/pinot/common/function/DateTimeFunctions.java
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/function/FunctionUtils.java
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java
rename pinot-common/src/main/java/org/apache/pinot/common/function/{ =>
scalar}/JsonFunctions.java (90%)
rename pinot-common/src/main/java/org/apache/pinot/common/function/{ =>
scalar}/StringFunctions.java (84%)
copy
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/query/src/parser/QueryParser.java
=>
pinot-common/src/main/java/org/apache/pinot/common/tier/PinotServerTierStorage.java
(63%)
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/tier/Tier.java
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/tier/TierFactory.java
rename
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/query/src/InputIterator.java
=>
pinot-common/src/main/java/org/apache/pinot/common/tier/TierSegmentSelector.java
(63%)
copy
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/query/src/stats/wrapper/AbstractQueryStats.java
=> pinot-common/src/main/java/org/apache/pinot/common/tier/TierStorage.java
(73%)
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/tier/TimeBasedTierSegmentSelector.java
rename {pinot-core/src/main/java/org/apache/pinot/core/data/recordtransformer
=> pinot-common/src/main/java/org/apache/pinot/common/utils}/PinotDataType.java
(95%)
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/utils/config/TierConfigUtils.java
create mode 100644
pinot-common/src/test/java/org/apache/pinot/common/tier/TierConfigUtilsTest.java
create mode 100644
pinot-common/src/test/java/org/apache/pinot/common/tier/TierSegmentSelectorTest.java
rename {pinot-core/src/test/java/org/apache/pinot/core/data/recordtransformer
=>
pinot-common/src/test/java/org/apache/pinot/common/utils}/PinotDataTypeTest.java
(98%)
create mode 100644 pinot-connectors/pinot-spark-connector/README.md
create mode 100644
pinot-connectors/pinot-spark-connector/documentation/images/spark-pinot-connector-executor-server-interaction.jpg
create mode 100644
pinot-connectors/pinot-spark-connector/documentation/read_model.md
create mode 100644 pinot-connectors/pinot-spark-connector/pom.xml
create mode 100644
pinot-connectors/pinot-spark-connector/src/main/resources/META-INF/services/org.apache.spark.sql.sources.DataSourceRegister
create mode 100644
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector/FilterPushDown.scala
create mode 100644
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector/PinotClusterClient.scala
create mode 100644
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector/PinotServerDataFetcher.scala
create mode 100644
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector/PinotSplitter.scala
create mode 100644
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector/PinotUtils.scala
copy
pinot-minion/src/main/java/org/apache/pinot/minion/executor/BaseTaskExecutor.java
=>
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector/query/GeneratedSQLs.scala
(66%)
create mode 100644
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector/query/SQLSelectionQueryGenerator.scala
create mode 100644
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/datasource/PinotDataSourceReadOptions.scala
create mode 100644
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/datasource/PinotDataSourceReader.scala
rename
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/views/contributor/ContributionViewTable.java
=>
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/datasource/PinotDataSourceV2.scala
(53%)
copy
pinot-minion/src/main/java/org/apache/pinot/minion/executor/BaseTaskExecutor.java
=>
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/datasource/PinotInputPartition.scala
(53%)
create mode 100644
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/datasource/PinotInputPartitionReader.scala
copy
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/query/src/stats/wrapper/AbstractQueryStats.java
=>
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/exceptions/exceptions.scala
(69%)
copy
pinot-minion/src/main/java/org/apache/pinot/minion/executor/BaseTaskExecutor.java
=>
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/package.scala
(60%)
create mode 100644
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/utils/HttpUtils.scala
create mode 100644
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/utils/Logging.scala
copy {pinot-common =>
pinot-connectors/pinot-spark-connector}/src/test/resources/log4j2.xml (100%)
create mode 100644
pinot-connectors/pinot-spark-connector/src/test/resources/schema/pinot-schema.json
create mode 100644
pinot-connectors/pinot-spark-connector/src/test/resources/schema/spark-schema.json
rename
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/views/contributor/ContributorViewRequest.java
=>
pinot-connectors/pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark/BaseTest.scala
(79%)
create mode 100644
pinot-connectors/pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark/ExampleSparkPinotConnectorTest.scala
create mode 100644
pinot-connectors/pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark/connector/FilterPushDownTest.scala
create mode 100644
pinot-connectors/pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark/connector/PinotSplitterTest.scala
create mode 100644
pinot-connectors/pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark/connector/PinotUtilsTest.scala
create mode 100644
pinot-connectors/pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark/connector/query/SQLSelectionQueryGeneratorTest.scala
create mode 100644
pinot-connectors/pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark/datasource/PinotDataSourceReadOptionsTest.scala
create mode 100644 pinot-connectors/pom.xml
rename
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/relocation/{RealtimeSegmentRelocator.java
=> SegmentRelocator.java} (66%)
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/RecommenderDriver.java
rename
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/query/src/stats/wrapper/AbstractQueryStats.java
=>
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/exceptions/InvalidInputException.java
(70%)
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/io/ConfigManager.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/io/InputManager.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/io/metadata/FieldMetadata.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/io/metadata/SchemaWithMetaData.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/io/metadata/TimeFieldSpecMetadata.java
rename
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/query/src/parser/QueryParser.java
=>
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/AbstractRule.java
(59%)
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/RulesToExecute.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/impl/BloomFilterRule.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/impl/FlagQueryRule.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/impl/InvertedSortedIndexJointRule.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/impl/KafkaPartitionRule.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/impl/NoDictionaryOnHeapDictionaryJointRule.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/impl/PinotTablePartitionRule.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/impl/VariedLengthDictionaryRule.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/io/FlaggedQueries.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/io/configs/IndexConfig.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/io/configs/PartitionConfig.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/io/params/BloomFilterRuleParams.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/io/params/FlagQueryRuleParams.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/io/params/InvertedSortedIndexJointRuleParams.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/io/params/NoDictionaryOnHeapDictionaryJointRuleParams.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/io/params/PartitionRuleParams.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/io/params/RecommenderConstants.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/utils/FixedLenBitset.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/utils/PredicateParseResult.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/utils/QueryInvertedSortedIndexRecommender.java
create mode 100644
pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/assignment/segment/OfflineNonReplicaGroupTieredSegmentAssignmentTest.java
create mode 100644
pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/assignment/segment/RealtimeNonReplicaGroupTieredSegmentAssignmentTest.java
create mode 100644
pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/retention/SegmentLineageCleanupTest.java
create mode 100644
pinot-controller/src/test/java/org/apache/pinot/controller/recommender/TestConfigEngine.java
create mode 100644
pinot-controller/src/test/resources/recommenderInput/BloomFilterInput.json
create mode 100644
pinot-controller/src/test/resources/recommenderInput/DataSizeCalculationInput.json
create mode 100644
pinot-controller/src/test/resources/recommenderInput/FlagQueryInput.json
create mode 100644
pinot-controller/src/test/resources/recommenderInput/InvalidInput1.json
create mode 100644
pinot-controller/src/test/resources/recommenderInput/InvalidInput2.json
create mode 100644
pinot-controller/src/test/resources/recommenderInput/KafkaPartitionRuleInput.json
create mode 100644
pinot-controller/src/test/resources/recommenderInput/KafkaPartitionRuleInput2.json
create mode 100644
pinot-controller/src/test/resources/recommenderInput/NoDictionaryOnHeapDictionaryJointRuleInput.json
create mode 100644
pinot-controller/src/test/resources/recommenderInput/PinotTablePartitionRuleInput.json
create mode 100644
pinot-controller/src/test/resources/recommenderInput/SortedInvertedIndexInput.json
create mode 100644
pinot-controller/src/test/resources/recommenderInput/VariedLengthDictionaryInput.json
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/query/postaggregation/PostAggregationFunction.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/query/postaggregation/PostAggregationFunctionTest.java
create mode 100644
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/AggregateMetricsClusterIntegrationTest.java
create mode 100644
pinot-spi/src/main/java/org/apache/pinot/spi/config/table/TierConfig.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/CollectMetadataForIndexTuning.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/EntriesScannedQuantileReport.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/IndexTunerCommand.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/driver/TunerDriver.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/meta/manager/JsonFileMetaManagerImpl.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/meta/manager/MetaManager.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/meta/manager/collector/ColStatsAccumulatorObj.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/meta/manager/collector/CompressedFilePathIter.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/meta/manager/collector/PathWrapper.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/meta/manager/collector/SegmentMetadataCollector.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/query/src/LogInputIteratorImpl.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/query/src/parser/BrokerLogParserImpl.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/query/src/stats/wrapper/IndexSuggestQueryStatsImpl.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/strategy/AbstractAccumulator.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/strategy/FrequencyAccumulator.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/strategy/FrequencyImpl.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/strategy/ParseBasedAccumulator.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/strategy/ParserBasedImpl.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/strategy/QuantileAnalysisAccumulator.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/strategy/QuantileAnalysisImpl.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/tuner/strategy/TuningStrategy.java
delete mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/anomaly/alert/util/DataReportHelper.java
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/api/detection/AnomalyDetectionResource.java
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/resources/RootResource.java
delete mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/views/contributor/ContributionCell.java
delete mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/views/contributor/ContributionViewTableBuilder.java
delete mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/views/contributor/ContributorViewHandler.java
delete mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/views/contributor/ContributorViewResponse.java
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/datasource/online/OnlineThirdEyeDataSource.java
create mode 100644
thirdeye/thirdeye-pinot/src/main/resources/detection-config-template.yml
delete mode 100644
thirdeye/thirdeye-pinot/src/main/resources/org/apache/pinot/thirdeye/detector/anomaly-report-v2.ftl
delete mode 100644
thirdeye/thirdeye-pinot/src/main/resources/org/apache/pinot/thirdeye/detector/anomaly-report.ftl
delete mode 100644
thirdeye/thirdeye-pinot/src/main/resources/org/apache/pinot/thirdeye/detector/custom-anomaly-report.ftl
delete mode 100644
thirdeye/thirdeye-pinot/src/main/resources/org/apache/pinot/thirdeye/detector/data-report-by-metric-dimension.ftl
create mode 100644
thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/api/detection/AnomalyDetectionResourceTest.java
delete mode 100644
thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/dashboard/handler/ContributorTest.java
create mode 100644
thirdeye/thirdeye-pinot/src/test/resources/org/apache/pinot/thirdeye/api/detection/payload-bad-custom.json
create mode 100644
thirdeye/thirdeye-pinot/src/test/resources/org/apache/pinot/thirdeye/api/detection/payload-bad.json
create mode 100644
thirdeye/thirdeye-pinot/src/test/resources/org/apache/pinot/thirdeye/api/detection/payload-good-custom.json
create mode 100644
thirdeye/thirdeye-pinot/src/test/resources/org/apache/pinot/thirdeye/api/detection/payload-good.json
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
