This is an automated email from the ASF dual-hosted git repository.
jlli pushed a change to branch add-counter-for-detecting-schema-mismatch
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard 8b0b778 Add Hadoop counter for detecting schema mismatch
add 2cfaed3 Support type conversion for all scalar functions (#5849)
add 09e9804 [TE] Fix wrong task pickup logic (#5855)
add 5469a84 add timeColumnName to tableConfig to enable TE auto-detection
(#5860)
add 449bf94 Fix NPE for aggregate metrics (#5862)
add 45d5d29 Add additional datetime functionality (#5438)
add 6dd54f8 Fix the variable names for off-heap alloc configs (#5852)
add 2b58bfb [TE] clean up legacy code (#5842)
add da1fe59 [TE] frontend - harleyjj/rca - fix heatmap click bug (#5880)
add 81e858b Adding integration test for null handling from realtime
source (#5870)
add 2d94cb9 Support post-aggregation in ORDER-BY (#5856)
add bae21bf [TE] anomaly search - hide child anomaly unless queried with
anomaly ids (#5865)
add 7eff806 [TE] merge time series snapshot when merging anomalies (#5861)
add 3173983 [TE] change dimension explore early stop to be 10% of top k
(#5882)
add aab087c Fixing backward-compatible issue of schema fetch call (#5885)
add ed2daaa Change group key delimiter from '\t' to '\0' (#5858)
add 87b5b77 Fix race condition in MetricsHelper (#5887)
add 41dc432 Bump up swagger ui version to 3.18.2 (#5888)
add a7f7f46 Fix gcsFS copyToLocal to precreate local destination
directory (#5890)
add 7a30a80 Roundup decimal points when compressing AnomalyTimelinesView
(#5869)
add e9ad1e0 Update swagger ui path (#5896)
add 89cd958 Support post-aggregation in SELECT (#5867)
add 4c659bf Add Hadoop counter for detecting schema mismatch
add d638096 Add SchemaValidator
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (8b0b778)
\
N -- N -- N refs/heads/add-counter-for-detecting-schema-mismatch
(d638096)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
LICENSE-binary | 2 +-
.../broker/broker/BrokerAdminApiApplication.java | 2 +-
.../pinot/common/function/DateTimeFunctions.java | 262 ----------
.../common/function/DateTimePatternHandler.java | 4 +-
.../apache/pinot/common/function/FunctionInfo.java | 44 +-
.../pinot/common/function/FunctionInvoker.java | 137 +++--
.../pinot/common/function/FunctionRegistry.java | 29 +-
.../pinot/common/function/FunctionUtils.java | 118 +++++
.../common/function/TransformFunctionType.java | 2 +-
.../function/annotations/ScalarFunction.java | 23 +-
.../function/scalar/ArithmeticFunctions.java | 95 ++++
.../common/function/scalar/DateTimeFunctions.java | 555 +++++++++++++++++++++
.../function/{ => scalar}/JsonFunctions.java | 9 +-
.../function/{ => scalar}/StringFunctions.java | 40 +-
.../pinot/common/metadata/ZKMetadataProvider.java | 5 +-
.../pinot/common/metrics/AbstractMetrics.java | 2 +
...eporterMetricsRegistryRegistrationListener.java | 6 +
.../apache/pinot/common/metrics/MetricsHelper.java | 15 +-
.../apache/pinot/common/utils}/PinotDataType.java | 32 +-
.../apache/pinot/sql/parsers/CalciteSqlParser.java | 35 +-
.../pinot/common/utils}/PinotDataTypeTest.java | 4 +-
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 57 ++-
.../api/ControllerAdminApiApplication.java | 2 +-
.../data/function/FunctionEvaluatorFactory.java | 15 +-
.../data/function/InbuiltFunctionEvaluator.java | 78 +--
.../manager/config/InstanceDataManagerConfig.java | 2 +-
.../realtime/HLRealtimeSegmentDataManager.java | 6 +-
.../realtime/LLRealtimeSegmentDataManager.java | 4 +-
.../recordtransformer/DataTypeTransformer.java | 1 +
.../apache/pinot/core/data/table/BaseTable.java | 47 +-
.../core/data/table/ConcurrentIndexedTable.java | 26 +-
.../apache/pinot/core/data/table/IndexedTable.java | 76 +--
.../pinot/core/data/table/SimpleIndexedTable.java | 22 +-
.../org/apache/pinot/core/data/table/Table.java | 9 +-
.../apache/pinot/core/data/table/TableResizer.java | 234 ++++++---
.../operator/combine/GroupByCombineOperator.java | 2 +-
.../combine/GroupByOrderByCombineOperator.java | 72 ++-
.../function/ScalarTransformFunctionWrapper.java | 331 ++++++------
.../function/TransformFunctionFactory.java | 25 +-
.../groupby/AggregationGroupByTrimmingService.java | 81 +--
.../aggregation/groupby/GroupKeyGenerator.java | 14 +-
.../postaggregation/PostAggregationFunction.java | 81 +++
.../query/reduce/AggregationDataTableReducer.java | 79 +--
.../core/query/reduce/GroupByDataTableReducer.java | 208 ++++----
.../core/query/reduce/PostAggregationHandler.java | 243 +++++++++
.../core/query/reduce/ResultReducerFactory.java | 4 +-
.../creator/SegmentIndexCreationDriver.java | 3 +
.../creator/impl/SegmentColumnarIndexCreator.java | 25 +-
.../impl/SegmentIndexCreationDriverImpl.java | 20 +-
.../segment/index/loader/IndexLoadingConfig.java | 16 +-
.../function/InbuiltFunctionEvaluatorTest.java | 91 ++--
.../core/data/function/InbuiltFunctionsTest.java | 160 +++++-
.../pinot/core/data/table/IndexedTableTest.java | 85 +---
.../pinot/core/data/table/TableResizerTest.java | 469 +++++++----------
.../ScalarTransformFunctionWrapperTest.java | 8 +-
.../PostAggregationFunctionTest.java | 62 +++
.../query/reduce/PostAggregationHandlerTest.java | 120 +++++
.../SegmentGenerationWithNullValueVectorTest.java | 3 +
...nerSegmentAggregationMultiValueQueriesTest.java | 19 +-
...erSegmentAggregationSingleValueQueriesTest.java | 17 +-
.../AggregationGroupByTrimmingServiceTest.java | 15 +-
.../query/aggregation/groupby/GroupKeyTest.java | 17 +-
.../pinot/query/executor/QueryExecutorTest.java | 5 +
.../AggregateMetricsClusterIntegrationTest.java | 130 +++++
.../tests/BaseClusterIntegrationTest.java | 10 +-
.../tests/BaseClusterIntegrationTestSet.java | 16 +
.../tests/NullHandlingIntegrationTest.java | 145 ++++++
.../src/test/resources/avro_data_with_nulls.tar.gz | Bin 0 -> 2973 bytes
.../src/test/resources/test_null_handling.schema | 33 ++
.../apache/pinot/perf/BenchmarkCombineGroupBy.java | 17 +-
.../apache/pinot/perf/BenchmarkIndexedTable.java | 21 +-
.../hadoop/job/mappers/SegmentCreationMapper.java | 71 +--
.../pinot/hadoop/data/SchemaValidatorTest.java | 50 ++
.../src/test/resources/data/test_sample_data.avro | Bin
.../v0_deprecated/pinot-ingestion-common/pom.xml | 61 +++
.../apache/pinot/plugin/filesystem/GcsPinotFS.java | 2 +
.../inputformat/avro/AvroSchemaValidator.java | 132 +++++
.../pinot/plugin/inputformat/avro/AvroUtils.java | 1 -
.../server/starter/helix/AdminApiApplication.java | 2 +-
.../helix/HelixInstanceDataManagerConfig.java | 4 +-
.../org/apache/pinot/spi/data/SchemaValidator.java | 21 +-
.../pinot/spi/data/SchemaValidatorFactory.java | 61 +++
.../spi/utils/builder/TableConfigBuilder.java | 7 +
.../PinotServiceManagerAdminApiApplication.java | 2 +-
.../resources/generator/complexWebsite_config.json | 3 +-
.../resources/generator/simpleWebsite_config.json | 3 +-
pom.xml | 4 +-
.../app/pods/components/heatmap-chart/component.js | 8 +-
.../anomaly/alert/util/DataReportHelper.java | 276 ----------
.../thirdeye/anomaly/alert/util/EmailHelper.java | 130 -----
.../pinot/thirdeye/anomaly/task/TaskDriver.java | 15 +-
.../views/CondensedAnomalyTimelinesView.java | 30 +-
.../dashboard/resources/SummaryResource.java | 24 +-
.../resources/v2/anomalies/AnomalySearcher.java | 8 +-
.../views/contributor/ContributionCell.java | 183 -------
.../views/contributor/ContributionViewTable.java | 48 --
.../contributor/ContributionViewTableBuilder.java | 142 ------
.../views/contributor/ContributorViewHandler.java | 331 ------------
.../views/contributor/ContributorViewRequest.java | 27 -
.../views/contributor/ContributorViewResponse.java | 109 ----
.../pinot/thirdeye/datalayer/bao/TaskManager.java | 2 +
.../datalayer/bao/jdbc/TaskManagerImpl.java | 27 +-
.../datalayer/dto/MergedAnomalyResultDTO.java | 1 +
.../detection/algorithm/DimensionWrapper.java | 5 +-
.../thirdeye/detection/yaml/YamlResource.java | 8 +-
.../content/BaseNotificationContent.java | 2 -
.../apache/pinot/thirdeye/util/ThirdEyeUtils.java | 89 +++-
.../pinot/thirdeye/detector/anomaly-report-v2.ftl | 136 -----
.../pinot/thirdeye/detector/anomaly-report.ftl | 150 ------
.../thirdeye/detector/custom-anomaly-report.ftl | 96 ----
.../detector/data-report-by-metric-dimension.ftl | 105 ----
.../views/TestCondensedAnomalyTimelinesView.java | 94 +++-
.../dashboard/handler/ContributorTest.java | 70 ---
.../v2/anomalies/AnomalySearcherTest.java | 22 +-
.../pinot/thirdeye/util/ThirdEyeUtilsTest.java | 68 +++
115 files changed, 3708 insertions(+), 3569 deletions(-)
delete mode 100644
pinot-common/src/main/java/org/apache/pinot/common/function/DateTimeFunctions.java
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/function/FunctionUtils.java
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java
rename pinot-common/src/main/java/org/apache/pinot/common/function/{ =>
scalar}/JsonFunctions.java (90%)
rename pinot-common/src/main/java/org/apache/pinot/common/function/{ =>
scalar}/StringFunctions.java (84%)
rename {pinot-core/src/main/java/org/apache/pinot/core/data/recordtransformer
=> pinot-common/src/main/java/org/apache/pinot/common/utils}/PinotDataType.java
(95%)
rename {pinot-core/src/test/java/org/apache/pinot/core/data/recordtransformer
=>
pinot-common/src/test/java/org/apache/pinot/common/utils}/PinotDataTypeTest.java
(98%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/query/postaggregation/PostAggregationFunction.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/query/reduce/PostAggregationHandler.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/query/postaggregation/PostAggregationFunctionTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/query/reduce/PostAggregationHandlerTest.java
create mode 100644
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/AggregateMetricsClusterIntegrationTest.java
create mode 100644
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/NullHandlingIntegrationTest.java
create mode 100644
pinot-integration-tests/src/test/resources/avro_data_with_nulls.tar.gz
create mode 100644
pinot-integration-tests/src/test/resources/test_null_handling.schema
create mode 100644
pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop/src/test/java/org/apache/pinot/hadoop/data/SchemaValidatorTest.java
copy {pinot-core =>
pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop}/src/test/resources/data/test_sample_data.avro
(100%)
create mode 100644
pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroSchemaValidator.java
copy pinot-core/src/main/java/org/apache/pinot/core/plan/Plan.java =>
pinot-spi/src/main/java/org/apache/pinot/spi/data/SchemaValidator.java (71%)
create mode 100644
pinot-spi/src/main/java/org/apache/pinot/spi/data/SchemaValidatorFactory.java
delete mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/anomaly/alert/util/DataReportHelper.java
delete mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/views/contributor/ContributionCell.java
delete mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/views/contributor/ContributionViewTable.java
delete mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/views/contributor/ContributionViewTableBuilder.java
delete mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/views/contributor/ContributorViewHandler.java
delete mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/views/contributor/ContributorViewRequest.java
delete mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/views/contributor/ContributorViewResponse.java
delete mode 100644
thirdeye/thirdeye-pinot/src/main/resources/org/apache/pinot/thirdeye/detector/anomaly-report-v2.ftl
delete mode 100644
thirdeye/thirdeye-pinot/src/main/resources/org/apache/pinot/thirdeye/detector/anomaly-report.ftl
delete mode 100644
thirdeye/thirdeye-pinot/src/main/resources/org/apache/pinot/thirdeye/detector/custom-anomaly-report.ftl
delete mode 100644
thirdeye/thirdeye-pinot/src/main/resources/org/apache/pinot/thirdeye/detector/data-report-by-metric-dimension.ftl
delete mode 100644
thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/dashboard/handler/ContributorTest.java
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
