This is an automated email from the ASF dual-hosted git repository.
jackie pushed a change to branch multi_stage_query_engine
in repository https://gitbox.apache.org/repos/asf/pinot.git
omit 8ff2e098df add pinot-query-runtime (#8412)
omit 5984af97d6 Add pinot-query-planner module (#8340)
omit 66de3ba28e make github action trigger on multi_stage_query_engine
branch (#8342)
add 5e7c0050f6 Fix upsert inconsistency by snapshotting the validDocIds
before reading the numDocs (#8392)
add 465b0c3c7d Set numGroupsLimitReached flag for SQL query (#8393)
add a388f670b6 install sysstat, perf, openjdk11 debug symbols,
async-profiler (#8395)
add a9ae53e34b Fix ingestion transform config bugs. (#8394)
add 65b9804656 Datetime transform functions (#8397)
add 3ebbfcad98 reenable highest stream offset metrics (#8372)
add 2ac2883bce aggregation test for aggregation over least/greatest (#8401)
add 975b11a9a0 add sysstats and procps to Docker image (#8400)
add 68033a686d Revert "add segment size metric on segment push (#8387)"
(#8405)
add 95d88f27ba Raw data can include the data out of gapfill boundary.
(#8403)
add 4bb7e4ff49 fix async-profiler install (#8404)
add 37fcfb8fd5 Fixing TIMESTAMP data type usage during segment creation
(#8407)
add 54db0915c9 builtin datetime functions to avoid throwing
NullPointerException when input value is null (#8382)
add e02bdda150 New Pinot byte metrics for compressed tar.gz and table size
w/o replicas (#8358)
add ec95df4844 use lower level Joda API in transform function for 10-15%
performance boost (#8414)
add bf70c654b7 Make maven build in parallel mode during docker image build
(#8415)
add 515f7a2f95 Fix the flaky DistinctCountQueriesTest (#8416)
add 0d6b5da7c6 FixedSegmentSelector for tier configs (#8389)
add 649f5988d5 [PQL Cleanup] Cleanup PQL related tools (#8417)
add 7655fcce5d Adding new field expressionOverrideHints into PinotQuery
(#8421)
add a4794d4f19 upgrade thrift to 0.15.0 (#8427)
add 189fd7a818 Move Other HttpUtils to use HttpClient (#8390)
add ebd20a4e00 Add commonly used math, string and date scalar functions in
Pinot (#8304)
add b4da2dd83f collapse redundant function types and add missing aliases
(#8430)
add 878cf75d71 Update master branch with the next release pom changes
(#8432)
add 6ec6d4a473 Adding pinot helm 0.2.5 (#8435)
add 24cc357b2a update 0.2.5 index with new zookeeper dependency (#8436)
add d2ca07ee57 Combine Metadata and Dictionary based plans into single
operator (#8408)
add 5b44c22b46 Remove secrets from job spec (#8420)
add 46b821c312 Avoid creation of max length result arrays (#8419)
add 67fa9dc4e7 [PQL Cleanup] Cleanup PQL in integration tests (#8425)
add b2f0cd8928 Allow ingesting Avro complex types into JSON column. (#8426)
add deaccf7768 Adjust comparison of numDocsScanned (#8433)
add 2279e71416 accelerate counts over bitmap based filters (#8411)
add e2053f6776 Refactor getMin/getMax value to IndexCreationContext from
Range (#8442)
add 51904fb99e Remove the tableDataManager deletion mechanism when the
number of segments drops to 0 to avoid race conditions. (#8422)
add c6f07626ad Make codestyle keep line break when reformat the code
(#8447)
add 91c9ce4aeb Fix a typo in BaseBrokerRequestHandler (#8448)
add 86958b6e0b Adding min/max value for column stats during segment
creation (#8452)
add 8dcb25580e Fix the group-by reduce handling when query times out
(#8450)
add 8a7a64d98d Add DocIdSetOprator to explain plan (#8446)
add 94a869b4c2 Fix ChildTraceId when using multiple child threads, make
them unique (#8443)
add d454acdd44 adding a constructor override (#8454)
add e7a2f5b2c0 Disable GitHub action on forks (#8456)
add 370d86c298 add metric for largest segment size (#8445)
add 1f5c4cf3de Rewrite PinotQuery based on expression hints at
instance/segment level (#8451)
add 5e56b9a155 Make ExpressionContext using ArrayList for replacement
(#8459)
add 164b7a4cd0 Adding Github Actions to build Pinot/Presto/Superset docker
images (#8458)
add 2d22c6d12c Bump up frontend-maven-plugin (#8461)
add d7be2ef412 Fetch Pulsar offsets from Consumer interface instead of
Reader (#8017)
add 9178618ff0 Fix to query inconsistencies under heavy upsert load
(resolves #7958) (#7971)
add 52e5a2c3a6 make tableNameMap always cache list of tables (#8475)
new b6763beee2 make github action trigger on multi_stage_query_engine
branch (#8342)
new 4874e203c9 Add pinot-query-planner module (#8340)
new 005d5b9965 add pinot-query-runtime (#8412)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (8ff2e098df)
\
N -- N -- N refs/heads/multi_stage_query_engine (005d5b9965)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/build-pinot-docker-image.yml | 69 +
.github/workflows/build-presto-docker-image.yml | 70 +
.github/workflows/build-superset-docker-image.yml | 65 +
.github/workflows/pinot_tests.yml | 12 +
.../docker/.pinot_docker_image_build_and_push.sh | 61 +
.../docker/.presto_docker_image_build_and_push.sh | 70 +
.../.superset_docker_image_build_and_push.sh | 59 +
config/codestyle-intellij.xml | 2 +-
docker/images/pinot/Dockerfile | 16 +-
.../etc/jmx_prometheus_javaagent/configs/pinot.yml | 22 +-
kubernetes/helm/index.yaml | 28 +
kubernetes/helm/pinot-0.2.5.tgz | Bin 0 -> 50178 bytes
kubernetes/helm/pinot/Chart.yaml | 2 +-
kubernetes/helm/pinot/values.yaml | 2 +-
pinot-broker/pom.xml | 2 +-
.../requesthandler/BaseBrokerRequestHandler.java | 2 +-
.../broker/broker/HelixBrokerStarterTest.java | 2 +-
pinot-clients/pinot-java-client/pom.xml | 2 +-
.../java/org/apache/pinot/client/Connection.java | 86 +-
.../client/JsonAsyncHttpPinotClientTransport.java | 26 +-
.../apache/pinot/client/PinotClientTransport.java | 3 +
.../org/apache/pinot/client/PreparedStatement.java | 13 +-
.../main/java/org/apache/pinot/client/Request.java | 1 +
.../java/org/apache/pinot/client/package-info.java | 4 +-
.../apache/pinot/client/PreparedStatementTest.java | 3 +-
pinot-clients/pinot-jdbc-client/pom.xml | 2 +-
.../pinot/client/PinotPreparedStatement.java | 12 +-
.../org/apache/pinot/client/PinotStatement.java | 17 +-
pinot-clients/pom.xml | 2 +-
pinot-common/pom.xml | 31 +-
.../pinot/common/config/provider/TableCache.java | 52 +-
.../common/function/DateTimePatternHandler.java | 8 +
.../apache/pinot/common/function/FunctionInfo.java | 8 +-
.../pinot/common/function/FunctionRegistry.java | 13 +-
.../common/function/TransformFunctionType.java | 73 +-
.../function/scalar/ArithmeticFunctions.java | 53 +-
.../common/function/scalar/DateTimeFunctions.java | 38 +-
.../common/function/scalar/JsonFunctions.java | 27 +-
.../common/function/scalar/StringFunctions.java | 110 +-
.../function/scalar/TrigonometricFunctions.java | 92 +
...eloadMessage.java => TableDeletionMessage.java} | 32 +-
.../common/metadata/segment/SegmentZKMetadata.java | 8 +
.../pinot/common/metrics/ControllerGauge.java | 12 +-
.../apache/pinot/common/metrics/ServerGauge.java | 3 -
.../apache/pinot/common/metrics/ServerMeter.java | 1 +
.../pinot/common/request/AggregationInfo.java | 48 +-
.../apache/pinot/common/request/BrokerRequest.java | 150 +-
.../apache/pinot/common/request/DataSource.java | 36 +-
.../apache/pinot/common/request/Expression.java | 42 +-
.../pinot/common/request/ExpressionType.java | 6 +-
.../pinot/common/request/FilterOperator.java | 8 +-
.../apache/pinot/common/request/FilterQuery.java | 58 +-
.../pinot/common/request/FilterQueryMap.java | 26 +-
.../org/apache/pinot/common/request/Function.java | 62 +-
.../org/apache/pinot/common/request/GroupBy.java | 40 +-
.../pinot/common/request/HavingFilterQuery.java | 58 +-
.../pinot/common/request/HavingFilterQueryMap.java | 24 +-
.../apache/pinot/common/request/Identifier.java | 18 +-
.../pinot/common/request/InstanceRequest.java | 48 +-
.../org/apache/pinot/common/request/Literal.java | 28 +-
.../apache/pinot/common/request/PinotQuery.java | 402 +-
.../apache/pinot/common/request/QuerySource.java | 20 +-
.../org/apache/pinot/common/request/QueryType.java | 44 +-
.../org/apache/pinot/common/request/Selection.java | 48 +-
.../apache/pinot/common/request/SelectionSort.java | 26 +-
.../request/context/RequestContextUtils.java | 8 +-
.../request/context/predicate/BasePredicate.java | 28 +-
.../request/context/predicate/EqPredicate.java | 9 +-
.../request/context/predicate/InPredicate.java | 10 +-
.../context/predicate/IsNotNullPredicate.java | 10 +-
.../request/context/predicate/IsNullPredicate.java | 10 +-
.../context/predicate/JsonMatchPredicate.java | 10 +-
.../request/context/predicate/NotEqPredicate.java | 10 +-
.../request/context/predicate/NotInPredicate.java | 10 +-
.../request/context/predicate/Predicate.java | 5 +
.../request/context/predicate/RangePredicate.java | 12 +-
.../context/predicate/RegexpLikePredicate.java | 10 +-
.../context/predicate/TextMatchPredicate.java | 10 +-
.../pinot/common/response/ProcessingException.java | 26 +-
...Selector.java => FixedTierSegmentSelector.java} | 54 +-
.../org/apache/pinot/common/tier/TierFactory.java | 8 +
.../apache/pinot/common/utils/PinotDataType.java | 22 +-
.../pinot/common/utils/SqlResultComparator.java | 66 +-
.../pinot/common/utils/config/TierConfigUtils.java | 25 +-
.../apache/pinot/common/utils/http/HttpClient.java | 76 +-
.../parsers/PinotQuery2BrokerRequestConverter.java | 2 +-
.../apache/pinot/sql/parsers/CalciteSqlParser.java | 3 +-
.../pinot/common/function/JsonFunctionsTest.java | 2 +-
.../pinot/common/tier/TierConfigUtilsTest.java | 63 +-
.../pinot/common/tier/TierSegmentSelectorTest.java | 54 +
.../pinot/common/utils/PinotDataTypeTest.java | 7 +-
.../common/utils/config/TableConfigSerDeTest.java | 18 +-
pinot-common/src/thrift/query.thrift | 1 +
pinot-compatibility-verifier/pom.xml | 2 +-
.../main/java/org/apache/pinot/compat/TableOp.java | 7 +-
pinot-connectors/pinot-flink-connector/pom.xml | 23 +-
.../pinot/connector/flink/FlinkQuickStart.java | 9 +-
.../pinot/connector/flink/http/HttpClient.java | 130 -
.../pinot/connector/flink/http/HttpException.java | 38 -
.../connector/flink/http/PinotConnectionUtils.java | 27 +-
.../flink/http/PinotControllerClient.java | 146 -
.../flink/http/PinotControllerClientTest.java | 165 -
.../flink/sink/PinotSinkIntegrationTest.java | 7 +-
pinot-connectors/pinot-spark-connector/pom.xml | 2 +-
pinot-connectors/pom.xml | 2 +-
.../pinot-common-jdk8/pom.xml | 31 +-
.../pinot-core-jdk8/pom.xml | 2 +-
.../pinot-segment-local-jdk8/pom.xml | 2 +-
.../pinot-segment-spi-jdk8/pom.xml | 2 +-
.../pinot-spi-jdk8/pom.xml | 2 +-
.../prestodb-pinot-dependencies/pom.xml | 2 +-
.../presto-pinot-driver/pom.xml | 2 +-
pinot-controller/pom.xml | 4 +-
.../pinot/controller/BaseControllerStarter.java | 3 +-
.../apache/pinot/controller/ControllerConf.java | 2 +-
.../PinotSegmentUploadDownloadRestletResource.java | 26 +-
.../pinot/controller/api/upload/ZKOperator.java | 20 +-
.../controller/helix/ControllerRequestClient.java | 101 +
.../helix/ControllerRequestURLBuilder.java | 16 +-
.../controller/helix/SegmentStatusChecker.java | 35 +-
.../helix/core/PinotHelixResourceManager.java | 84 +-
.../pinot/controller/util/TableSizeReader.java | 76 +-
.../controller/validation/StorageQuotaChecker.java | 14 +-
.../pinot/controller/ControllerTestUtils.java | 60 +-
.../pinot/controller/api/TableSizeReaderTest.java | 85 +-
.../controller/api/upload/ZKOperatorTest.java | 16 +-
.../pinot/controller/helix/ControllerTest.java | 256 +-
.../controller/helix/SegmentStatusCheckerTest.java | 95 +-
.../pinot/controller/helix/TableCacheTest.java | 38 +-
...NonReplicaGroupTieredSegmentAssignmentTest.java | 6 +-
...NonReplicaGroupTieredSegmentAssignmentTest.java | 6 +-
.../core/minion/PinotTaskManagerStatelessTest.java | 9 +
.../TableRebalancerClusterStatelessTest.java | 24 +-
.../validation/StorageQuotaCheckerTest.java | 7 +-
.../validation/ValidationManagerTest.java | 3 +-
pinot-core/pom.xml | 2 +-
.../core/data/manager/InstanceDataManager.java | 6 +
.../realtime/LLRealtimeSegmentDataManager.java | 10 +-
.../pinot/core/operator/DocIdSetOperator.java | 2 +-
.../combine/GroupByOrderByCombineOperator.java | 12 +-
.../core/operator/filter/AndFilterOperator.java | 24 +
.../core/operator/filter/BaseFilterOperator.java | 28 +
.../operator/filter/BitmapBasedFilterOperator.java | 70 +-
.../core/operator/filter/BitmapCollection.java | 129 +
.../core/operator/filter/EmptyFilterOperator.java | 10 +
.../operator/filter/JsonMatchFilterOperator.java | 20 +
.../operator/filter/MatchAllFilterOperator.java | 10 +
.../core/operator/filter/NotFilterOperator.java | 20 +
.../core/operator/filter/OrFilterOperator.java | 23 +
.../filter/SortedIndexBasedFilterOperator.java | 87 +
.../operator/filter/TextMatchFilterOperator.java | 20 +
.../query/AggregationGroupByOrderByOperator.java | 13 +-
...perator.java => FastFilteredCountOperator.java} | 64 +-
...r.java => NonScanBasedAggregationOperator.java} | 85 +-
.../function/AdditionTransformFunction.java | 8 +-
.../function/ArrayAverageTransformFunction.java | 16 +-
.../function/ArrayLengthTransformFunction.java | 18 +-
.../function/ArrayMaxTransformFunction.java | 41 +-
.../function/ArrayMinTransformFunction.java | 45 +-
.../function/ArraySumTransformFunction.java | 9 +-
.../transform/function/BaseTransformFunction.java | 56 +-
.../transform/function/CastTransformFunction.java | 86 +-
.../DateTimeConversionTransformFunction.java | 15 +-
.../function/DateTimeTransformFunction.java | 265 +
.../function/DateTruncTransformFunction.java | 6 +-
.../function/DivisionTransformFunction.java | 9 +-
.../function/InIdSetTransformFunction.java | 8 +-
.../transform/function/InTransformFunction.java | 6 +-
.../function/JsonExtractKeyTransformFunction.java | 10 +-
.../function/LogicalOperatorTransformFunction.java | 8 +-
.../function/MapValueTransformFunction.java | 8 +-
.../function/ModuloTransformFunction.java | 9 +-
.../function/MultiplicationTransformFunction.java | 8 +-
...rmFunction.java => PowerTransformFunction.java} | 59 +-
.../function/RoundDecimalTransformFunction.java | 113 +
.../function/ScalarTransformFunctionWrapper.java | 76 +-
.../function/SingleParamMathTransformFunction.java | 75 +-
.../function/SubtractionTransformFunction.java | 9 +-
.../function/TimeConversionTransformFunction.java | 9 +-
.../function/TransformFunctionFactory.java | 241 +-
.../function/TrigonometricTransformFunctions.java | 268 +
.../function/TruncateDecimalTransformFunction.java | 111 +
.../function/ValueInTransformFunction.java | 38 +-
.../datetime/DateTimeTransformerFactory.java | 6 +-
.../pinot/core/plan/AggregationPlanNode.java | 85 +-
.../org/apache/pinot/core/plan/FilterPlanNode.java | 80 +-
.../core/plan/maker/InstancePlanMakerImplV2.java | 83 +
.../pinot/core/query/reduce/GapfillProcessor.java | 30 +-
.../core/query/reduce/GroupByDataTableReducer.java | 52 +-
.../core/query/request/context/QueryContext.java | 31 +-
.../BrokerRequestToQueryContextConverter.java | 15 +-
.../query/selection/SelectionOperatorUtils.java | 2 +-
.../apache/pinot/core/util/trace/TraceContext.java | 5 +-
.../data/function/ArithmeticFunctionsTest.java | 15 +
.../core/data/function/DateTimeFunctionsTest.java | 133 +-
.../function/InbuiltFunctionEvaluatorTest.java | 33 +-
.../core/operator/filter/BitmapCollectionTest.java | 237 +
.../function/DateTimeTransformFunctionTest.java | 110 +
.../function/PowerTransformFunctionTest.java | 93 +
.../RoundDecimalTransformFunctionTest.java | 68 +
.../ScalarTransformFunctionWrapperTest.java | 126 +-
.../SingleParamMathTransformFunctionTest.java | 138 +
.../function/TrigonometricFunctionsTest.java | 163 +
.../TruncateDecimalTransformFunctionTest.java | 68 +
.../apache/pinot/core/plan/FilterPlanNodeTest.java | 90 +
...adataAndDictionaryAggregationPlanMakerTest.java | 24 +-
.../plan/maker/QueryOverrideWithHintsTest.java | 233 +
.../core/query/reduce/BrokerReduceServiceTest.java | 79 +
.../pinot/core/query/scheduler/TestHelper.java | 3 +-
.../selection/SelectionOperatorServiceTest.java | 2 +-
.../pinot/core/util/trace/TraceContextTest.java | 8 +-
.../pinot/queries/DistinctCountQueriesTest.java | 39 +-
.../pinot/queries/ExplainPlanQueriesTest.java | 177 +-
.../pinot/queries/FastFilteredCountTest.java | 300 +
.../apache/pinot/queries/GapfillQueriesTest.java | 339 +
...terSegmentResultTableMultiValueQueriesTest.java | 18 +
...erSegmentResultTableSingleValueQueriesTest.java | 18 +
.../queries/JsonIngestionFromAvroQueriesTest.java | 341 +
.../pinot/queries/NotOperatorQueriesTest.java | 5 +-
...SegmentPartitionedDistinctCountQueriesTest.java | 6 +-
.../pinot/queries/SumGreatestLeastQueriesTest.java | 161 +
.../pinot/queries/TextSearchQueriesTest.java | 4 +-
pinot-distribution/pom.xml | 2 +-
pinot-integration-test-base/pom.xml | 2 +-
.../tests/BaseClusterIntegrationTest.java | 84 +-
.../tests/ClusterIntegrationTestUtils.java | 345 +-
.../pinot/integration/tests/ClusterTest.java | 105 +-
...ridClusterIntegrationTestCommandLineRunner.java | 441 -
.../pinot/integration/tests/QueryGenerator.java | 360 +-
pinot-integration-tests/pom.xml | 10 +-
.../tests/AdminConsoleIntegrationTest.java | 9 +-
.../AggregateMetricsClusterIntegrationTest.java | 14 +-
.../tests/BaseClusterIntegrationTestSet.java | 319 +-
.../tests/BasicAuthBatchIntegrationTest.java | 1 -
.../tests/BasicAuthRealtimeIntegrationTest.java | 7 +-
.../tests/GrpcBrokerClusterIntegrationTest.java | 17 +-
.../tests/HybridClusterIntegrationTest.java | 22 +-
.../IngestionConfigHybridIntegrationTest.java | 14 +-
.../tests/JsonPathClusterIntegrationTest.java | 235 +-
.../tests/LLCRealtimeClusterIntegrationTest.java | 4 +-
.../LuceneRealtimeClusterIntegrationTest.java | 2 +-
.../tests/MapTypeClusterIntegrationTest.java | 182 +-
.../MergeRollupMinionClusterIntegrationTest.java | 16 +-
...onaryAggregationPlanClusterIntegrationTest.java | 341 -
.../tests/NullHandlingIntegrationTest.java | 17 +-
.../tests/OfflineClusterIntegrationTest.java | 1110 +-
.../tests/RealtimeClusterIntegrationTest.java | 73 +-
.../tests/RealtimeKinesisIntegrationTest.java | 11 +-
...fflineSegmentsMinionClusterIntegrationTest.java | 59 +-
...PartitionLLCRealtimeClusterIntegrationTest.java | 16 +-
.../SegmentWriterUploaderIntegrationTest.java | 2 +-
.../tests/StarTreeClusterIntegrationTest.java | 34 +-
.../tests/ThetaSketchIntegrationTest.java | 36 +-
.../integration/tests/TlsIntegrationTest.java | 7 +-
.../tests/startree}/SegmentInfoProvider.java | 6 +-
.../tests/startree}/StarTreeQueryGenerator.java | 76 +-
...rformance_2014_100k_subset.test_queries_10K.pql | 10019 -------------------
...rformance_2014_100k_subset.test_queries_200.pql | 234 -
pinot-minion/pom.xml | 2 +-
pinot-perf/pom.xml | 2 +-
.../org/apache/pinot/perf/BenchmarkQueries.java | 34 +-
.../perf/BenchmarkRealtimeConsumptionSpeed.java | 8 +-
.../pinot-batch-ingestion-common/pom.xml | 2 +-
.../pinot-batch-ingestion-hadoop/pom.xml | 2 +-
.../pinot-batch-ingestion-spark/pom.xml | 2 +-
.../pinot-batch-ingestion-standalone/pom.xml | 2 +-
pinot-plugins/pinot-batch-ingestion/pom.xml | 2 +-
.../v0_deprecated/pinot-hadoop/pom.xml | 2 +-
.../job/mappers/SegmentPreprocessingMapper.java | 3 +-
.../v0_deprecated/pinot-ingestion-common/pom.xml | 2 +-
.../v0_deprecated/pinot-spark/pom.xml | 2 +-
.../pinot-batch-ingestion/v0_deprecated/pom.xml | 2 +-
.../pinot-environment/pinot-azure/pom.xml | 2 +-
pinot-plugins/pinot-environment/pom.xml | 2 +-
pinot-plugins/pinot-file-system/pinot-adls/pom.xml | 2 +-
pinot-plugins/pinot-file-system/pinot-gcs/pom.xml | 2 +-
pinot-plugins/pinot-file-system/pinot-hdfs/pom.xml | 2 +-
pinot-plugins/pinot-file-system/pinot-s3/pom.xml | 2 +-
pinot-plugins/pinot-file-system/pom.xml | 2 +-
.../pinot-input-format/pinot-avro-base/pom.xml | 2 +-
.../avro/AvroIngestionSchemaValidator.java | 7 +
.../plugin/inputformat/avro/AvroSchemaUtil.java | 7 +
.../inputformat/avro/AvroRecordExtractorTest.java | 8 +-
.../pinot-input-format/pinot-avro/pom.xml | 2 +-
.../pinot-confluent-avro/pom.xml | 2 +-
pinot-plugins/pinot-input-format/pinot-csv/pom.xml | 2 +-
.../pinot-input-format/pinot-json/pom.xml | 2 +-
pinot-plugins/pinot-input-format/pinot-orc/pom.xml | 2 +-
.../pinot-input-format/pinot-parquet/pom.xml | 2 +-
.../pinot-input-format/pinot-protobuf/pom.xml | 2 +-
.../pinot-input-format/pinot-thrift/pom.xml | 2 +-
pinot-plugins/pinot-input-format/pom.xml | 2 +-
.../pinot-metrics/pinot-dropwizard/pom.xml | 2 +-
pinot-plugins/pinot-metrics/pinot-yammer/pom.xml | 2 +-
pinot-plugins/pinot-metrics/pom.xml | 2 +-
.../pinot-minion-builtin-tasks/pom.xml | 2 +-
pinot-plugins/pinot-minion-tasks/pom.xml | 2 +-
.../pinot-segment-uploader-default/pom.xml | 2 +-
pinot-plugins/pinot-segment-uploader/pom.xml | 2 +-
.../pinot-segment-writer-file-based/pom.xml | 2 +-
pinot-plugins/pinot-segment-writer/pom.xml | 2 +-
.../pinot-stream-ingestion/pinot-kafka-0.9/pom.xml | 2 +-
.../pinot-stream-ingestion/pinot-kafka-2.0/pom.xml | 2 +-
.../pinot-kafka-base/pom.xml | 2 +-
.../pinot-stream-ingestion/pinot-kinesis/pom.xml | 2 +-
.../pinot-stream-ingestion/pinot-pulsar/pom.xml | 2 +-
.../pinot/plugin/stream/pulsar/PulsarConfig.java | 21 +-
.../plugin/stream/pulsar/PulsarMessageBatch.java | 26 +-
.../PulsarPartitionLevelConnectionHandler.java | 2 +-
.../pulsar/PulsarStreamMetadataProvider.java | 59 +-
.../pinot/plugin/stream/pulsar/PulsarUtils.java | 54 +
.../plugin/stream/pulsar/PulsarConsumerTest.java | 117 +-
pinot-plugins/pinot-stream-ingestion/pom.xml | 2 +-
pinot-plugins/pom.xml | 2 +-
pinot-segment-local/pom.xml | 2 +-
.../local/function/InbuiltFunctionEvaluator.java | 20 +
.../recordtransformer/DataTypeTransformer.java | 5 +-
.../recordtransformer/ExpressionTransformer.java | 47 +-
.../creator/impl/DefaultIndexCreatorProvider.java | 4 +-
.../creator/impl/SegmentColumnarIndexCreator.java | 2 +
.../loader/invertedindex/RangeIndexHandler.java | 2 +-
.../segment/local/utils/TableConfigUtils.java | 2 +-
.../ExpressionTransformerTest.java | 71 +
.../segment/local/utils/TableConfigUtilsTest.java | 85 +-
pinot-segment-spi/pom.xml | 2 +-
.../segment/spi/creator/IndexCreationContext.java | 75 +-
.../name/NormalizedDateSegmentNameGenerator.java | 4 +-
pinot-server/pom.xml | 2 +-
.../server/starter/helix/BaseServerStarter.java | 6 +
.../starter/helix/HelixInstanceDataManager.java | 39 +-
.../helix/SegmentMessageHandlerFactory.java | 25 +
pinot-spi/pom.xml | 2 +-
.../pinot/spi/annotations/ScalarFunction.java | 3 +
.../apache/pinot/spi/config/table/TierConfig.java | 12 +
.../java/org/apache/pinot/spi/data/Schema.java | 4 +-
.../spi/ingestion/batch/IngestionJobLauncher.java | 5 +-
.../batch/spec/SegmentGenerationJobSpec.java | 10 +
.../apache/pinot/spi/utils/CommonConstants.java | 1 +
.../data/readers/AbstractRecordExtractorTest.java | 22 +-
pinot-tools/pom.xml | 64 +-
.../tools/query/comparison/ClusterStarter.java | 270 -
.../tools/query/comparison/QueryComparison.java | 571 --
.../query/comparison/QueryComparisonConfig.java | 209 -
.../tools/query/comparison/StatsGenerator.java | 87 -
.../apache/pinot/tools/scan/query/Aggregation.java | 199 -
.../pinot/tools/scan/query/AggregationFunc.java | 33 -
.../tools/scan/query/AggregationFuncFactory.java | 60 -
.../apache/pinot/tools/scan/query/AvgFunction.java | 57 -
.../pinot/tools/scan/query/CountFunction.java | 44 -
.../tools/scan/query/DistinctCountFunction.java | 57 -
.../tools/scan/query/EqualsPredicateFilter.java | 46 -
.../pinot/tools/scan/query/GroupByOperator.java | 75 -
.../pinot/tools/scan/query/InPredicateFilter.java | 52 -
.../apache/pinot/tools/scan/query/MaxFunction.java | 45 -
.../apache/pinot/tools/scan/query/MinFunction.java | 45 -
.../tools/scan/query/MinMaxRangeFunction.java | 47 -
.../tools/scan/query/NotInPredicateFilter.java | 52 -
.../pinot/tools/scan/query/NotPredicateFilter.java | 46 -
.../pinot/tools/scan/query/PredicateFilter.java | 26 -
.../apache/pinot/tools/scan/query/Projection.java | 141 -
.../pinot/tools/scan/query/QueryResponse.java | 271 -
.../tools/scan/query/RangePredicateFilter.java | 87 -
.../apache/pinot/tools/scan/query/ResultTable.java | 297 -
.../tools/scan/query/ScanBasedQueryProcessor.java | 184 -
.../tools/scan/query/SegmentQueryProcessor.java | 318 -
.../apache/pinot/tools/scan/query/Selection.java | 62 -
.../apache/pinot/tools/scan/query/SumFunction.java | 45 -
.../org/apache/pinot/tools/scan/query/Utils.java | 70 -
.../PullRequestMergedEventsStream.java | 15 +
pom.xml | 2 +-
370 files changed, 9740 insertions(+), 19204 deletions(-)
create mode 100644 .github/workflows/build-pinot-docker-image.yml
create mode 100644 .github/workflows/build-presto-docker-image.yml
create mode 100644 .github/workflows/build-superset-docker-image.yml
create mode 100755
.github/workflows/scripts/docker/.pinot_docker_image_build_and_push.sh
create mode 100755
.github/workflows/scripts/docker/.presto_docker_image_build_and_push.sh
create mode 100755
.github/workflows/scripts/docker/.superset_docker_image_build_and_push.sh
create mode 100644 kubernetes/helm/pinot-0.2.5.tgz
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/function/scalar/TrigonometricFunctions.java
copy
pinot-common/src/main/java/org/apache/pinot/common/messages/{SegmentReloadMessage.java
=> TableDeletionMessage.java} (57%)
copy
pinot-core/src/main/java/org/apache/pinot/core/query/reduce/filter/ValueExtractorFactory.java
=>
pinot-common/src/main/java/org/apache/pinot/common/request/context/predicate/BasePredicate.java
(67%)
copy
pinot-common/src/main/java/org/apache/pinot/common/tier/{TimeBasedTierSegmentSelector.java
=> FixedTierSegmentSelector.java} (50%)
delete mode 100644
pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/http/HttpClient.java
delete mode 100644
pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/http/HttpException.java
delete mode 100644
pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/http/PinotControllerClient.java
delete mode 100644
pinot-connectors/pinot-flink-connector/src/test/java/org/apache/pinot/connector/flink/http/PinotControllerClientTest.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/filter/BitmapCollection.java
rename
pinot-core/src/main/java/org/apache/pinot/core/operator/query/{MetadataBasedAggregationOperator.java
=> FastFilteredCountOperator.java} (51%)
rename
pinot-core/src/main/java/org/apache/pinot/core/operator/query/{DictionaryBasedAggregationOperator.java
=> NonScanBasedAggregationOperator.java} (69%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/DateTimeTransformFunction.java
copy
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/{MultiplicationTransformFunction.java
=> PowerTransformFunction.java} (51%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/RoundDecimalTransformFunction.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/TrigonometricTransformFunctions.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/TruncateDecimalTransformFunction.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/operator/filter/BitmapCollectionTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/operator/transform/function/DateTimeTransformFunctionTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunctionTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/operator/transform/function/RoundDecimalTransformFunctionTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/operator/transform/function/TrigonometricFunctionsTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/operator/transform/function/TruncateDecimalTransformFunctionTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/plan/FilterPlanNodeTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/plan/maker/QueryOverrideWithHintsTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/query/reduce/BrokerReduceServiceTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/queries/FastFilteredCountTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/queries/JsonIngestionFromAvroQueriesTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/queries/SumGreatestLeastQueriesTest.java
delete mode 100644
pinot-integration-test-base/src/test/java/org/apache/pinot/integration/tests/HybridClusterIntegrationTestCommandLineRunner.java
delete mode 100644
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/MetadataAndDictionaryAggregationPlanClusterIntegrationTest.java
rename {pinot-tools/src/main/java/org/apache/pinot/tools/query/comparison =>
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/startree}/SegmentInfoProvider.java
(96%)
rename {pinot-tools/src/main/java/org/apache/pinot/tools/query/comparison =>
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/startree}/StarTreeQueryGenerator.java
(82%)
delete mode 100644
pinot-integration-tests/src/test/resources/On_Time_On_Time_Performance_2014_100k_subset.test_queries_10K.pql
delete mode 100644
pinot-integration-tests/src/test/resources/On_Time_On_Time_Performance_2014_100k_subset.test_queries_200.pql
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarUtils.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/query/comparison/ClusterStarter.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/query/comparison/QueryComparison.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/query/comparison/QueryComparisonConfig.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/query/comparison/StatsGenerator.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/Aggregation.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/AggregationFunc.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/AggregationFuncFactory.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/AvgFunction.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/CountFunction.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/DistinctCountFunction.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/EqualsPredicateFilter.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/GroupByOperator.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/InPredicateFilter.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/MaxFunction.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/MinFunction.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/MinMaxRangeFunction.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/NotInPredicateFilter.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/NotPredicateFilter.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/PredicateFilter.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/Projection.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/QueryResponse.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/RangePredicateFilter.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/ResultTable.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/ScanBasedQueryProcessor.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/SegmentQueryProcessor.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/Selection.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/SumFunction.java
delete mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/Utils.java
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
