This is an automated email from the ASF dual-hosted git repository.
jlli pushed a change to branch support-spark-preprocessing
in repository https://gitbox.apache.org/repos/asf/pinot.git.
discard 6231d1a Support data preprocessing in Spark framework
add dbaac00 Fix errors happening due to parallel builds (#7172)
add 480395f Add Apache Pulsar low level and high level connector (#7026)
add a766c65 Updated CONTRIBUTING.md with Code Formatting Guidelines
(#7170)
add db5561a Improve the segment aggregated metadata controller/server API
(#7157)
add 738a584 Extract time handling for SegmentProcessorFramework (#7158)
add b6907fc add groovy unit test for sha256 data masking (#7179)
add fe83e95 Added a query param to /tables for getting sorted table names
based on time metadata (#7142)
add d6462cd Bug fix: JDBC driver throws null pointer exception when
metric columns are not present in Schema (#7175)
add febdcbc Use resourceAsStream to fetch resources in tests (#7181)
add e0163f4 Integrate enhanced SegmentProcessorFramework into
MergeRollupTaskExecutor (#7180)
add a0ad49d Make some things public (#7186)
add 4c04e1f [7156] Human Readable Controller Configs (#7173)
add 9e81326 Fix the config key for TASK_MANAGER_FREQUENCY_PERIOD (#7198)
add 039bd7b Move pinot url away from incubator (#7199)
add 2f3774b Reduce the disk usage for segment conversion task (#7193)
add d09c0ad Refactoring SegmentGenerationAndPushTask for extensibility
(#7188)
add ac7e5f9 Fixed pinot java client to add zkClient close (#7196)
add c6c407d Change build status badge to use github-action (#7201)
add 7e6cd9e Add the support of geoToH3 function (#7182)
add 25d14c2 Added ParquetNativeRecordReader in config's doc (#7204)
add 87ee80a Updating maven binary download link (#7209)
add 717940b Introduce OR Predicate Execution On Star Tree Index (#7184)
add c2c887a allow to extract values from array of objects with
jsonPathArray (#7208)
add 069bae0 Bump up the "grpc-protobuf-lite" version to 1.19.0 in Apache
Pulsar plugin (#7220)
add 5af6078 Fix the shaded issue with Apache Pulsar (#7223)
add 573651b Update license, notice for 0.8.0 release (#7225)
add 5e0dcff 7228 /sql REST call now compatible with auth (#7230)
add df9f5ef moving the map of pinot data type and java type to common
util (#7231)
add 95a6ae8 add cli args for auth tokens and user/password to ingestion
job command (#7233)
add 78b08e4 convert types per value for array with mixing types (#7234)
add 06ac649 ignore query json parse errors (#7165)
add 2831223 Support get metadata and indexes API for REALTIME tables.
(#7169)
add 29a6ba9 Make group trim size configurable for better accuracy (#7241)
add c1ac8a1 Do not shade jar for pinot native plug-in components (#7245)
add 1f24630 Cleanup: Make `jobType` case insensitive. (#7244)
add 6274e86 Allow any column as the upsert comparison column (#7200)
add 4da1dae Fix pulsar connector issue with `NoSuchMethodError` (#7247)
add 066c793 Adding shutdownhook for PinotServiceManager cmd (#7251)
add f99a2e6 move PinotSM shutdownhook to cleanup method (#7253)
add 329a419 Add missing `SegmentMetadataPushJobRunner` config in all the
example yaml files. (#7258)
add 0d96c7f Treat STRING to BOOLEAN data type change as backward
compatible schema change (#7259)
add 82fe4fa Replace gcp hardcoded values with generic annotations (#6985)
add d7e08b2 Split tests to multiple test set for github action (#7257)
add 337c58c Fix segment conversion executor for in-place conversion
(#7265)
add 5227266 extend Reload rest API with option to force download segment
(#7249)
add e4d391c Merge/Rollup task scheduler for offline tables. (#7178)
add 7add772 Add option to store the hash of the upsert primary key (#7246)
add c2637d1 Remove pinot-pulsar module from the binary release (#7272)
add d563611 Reformat tests (#7278)
add 6ef7f94 Enable hit counter and max hit rate tracker (#7236)
new 3bdd50c Support data preprocessing in Spark framework
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (6231d1a)
\
N -- N -- N refs/heads/support-spark-preprocessing (3bdd50c)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/pinot_tests.yml | 40 +-
...ot_test.sh => .pinot_compatibility_verifier.sh} | 11 +-
.github/workflows/scripts/.pinot_test.sh | 51 +-
CONTRIBUTING.md | 5 +
DISCLAIMER | 10 -
LICENSE-binary | 119 ++-
NOTICE | 2 +-
NOTICE-binary | 9 +-
README.md | 15 +-
compatibility-verifier/checkoutAndBuild.sh | 68 +-
compatibility-verifier/compCheck.sh | 21 +
docker/images/pinot/Dockerfile | 2 +-
.../docker/ingestion-job-specs/airlineStats.yaml | 12 +-
.../docker/ingestion-job-specs/baseballStats.yaml | 12 +-
.../pinot/templates/broker/service-external.yaml | 5 +-
.../helm/pinot/templates/broker/service.yaml | 5 +-
.../templates/controller/service-external.yaml | 5 +-
.../helm/pinot/templates/controller/service.yaml | 5 +-
.../helm/pinot/templates/minion/service.yaml | 5 +-
.../helm/pinot/templates/server/service.yaml | 5 +-
kubernetes/helm/pinot/values.yaml | 9 +-
licenses-binary/LICENSE-antlr4-runtime.txt | 28 +
...active-streams.txt => LICENSE-bouncycastle.txt} | 8 +-
...ENSE-google-auth.txt => LICENSE-google-api.txt} | 5 +-
licenses-binary/LICENSE-jcip.txt | 7 +
.../HelixExternalViewBasedQueryQuotaManager.java | 100 ++-
.../pinot/broker/queryquota/QueryQuotaEntity.java | 4 +
.../SegmentLineageBasedSegmentPreSelector.java | 14 +-
.../broker/broker/HelixBrokerStarterTest.java | 3 +-
...elixExternalViewBasedQueryQuotaManagerTest.java | 28 +-
.../org/apache/pinot/client/AbstractResultSet.java | 2 +-
.../org/apache/pinot/client/BrokerSelector.java | 5 +
.../java/org/apache/pinot/client/Connection.java | 1 +
.../apache/pinot/client/DynamicBrokerSelector.java | 24 +-
.../org/apache/pinot/client/ExecutionStats.java | 2 +-
.../apache/pinot/client/ResultTableResultSet.java | 2 +-
.../apache/pinot/client/SimpleBrokerSelector.java | 4 +
.../pinot/client/PinotConnectionMetaData.java | 24 +-
.../common/function/TransformFunctionType.java | 5 +-
.../common/function/scalar/JsonFunctions.java | 48 +-
.../pinot/common/lineage/SegmentLineageUtils.java | 18 +
.../common/messages/SegmentReloadMessage.java | 12 +-
.../apache/pinot/common/minion/Granularity.java | 27 -
.../common/minion/MergeRollupTaskMetadata.java | 19 +-
.../common/minion/MinionTaskMetadataUtils.java | 11 -
.../restlet/resources/TableMetadataInfo.java | 2 +
.../apache/pinot/common/utils/PinotDataType.java | 173 +++-
.../pinot/common/utils/TarGzCompressionUtils.java | 14 +-
.../org/apache/pinot/common/data/SchemaTest.java | 48 ++
.../pinot/common/function/JsonFunctionsTest.java | 78 ++
.../scalar/ArrayAwareJacksonJsonProviderTest.java | 114 +++
.../metadata/MetadataEqualsHashCodeTest.java | 34 +-
.../pinot/common/utils/PinotDataTypeTest.java | 106 ++-
.../common/utils/config/TableConfigSerDeTest.java | 3 +-
.../apache/pinot/controller/ControllerConf.java | 263 +++---
.../api/resources/PinotSegmentRestletResource.java | 94 ++-
.../api/resources/PinotTableRestletResource.java | 75 +-
.../controller/api/resources/ResourceUtils.java | 43 +
.../helix/ControllerRequestURLBuilder.java | 19 +-
.../helix/core/PinotHelixResourceManager.java | 42 +-
.../helix/core/minion/ClusterInfoAccessor.java | 25 +-
.../src/main/resources/app/requests/index.ts | 2 +-
.../main/resources/app/utils/PinotMethodUtils.ts | 20 +-
.../src/main/resources/app/utils/axios-config.ts | 4 +-
.../pinot/controller/ControllerConfTest.java | 205 +++++
.../controller/api/upload/ZKOperatorTest.java | 23 +-
.../pinot/controller/helix/ControllerTest.java | 19 +-
.../controller/helix/PinotResourceManagerTest.java | 64 +-
.../helix/core/PinotHelixResourceManagerTest.java | 219 ++---
.../helix/core/realtime/SegmentCompletionTest.java | 605 ++++++--------
.../helix/core/retention/RetentionManagerTest.java | 6 +-
.../validation/ValidationManagerTest.java | 38 +-
.../apache/pinot/core/common/MinionConstants.java | 65 +-
.../realtime/LLRealtimeSegmentDataManager.java | 4 +-
.../manager/realtime/RealtimeTableDataManager.java | 22 +-
.../transform/function/GeoToH3Function.java | 121 +++
.../transform/function/ScalarFunctions.java | 13 +
.../combine/GroupByOrderByCombineOperator.java | 29 +-
.../query/AggregationGroupByOrderByOperator.java | 58 +-
.../function/TransformFunctionFactory.java | 4 +
.../plan/AggregationGroupByOrderByPlanNode.java | 22 +-
.../core/plan/AggregationGroupByPlanNode.java | 4 +-
.../pinot/core/plan/AggregationPlanNode.java | 5 +-
.../apache/pinot/core/plan/CombinePlanNode.java | 24 +-
.../core/plan/maker/InstancePlanMakerImplV2.java | 85 +-
.../processing/framework/SegmentConfig.java | 2 +-
.../framework/SegmentProcessorConfig.java | 100 +--
.../framework/SegmentProcessorFramework.java | 123 +--
.../segment/processing/mapper/SegmentMapper.java | 52 +-
.../partitioner/ColumnValuePartitioner.java | 1 -
.../processing/partitioner/NoOpPartitioner.java | 1 -
.../processing/partitioner}/Partitioner.java | 2 +-
.../processing/partitioner/PartitionerFactory.java | 1 -
.../partitioner/RoundRobinPartitioner.java | 1 -
.../partitioner/TableConfigPartitioner.java | 1 -
.../partitioner/TransformFunctionPartitioner.java | 1 -
.../processing/timehandler/EpochTimeHandler.java | 69 ++
.../NoOpTimeHandler.java} | 11 +-
.../processing/timehandler/TimeHandler.java | 24 +-
.../processing/timehandler/TimeHandlerConfig.java | 106 +++
.../processing/timehandler/TimeHandlerFactory.java | 57 ++
.../core/startree/CompositePredicateEvaluator.java | 57 ++
.../apache/pinot/core/startree/StarTreeUtils.java | 151 +++-
.../startree/operator/StarTreeFilterOperator.java | 117 ++-
.../startree/plan/StarTreeDocIdSetPlanNode.java | 7 +-
.../core/startree/plan/StarTreeFilterPlanNode.java | 9 +-
.../startree/plan/StarTreeProjectionPlanNode.java | 4 +-
.../startree/plan/StarTreeTransformPlanNode.java | 5 +-
.../org/apache/pinot/core/util/QueryOptions.java | 38 +-
.../data/function/GroovyFunctionEvaluatorTest.java | 4 +
.../core/geospatial/transform/GeoFunctionTest.java | 17 +-
...lFunctionTest.java => GeoToH3FunctionTest.java} | 29 +-
.../operator/combine/CombineSlowOperatorsTest.java | 3 +-
.../combine/SelectionCombineOperatorTest.java | 3 +-
.../pinot/core/plan/CombinePlanNodeTest.java | 12 +-
...adataAndDictionaryAggregationPlanMakerTest.java | 6 +-
...InSegmentTrimTest.java => GroupByTrimTest.java} | 208 +++--
.../processing/framework/PartitionerTest.java | 2 +-
.../processing/framework/SegmentMapperTest.java | 140 ++--
.../framework/SegmentProcessorFrameworkTest.java | 798 ++++++------------
.../pinot/core/startree/v2/BaseStarTreeV2Test.java | 28 +-
...terSegmentAggregationMultiValueQueriesTest.java | 5 +-
...erSegmentAggregationSingleValueQueriesTest.java | 9 +-
.../InterSegmentOrderByMultiValueQueriesTest.java | 6 +-
.../InterSegmentOrderBySingleValueQueriesTest.java | 6 +-
pinot-distribution/pinot-assembly.xml | 24 +-
pinot-distribution/pom.xml | 4 +-
.../org/apache/pinot/compat/tests/StreamOp.java | 11 +-
.../tests/BaseClusterIntegrationTest.java | 35 +-
.../tests/BaseClusterIntegrationTestSet.java | 19 +-
.../ControllerPeriodicTasksIntegrationTest.java | 6 +-
.../tests/OfflineClusterIntegrationTest.java | 72 +-
...rDownloadLLCRealtimeClusterIntegrationTest.java | 12 +-
.../UpsertTableSegmentUploadIntegrationTest.java | 4 +-
.../segmentCreationAndTarPushJobSpec.yaml | 1 +
.../segmentCreationAndUriPushJobSpec.yaml | 1 +
.../src/main/resources/segmentCreationJobSpec.yaml | 1 +
.../src/main/resources/segmentTarPushJobSpec.yaml | 1 +
.../src/main/resources/segmentUriPushJobSpec.yaml | 1 +
.../segmentCreationAndTarPushJobSpec.yaml | 1 +
.../segmentCreationAndUriPushJobSpec.yaml | 1 +
.../src/main/resources/segmentCreationJobSpec.yaml | 1 +
.../src/main/resources/segmentTarPushJobSpec.yaml | 1 +
.../src/main/resources/segmentUriPushJobSpec.yaml | 1 +
.../segmentCreationAndTarPushJobSpec.yaml | 1 +
.../segmentCreationAndUriPushJobSpec.yaml | 1 +
.../src/main/resources/segmentCreationJobSpec.yaml | 1 +
.../src/main/resources/segmentTarPushJobSpec.yaml | 1 +
.../src/main/resources/segmentUriPushJobSpec.yaml | 1 +
.../hadoop/job/HadoopSegmentPreprocessingJob.java | 4 +-
.../hadoop/job/mappers/SegmentCreationMapper.java | 2 +-
.../HadoopAvroDataPreprocessingHelper.java | 18 +
.../preprocess/HadoopDataPreprocessingHelper.java | 22 +-
.../HadoopDataPreprocessingHelperFactory.java | 18 +
.../hadoop/job/preprocess/HadoopJobPreparer.java | 18 +
.../HadoopOrcDataPreprocessingHelper.java | 18 +
.../v0_deprecated/pinot-ingestion-common/pom.xml | 7 +
.../preprocess/AvroDataPreprocessingHelper.java | 10 -
.../preprocess/OrcDataPreprocessingHelper.java | 11 -
.../preprocess/SampleTimeColumnExtractable.java | 18 +
.../mappers/AvroDataPreprocessingMapper.java | 2 +-
.../mappers/OrcDataPreprocessingMapper.java | 2 +-
.../mappers/SegmentPreprocessingMapper.java | 2 +-
.../v0_deprecated/pinot-spark/pom.xml | 28 +-
.../spark/jobs/SparkSegmentPreprocessingJob.java | 24 +-
.../SparkAvroDataPreprocessingHelper.java | 46 +-
.../SparkDataPreprocessingComparator.java | 48 ++
.../preprocess/SparkDataPreprocessingHelper.java | 128 ++-
.../SparkDataPreprocessingHelperFactory.java | 18 +
.../preprocess/SparkDataPreprocessingJobKey.java | 22 +-
.../SparkDataPreprocessingPartitioner.java | 56 ++
.../SparkOrcDataPreprocessingHelper.java | 23 +-
.../pinot-minion-builtin-tasks/pom.xml | 2 +-
.../BaseMultipleSegmentsConversionExecutor.java | 36 +-
.../tasks/BaseSingleSegmentConversionExecutor.java | 24 +-
.../pinot/plugin/minion/tasks/MergeTaskUtils.java | 147 ++++
.../minion/tasks/merge_rollup/MergeProperties.java | 50 --
.../merge_rollup/MergeRollupTaskExecutor.java | 95 +--
.../merge_rollup/MergeRollupTaskGenerator.java | 452 +++++++++++
.../tasks/merge_rollup/MergeRollupTaskUtils.java | 108 +--
.../RealtimeToOfflineSegmentsTaskExecutor.java | 232 ++----
.../RealtimeToOfflineSegmentsTaskGenerator.java | 43 +-
.../SegmentGenerationAndPushTaskExecutor.java | 42 +-
.../plugin/minion/tasks/MergeTaskUtilsTest.java | 172 ++++
.../plugin/minion/tasks/TaskRegistryTest.java | 4 +-
.../merge_rollup/MergeRollupTaskExecutorTest.java | 3 +-
.../merge_rollup/MergeRollupTaskGeneratorTest.java | 696 ++++++++++++++++
.../merge_rollup/MergeRollupTaskUtilsTest.java | 86 +-
.../RealtimeToOfflineSegmentsTaskExecutorTest.java | 27 +-
...RealtimeToOfflineSegmentsTaskGeneratorTest.java | 8 +-
.../pinot-segment-uploader-default/pom.xml | 2 +-
.../pinot-segment-writer-file-based/pom.xml | 2 +-
.../pinot-stream-ingestion/pinot-pulsar/pom.xml | 397 +++++++++
.../stream/pulsar/MessageIdStreamOffset.java | 72 ++
.../pulsar/MessageIdStreamOffsetFactory.java} | 20 +-
.../pinot/plugin/stream/pulsar/PulsarConfig.java | 83 ++
.../stream/pulsar/PulsarConsumerFactory.java} | 25 +-
.../plugin/stream/pulsar/PulsarMessageBatch.java | 86 ++
.../PulsarPartitionLevelConnectionHandler.java | 79 ++
.../pulsar/PulsarPartitionLevelConsumer.java | 138 ++++
.../stream/pulsar/PulsarStreamLevelConsumer.java} | 71 +-
.../pulsar/PulsarStreamLevelConsumerManager.java | 175 ++++
.../pulsar/PulsarStreamMetadataProvider.java | 158 ++++
.../plugin/stream/pulsar/PulsarConsumerTest.java | 191 +++++
.../stream/pulsar/PulsarStandaloneCluster.java | 132 +++
.../src/test/resources/standalone.properties | 898 +++++++++++++++++++++
pinot-plugins/pinot-stream-ingestion/pom.xml | 1 +
.../immutable/ImmutableSegmentImpl.java | 4 -
.../indexsegment/mutable/MutableSegmentImpl.java | 15 +-
.../local/realtime/impl/RealtimeSegmentConfig.java | 30 +-
.../recordtransformer/DataTypeTransformer.java | 51 +-
.../upsert/PartitionUpsertMetadataManager.java | 76 +-
.../pinot/segment/local/upsert/RecordLocation.java | 11 +-
.../local/upsert/TableUpsertMetadataManager.java | 8 +-
.../local/utils/{H3Utils.java => HashUtils.java} | 19 +-
.../segment/local/utils/TableConfigUtils.java | 6 +
.../mutable/MutableSegmentImplTestUtils.java | 4 +
...MutableSegmentImplUpsertComparisonColTest.java} | 14 +-
.../mutable/MutableSegmentImplUpsertTest.java | 32 +-
.../recordtransformer/RecordTransformerTest.java | 6 +-
.../upsert/PartitionUpsertMetadataManagerTest.java | 150 ++--
.../pinot/segment/local/utils/HashUtilsTest.java | 18 +-
.../segment/local/utils/TableConfigUtilsTest.java | 14 +-
...a.json => test_upsert_comparison_col_data.json} | 8 +-
...json => test_upsert_comparison_col_schema.json} | 4 +
.../spi/index/metadata/SegmentMetadataImpl.java | 2 +-
.../api/resources/SegmentMetadataFetcher.java | 121 +--
.../pinot/server/api/resources/TablesResource.java | 32 +-
.../starter/helix/SegmentFetcherAndLoader.java | 33 +-
.../helix/SegmentMessageHandlerFactory.java | 21 +-
.../pinot/server/api/TablesResourceTest.java | 14 +-
.../apache/pinot/spi/config/table/TableConfig.java | 11 +
.../pinot/spi/config/table/UpsertConfig.java | 26 +-
.../java/org/apache/pinot/spi/data/Schema.java | 31 +-
.../apache/pinot/spi/data/readers/PrimaryKey.java | 5 +
.../spi/ingestion/batch/IngestionJobLauncher.java | 36 +-
.../batch/spec/SegmentGenerationJobSpec.java | 19 +-
.../java/org/apache/pinot/spi/utils/ByteArray.java | 3 +-
.../apache/pinot/spi/utils/CommonConstants.java | 4 +-
.../pinot/spi/config/table/UpsertConfigTest.java | 10 +-
.../pinot/spi/data/readers/PrimaryKeyTest.java | 11 +
.../resources/ingestion_job_spec_template.yaml | 1 +
.../command/LaunchDataIngestionJobCommand.java | 11 +
.../command/SegmentProcessorFrameworkCommand.java | 58 +-
.../admin/command/StartServiceManagerCommand.java | 2 +-
.../processor/SegmentProcessorFrameworkSpec.java | 40 +-
.../pinot/tools/service/PinotServiceManager.java | 12 +-
.../apache/pinot/tools/utils/PinotConfigUtils.java | 8 +-
.../batch/airlineStats/hadoopIngestionJobSpec.yaml | 12 +-
.../batch/airlineStats/ingestionJobSpec.yaml | 12 +-
.../batch/airlineStats/sparkIngestionJobSpec.yaml | 12 +-
.../batch/baseballStats/ingestionJobSpec.yaml | 13 +-
.../batch/baseballStats/sparkIngestionJobSpec.yaml | 12 +-
.../batch/dimBaseballTeams/ingestionJobSpec.yaml | 12 +-
.../ingestionJobComplexTypeHandlingSpec.yaml | 12 +-
.../batch/githubEvents/ingestionJobSpec.yaml | 12 +-
.../batch/githubEvents/sparkIngestionJobSpec.yaml | 12 +-
.../batch/starbucksStores/ingestionJobSpec.yaml | 12 +-
pom.xml | 12 +-
259 files changed, 9064 insertions(+), 3381 deletions(-)
copy .github/workflows/scripts/{.pinot_test.sh =>
.pinot_compatibility_verifier.sh} (78%)
delete mode 100644 DISCLAIMER
create mode 100644 licenses-binary/LICENSE-antlr4-runtime.txt
copy licenses-binary/{LICENSE-reactive-streams.txt =>
LICENSE-bouncycastle.txt} (74%)
copy licenses-binary/{LICENSE-google-auth.txt => LICENSE-google-api.txt} (96%)
create mode 100644 licenses-binary/LICENSE-jcip.txt
delete mode 100644
pinot-common/src/main/java/org/apache/pinot/common/minion/Granularity.java
create mode 100644
pinot-common/src/test/java/org/apache/pinot/common/function/scalar/ArrayAwareJacksonJsonProviderTest.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ResourceUtils.java
create mode 100644
pinot-controller/src/test/java/org/apache/pinot/controller/ControllerConfTest.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/geospatial/transform/function/GeoToH3Function.java
rename {pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/partition
=>
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/partitioner}/Partitioner.java
(94%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/timehandler/EpochTimeHandler.java
copy
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/{filter/NoOpRecordFilter.java
=> timehandler/NoOpTimeHandler.java} (75%)
copy
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/RecordTransformer.java
=>
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/timehandler/TimeHandler.java
(58%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/timehandler/TimeHandlerConfig.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/timehandler/TimeHandlerFactory.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/startree/CompositePredicateEvaluator.java
copy
pinot-core/src/test/java/org/apache/pinot/core/geospatial/transform/{StEqualFunctionTest.java
=> GeoToH3FunctionTest.java} (51%)
rename
pinot-core/src/test/java/org/apache/pinot/core/query/aggregation/groupby/{GroupByInSegmentTrimTest.java
=> GroupByTrimTest.java} (55%)
create mode 100644
pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-spark/src/main/java/org/apache/pinot/spark/jobs/preprocess/SparkDataPreprocessingComparator.java
copy
pinot-broker/src/main/java/org/apache/pinot/broker/routing/timeboundary/TimeBoundaryInfo.java
=>
pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-spark/src/main/java/org/apache/pinot/spark/jobs/preprocess/SparkDataPreprocessingJobKey.java
(65%)
create mode 100644
pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-spark/src/main/java/org/apache/pinot/spark/jobs/preprocess/SparkDataPreprocessingPartitioner.java
create mode 100644
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/MergeTaskUtils.java
delete mode 100644
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/merge_rollup/MergeProperties.java
create mode 100644
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/merge_rollup/MergeRollupTaskGenerator.java
create mode 100644
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/test/java/org/apache/pinot/plugin/minion/tasks/MergeTaskUtilsTest.java
create mode 100644
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/test/java/org/apache/pinot/plugin/minion/tasks/merge_rollup/MergeRollupTaskGeneratorTest.java
create mode 100644 pinot-plugins/pinot-stream-ingestion/pinot-pulsar/pom.xml
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/MessageIdStreamOffset.java
copy
pinot-plugins/pinot-stream-ingestion/{pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisMsgOffsetFactory.java
=>
pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/MessageIdStreamOffsetFactory.java}
(67%)
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarConfig.java
copy
pinot-plugins/pinot-stream-ingestion/{pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisConsumerFactory.java
=>
pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarConsumerFactory.java}
(74%)
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarMessageBatch.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarPartitionLevelConnectionHandler.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarPartitionLevelConsumer.java
copy
pinot-plugins/pinot-stream-ingestion/{pinot-kafka-0.9/src/main/java/org/apache/pinot/plugin/stream/kafka09/KafkaStreamLevelConsumer.java
=>
pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarStreamLevelConsumer.java}
(51%)
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarStreamLevelConsumerManager.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarStreamMetadataProvider.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/test/java/org/apache/pinot/plugin/stream/pulsar/PulsarConsumerTest.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/test/java/org/apache/pinot/plugin/stream/pulsar/PulsarStandaloneCluster.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/test/resources/standalone.properties
copy
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/{H3Utils.java
=> HashUtils.java} (73%)
copy
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/indexsegment/mutable/{MutableSegmentImplUpsertTest.java
=> MutableSegmentImplUpsertComparisonColTest.java} (91%)
copy
pinot-core/src/test/java/org/apache/pinot/core/metadata/MetadataExtractorFactoryTest.java
=>
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/utils/HashUtilsTest.java
(66%)
copy pinot-segment-local/src/test/resources/data/{test_upsert_data.json =>
test_upsert_comparison_col_data.json} (68%)
copy pinot-segment-local/src/test/resources/data/{test_upsert_schema.json =>
test_upsert_comparison_col_schema.json} (86%)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
