This is an automated email from the ASF dual-hosted git repository. rongr pushed a change to branch multi_stage_query_engine in repository https://gitbox.apache.org/repos/asf/pinot.git
omit 08da4ee9e7 [hotfix] multi stage engine serde (#8689) omit 77b59bf9db row/columnar compatible block (#8583) omit c9bebeb504 [hotfix] fix server discovery (#8664) omit 6ae5f76d77 fix rebase issue after PQL deprecation (#8652) omit 58df2c2003 add support for project/filter pushdown (#8558) omit 1bbded3eac Use proto for query plan serialization (#8479) omit 0e38a5f4d5 add pinot-query-runtime (#8412) omit e819d11fa8 Add pinot-query-planner module (#8340) omit e8ddc1dbe8 make github action trigger on multi_stage_query_engine branch (#8342) add 6192229a60 Revert "Enable minion liveness probe (#8593)" (#8650) add ec42a3a6a4 fix column data type store for data table (#8648) add d0f9aaf5b3 fix GcsPinotFs listFiles by using bucket directly (#8656) add b4943458a5 [issue-7357] Add support for Avro logical types (#7358) add c0056dc278 Allow broker to automatically rewrite expensive function to its approximate counterpart (#8655) add 8fac50c038 [CI] Reduce quickstart testing to only LTS (#8665) add b9bfb8e752 Support ADLS authentication with AAD + proxy (#8654) add 497d0517ab Introduce Real Time Native Text Index (#8636) add d208f577ed Adding pinot file system command (#8659) add abf62f571d fix pruning in expressions by max/min/bloom (#8672) add 65cecd259e Randomly pick the first URI for RoundRobinURIProvider (#8666) add f4c85e9596 Fix the flaky UpsertTableSegmentUploadIntegrationTest (#8675) add 5135acb95d refine error msg related to segment name generation to be more actionable (#8676) add d616a097e5 Add BigDecimal OnHeap/OffHeap mutable dictionaries (#8678) add a12b760b74 Amend schema to add timestamp index during segment reloading (#8679) add d7f2054b6e Import dry run (#8673) add 23a81d07b5 Add TLS configuration to JDBC driver (#8578) add a7e2c12c99 Add InstanceTypeUtils to get instance type from the id (#8681) add 59bf3827db By default start 1 minion for all quickstart (#8682) add b58650658c Allow updating start/end offset for pushed segments (#8653) add 56282e8b2f track group by evaluation statistics (#8683) add 6d785e0eb3 Upgrade jersey to fix parallel segment upload (#8677) add e12bda1cf6 Fix the race condition of reading time boundary info (#8685) add 24a86acec0 Fixing GRPC ITCase timeout (#8686) add f9e55ce7a6 Handle the race condition for common dest segments in startReplaceSegments(). (#8667) add df29b81c27 Change minion's default port, enable minion health and readiness probe via helm chart (#8669) add 088da3f8c2 deduplicate predicates optimistically (#8687) add c3d17b9ed7 Fix PulsarConsumerTest (#8691) add cfb0074a2a Upgrade h3 to 3.7.2 for M1 Mac support (#8696) add f65b401dc7 Set max number of parallel segment downloads per table in pinot-server (#8694) add d06338acb8 avoid putting null into segment download uri (#8693) add 3983728e64 make exclusive bounds exclusive (#8703) add 078d8969bf Reduce wait time for pulsar tests (#8702) add d3c97d54d2 Add Pulsar plugin back to distribution (#8657) add f90137b1d5 rationalise Pair types (#8709) add e4a8e6ed60 throw when integer overflow occurs (#8710) add e788a0afc6 Fix when there're two index filter operator h3 inclusion index throw exception (#8707) add 159afb7820 Cleanup segment upload logic and allow validation on real-time table (#8695) add 7548b67592 CSV Realtime Decoder (#8658) add 70f505f028 Bugfixing, don't remove timestmap index during server restart. (#8705) add 09bae15030 Scalable Gapfill Implementation for Avg/Count/Sum (#8647) add d62438c36c [Minor] Make warning log more explicit for segment upload (#8715) add b8af790c2f Enable uploading segments to realtime tables (#8584) add b672af1b9f Disable Groovy function by default (#8711) add 3bbae6ef61 Remove big-decimal getters for lower/upper bound (#8712) add 2c3813bb97 Add pluggable client auth provider (#8670) new 90e722a6b0 make github action trigger on multi_stage_query_engine branch (#8342) new 8c8380005b Add pinot-query-planner module (#8340) new dacd28187a add pinot-query-runtime (#8412) new c06bbfbaf3 Use proto for query plan serialization (#8479) new 3b01b2dec8 add support for project/filter pushdown (#8558) new 30bc434343 fix rebase issue after PQL deprecation (#8652) new 4d77924e59 [hotfix] fix server discovery (#8664) new 2c07cb04e2 row/columnar compatible block (#8583) new fa2b2fa7cf [hotfix] multi stage engine serde (#8689) This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (08da4ee9e7) \ N -- N -- N refs/heads/multi_stage_query_engine (fa2b2fa7cf) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 9 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .github/workflows/pinot_tests.yml | 4 +- .github/workflows/scripts/.pinot_quickstart.sh | 33 +- LICENSE-binary | 42 +- .../config/BrokerConfig.properties | 1 + .../config/ControllerConfig.properties | 1 + .../broker/broker/helix/BaseBrokerStarter.java | 9 +- .../requesthandler/BaseBrokerRequestHandler.java | 299 +++++--- .../pinot/broker/routing/BrokerRoutingManager.java | 9 +- .../broker/broker/HelixBrokerStarterTest.java | 2 +- .../requesthandler/DistinctCountRewriteTest.java | 44 -- .../requesthandler/QueryLimitOverrideTest.java | 50 -- .../broker/requesthandler/QueryOverrideTest.java | 123 +++ pinot-clients/pinot-jdbc-client/pom.xml | 4 + .../java/org/apache/pinot/client/PinotDriver.java | 35 +- .../controller/PinotControllerTransport.java | 31 +- .../PinotControllerTransportFactory.java} | 12 +- .../org/apache/pinot/client/utils/DriverUtils.java | 19 +- .../org/apache/pinot/common/auth/AuthConfig.java | 28 +- .../pinot/common/auth/AuthProviderUtils.java | 171 +++++ .../NullAuthProvider.java} | 27 +- .../pinot/common/auth/StaticTokenAuthProvider.java | 68 ++ .../apache/pinot/common/auth/UrlAuthProvider.java | 87 +++ .../common/metadata/segment/SegmentZKMetadata.java | 3 +- .../org/apache/pinot/common/utils/DataSchema.java | 4 + .../common/utils/FileUploadDownloadClient.java | 45 +- .../java/org/apache/pinot/common/utils/Pair.java | 49 -- .../pinot/common/utils/RoundRobinURIProvider.java | 4 +- .../apache/pinot/common/utils/SegmentUtils.java | 32 +- .../org/apache/pinot/common/utils/URIUtils.java | 14 +- .../common/utils/fetcher/BaseSegmentFetcher.java | 7 +- .../common/utils/fetcher/HttpSegmentFetcher.java | 4 +- .../utils/fetcher/SegmentFetcherFactory.java | 15 +- .../pinot/common/utils/helix/HelixHelper.java | 4 +- .../apache/pinot/common/utils/http/HttpClient.java | 70 +- .../common/utils/RoundRobinURIProviderTest.java | 151 ++-- .../common/utils/config/TableConfigSerDeTest.java | 6 +- .../pinot/controller/BaseControllerStarter.java | 8 +- .../apache/pinot/controller/ControllerConf.java | 6 +- .../resources/PinotIngestionRestletResource.java | 32 +- .../PinotSegmentUploadDownloadRestletResource.java | 268 +++---- .../api/upload/SegmentValidationUtils.java | 94 +++ .../controller/api/upload/SegmentValidator.java | 122 --- .../pinot/controller/api/upload/ZKOperator.java | 246 +++--- .../helix/core/PinotHelixResourceManager.java | 189 +++-- .../segment/RealtimeSegmentAssignment.java | 50 +- .../core/minion/MinionInstancesCleanupTask.java | 4 +- .../helix/core/minion/TaskMetricsEmitter.java | 20 +- .../realtime/PinotLLCRealtimeSegmentManager.java | 16 +- .../core/realtime/PinotRealtimeSegmentManager.java | 10 +- .../helix/core/util/ZKMetadataUtils.java | 105 ++- .../pinot/controller/util/FileIngestionHelper.java | 10 +- .../pinot/controller/api/TableViewsTest.java | 9 +- .../controller/api/upload/ZKOperatorTest.java | 40 +- .../pinot/controller/helix/ControllerTest.java | 3 +- .../helix/core/PinotHelixResourceManagerTest.java | 62 +- ...altimeNonReplicaGroupSegmentAssignmentTest.java | 114 ++- ...NonReplicaGroupTieredSegmentAssignmentTest.java | 5 +- .../RealtimeReplicaGroupSegmentAssignmentTest.java | 132 +++- .../helix/core/realtime/SegmentCompletionTest.java | 10 +- .../helix/core/retention/RetentionManagerTest.java | 38 +- .../validation/ValidationManagerTest.java | 3 +- pinot-core/pom.xml | 12 + .../core/data/manager/BaseTableDataManager.java | 58 +- .../manager/offline/OfflineTableDataManager.java | 12 - .../manager/offline/TableDataManagerProvider.java | 7 +- .../manager/realtime/RealtimeTableDataManager.java | 5 +- .../manager/realtime/SegmentCommitterFactory.java | 2 +- .../realtime/Server2ControllerSegmentUploader.java | 11 +- .../core/operator/filter/FilterOperatorUtils.java | 3 +- .../filter/H3InclusionIndexFilterOperator.java | 52 +- .../predicate/RangePredicateEvaluatorFactory.java | 48 +- .../query/AggregationGroupByOrderByOperator.java | 2 + .../plan/AggregationGroupByOrderByPlanNode.java | 2 +- .../pinot/core/plan/AggregationPlanNode.java | 2 +- .../org/apache/pinot/core/plan/FilterPlanNode.java | 29 +- .../statement/JsonStatementOptimizer.java | 20 +- .../query/pruner/ColumnValueSegmentPruner.java | 19 +- .../core/query/reduce/BaseGapfillProcessor.java | 229 ++++++ .../core/query/reduce/BrokerReduceService.java | 7 +- .../core/query/reduce/CountGapfillProcessor.java | 131 ++++ .../pinot/core/query/reduce/GapfillProcessor.java | 138 +--- .../core/query/reduce/GapfillProcessorFactory.java | 65 ++ .../core/query/reduce/SumAvgGapfillProcessor.java | 212 ++++++ .../apache/pinot/core/startree/StarTreeUtils.java | 24 +- .../apache/pinot/core/util/ListenerConfigUtil.java | 19 +- .../server/realtime/ControllerLeaderLocator.java | 6 +- .../ServerSegmentCompletionProtocolHandler.java | 26 +- .../core/common/datatable/DataTableSerDeTest.java | 75 +- .../BaseTableDataManagerAcquireSegmentTest.java | 5 +- .../data/manager/BaseTableDataManagerTest.java | 5 +- .../offline/DimensionTableDataManagerTest.java | 2 +- .../query/pruner/ColumnValueSegmentPrunerTest.java | 32 +- .../pinot/core/startree/v2/BaseStarTreeV2Test.java | 2 +- .../queries/GapfillQueriesScalabilityTest.java | 822 +++++++++++++++++++++ .../apache/pinot/queries/GapfillQueriesTest.java | 1 + .../apache/pinot/queries/H3IndexQueriesTest.java | 28 + .../queries/JsonIngestionFromAvroQueriesTest.java | 10 +- .../org/apache/pinot/queries/RangeQueriesTest.java | 72 +- .../realtime/ControllerLeaderLocatorTest.java | 30 +- pinot-distribution/pinot-assembly.xml | 2 - pinot-distribution/pom.xml | 12 - .../tests/BaseClusterIntegrationTest.java | 15 +- .../pinot/integration/tests/ClusterTest.java | 100 +-- pinot-integration-tests/pom.xml | 13 +- .../tests/BaseClusterIntegrationTestSet.java | 3 +- .../tests/BasicAuthBatchIntegrationTest.java | 4 +- .../tests/LLCRealtimeClusterIntegrationTest.java | 96 ++- .../tests/OfflineClusterIntegrationTest.java | 48 +- .../tests/OfflineGRPCServerIntegrationTest.java | 2 +- .../tests/RealtimeClusterIntegrationTest.java | 16 +- ...TextIndicesRealtimeClusterIntegrationTest.java} | 50 +- .../integration/tests/TlsIntegrationTest.java | 58 ++ .../UpsertTableSegmentUploadIntegrationTest.java | 218 +++--- ...st.java => UrlAuthRealtimeIntegrationTest.java} | 55 +- .../ControllerLeaderLocatorIntegrationTest.java | 8 +- .../src/test/resources/url-auth-token-prefixed.txt | 1 + .../src/test/resources/url-auth-token.txt | 1 + .../org/apache/pinot/minion/BaseMinionStarter.java | 7 +- .../org/apache/pinot/perf/BenchmarkQueries.java | 6 +- .../v0_deprecated/pinot-hadoop/pom.xml | 12 - .../pinot/plugin/filesystem/ADLSGen2PinotFS.java | 83 ++- .../org/apache/pinot/plugin/filesystem/GcsUri.java | 16 +- .../pinot/plugin/filesystem/TestGcsPinotFS.java | 10 +- .../apache/pinot/plugin/filesystem/TestGcsUri.java | 20 +- .../inputformat/avro/AvroRecordExtractor.java | 32 + .../avro/AvroRecordExtractorConfig.java} | 20 +- .../plugin/inputformat/avro/AvroRecordReader.java | 8 +- .../inputformat/avro/AvroRecordReaderConfig.java} | 19 +- .../plugin/inputformat/avro/AvroSchemaUtil.java | 58 ++ .../inputformat/avro/AvroRecordExtractorTest.java | 143 +++- .../inputformat/avro/AvroSchemaUtilTest.java | 83 +++ .../inputformat/avro/KafkaAvroMessageDecoder.java | 10 +- .../inputformat/avro/SimpleAvroMessageDecoder.java | 10 +- ...aConfluentSchemaRegistryAvroMessageDecoder.java | 5 +- .../plugin/inputformat/csv/CSVMessageDecoder.java | 146 ++++ .../inputformat/csv/CSVMessageDecoderTest.java | 157 ++++ .../BaseMultipleSegmentsConversionExecutor.java | 19 +- .../tasks/BaseSingleSegmentConversionExecutor.java | 7 +- .../minion/tasks/SegmentConversionUtils.java | 19 +- .../segmentuploader/SegmentUploaderDefault.java | 10 +- .../pinot-stream-ingestion/pinot-pulsar/pom.xml | 2 +- .../plugin/stream/pulsar/PulsarConsumerTest.java | 77 +- .../local/data/manager/TableDataManager.java | 4 +- .../local/data/manager/TableDataManagerConfig.java | 14 +- .../immutable/ImmutableSegmentLoader.java | 2 + .../indexsegment/mutable/MutableSegmentImpl.java | 25 +- .../writer/impl/BaseChunkSVForwardIndexWriter.java | 1 + ...ava => BigDecimalOffHeapMutableDictionary.java} | 130 ++-- ...java => BigDecimalOnHeapMutableDictionary.java} | 105 +-- .../impl/dictionary/MutableDictionaryFactory.java | 5 + .../impl/invertedindex/NativeMutableTextIndex.java | 123 +++ .../invertedindex/RealtimeLuceneTextIndex.java | 2 +- .../creator/impl/SegmentColumnarIndexCreator.java | 9 +- .../creator/impl/inv/RangeIndexCreator.java | 18 +- .../pinot/segment/local/utils/IngestionUtils.java | 17 +- .../segment/local/utils/SegmentPushUtils.java | 16 +- .../impl/dictionary/MutableDictionaryTest.java | 15 +- .../NativeAndLuceneMutableTextIndexTest.java | 107 +++ .../NativeMutableTextIndexConcurrentTest.java | 251 +++++++ .../NativeMutableTextIndexReaderWriterTest.java} | 44 +- .../forward/VarByteChunkSVForwardIndexTest.java | 24 + .../mutablefst/MutableFSTConcurrentTest.java | 74 +- .../apache/pinot/segment/spi/SegmentMetadata.java | 4 + .../org/apache/pinot/segment/spi/V1Constants.java | 5 + .../creator/name/FixedSegmentNameGenerator.java | 6 +- .../name/InputFileSegmentNameGenerator.java | 31 +- .../name/NormalizedDateSegmentNameGenerator.java | 22 +- .../creator/name/SimpleSegmentNameGenerator.java | 17 +- .../spi/index/metadata/SegmentMetadataImpl.java | 29 +- .../name/FixedSegmentNameGeneratorTest.java} | 22 +- .../name/InputFileSegmentNameGeneratorTest.java | 3 +- .../NormalizedDateSegmentNameGeneratorTest.java | 17 +- .../name/SimpleSegmentNameGeneratorTest.java | 1 + .../server/starter/helix/BaseServerStarter.java | 4 + .../starter/helix/HelixInstanceDataManager.java | 7 +- .../helix/HelixInstanceDataManagerConfig.java | 15 +- .../apache/pinot/server/api/BaseResourceTest.java | 2 +- .../auth/{AuthContext.java => AuthProvider.java} | 21 +- .../config/instance/InstanceDataManagerConfig.java | 2 +- .../apache/pinot/spi/config/table/QueryConfig.java | 23 +- .../spi/data/readers/RecordExtractorConfig.java | 4 + .../spi/data/readers/RecordReaderFactory.java | 6 +- .../batch/spec/SegmentGenerationJobSpec.java | 6 + .../segment/uploader/SegmentUploader.java | 10 +- .../org/apache/pinot/spi/plugin/PluginManager.java | 1 + .../pinot/spi/stream/StreamMessageDecoder.java | 1 + .../pinot/spi/trace/InvocationRecording.java | 9 + .../apache/pinot/spi/utils/CommonConstants.java | 14 +- .../apache/pinot/spi/utils/InstanceTypeUtils.java | 62 ++ .../main/java/org/apache/pinot/spi/utils/Pair.java | 73 -- .../data/readers/AbstractRecordExtractorTest.java | 12 +- .../spi/data/readers/RecordReaderFactoryTest.java | 4 +- pinot-tools/pom.xml | 20 +- .../org/apache/pinot/tools/AuthQuickstart.java | 6 +- .../pinot/tools/BatchQuickstartWithMinion.java | 49 -- .../org/apache/pinot/tools/BootstrapTableTool.java | 13 +- .../org/apache/pinot/tools/EmptyQuickstart.java | 7 +- .../org/apache/pinot/tools/GenericQuickstart.java | 2 +- .../apache/pinot/tools/GitHubEventsQuickstart.java | 2 +- .../org/apache/pinot/tools/HybridQuickstart.java | 2 +- .../org/apache/pinot/tools/JoinQuickStart.java | 2 +- .../apache/pinot/tools/JsonIndexQuickStart.java | 2 +- .../OfflineComplexTypeHandlingQuickStart.java | 2 +- .../pinot/tools/PartialUpsertQuickStart.java | 2 +- .../java/org/apache/pinot/tools/Quickstart.java | 12 +- .../RealtimeComplexTypeHandlingQuickStart.java | 2 +- .../pinot/tools/RealtimeJsonIndexQuickStart.java | 2 +- .../org/apache/pinot/tools/RealtimeQuickStart.java | 2 +- .../pinot/tools/TimestampIndexQuickstart.java | 2 +- .../apache/pinot/tools/UpsertJsonQuickStart.java | 2 +- .../org/apache/pinot/tools/UpsertQuickStart.java | 2 +- .../pinot/tools/admin/PinotAdministrator.java | 4 + .../admin/command/AbstractBaseAdminCommand.java | 38 +- .../tools/admin/command/AddSchemaCommand.java | 14 +- .../pinot/tools/admin/command/AddTableCommand.java | 20 +- .../tools/admin/command/AddTenantCommand.java | 17 +- .../tools/admin/command/BootstrapTableCommand.java | 16 +- .../tools/admin/command/ChangeTableState.java | 20 +- .../admin/command/DataImportDryRunCommand.java | 96 +++ .../tools/admin/command/FileSystemCommand.java | 111 +++ .../tools/admin/command/ImportDataCommand.java | 17 +- .../command/LaunchDataIngestionJobCommand.java | 11 +- .../admin/command/OperateClusterConfigCommand.java | 19 +- .../tools/admin/command/PostQueryCommand.java | 17 +- .../tools/admin/command/QuickstartRunner.java | 59 +- .../tools/admin/command/UploadSegmentCommand.java | 20 +- .../command/filesystem/BaseFileOperation.java | 58 ++ .../tools/admin/command/filesystem/CopyFiles.java | 113 +++ .../admin/command/filesystem/DeleteFiles.java | 81 ++ .../tools/admin/command/filesystem/ListFiles.java | 93 +++ .../tools/admin/command/filesystem/MoveFiles.java | 93 +++ .../tools/admin/command/filesystem/Utils.java | 26 +- .../src/main/resources/conf/pinot-minion.conf | 5 +- .../tools/admin/command/TestQuickStartCommand.java | 159 ++-- pom.xml | 36 +- 235 files changed, 7076 insertions(+), 2628 deletions(-) delete mode 100644 pinot-broker/src/test/java/org/apache/pinot/broker/requesthandler/DistinctCountRewriteTest.java delete mode 100644 pinot-broker/src/test/java/org/apache/pinot/broker/requesthandler/QueryLimitOverrideTest.java create mode 100644 pinot-broker/src/test/java/org/apache/pinot/broker/requesthandler/QueryOverrideTest.java copy pinot-clients/{pinot-java-client/src/main/java/org/apache/pinot/client/JsonAsyncHttpPinotClientTransportFactory.java => pinot-jdbc-client/src/main/java/org/apache/pinot/client/controller/PinotControllerTransportFactory.java} (81%) copy pinot-spi/src/main/java/org/apache/pinot/spi/services/ServiceStartable.java => pinot-common/src/main/java/org/apache/pinot/common/auth/AuthConfig.java (64%) create mode 100644 pinot-common/src/main/java/org/apache/pinot/common/auth/AuthProviderUtils.java copy pinot-common/src/main/java/org/apache/pinot/common/{request/context/predicate/BasePredicate.java => auth/NullAuthProvider.java} (64%) create mode 100644 pinot-common/src/main/java/org/apache/pinot/common/auth/StaticTokenAuthProvider.java create mode 100644 pinot-common/src/main/java/org/apache/pinot/common/auth/UrlAuthProvider.java delete mode 100644 pinot-common/src/main/java/org/apache/pinot/common/utils/Pair.java create mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/api/upload/SegmentValidationUtils.java delete mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/api/upload/SegmentValidator.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/reduce/BaseGapfillProcessor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/reduce/CountGapfillProcessor.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/reduce/GapfillProcessorFactory.java create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/reduce/SumAvgGapfillProcessor.java create mode 100644 pinot-core/src/test/java/org/apache/pinot/queries/GapfillQueriesScalabilityTest.java rename pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/{LuceneRealtimeClusterIntegrationTest.java => TextIndicesRealtimeClusterIntegrationTest.java} (75%) rename pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/{BasicAuthRealtimeIntegrationTest.java => UrlAuthRealtimeIntegrationTest.java} (70%) create mode 100644 pinot-integration-tests/src/test/resources/url-auth-token-prefixed.txt create mode 100644 pinot-integration-tests/src/test/resources/url-auth-token.txt copy pinot-plugins/pinot-input-format/{pinot-thrift/src/main/java/org/apache/pinot/plugin/inputformat/thrift/ThriftRecordExtractorConfig.java => pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroRecordExtractorConfig.java} (62%) copy pinot-plugins/pinot-input-format/{pinot-protobuf/src/main/java/org/apache/pinot/plugin/inputformat/protobuf/ProtoBufRecordReaderConfig.java => pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroRecordReaderConfig.java} (69%) create mode 100644 pinot-plugins/pinot-input-format/pinot-avro-base/src/test/java/org/apache/pinot/plugin/inputformat/avro/AvroSchemaUtilTest.java create mode 100644 pinot-plugins/pinot-input-format/pinot-csv/src/main/java/org/apache/pinot/plugin/inputformat/csv/CSVMessageDecoder.java create mode 100644 pinot-plugins/pinot-input-format/pinot-csv/src/test/java/org/apache/pinot/plugin/inputformat/csv/CSVMessageDecoderTest.java copy pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/dictionary/{IntOffHeapMutableDictionary.java => BigDecimalOffHeapMutableDictionary.java} (54%) copy pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/dictionary/{LongOnHeapMutableDictionary.java => BigDecimalOnHeapMutableDictionary.java} (58%) create mode 100644 pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/NativeMutableTextIndex.java create mode 100644 pinot-segment-local/src/test/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/NativeAndLuceneMutableTextIndexTest.java create mode 100644 pinot-segment-local/src/test/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/NativeMutableTextIndexConcurrentTest.java copy pinot-segment-local/src/test/java/org/apache/pinot/segment/local/{segment/index/creator/NativeTextIndexCreatorTest.java => realtime/impl/invertedindex/NativeMutableTextIndexReaderWriterTest.java} (52%) copy pinot-segment-spi/src/test/java/org/apache/pinot/segment/spi/{reader/H3IndexResolutionTest.java => creator/name/FixedSegmentNameGeneratorTest.java} (60%) rename pinot-spi/src/main/java/org/apache/pinot/spi/auth/{AuthContext.java => AuthProvider.java} (62%) create mode 100644 pinot-spi/src/main/java/org/apache/pinot/spi/utils/InstanceTypeUtils.java delete mode 100644 pinot-spi/src/main/java/org/apache/pinot/spi/utils/Pair.java delete mode 100644 pinot-tools/src/main/java/org/apache/pinot/tools/BatchQuickstartWithMinion.java create mode 100644 pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/DataImportDryRunCommand.java create mode 100644 pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/FileSystemCommand.java create mode 100644 pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/filesystem/BaseFileOperation.java create mode 100644 pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/filesystem/CopyFiles.java create mode 100644 pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/filesystem/DeleteFiles.java create mode 100644 pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/filesystem/ListFiles.java create mode 100644 pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/filesystem/MoveFiles.java copy pinot-common/src/main/java/org/apache/pinot/common/utils/fetcher/PinotFSSegmentFetcher.java => pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/filesystem/Utils.java (57%) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org