[GitHub] [pinot] saurabhd336 commented on a diff in pull request #8502: Add zk metadata to segment's metadata.properties file

saurabhd336 commented on code in PR #8502: URL: https://github.com/apache/pinot/pull/8502#discussion_r848031959 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/SegmentZKMetadataConfig.java: ## @@ -0,0 +1,44 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[pinot] branch helix-1.0 updated (aa653c139f -> 67e49969ba)

This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch helix-1.0 in repository https://gitbox.apache.org/repos/asf/pinot.git omit aa653c139f Upgrade Helix to 1.0.2 add c0309a2737 Expose instance config and starter in ClusterTest for easier ex

[GitHub] [pinot] timsants commented on a diff in pull request #8509: Fix table creation bug for invalid realtime consumer props

timsants commented on code in PR #8509: URL: https://github.com/apache/pinot/pull/8509#discussion_r847965981 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java: ## @@ -1448,7 +1449,14 @@ public void addTable(TableConfig tableC

[GitHub] [pinot] timsants commented on issue #8500: Bad table config may result in Zombie tables

timsants commented on issue #8500: URL: https://github.com/apache/pinot/issues/8500#issuecomment-1096000480 @npawar, this was a change that was merged in Feb [#8103](https://github.com/apache/pinot/pull/8103/files). Before that, realtime tables didn't have this check. Previously, the table

[pinot] branch master updated: Fixing the bug of missing dot to extract sub props from ingestion job filesytem spec and minion segmentNameGeneratorSpec (#8511)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new bb46e466af Fixing the bug of missing dot to extr

[GitHub] [pinot] xiangfu0 merged pull request #8511: bugfix: fixing the bug of missing dot to extract sub props from ingestion job and minion

xiangfu0 merged PR #8511: URL: https://github.com/apache/pinot/pull/8511 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

[GitHub] [pinot] yupeng9 merged pull request #8128: Improve segment metadata fetch API

yupeng9 merged PR #8128: URL: https://github.com/apache/pinot/pull/8128 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache

[pinot] branch master updated: Improve segment metadata fetch API (#8128)

This is an automated email from the ASF dual-hosted git repository. yupeng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 3556e7149e Improve segment metadata fetch API (#8

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8509: Fix table creation bug for invalid realtime consumer props

Jackie-Jiang commented on code in PR #8509: URL: https://github.com/apache/pinot/pull/8509#discussion_r847909780 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java: ## @@ -1448,7 +1449,14 @@ public void addTable(TableConfig ta

[GitHub] [pinot] npawar commented on issue #8500: Bad table config may result in Zombie tables

npawar commented on issue #8500: URL: https://github.com/apache/pinot/issues/8500#issuecomment-1095889721 @timsants there was a report that this worked fine in 0.9.3 (i.e. table creation failed cleanly) and in 0.10.0 this fails with zombie states. Any idea why that is? -- This is an auto

[GitHub] [pinot] WangCHX commented on a diff in pull request #8498: Support st_contains using H3 index

WangCHX commented on code in PR #8498: URL: https://github.com/apache/pinot/pull/8498#discussion_r847895120 ## pinot-core/src/main/java/org/apache/pinot/core/operator/filter/H3InclusionIndexFilterOperator.java: ## @@ -0,0 +1,128 @@ +/** + * Licensed to the Apache Software Founda

[GitHub] [pinot] WangCHX commented on a diff in pull request #8498: Support st_contains using H3 index

WangCHX commented on code in PR #8498: URL: https://github.com/apache/pinot/pull/8498#discussion_r847895120 ## pinot-core/src/main/java/org/apache/pinot/core/operator/filter/H3InclusionIndexFilterOperator.java: ## @@ -0,0 +1,128 @@ +/** + * Licensed to the Apache Software Founda

[GitHub] [pinot] mayankshriv commented on issue #8281: Long running rebalancing task reverts ideal states in offline table

mayankshriv commented on issue #8281: URL: https://github.com/apache/pinot/issues/8281#issuecomment-1095847461 Fixed by #8921 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [pinot] mayankshriv closed issue #8281: Long running rebalancing task reverts ideal states in offline table

mayankshriv closed issue #8281: Long running rebalancing task reverts ideal states in offline table URL: https://github.com/apache/pinot/issues/8281 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [pinot] timsants commented on a diff in pull request #8509: Fix table creation bug for invalid realtime consumer props

timsants commented on code in PR #8509: URL: https://github.com/apache/pinot/pull/8509#discussion_r847866608 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java: ## @@ -1384,7 +1384,7 @@ public List getSchemaNames() { public

[GitHub] [pinot] walterddr commented on pull request #8484: Add customizable parser module

walterddr commented on PR #8484: URL: https://github.com/apache/pinot/pull/8484#issuecomment-1095758884 > High level question: Do we introduce this custom parser to accelerate the parsing but keep the same preserved words as `BABEL`? the goal is not accelerate parsing but to get prec

[GitHub] [pinot] Jackie-Jiang merged pull request #8510: Expose instance config and starter in ClusterTest for easier extension

Jackie-Jiang merged PR #8510: URL: https://github.com/apache/pinot/pull/8510 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

[pinot] branch master updated: Expose instance config and starter in ClusterTest for easier extension (#8510)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new c0309a2737 Expose instance config and starter in

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8503: Support single-valued BigDecimal in schema, type conversion, SQL statements and minimum set of transforms.

Jackie-Jiang commented on code in PR #8503: URL: https://github.com/apache/pinot/pull/8503#discussion_r847842149 ## pinot-core/src/main/java/org/apache/pinot/core/common/BlockValSet.java: ## @@ -29,6 +29,13 @@ */ public interface BlockValSet { + /** + * Returns the numbe

[GitHub] [pinot] codecov-commenter commented on pull request #8510: Expose instance config and starter in ClusterTest for easier extension

codecov-commenter commented on PR #8510: URL: https://github.com/apache/pinot/pull/8510#issuecomment-1095743830 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8510?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[GitHub] [pinot] codecov-commenter commented on pull request #8511: bugfix: fixing the bug of missing dot to extract sub props from ingestion job and minion

codecov-commenter commented on PR #8511: URL: https://github.com/apache/pinot/pull/8511#issuecomment-1095736189 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8511?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[GitHub] [pinot] xiangfu0 commented on pull request #8484: Add customizable parser module

xiangfu0 commented on PR #8484: URL: https://github.com/apache/pinot/pull/8484#issuecomment-1095733268 > High level question: Do we introduce this custom parser to accelerate the parsing but keep the same preserved words as `BABEL`? Can we get the same performance if switching to `DEFAULT`?

[GitHub] [pinot] timsants commented on pull request #8509: Fix table creation bug for invalid realtime consumer props

timsants commented on PR #8509: URL: https://github.com/apache/pinot/pull/8509#issuecomment-1095730780 > I think this behavior is intentional to avoid creating a table on a recently deleted table that doesn't have all entries cleaned up. We remove the table config as the last step when dele

[GitHub] [pinot] Jackie-Jiang commented on issue #8500: Bad table config may result in Zombie tables

Jackie-Jiang commented on issue #8500: URL: https://github.com/apache/pinot/issues/8500#issuecomment-1095728788 Checking table config existence is intentional to prevent race condition of re-creating a recently not fully deleted table (see #8103 for details) The correct fix here should be

[GitHub] [pinot] navina commented on a diff in pull request #8509: Fix table creation bug for invalid realtime consumer props

navina commented on code in PR #8509: URL: https://github.com/apache/pinot/pull/8509#discussion_r847837277 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java: ## @@ -1384,7 +1384,7 @@ public List getSchemaNames() { public v

[GitHub] [pinot] Jackie-Jiang commented on pull request #8509: Fix table creation bug for invalid realtime consumer props

Jackie-Jiang commented on PR #8509: URL: https://github.com/apache/pinot/pull/8509#issuecomment-1095725223 I think this behavior is intentional to avoid creating a table on a recently deleted table that doesn't have all entries cleaned up. We remove the table config as the last step when de

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8484: Add customizable parser module

Jackie-Jiang commented on code in PR #8484: URL: https://github.com/apache/pinot/pull/8484#discussion_r847828548 ## pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java: ## @@ -79,13 +82,9 @@ public void testCanonicalFunctionName() { public void

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #8465: Add adhoc minion task creation endpoint

xiangfu0 commented on code in PR #8465: URL: https://github.com/apache/pinot/pull/8465#discussion_r847830168 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/generator/PinotTaskGenerator.java: ## @@ -45,6 +46,12 @@ public interface PinotTaskGenerat

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #8465: Add adhoc minion task creation endpoint

xiangfu0 commented on code in PR #8465: URL: https://github.com/apache/pinot/pull/8465#discussion_r847830015 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotTaskManager.java: ## @@ -128,6 +132,68 @@ public PinotTaskManager(PinotHelixTaskResou

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #8465: Add adhoc minion task creation endpoint

xiangfu0 commented on code in PR #8465: URL: https://github.com/apache/pinot/pull/8465#discussion_r847829828 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotTaskManager.java: ## @@ -27,8 +27,10 @@ import java.util.List; import java.util.Map

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #8465: Add adhoc minion task creation endpoint

xiangfu0 commented on code in PR #8465: URL: https://github.com/apache/pinot/pull/8465#discussion_r847829513 ## pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/segmentgenerationandpush/SegmentGenerationAndPushTaskGenerator.j

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #8465: Add adhoc minion task creation endpoint

xiangfu0 commented on code in PR #8465: URL: https://github.com/apache/pinot/pull/8465#discussion_r847829124 ## pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/segmentgenerationandpush/SegmentGenerationAndPushTaskExecutor.ja

[GitHub] [pinot] codecov-commenter commented on pull request #8509: Fix table creation bug for invalid realtime consumer props

codecov-commenter commented on PR #8509: URL: https://github.com/apache/pinot/pull/8509#issuecomment-1095707470 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8509?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[GitHub] [pinot] xiangfu0 opened a new pull request, #8511: bugfix: fixing the bug of missing dot to extract sub props from ingestion job and minion

xiangfu0 opened a new pull request, #8511: URL: https://github.com/apache/pinot/pull/8511 fixing the bug of missing dot to extract sub props from ingestion job filesytem spec and minion segmentNameGeneratorSpec -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #8465: Add adhoc minion task creation endpoint

xiangfu0 commented on code in PR #8465: URL: https://github.com/apache/pinot/pull/8465#discussion_r847823265 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotHelixTaskResourceManager.java: ## @@ -611,6 +636,10 @@ private static String getTaskT

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #8465: Add adhoc minion task creation endpoint

xiangfu0 commented on code in PR #8465: URL: https://github.com/apache/pinot/pull/8465#discussion_r847822790 ## pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/segmentgenerationandpush/SegmentGenerationAndPushTaskGenerator.j

[GitHub] [pinot] codecov-commenter commented on pull request #8508: add shouldNotMerge field in segment metadata to indicate if it is safe to merge

codecov-commenter commented on PR #8508: URL: https://github.com/apache/pinot/pull/8508#issuecomment-1095697491 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8508?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[pinot] branch helix-1.0 updated (4aa4dd9d31 -> aa653c139f)

This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch helix-1.0 in repository https://gitbox.apache.org/repos/asf/pinot.git discard 4aa4dd9d31 Upgrade Helix to 1.0.2 add d7be2ef412 Fetch Pulsar offsets from Consumer interface instead of Reader

[GitHub] [pinot] timsants opened a new pull request, #8509: Fix table creation bug for invalid realtime consumer props

timsants opened a new pull request, #8509: URL: https://github.com/apache/pinot/pull/8509 Instructions: 1. The PR has to be tagged with at least one of the following labels (*): 1. `feature` 2. `bugfix` 3. `performance` 4. `ui` 5. `backward-incompat` 6.

[GitHub] [pinot] timsants commented on issue #8500: Bad table config may result in Zombie tables

timsants commented on issue #8500: URL: https://github.com/apache/pinot/issues/8500#issuecomment-1095670691 When a realtime table is being created, the table config is first uploaded to the ZK property store. After that, we try to create the realtime consumer. But as pointed out, if there i

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8508: add shouldNotMerge field in segment metadata to indicate if it is safe to merge

Jackie-Jiang commented on code in PR #8508: URL: https://github.com/apache/pinot/pull/8508#discussion_r847787254 ## pinot-core/src/main/java/org/apache/pinot/core/common/MinionConstants.java: ## @@ -101,6 +101,11 @@ public static abstract class MergeTask { public static fin

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #8499: DataTable deserialization improvements

siddharthteotia commented on code in PR #8499: URL: https://github.com/apache/pinot/pull/8499#discussion_r847771725 ## pinot-core/src/main/java/org/apache/pinot/core/common/datatable/DataTableImplV3.java: ## @@ -350,32 +340,29 @@ private byte[] serializeMetadata() * This is

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #8499: DataTable deserialization improvements

siddharthteotia commented on code in PR #8499: URL: https://github.com/apache/pinot/pull/8499#discussion_r847772108 ## pinot-core/src/main/java/org/apache/pinot/core/common/datatable/DataTableImplV3.java: ## @@ -350,32 +340,29 @@ private byte[] serializeMetadata() * This is

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #8499: DataTable deserialization improvements

siddharthteotia commented on code in PR #8499: URL: https://github.com/apache/pinot/pull/8499#discussion_r847771725 ## pinot-core/src/main/java/org/apache/pinot/core/common/datatable/DataTableImplV3.java: ## @@ -350,32 +340,29 @@ private byte[] serializeMetadata() * This is

[GitHub] [pinot] richardstartin commented on a diff in pull request #8491: Add connection based FailureDetector

richardstartin commented on code in PR #8491: URL: https://github.com/apache/pinot/pull/8491#discussion_r847762088 ## pinot-core/src/test/java/org/apache/pinot/core/query/reduce/StreamingReduceServiceTest.java: ## @@ -78,19 +74,14 @@ public Void answer(InvocationOnMock invocatio

[GitHub] [pinot] richardstartin commented on a diff in pull request #8491: Add connection based FailureDetector

richardstartin commented on code in PR #8491: URL: https://github.com/apache/pinot/pull/8491#discussion_r847761220 ## pinot-core/src/main/java/org/apache/pinot/core/transport/ServerRoutingInstance.java: ## @@ -87,17 +100,20 @@ public boolean equals(Object o) { if (this == o

[GitHub] [pinot] richardstartin commented on a diff in pull request #8491: Add connection based FailureDetector

richardstartin commented on code in PR #8491: URL: https://github.com/apache/pinot/pull/8491#discussion_r847759869 ## pinot-core/src/main/java/org/apache/pinot/core/transport/ServerChannels.java: ## @@ -181,24 +191,43 @@ private void sendRequest(String rawTableName, AsyncQueryR

[GitHub] [pinot] richardstartin commented on a diff in pull request #8491: Add connection based FailureDetector

richardstartin commented on code in PR #8491: URL: https://github.com/apache/pinot/pull/8491#discussion_r847755763 ## pinot-broker/src/test/java/org/apache/pinot/broker/failuredetector/ConnectionFailureDetectorTest.java: ## @@ -0,0 +1,160 @@ +/** + * Licensed to the Apache Softw

[GitHub] [pinot] klsince opened a new pull request, #8508: add shouldNotMerge field in segment metadata to indicate if it is safe to merge

klsince opened a new pull request, #8508: URL: https://github.com/apache/pinot/pull/8508 Add shouldNotMerge field in SegmentZKMetadata customMap to indicate if segment can be merged. Tasks can take use of this field to coordinate with the merge task. By default, segment is safe to merge, so

[GitHub] [pinot] richardstartin commented on a diff in pull request #8491: Add connection based FailureDetector

richardstartin commented on code in PR #8491: URL: https://github.com/apache/pinot/pull/8491#discussion_r847754858 ## pinot-broker/src/main/java/org/apache/pinot/broker/routing/BrokerRoutingManager.java: ## @@ -214,87 +219,178 @@ private void processInstanceConfigChange() {

[GitHub] [pinot] richardstartin commented on a diff in pull request #8491: Add connection based FailureDetector

richardstartin commented on code in PR #8491: URL: https://github.com/apache/pinot/pull/8491#discussion_r847752986 ## pinot-broker/src/main/java/org/apache/pinot/broker/routing/BrokerRoutingManager.java: ## @@ -214,87 +219,178 @@ private void processInstanceConfigChange() {

[GitHub] [pinot] kkrugler commented on a diff in pull request #8491: Add connection based FailureDetector

kkrugler commented on code in PR #8491: URL: https://github.com/apache/pinot/pull/8491#discussion_r847750948 ## pinot-broker/src/main/java/org/apache/pinot/broker/failuredetector/BaseExponentialBackoffRetryFailureDetector.java: ## @@ -0,0 +1,174 @@ +/** + * Licensed to the Apach

[GitHub] [pinot] richardstartin commented on a diff in pull request #8491: Add connection based FailureDetector

richardstartin commented on code in PR #8491: URL: https://github.com/apache/pinot/pull/8491#discussion_r847750591 ## pinot-broker/src/main/java/org/apache/pinot/broker/failuredetector/FailureDetectorFactory.java: ## @@ -0,0 +1,56 @@ +/** + * Licensed to the Apache Software Foun

[GitHub] [pinot] richardstartin commented on a diff in pull request #8491: Add connection based FailureDetector

richardstartin commented on code in PR #8491: URL: https://github.com/apache/pinot/pull/8491#discussion_r847750216 ## pinot-broker/src/main/java/org/apache/pinot/broker/failuredetector/FailureDetectorFactory.java: ## @@ -0,0 +1,56 @@ +/** + * Licensed to the Apache Software Foun

[GitHub] [pinot] richardstartin commented on a diff in pull request #8491: Add connection based FailureDetector

richardstartin commented on code in PR #8491: URL: https://github.com/apache/pinot/pull/8491#discussion_r847748944 ## pinot-broker/src/main/java/org/apache/pinot/broker/failuredetector/BaseExponentialBackoffRetryFailureDetector.java: ## @@ -0,0 +1,174 @@ +/** + * Licensed to the

[GitHub] [pinot] richardstartin commented on a diff in pull request #8491: Add connection based FailureDetector

richardstartin commented on code in PR #8491: URL: https://github.com/apache/pinot/pull/8491#discussion_r847746972 ## pinot-broker/src/main/java/org/apache/pinot/broker/failuredetector/BaseExponentialBackoffRetryFailureDetector.java: ## @@ -0,0 +1,174 @@ +/** + * Licensed to the

[pinot] branch master updated: Cleanup and simplify the integration test instance start (#8507)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 68c89b74f5 Cleanup and simplify the integration t

[GitHub] [pinot] Jackie-Jiang merged pull request #8507: Cleanup and simplify the integration test instance start

Jackie-Jiang merged PR #8507: URL: https://github.com/apache/pinot/pull/8507 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

[pinot] branch master updated (2935bb5c8c -> be163f6f55)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 2935bb5c8c upgrade snappy version (#8494) add be163f6f55 Timestamp type index (#8343) No new revisions were adde

[GitHub] [pinot] xiangfu0 merged pull request #8343: Timestamp type index

xiangfu0 merged PR #8343: URL: https://github.com/apache/pinot/pull/8343 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

[GitHub] [pinot] jackjlli commented on issue #8492: Need ways to check whether segment reload/refresh is done on server side

jackjlli commented on issue #8492: URL: https://github.com/apache/pinot/issues/8492#issuecomment-1095529070 We do have a `SegmentRefreshSemaphore` class which is initialized only once per pinot-server, something that we can use to check the number of segment reloads / refreshes per server.

[GitHub] [pinot] jackjlli commented on issue #8492: Need ways to check whether segment reload/refresh is done on server side

jackjlli commented on issue #8492: URL: https://github.com/apache/pinot/issues/8492#issuecomment-1095526012 It's not very intuitive nor straightforward to retrieve other information in order to check the state of a reload/refresh request. What's worse, one corner scenario I can think

[GitHub] [pinot] kevin-kosmos commented on pull request #8338: Handle authentication in pulsar pinot connector

kevin-kosmos commented on PR #8338: URL: https://github.com/apache/pinot/pull/8338#issuecomment-1095520712 > LGTM! Please resolve the merge conflicts. Done! Thank you :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [pinot] KKcorps commented on a diff in pull request #8502: Add zk metadata to segment's metadata.properties file

KKcorps commented on code in PR #8502: URL: https://github.com/apache/pinot/pull/8502#discussion_r847692284 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/SegmentZKMetadataConfig.java: ## @@ -0,0 +1,44 @@ +/** + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [pinot] KKcorps commented on pull request #8338: Handle authentication in pulsar pinot connector

KKcorps commented on PR #8338: URL: https://github.com/apache/pinot/pull/8338#issuecomment-1095509391 LGTM! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [pinot] richardstartin commented on a diff in pull request #8498: Support st_contains using H3 index

richardstartin commented on code in PR #8498: URL: https://github.com/apache/pinot/pull/8498#discussion_r847665422 ## pinot-core/src/main/java/org/apache/pinot/core/operator/filter/H3InclusionIndexFilterOperator.java: ## @@ -0,0 +1,128 @@ +/** + * Licensed to the Apache Software

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #7358: [issue-7357] Add support for Avro logical types in realtime ingestion

Jackie-Jiang commented on code in PR #7358: URL: https://github.com/apache/pinot/pull/7358#discussion_r847650708 ## pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroRecordExtractor.java: ## @@ -31,16 +34,19 @@ import or

[GitHub] [pinot] codecov-commenter commented on pull request #8507: Cleanup and simplify the integration test instance start

codecov-commenter commented on PR #8507: URL: https://github.com/apache/pinot/pull/8507#issuecomment-1095440506 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8507?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[pinot] branch master updated (b2e2a7e503 -> 2935bb5c8c)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from b2e2a7e503 Remove cron for github action to build and push docker images (#8501) add 2935bb5c8c upgrade snappy ve

[GitHub] [pinot] Jackie-Jiang merged pull request #8494: Upgrade snappy version to 1.1.8.2

Jackie-Jiang merged PR #8494: URL: https://github.com/apache/pinot/pull/8494 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

[GitHub] [pinot] Jackie-Jiang commented on issue #8492: Need ways to check whether segment reload/refresh is done on server side

Jackie-Jiang commented on issue #8492: URL: https://github.com/apache/pinot/issues/8492#issuecomment-1095432819 @mcvsubbu We built this API as a work-around to check the reload status, but it does have some limitations/drawbacks: - Not able to show refresh status - Not user friendly. U

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #8036: Add new compat test cases and data (#7235)

jasperjiaguo commented on code in PR #8036: URL: https://github.com/apache/pinot/pull/8036#discussion_r847636575 ## compatibility-verifier/sample-test-suite/post-server-rollback.yaml: ## @@ -44,3 +58,14 @@ operations: description: Run query on FeatureTest2 using SQL qu

[GitHub] [pinot] mcvsubbu commented on issue #8492: Need ways to check whether segment reload/refresh is done on server side

mcvsubbu commented on issue #8492: URL: https://github.com/apache/pinot/issues/8492#issuecomment-1095424681 Can we use this API? https://github.com/apache/pinot/blob/master/pinot-server/src/main/java/org/apache/pinot/server/api/resources/SegmentMetadataFetcher.java -- This is an autom

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8483: Minimize data movement between instances in pools

Jackie-Jiang commented on code in PR #8483: URL: https://github.com/apache/pinot/pull/8483#discussion_r847627440 ## pinot-common/src/main/java/org/apache/pinot/common/assignment/InstancePartitions.java: ## @@ -58,24 +60,39 @@ @JsonIgnoreProperties(ignoreUnknown = true) public

[GitHub] [pinot] Jackie-Jiang opened a new pull request, #8507: Cleanup and simplify the integration test instance start

Jackie-Jiang opened a new pull request, #8507: URL: https://github.com/apache/pinot/pull/8507 NOTE: Some methods within the integration tests are changed/removed/simplified. Custom tests that extends the integration test might need to revised. -- This is an automated message from the Apa

[GitHub] [pinot] codecov-commenter commented on pull request #8506: Upgrade helix to 0.9.10

codecov-commenter commented on PR #8506: URL: https://github.com/apache/pinot/pull/8506#issuecomment-1095353201 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8506?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[GitHub] [pinot] mcvsubbu commented on issue #8492: Need ways to check whether segment reload/refresh is done on server side

mcvsubbu commented on issue #8492: URL: https://github.com/apache/pinot/issues/8492#issuecomment-1095345667 I thought we already added several server level APIs to fetch the state exactly for this purpose? Are you sure it is not there already? -- This is an automated message from the Apac

[GitHub] [pinot] saurabhd336 commented on a diff in pull request #8502: Add zk metadata to segment's metadata.properties file

saurabhd336 commented on code in PR #8502: URL: https://github.com/apache/pinot/pull/8502#discussion_r847584943 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/SegmentZKMetadataConfig.java: ## @@ -0,0 +1,44 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] saurabhd336 commented on a diff in pull request #8502: Add zk metadata to segment's metadata.properties file

saurabhd336 commented on code in PR #8502: URL: https://github.com/apache/pinot/pull/8502#discussion_r847583488 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/SegmentZKMetadataConfig.java: ## @@ -0,0 +1,44 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] saurabhd336 commented on a diff in pull request #8502: Add zk metadata to segment's metadata.properties file

saurabhd336 commented on code in PR #8502: URL: https://github.com/apache/pinot/pull/8502#discussion_r847582952 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/HLRealtimeSegmentDataManager.java: ## @@ -285,11 +286,13 @@ public void run() { _seg

[GitHub] [pinot] dongxiaoman commented on pull request #8506: Upgrade helix to 0.9.10

dongxiaoman commented on PR #8506: URL: https://github.com/apache/pinot/pull/8506#issuecomment-1095331058 (cannot add labels in the UI, and somehow cannot find how to markdown a label) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [pinot] npawar commented on a diff in pull request #8502: Add zk metadata to segment's metadata.properties file

npawar commented on code in PR #8502: URL: https://github.com/apache/pinot/pull/8502#discussion_r847567661 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/SegmentZKMetadataConfig.java: ## @@ -0,0 +1,44 @@ +/** + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] [pinot] jackjlli commented on a diff in pull request #8483: Minimize data movement between instances in pools

jackjlli commented on code in PR #8483: URL: https://github.com/apache/pinot/pull/8483#discussion_r847570196 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/instance/MinimizedDataMovementInstanceReplicaGroupPartitionSelector.java: ## @@ -0,0 +

[GitHub] [pinot] jackjlli commented on a diff in pull request #8483: Minimize data movement between instances in pools

jackjlli commented on code in PR #8483: URL: https://github.com/apache/pinot/pull/8483#discussion_r847569049 ## pinot-common/src/main/java/org/apache/pinot/common/assignment/InstancePartitions.java: ## @@ -58,24 +60,39 @@ @JsonIgnoreProperties(ignoreUnknown = true) public clas

[GitHub] [pinot] dongxiaoman opened a new pull request, #8506: Upgrade helix to 0.9.10

dongxiaoman opened a new pull request, #8506: URL: https://github.com/apache/pinot/pull/8506 == Description Upgrade helix dependency to `0.9.10` for fixing the issues related to ZNode size limit. 1. Stripe has tested out helix `0.9.9` on private build and surpassed the ZNode limit. (D

[GitHub] [pinot] mcvsubbu commented on a diff in pull request #8465: Add adhoc minion task creation endpoint

mcvsubbu commented on code in PR #8465: URL: https://github.com/apache/pinot/pull/8465#discussion_r847525990 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotTaskManager.java: ## @@ -27,8 +27,10 @@ import java.util.List; import java.util.Map

[pinot] branch master updated (fd3eceaea0 -> b2e2a7e503)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from fd3eceaea0 bump roaringbitmap 0.9.26 (#8504) add b2e2a7e503 Remove cron for github action to build and push docke

[GitHub] [pinot] xiangfu0 merged pull request #8501: docker: Remove cron for github action to build and push docker images

xiangfu0 merged PR #8501: URL: https://github.com/apache/pinot/pull/8501 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

[GitHub] [pinot] siddharthteotia merged pull request #8504: bump roaringbitmap 0.9.26

siddharthteotia merged PR #8504: URL: https://github.com/apache/pinot/pull/8504 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pino

[pinot] branch master updated (436e8ee85c -> fd3eceaea0)

This is an automated email from the ASF dual-hosted git repository. siddteotia pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 436e8ee85c Error message if the dimension table in memory hasn't been populated (#8505) add fd3eceaea0 bump r

[GitHub] [pinot] richardstartin merged pull request #8505: Error message if the dimension table in memory hasn't been populated

richardstartin merged PR #8505: URL: https://github.com/apache/pinot/pull/8505 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[pinot] branch master updated: Error message if the dimension table in memory hasn't been populated (#8505)

This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 436e8ee85c Error message if the dimension

[GitHub] [pinot] mneedham opened a new pull request, #8505: Error message if the dimension table in memory hasn't been populated

mneedham opened a new pull request, #8505: URL: https://github.com/apache/pinot/pull/8505 There's a scenario where the Pinot Server previously downloaded the segment for a dimension table and is then restarted, which means the `addSegment` flow doesn't get called. This means that th

[GitHub] [pinot] saurabhd336 commented on a diff in pull request #8502: Add zk metadata to segment's metadata.properties file

saurabhd336 commented on code in PR #8502: URL: https://github.com/apache/pinot/pull/8502#discussion_r847349563 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/SegmentZKMetadataConfig.java: ## @@ -0,0 +1,44 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] richardstartin opened a new pull request, #8504: bump roaringbitmap 0.9.26

richardstartin opened a new pull request, #8504: URL: https://github.com/apache/pinot/pull/8504 Upgrade RoaringBitmap to pick up a bug fix which affects range indexes in very rare edge cases: https://github.com/RoaringBitmap/RoaringBitmap/pull/557 This release also makes it possible t

[GitHub] [pinot] codecov-commenter commented on pull request #8502: Add zk metadata to segment's metadata.properties file

codecov-commenter commented on PR #8502: URL: https://github.com/apache/pinot/pull/8502#issuecomment-1094814279 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8502?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[GitHub] [pinot] nicovak commented on issue #8500: Bad table config may result in Zombie tables

nicovak commented on issue #8500: URL: https://github.com/apache/pinot/issues/8500#issuecomment-1094811870 I had the same issue multiple time last week : - Try to addTable if config error or internal error happens (specify an SSL path for example that does not exist) - It returns a

[GitHub] [pinot] codecov-commenter commented on pull request #8503: Support single-valued BigDecimal in schema, type conversion, SQL statements and minimum set of transforms.

codecov-commenter commented on PR #8503: URL: https://github.com/apache/pinot/pull/8503#issuecomment-1094717065 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8503?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[GitHub] [pinot] saurabhd336 commented on a diff in pull request #8502: Add zk metadata to segment's metadata.properties file

saurabhd336 commented on code in PR #8502: URL: https://github.com/apache/pinot/pull/8502#discussion_r847082067 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/SegmentZKMetadataConfig.java: ## @@ -0,0 +1,44 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] nizarhejazi opened a new pull request, #8503: Support single-valued BigDecimal in schema, type conversion, SQL statements and minimum set of transforms.

nizarhejazi opened a new pull request, #8503: URL: https://github.com/apache/pinot/pull/8503 `feature`, `bugfix`, `release-notes`. - Support single-valued BigDecimal in schema, and type conversion so that data can be ingested as `BIG_DECIMAL`. - Support `BIG_DECIMAL` in SQL stateme

  1   2   >