[pinot] branch master updated: Remove the tableDataManager deletion mechanism when the number of segments drops to 0 to avoid race conditions. (#8422)

This is an automated email from the ASF dual-hosted git repository. snlee pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 51904fb Remove the tableDataManager deletion mech

[GitHub] [pinot] snleee merged pull request #8422: Change TableDataManger deletion to message based

snleee merged pull request #8422: URL: https://github.com/apache/pinot/pull/8422 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr.

[pinot] branch retain-instance-sequence updated (2a6411d -> 072da30)

This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch retain-instance-sequence in repository https://gitbox.apache.org/repos/asf/pinot.git. discard 2a6411d Add retainInstancesSequence feature to table rebalance to minimize data movement between ins

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8422: Change TableDataManger deletion to message based

codecov-commenter edited a comment on pull request #8422: URL: https://github.com/apache/pinot/pull/8422#issuecomment-1081038136 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8422?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[pinot] branch retain-instance-sequence updated (c9591ed -> 2a6411d)

This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch retain-instance-sequence in repository https://gitbox.apache.org/repos/asf/pinot.git. discard c9591ed Add retainInstancesSequence feature to table rebalance to minimize data movement between ins

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8445: add metric for largest segment size

codecov-commenter edited a comment on pull request #8445: URL: https://github.com/apache/pinot/pull/8445#issuecomment-1084055024 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8445?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter commented on pull request #8445: add metric for largest segment size

codecov-commenter commented on pull request #8445: URL: https://github.com/apache/pinot/pull/8445#issuecomment-1084055024 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8445?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apa

[GitHub] [pinot] snleee commented on a change in pull request #8443: Fix ChildTraceId in multithreaded environment

snleee commented on a change in pull request #8443: URL: https://github.com/apache/pinot/pull/8443#discussion_r839151107 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/trace/TraceContext.java ## @@ -67,7 +68,7 @@ JsonNode toJson() { final String _trac

[GitHub] [pinot] jtao15 closed pull request #8422: Change TableDataManger deletion to message based

jtao15 closed pull request #8422: URL: https://github.com/apache/pinot/pull/8422 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr.

[GitHub] [pinot] jadami10 opened a new pull request #8445: add metric for largest segment size

jadami10 opened a new pull request #8445: URL: https://github.com/apache/pinot/pull/8445 ## Description This adds a metric for the largest segment size on disk by table - it only publishes the metric if there is at least 1 segment - the gauge is not reset if all segments are del

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839131605 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -1757,6 +1758,16 @@ public void del

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839131473 ## File path: pinot-server/src/main/java/org/apache/pinot/server/starter/helix/HelixInstanceDataManager.java ## @@ -73,6 +75,9 @@ @ThreadSafe public class

[GitHub] [pinot] snleee commented on a change in pull request #8422: Change TableDataManger deletion to message based

snleee commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839119100 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -1757,6 +1758,16 @@ public void del

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8422: Change TableDataManger deletion to message based

codecov-commenter edited a comment on pull request #8422: URL: https://github.com/apache/pinot/pull/8422#issuecomment-1081038136 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8422?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8412: add pinot-query-runtime

codecov-commenter edited a comment on pull request #8412: URL: https://github.com/apache/pinot/pull/8412#issuecomment-1079168510 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8412?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839090215 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -1757,6 +1758,10 @@ public void del

[GitHub] [pinot] siddharthteotia commented on a change in pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

siddharthteotia commented on a change in pull request #8441: URL: https://github.com/apache/pinot/pull/8441#discussion_r839088155 ## File path: pinot-common/src/main/java/org/apache/pinot/common/assignment/InstancePartitions.java ## @@ -58,25 +60,37 @@ @JsonIgnoreProperties(i

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839086828 ## File path: pinot-common/src/main/java/org/apache/pinot/common/metrics/ServerMeter.java ## @@ -33,6 +33,7 @@ QUERY_EXECUTION_EXCEPTIONS("exceptions", f

[GitHub] [pinot] mcvsubbu edited a comment on issue #8423: The race condition that TableDataManager is removed during segment download because of incorrect segment count.

mcvsubbu edited a comment on issue #8423: URL: https://github.com/apache/pinot/issues/8423#issuecomment-1083881269 Lol, I asked this same question in https://github.com/apache/pinot/pull/6608 @Jackie-Jiang and @jtao15 is it fair to say that the root cause is this PR? -- This is an

[GitHub] [pinot] mcvsubbu commented on issue #8423: The race condition that TableDataManager is removed during segment download because of incorrect segment count.

mcvsubbu commented on issue #8423: URL: https://github.com/apache/pinot/issues/8423#issuecomment-1083881269 Lol, I asked this same question in https://github.com/apache/pinot/pull/6608 -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [pinot] siddharthteotia commented on a change in pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

siddharthteotia commented on a change in pull request #8441: URL: https://github.com/apache/pinot/pull/8441#discussion_r839066546 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/instance/InstanceTagPoolSelector.java ## @@ -48,52 +

[GitHub] [pinot] siddharthteotia commented on a change in pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

siddharthteotia commented on a change in pull request #8441: URL: https://github.com/apache/pinot/pull/8441#discussion_r839066546 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/instance/InstanceTagPoolSelector.java ## @@ -48,52 +

[GitHub] [pinot] siddharthteotia commented on a change in pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

siddharthteotia commented on a change in pull request #8441: URL: https://github.com/apache/pinot/pull/8441#discussion_r839050322 ## File path: pinot-common/src/main/java/org/apache/pinot/common/assignment/InstancePartitions.java ## @@ -58,25 +60,37 @@ @JsonIgnoreProperties(i

[GitHub] [pinot] jasperjiaguo commented on a change in pull request #8036: Add new compat test cases and data (#7235)

jasperjiaguo commented on a change in pull request #8036: URL: https://github.com/apache/pinot/pull/8036#discussion_r839060259 ## File path: compatibility-verifier/sample-test-suite/config/dataGenerator.py ## @@ -0,0 +1,90 @@ +# +# Licensed to the Apache Software Foundation (AS

[GitHub] [pinot] mcvsubbu commented on a change in pull request #8036: Add new compat test cases and data (#7235)

mcvsubbu commented on a change in pull request #8036: URL: https://github.com/apache/pinot/pull/8036#discussion_r839056621 ## File path: compatibility-verifier/sample-test-suite/post-server-rollback.yaml ## @@ -44,3 +58,14 @@ operations: description: Run query on FeatureTe

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839056884 ## File path: pinot-server/src/main/java/org/apache/pinot/server/starter/helix/SegmentMessageHandlerFactory.java ## @@ -144,6 +147,28 @@ public HelixTaskRes

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839056884 ## File path: pinot-server/src/main/java/org/apache/pinot/server/starter/helix/SegmentMessageHandlerFactory.java ## @@ -144,6 +147,28 @@ public HelixTaskRes

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839056385 ## File path: pinot-server/src/main/java/org/apache/pinot/server/starter/helix/SegmentMessageHandlerFactory.java ## @@ -144,6 +147,28 @@ public HelixTaskRes

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839056123 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -2053,6 +2062,35 @@ public boolean

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839055818 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -1800,6 +1805,10 @@ public void del

[GitHub] [pinot] siddharthteotia commented on a change in pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

siddharthteotia commented on a change in pull request #8441: URL: https://github.com/apache/pinot/pull/8441#discussion_r839051143 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/instance/InstanceAssignmentDriver.java ## @@ -51,16

[GitHub] [pinot] siddharthteotia commented on a change in pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

siddharthteotia commented on a change in pull request #8441: URL: https://github.com/apache/pinot/pull/8441#discussion_r839050322 ## File path: pinot-common/src/main/java/org/apache/pinot/common/assignment/InstancePartitions.java ## @@ -58,25 +60,37 @@ @JsonIgnoreProperties(i

[GitHub] [pinot] siddharthteotia commented on a change in pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

siddharthteotia commented on a change in pull request #8441: URL: https://github.com/apache/pinot/pull/8441#discussion_r839047121 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/instance/InstanceAssignmentDriver.java ## @@ -75,9 +

[GitHub] [pinot] siddharthteotia commented on a change in pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

siddharthteotia commented on a change in pull request #8441: URL: https://github.com/apache/pinot/pull/8441#discussion_r839047121 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/instance/InstanceAssignmentDriver.java ## @@ -75,9 +

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839040897 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -1757,6 +1758,10 @@ public void del

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

codecov-commenter edited a comment on pull request #8441: URL: https://github.com/apache/pinot/pull/8441#issuecomment-1083528279 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8441?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] mqliang commented on pull request #8036: Add new compat test cases and data (#7235)

mqliang commented on pull request #8036: URL: https://github.com/apache/pinot/pull/8036#issuecomment-1083717451 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

[GitHub] [pinot] siddharthteotia commented on a change in pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

siddharthteotia commented on a change in pull request #8441: URL: https://github.com/apache/pinot/pull/8441#discussion_r839028089 ## File path: pinot-common/src/main/java/org/apache/pinot/common/assignment/InstancePartitions.java ## @@ -58,25 +60,37 @@ @JsonIgnoreProperties(i

[GitHub] [pinot] siddharthteotia commented on a change in pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

siddharthteotia commented on a change in pull request #8441: URL: https://github.com/apache/pinot/pull/8441#discussion_r839022456 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/instance/InstanceAssignmentDriver.java ## @@ -51,16

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839022387 ## File path: pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java ## @@ -408,7 +413,41 @@ public voi

[GitHub] [pinot] siddharthteotia commented on a change in pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

siddharthteotia commented on a change in pull request #8441: URL: https://github.com/apache/pinot/pull/8441#discussion_r839022456 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/instance/InstanceAssignmentDriver.java ## @@ -51,16

[GitHub] [pinot] PrachiKhobragade commented on a change in pull request #8422: Change TableDataManger deletion to message based

PrachiKhobragade commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839021507 ## File path: pinot-server/src/main/java/org/apache/pinot/server/starter/helix/SegmentMessageHandlerFactory.java ## @@ -144,6 +147,28 @@ public He

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839019506 ## File path: pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java ## @@ -408,7 +413,41 @@ public voi

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839018545 ## File path: pinot-server/src/main/java/org/apache/pinot/server/starter/helix/SegmentMessageHandlerFactory.java ## @@ -58,6 +59,8 @@ public MessageHandler

[pinot] branch retain-instance-sequence updated (0c2c0e6 -> c9591ed)

This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch retain-instance-sequence in repository https://gitbox.apache.org/repos/asf/pinot.git. discard 0c2c0e6 Add retainInstancesSequence feature to table rebalance to minimize data movement between ins

[pinot] branch retain-instance-sequence updated (d31a416 -> 0c2c0e6)

This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch retain-instance-sequence in repository https://gitbox.apache.org/repos/asf/pinot.git. discard d31a416 Add retainInstancesSequence feature to table rebalance to minimize data movement between ins

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839010495 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -1757,6 +1758,10 @@ public void del

[GitHub] [pinot] jackjlli commented on a change in pull request #8422: Change TableDataManger deletion to message based

jackjlli commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839007738 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -2053,6 +2062,35 @@ public boolea

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r839008960 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -3285,10 +3324,11 @@ private void w

[GitHub] [pinot] xiangfu0 commented on a change in pull request #8343: Timestamp type index

xiangfu0 commented on a change in pull request #8343: URL: https://github.com/apache/pinot/pull/8343#discussion_r838997714 ## File path: pinot-common/src/main/java/org/apache/pinot/common/request/context/FunctionContext.java ## @@ -84,7 +84,7 @@ public boolean equals(Object o)

[GitHub] [pinot] xiangfu0 commented on a change in pull request #8343: Timestamp type index

xiangfu0 commented on a change in pull request #8343: URL: https://github.com/apache/pinot/pull/8343#discussion_r838996763 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/config/table/TimestampIndexGranularity.java ## @@ -0,0 +1,66 @@ +/** + * Licensed to the Apache

[GitHub] [pinot] xiangfu0 commented on a change in pull request #8343: Timestamp type index

xiangfu0 commented on a change in pull request #8343: URL: https://github.com/apache/pinot/pull/8343#discussion_r838996606 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/config/table/TimestampIndexGranularity.java ## @@ -0,0 +1,66 @@ +/** + * Licensed to the Apache

[GitHub] [pinot] PrachiKhobragade commented on a change in pull request #8422: Change TableDataManger deletion to message based

PrachiKhobragade commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r838994387 ## File path: pinot-server/src/main/java/org/apache/pinot/server/starter/helix/SegmentMessageHandlerFactory.java ## @@ -58,6 +59,8 @@ public Messa

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8422: Change TableDataManger deletion to message based

codecov-commenter edited a comment on pull request #8422: URL: https://github.com/apache/pinot/pull/8422#issuecomment-1081038136 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8422?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8422: Change TableDataManger deletion to message based

codecov-commenter edited a comment on pull request #8422: URL: https://github.com/apache/pinot/pull/8422#issuecomment-1081038136 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8422?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8036: Add new compat test cases and data (#7235)

codecov-commenter edited a comment on pull request #8036: URL: https://github.com/apache/pinot/pull/8036#issuecomment-1015902387 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8036?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] snleee commented on a change in pull request #8422: Change TableDataManger deletion to message based

snleee commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r838863260 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -1757,6 +1758,10 @@ public void del

[GitHub] [pinot] snleee commented on a change in pull request #8422: Change TableDataManger deletion to message based

snleee commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r838863260 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -1757,6 +1758,10 @@ public void del

[GitHub] [pinot] snleee commented on a change in pull request #8422: Change TableDataManger deletion to message based

snleee commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r838865953 ## File path: pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java ## @@ -408,7 +413,41 @@ public voi

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r838958405 ## File path: pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java ## @@ -408,7 +413,41 @@ public voi

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r838958003 ## File path: pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/RealtimeClusterIntegrationTest.java ## @@ -154,6 +156,7 @@ public voi

[GitHub] [pinot] jtao15 commented on a change in pull request #8422: Change TableDataManger deletion to message based

jtao15 commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r838957733 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -2053,6 +2062,36 @@ public boolean

[GitHub] [pinot] jadami10 opened a new issue #8444: Filtering on a boolean field with just the field name results in NullPointerException

jadami10 opened a new issue #8444: URL: https://github.com/apache/pinot/issues/8444 Running a query with just `select from where ` results in a null point exception. This should be a supported query. Stack trace from pinot ``` [2022-03-30 20:40:05.343615] java.lang.NullPoint

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8343: Timestamp type index

codecov-commenter edited a comment on pull request #8343: URL: https://github.com/apache/pinot/pull/8343#issuecomment-1065843913 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8343?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] richardstartin commented on a change in pull request #8343: Timestamp type index

richardstartin commented on a change in pull request #8343: URL: https://github.com/apache/pinot/pull/8343#discussion_r838941826 ## File path: pinot-common/src/test/java/org/apache/pinot/common/request/RequestContextUtilsTest.java ## @@ -0,0 +1,191 @@ +/** + * Licensed to the

[GitHub] [pinot] jasperjiaguo commented on a change in pull request #8036: Add new compat test cases and data (#7235)

jasperjiaguo commented on a change in pull request #8036: URL: https://github.com/apache/pinot/pull/8036#discussion_r838931671 ## File path: compatibility-verifier/sample-test-suite/post-broker-rollback.yaml ## @@ -20,6 +20,12 @@ # Operations to be done. description: Operatio

[GitHub] [pinot] PrachiKhobragade commented on pull request #8443: Fix ChildTraceId in multithreaded environment

PrachiKhobragade commented on pull request #8443: URL: https://github.com/apache/pinot/pull/8443#issuecomment-1083588072 cc: @snleee @sajjad-moradi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [pinot] jasperjiaguo commented on a change in pull request #8036: Add new compat test cases and data (#7235)

jasperjiaguo commented on a change in pull request #8036: URL: https://github.com/apache/pinot/pull/8036#discussion_r838931999 ## File path: compatibility-verifier/sample-test-suite/config/feature-test-3-realtime.json ## @@ -0,0 +1,63 @@ +{ + "tableIndexConfig": { +"aggre

[GitHub] [pinot] jasperjiaguo commented on a change in pull request #8036: Add new compat test cases and data (#7235)

jasperjiaguo commented on a change in pull request #8036: URL: https://github.com/apache/pinot/pull/8036#discussion_r838931671 ## File path: compatibility-verifier/sample-test-suite/post-broker-rollback.yaml ## @@ -20,6 +20,12 @@ # Operations to be done. description: Operatio

[GitHub] [pinot] PrachiKhobragade opened a new pull request #8443: Fix ChildTraceId in multithreaded environment

PrachiKhobragade opened a new pull request #8443: URL: https://github.com/apache/pinot/pull/8443 ## Description The current numChildren used to create a childTraceId is not threadSafe, and can cause different child threads to have same traceID, fix is to make it atomic ## Upgrade Note

[GitHub] [pinot] noon-stripe commented on pull request #8369: [Generalized Pre-Aggregation] Initial Config wiring

noon-stripe commented on pull request #8369: URL: https://github.com/apache/pinot/pull/8369#issuecomment-1083584430 @Jackie-Jiang and @mcvsubbu Thanks for your reviews. In hindsight, this PR will be better as one large change instead of doing it in parts. I should have that up in the next

[GitHub] [pinot] noon-stripe closed pull request #8369: [Generalized Pre-Aggregation] Initial Config wiring

noon-stripe closed pull request #8369: URL: https://github.com/apache/pinot/pull/8369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsu

[GitHub] [pinot] xiangfu0 commented on a change in pull request #8343: Timestamp type index

xiangfu0 commented on a change in pull request #8343: URL: https://github.com/apache/pinot/pull/8343#discussion_r838927613 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/config/table/TimestampIndexGranularity.java ## @@ -0,0 +1,66 @@ +/** + * Licensed to the Apache

[GitHub] [pinot] xiangfu0 commented on pull request #8343: Timestamp type index

xiangfu0 commented on pull request #8343: URL: https://github.com/apache/pinot/pull/8343#issuecomment-1083580586 > Can you explain if there is any dependency on distinct hash code values? I recall seeing hash codes being used as map keys but can’t find it. Got suggestions from @Jacki

[GitHub] [pinot] xiangfu0 commented on a change in pull request #8343: Timestamp type index

xiangfu0 commented on a change in pull request #8343: URL: https://github.com/apache/pinot/pull/8343#discussion_r838925835 ## File path: pinot-common/src/test/java/org/apache/pinot/common/request/RequestContextUtilsTest.java ## @@ -0,0 +1,191 @@ +/** + * Licensed to the Apache

[pinot] branch master updated: Refactor getMin/getMax value to IndexCreationContext from Range (#8442)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new e2053f6 Refactor getMin/getMax value to IndexCr

[GitHub] [pinot] xiangfu0 merged pull request #8442: Refactor getMin/getMax value to IndexCreationContext from Range

xiangfu0 merged pull request #8442: URL: https://github.com/apache/pinot/pull/8442 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[GitHub] [pinot] richardstartin closed issue #8410: Optimise filtered count queries when an inverted index is present

richardstartin closed issue #8410: URL: https://github.com/apache/pinot/issues/8410 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [pinot] jadami10 commented on issue #8423: The race condition that TableDataManager is removed during segment download because of incorrect segment count.

jadami10 commented on issue #8423: URL: https://github.com/apache/pinot/issues/8423#issuecomment-1083559400 Just to confirm my understanding, #7961 surfaced this bug because it's deleting tmp files or it's causing it? -- This is an automated message from the Apache Git Service. To respon

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8442: Refactor getMin/getMax value to IndexCreationContext from Range

codecov-commenter edited a comment on pull request #8442: URL: https://github.com/apache/pinot/pull/8442#issuecomment-1083538670 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8442?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

codecov-commenter edited a comment on pull request #8441: URL: https://github.com/apache/pinot/pull/8441#issuecomment-1083528279 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8441?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter commented on pull request #8442: Refactor getMin/getMax value to IndexCreationContext from Range

codecov-commenter commented on pull request #8442: URL: https://github.com/apache/pinot/pull/8442#issuecomment-1083538670 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8442?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apa

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

codecov-commenter edited a comment on pull request #8441: URL: https://github.com/apache/pinot/pull/8441#issuecomment-1083528279 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8441?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter commented on pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

codecov-commenter commented on pull request #8441: URL: https://github.com/apache/pinot/pull/8441#issuecomment-1083528279 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8441?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apa

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8343: Timestamp type index

codecov-commenter edited a comment on pull request #8343: URL: https://github.com/apache/pinot/pull/8343#issuecomment-1065843913 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8343?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] xiangfu0 opened a new pull request #8442: Refactor getMin/getMax value to IndexCreationContext from Range

xiangfu0 opened a new pull request #8442: URL: https://github.com/apache/pinot/pull/8442 ## Description Refactor getMin/getMax value to IndexCreationContext from Range ## Upgrade Notes Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Ser

[GitHub] [pinot] richardstartin commented on a change in pull request #8343: Timestamp type index

richardstartin commented on a change in pull request #8343: URL: https://github.com/apache/pinot/pull/8343#discussion_r838853680 ## File path: pinot-common/src/test/java/org/apache/pinot/common/request/RequestContextUtilsTest.java ## @@ -0,0 +1,191 @@ +/** + * Licensed to the

[pinot] branch retain-instance-sequence updated (137b00c -> d31a416)

This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch retain-instance-sequence in repository https://gitbox.apache.org/repos/asf/pinot.git. discard 137b00c Add retainInstancesSequence feature to table rebalance to minimize data movement between ins

[GitHub] [pinot] xiangfu0 commented on a change in pull request #8343: Timestamp type index

xiangfu0 commented on a change in pull request #8343: URL: https://github.com/apache/pinot/pull/8343#discussion_r838849486 ## File path: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/inv/RangeIndexCreator.java ## @@ -112,12 +112,31 @@

[GitHub] [pinot] jackjlli commented on a change in pull request #8422: Change TableDataManger deletion to message based

jackjlli commented on a change in pull request #8422: URL: https://github.com/apache/pinot/pull/8422#discussion_r838836727 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -2053,6 +2062,36 @@ public boolea

[GitHub] [pinot] richardstartin commented on pull request #8343: Timestamp type index

richardstartin commented on pull request #8343: URL: https://github.com/apache/pinot/pull/8343#issuecomment-1083474928 Can you explain if there is any dependency on distinct hash code values? I recall seeing hash codes being used as map keys but can’t find it. -- This is an automated mes

[GitHub] [pinot] mcvsubbu commented on a change in pull request #8036: Add new compat test cases and data (#7235)

mcvsubbu commented on a change in pull request #8036: URL: https://github.com/apache/pinot/pull/8036#discussion_r838840404 ## File path: compatibility-verifier/sample-test-suite/config/feature-test-3-realtime.json ## @@ -0,0 +1,63 @@ +{ + "tableIndexConfig": { +"aggregate

[GitHub] [pinot] richardstartin commented on a change in pull request #8343: Timestamp type index

richardstartin commented on a change in pull request #8343: URL: https://github.com/apache/pinot/pull/8343#discussion_r838843286 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/config/table/TimestampIndexGranularity.java ## @@ -0,0 +1,66 @@ +/** + * Licensed to the

[GitHub] [pinot] richardstartin commented on a change in pull request #8343: Timestamp type index

richardstartin commented on a change in pull request #8343: URL: https://github.com/apache/pinot/pull/8343#discussion_r838841673 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/config/table/TimestampIndexGranularity.java ## @@ -0,0 +1,66 @@ +/** + * Licensed to the

[GitHub] [pinot] richardstartin commented on a change in pull request #8343: Timestamp type index

richardstartin commented on a change in pull request #8343: URL: https://github.com/apache/pinot/pull/8343#discussion_r838836664 ## File path: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ExpressionTransformer.java ## @@ -65,6 +67,19 @@ pu

[GitHub] [pinot] richardstartin commented on a change in pull request #8343: Timestamp type index

richardstartin commented on a change in pull request #8343: URL: https://github.com/apache/pinot/pull/8343#discussion_r838833182 ## File path: pinot-common/src/main/java/org/apache/pinot/common/request/context/FunctionContext.java ## @@ -84,7 +84,7 @@ public boolean equals(Obj

[pinot] branch retain-instance-sequence updated (0214adf -> 137b00c)

This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch retain-instance-sequence in repository https://gitbox.apache.org/repos/asf/pinot.git. omit 0214adf Add retainInstancesSequence feature to table rebalance to minimize data movement between ins

[GitHub] [pinot] jackjlli opened a new pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

jackjlli opened a new pull request #8441: URL: https://github.com/apache/pinot/pull/8441 ## Description This PR adds `retainInstancesSequence` feature to table rebalance to minimize data movement between instances within the pool. Related issue: https://github.com/apache/pinot/iss

[pinot] 01/01: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch retain-instance-sequence in repository https://gitbox.apache.org/repos/asf/pinot.git commit 0214adf78c12aae29732aa0b94932854fca13b0e Author: Jack Li(Analytics Engineering) AuthorDate: Wed Mar 30 10

[pinot] branch retain-instance-sequence updated (3067f65 -> 0214adf)

This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch retain-instance-sequence in repository https://gitbox.apache.org/repos/asf/pinot.git. discard 3067f65 Add retainInstancesSequence feature to table rebalance to minimize data movement between ins

  1   2   >