(pinot) branch master updated: Add @JsonProperty annotation to LeadControllerEntry & LeadControllerResponse (#13950)

2024-09-09 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new ffc72b1019 Add @JsonProperty annotation to LeadCo

(pinot) branch master updated: Improve dimension table handing (#13967)

2024-09-09 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new d04e361ef1 Improve dimension table handing (#1396

Re: [PR] Add @JsonProperty annotation to LeadControllerEntry & LeadControllerResponse [pinot]

2024-09-09 Thread via GitHub
deepthi912 commented on code in PR #13950: URL: https://github.com/apache/pinot/pull/13950#discussion_r1751059419 ## pinot-controller/src/test/java/org/apache/pinot/controller/util/JsonSerializerTest.java: ## @@ -0,0 +1,187 @@ +/** + * Licensed to the Apache Software Foundation

Re: [PR] Add @JsonProperty annotation to LeadControllerEntry & LeadControllerResponse [pinot]

2024-09-09 Thread via GitHub
deepthi912 commented on code in PR #13950: URL: https://github.com/apache/pinot/pull/13950#discussion_r1751059419 ## pinot-controller/src/test/java/org/apache/pinot/controller/util/JsonSerializerTest.java: ## @@ -0,0 +1,187 @@ +/** + * Licensed to the Apache Software Foundation

Re: [PR] Improve dimension table handing [pinot]

2024-09-09 Thread via GitHub
Jackie-Jiang commented on code in PR #13967: URL: https://github.com/apache/pinot/pull/13967#discussion_r1751047670 ## pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/LookupTransformFunction.java: ## @@ -219,8 +218,7 @@ private void lookup(ValueBlock v

Re: [PR] Improve dimension table handing [pinot]

2024-09-09 Thread via GitHub
klsince commented on code in PR #13967: URL: https://github.com/apache/pinot/pull/13967#discussion_r1751039460 ## pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/LookupTransformFunction.java: ## @@ -219,8 +218,7 @@ private void lookup(ValueBlock valueB

Re: [PR] Remove Calcite EnumerableRules because we don't rely on enumerable calling convention [pinot]

2024-09-09 Thread via GitHub
Jackie-Jiang merged PR #13965: URL: https://github.com/apache/pinot/pull/13965 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[PR] [Multi-stage] Support lookup join [pinot]

2024-09-09 Thread via GitHub
Jackie-Jiang opened a new pull request, #13966: URL: https://github.com/apache/pinot/pull/13966 On top of #13965 - Add `lookup` join strategy as a hint - Improve dimension table performance TODO: Improve planner rules to not push down expression/filter into the dimension ta

[PR] Remove Calcite EnumerableRules because we don't rely on enumerable calling convention [pinot]

2024-09-09 Thread via GitHub
Jackie-Jiang opened a new pull request, #13965: URL: https://github.com/apache/pinot/pull/13965 Also cleans up the rule set for readability -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] Move untar method to BaseTaskExecutor and untar with peerDownload segment if deepstore corrupted [pinot]

2024-09-09 Thread via GitHub
codecov-commenter commented on PR #13964: URL: https://github.com/apache/pinot/pull/13964#issuecomment-2339100551 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13964?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

(pinot) branch master updated: Part-1: Pinot Timeseries Engine SPI (#13885)

2024-09-09 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new b828280882 Part-1: Pinot Timeseries Engine SPI (#

Re: [PR] Part-1: Pinot Timeseries Engine SPI [pinot]

2024-09-09 Thread via GitHub
Jackie-Jiang merged PR #13885: URL: https://github.com/apache/pinot/pull/13885 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Address comments for Segment Relocator Rebalance Config [pinot]

2024-09-09 Thread via GitHub
Jackie-Jiang merged PR #13963: URL: https://github.com/apache/pinot/pull/13963 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[PR] Move untar method to BaseTaskExecutor and untar with peerDownload segment if deepstore corrupted [pinot]

2024-09-09 Thread via GitHub
tibrewalpratik17 opened a new pull request, #13964: URL: https://github.com/apache/pinot/pull/13964 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] Optimize Adaptive Server Selection [pinot]

2024-09-09 Thread via GitHub
praveenc7 commented on code in PR #13952: URL: https://github.com/apache/pinot/pull/13952#discussion_r1750889890 ## pinot-broker/src/main/java/org/apache/pinot/broker/routing/instanceselector/ReplicaGroupInstanceSelector.java: ## @@ -152,21 +155,28 @@ private Pair, Map> selectS

Re: [PR] Address comments for Segment Relocator Rebalance Config [pinot]

2024-09-09 Thread via GitHub
codecov-commenter commented on PR #13963: URL: https://github.com/apache/pinot/pull/13963#issuecomment-2339007409 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13963?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.27.21 deleted (was 6c7ba52b48)

2024-09-09 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.27.21 in repository https://gitbox.apache.org/repos/asf/pinot.git was 6c7ba52b48 Bump software.amazon.awssdk:bom from 2.27.20 to 2.27.21 The r

(pinot) branch master updated (520eb991ae -> 05d7fd85cf)

2024-09-09 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 520eb991ae Rename dangling occurrences of StageNode to PlanNode (#13962) add 05d7fd85cf Bump software.amazon.awss

Re: [PR] Rename dangling occurrences of StageNode to PlanNode [pinot]

2024-09-09 Thread via GitHub
Jackie-Jiang merged PR #13962: URL: https://github.com/apache/pinot/pull/13962 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[PR] Address comments for Segment Relocator Rebalance Config [pinot]

2024-09-09 Thread via GitHub
Jackie-Jiang opened a new pull request, #13963: URL: https://github.com/apache/pinot/pull/13963 Address remaining comments for #13863 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] DISTINCTCOUNT and percentile doesn't utilize indexes to filter data [pinot]

2024-09-09 Thread via GitHub
Jackie-Jiang commented on issue #13833: URL: https://github.com/apache/pinot/issues/13833#issuecomment-2338854162 > There is no way to use ST index in this case but it also prevents inv index from being created/used. @mayankshriv Inverted index is being created/used. What is missing i

Re: [PR] Add @JsonProperty annotation to LeadControllerEntry & LeadControllerResponse [pinot]

2024-09-09 Thread via GitHub
Jackie-Jiang commented on code in PR #13950: URL: https://github.com/apache/pinot/pull/13950#discussion_r1750761802 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotLeadControllerRestletResource.java: ## @@ -211,12 +212,16 @@ public List getTableN

Re: [I] Bump org.apache.helix:helix-core from 1.3.1 to 1.4.0 [pinot]

2024-09-09 Thread via GitHub
Jackie-Jiang commented on issue #13472: URL: https://github.com/apache/pinot/issues/13472#issuecomment-2338830528 Helix has released [1.4.1](https://helix.apache.org/1.4.1-docs/releasenotes/release-1.4.1.html). We should directly bump to the latest version -- This is an automated message

(pinot) branch dependabot/maven/log4j.version-2.24.0 deleted (was cf2ab3a264)

2024-09-09 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/log4j.version-2.24.0 in repository https://gitbox.apache.org/repos/asf/pinot.git was cf2ab3a264 Bump log4j.version from 2.23.1 to 2.24.0 The revisions that were on this bra

Re: [PR] Retain existing segments for segmentsToCleanUp in startReplaceSegments method [pinot]

2024-09-09 Thread via GitHub
Jackie-Jiang commented on code in PR #13953: URL: https://github.com/apache/pinot/pull/13953#discussion_r1750726919 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java: ## @@ -3722,8 +3722,12 @@ public String startReplaceSegmen

(pinot) branch master updated: Add Segment Relocator Rebalance Configs to Controller Conf (#13863)

2024-09-09 Thread tingchen
This is an automated email from the ASF dual-hosted git repository. tingchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new d73aa1ee31 Add Segment Relocator Rebalance Conf

Re: [PR] Add Segment Relocator Rebalance Configs to Controller Conf [pinot]

2024-09-09 Thread via GitHub
chenboat merged PR #13863: URL: https://github.com/apache/pinot/pull/13863 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] Enforce removeSegment flow with _enableDeletedKeysCompactionConsistency [pinot]

2024-09-09 Thread via GitHub
tibrewalpratik17 commented on code in PR #13914: URL: https://github.com/apache/pinot/pull/13914#discussion_r1750640540 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java: ## @@ -544,9 +544,7 @@ public void addSegme

Re: [PR] Fix handling of empty server tags in controller's availability check [pinot]

2024-09-09 Thread via GitHub
anandheritage commented on code in PR #13954: URL: https://github.com/apache/pinot/pull/13954#discussion_r1750623794 ## pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java: ## @@ -542,6 +549,12 @@ public static List getInstancesWithTag(List instanceC

Re: [PR] Fix handling of empty server tags in controller's availability check [pinot]

2024-09-09 Thread via GitHub
mayankshriv commented on code in PR #13954: URL: https://github.com/apache/pinot/pull/13954#discussion_r1750606115 ## pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java: ## @@ -542,6 +549,12 @@ public static List getInstancesWithTag(List instanceCon

(pinot) branch master updated: [Backfill] flink based upsert table backfill support (#13837)

2024-09-09 Thread xbli
This is an automated email from the ASF dual-hosted git repository. xbli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new a9e4e5289f [Backfill] flink based upsert table back

Re: [PR] Improve performance of DataBlock serde [pinot]

2024-09-09 Thread via GitHub
gortiz commented on code in PR #13303: URL: https://github.com/apache/pinot/pull/13303#discussion_r1750400466 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/DataBlockUtils.java: ## @@ -18,20 +18,48 @@ */ package org.apache.pinot.common.datablock; +import co

Re: [PR] Rename dangling occurrences of StageNode to PlanNode [pinot]

2024-09-09 Thread via GitHub
codecov-commenter commented on PR #13962: URL: https://github.com/apache/pinot/pull/13962#issuecomment-2338290038 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13962?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Multi stage explain [pinot]

2024-09-09 Thread via GitHub
gortiz commented on code in PR #13733: URL: https://github.com/apache/pinot/pull/13733#discussion_r1750381217 ## pinot-common/src/main/java/org/apache/pinot/common/utils/DatabaseUtils.java: ## @@ -32,6 +32,10 @@ public class DatabaseUtils { private DatabaseUtils() { } +

Re: [PR] Multi stage explain [pinot]

2024-09-09 Thread via GitHub
gortiz commented on code in PR #13733: URL: https://github.com/apache/pinot/pull/13733#discussion_r1750350125 ## pinot-core/src/main/java/org/apache/pinot/core/operator/ProjectionOperator.java: ## @@ -100,6 +100,17 @@ public String toExplainString() { return stringBuilder.a

Re: [PR] Multi stage explain [pinot]

2024-09-09 Thread via GitHub
gortiz commented on code in PR #13733: URL: https://github.com/apache/pinot/pull/13733#discussion_r1750345793 ## pinot-core/src/main/java/org/apache/pinot/core/operator/query/AggregationOperator.java: ## @@ -100,4 +103,21 @@ public String toExplainString() { return string

Re: [PR] Multi stage explain [pinot]

2024-09-09 Thread via GitHub
gortiz commented on code in PR #13733: URL: https://github.com/apache/pinot/pull/13733#discussion_r1750325812 ## pinot-core/src/main/java/org/apache/pinot/core/operator/query/FilteredGroupByOperator.java: ## @@ -230,4 +232,26 @@ public String toExplainString() { return st

Re: [PR] Multi stage explain [pinot]

2024-09-09 Thread via GitHub
gortiz commented on code in PR #13733: URL: https://github.com/apache/pinot/pull/13733#discussion_r1750317381 ## pinot-common/src/main/proto/plan.proto: ## @@ -193,3 +194,29 @@ message WindowNode { int32 upperBound = 6; repeated Literal constants = 7; } + +message Explain

Re: [PR] Multi stage explain [pinot]

2024-09-09 Thread via GitHub
gortiz commented on code in PR #13733: URL: https://github.com/apache/pinot/pull/13733#discussion_r1750308034 ## pinot-core/src/main/java/org/apache/pinot/core/plan/PinotExplainedRelNode.java: ## @@ -0,0 +1,146 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] Multi stage explain [pinot]

2024-09-09 Thread via GitHub
gortiz commented on code in PR #13733: URL: https://github.com/apache/pinot/pull/13733#discussion_r1750300488 ## pinot-core/src/main/java/org/apache/pinot/core/plan/PinotExplainedRelNode.java: ## @@ -0,0 +1,146 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] Multi stage explain [pinot]

2024-09-09 Thread via GitHub
gortiz commented on code in PR #13733: URL: https://github.com/apache/pinot/pull/13733#discussion_r1750217340 ## pinot-core/src/main/java/org/apache/pinot/core/query/request/context/QueryContext.java: ## @@ -85,7 +85,7 @@ public class QueryContext { private final int _offset;

Re: [PR] Multi stage explain [pinot]

2024-09-09 Thread via GitHub
gortiz commented on code in PR #13733: URL: https://github.com/apache/pinot/pull/13733#discussion_r1750212242 ## pinot-core/src/main/java/org/apache/pinot/core/query/request/context/QueryContext.java: ## @@ -528,7 +538,16 @@ public Builder setExpressionOverrideHints(Map

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.27.21 created (now 6c7ba52b48)

2024-09-09 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.27.21 in repository https://gitbox.apache.org/repos/asf/pinot.git at 6c7ba52b48 Bump software.amazon.awssdk:bom from 2.27.20 to 2.27.21 N

[PR] Bump software.amazon.awssdk:bom from 2.27.20 to 2.27.21 [pinot]

2024-09-09 Thread via GitHub
dependabot[bot] opened a new pull request, #13961: URL: https://github.com/apache/pinot/pull/13961 Bumps software.amazon.awssdk:bom from 2.27.20 to 2.27.21. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=softwa

(pinot) branch dependabot/maven/circe.version-0.14.10 created (now acde6e0b3f)

2024-09-09 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/circe.version-0.14.10 in repository https://gitbox.apache.org/repos/asf/pinot.git at acde6e0b3f Bump circe.version from 0.14.9 to 0.14.10 No new revisions were added b

Re: [PR] Part-1: Pinot Timeseries Engine SPI [pinot]

2024-09-09 Thread via GitHub
ankitsultana commented on code in PR #13885: URL: https://github.com/apache/pinot/pull/13885#discussion_r1750045095 ## pinot-timeseries/pinot-timeseries-spi/src/main/java/org/apache/pinot/tsdb/spi/plan/ScanFilterAndProjectPlanNode.java: ## @@ -0,0 +1,127 @@ +/** + * Licensed to

Re: [PR] Part-1: Pinot Timeseries Engine SPI [pinot]

2024-09-09 Thread via GitHub
ankitsultana commented on code in PR #13885: URL: https://github.com/apache/pinot/pull/13885#discussion_r1750027760 ## pinot-timeseries/pinot-timeseries-spi/src/main/java/org/apache/pinot/tsdb/spi/RangeTimeSeriesRequest.java: ## @@ -0,0 +1,99 @@ +/** + * Licensed to the Apache S

Re: [PR] disable-shading-by-default: Move shading config to its own profile build-shaded-jar and disable it by default [pinot]

2024-09-09 Thread via GitHub
codecov-commenter commented on PR #13956: URL: https://github.com/apache/pinot/pull/13956#issuecomment-2337707769 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13956?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Multi stage explain [pinot]

2024-09-09 Thread via GitHub
gortiz commented on code in PR #13733: URL: https://github.com/apache/pinot/pull/13733#discussion_r1749925095 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/MultiStageEngineExplainIntegrationTest.java: ## @@ -0,0 +1,255 @@ +/** + * Licensed to the Ap

Re: [PR] executor-spi [pinot]

2024-09-09 Thread via GitHub
gortiz merged PR #13921: URL: https://github.com/apache/pinot/pull/13921 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

Re: [PR] Fixes in accounting memory usage in multi-stage queries [pinot]

2024-09-09 Thread via GitHub
codecov-commenter commented on PR #13955: URL: https://github.com/apache/pinot/pull/13955#issuecomment-2337309697 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13955?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u