Re: [PR] [WIP] Ensure Consistent Upsert Tie-Breaking using ZK Segment Creation Time from Controller ZK metadata [pinot]

2025-06-11 Thread via GitHub
tarun11Mavani closed pull request #16006: [WIP] Ensure Consistent Upsert Tie-Breaking using ZK Segment Creation Time from Controller ZK metadata URL: https://github.com/apache/pinot/pull/16006 -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] [WIP] Ensure Consistent Upsert Tie-Breaking using ZK Segment Creation Time from Controller ZK metadata [pinot]

2025-06-11 Thread via GitHub
tarun11Mavani commented on PR #16006: URL: https://github.com/apache/pinot/pull/16006#issuecomment-2965218919 We abandoned this change in favor of #16034. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [MSE] For constant expression query, solve it with a single random server [pinot]

2025-06-11 Thread via GitHub
yashmayya commented on code in PR #16083: URL: https://github.com/apache/pinot/pull/16083#discussion_r2141505775 ## pinot-query-planner/src/main/java/org/apache/pinot/query/routing/WorkerManager.java: ## @@ -319,48 +340,44 @@ private boolean isPrePartitionAssignment(List childr

Re: [PR] Reduce fanout of intermediate stage [pinot]

2025-06-11 Thread via GitHub
Jackie-Jiang commented on PR #16084: URL: https://github.com/apache/pinot/pull/16084#issuecomment-2964689808 @ankitsultana Can you help take a look and see if this is similar to how intermediate stage workers are picked in the new physical optimizer? -- This is an automated message from t

Re: [PR] Add multistage thread limiting configs at the broker and server level [pinot]

2025-06-11 Thread via GitHub
Jackie-Jiang commented on code in PR #16080: URL: https://github.com/apache/pinot/pull/16080#discussion_r2141347061 ## pinot-spi/src/main/java/org/apache/pinot/spi/executor/HardLimitExecutor.java: ## @@ -46,21 +46,28 @@ public HardLimitExecutor(int max, ExecutorService executor

Re: [PR] Add multistage thread limiting configs at the broker and server level [pinot]

2025-06-11 Thread via GitHub
satwik-pachigolla commented on code in PR #16080: URL: https://github.com/apache/pinot/pull/16080#discussion_r2141294629 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/QueryRunner.java: ## @@ -195,6 +195,7 @@ public void init(PinotConfiguration config, Inst

Re: [PR] Tenant Rebalance API Improvement [pinot]

2025-06-11 Thread via GitHub
somandal commented on PR #15891: URL: https://github.com/apache/pinot/pull/15891#issuecomment-2964546545 @J-HowHuang can you look into API deprecation vs. how to deprecate certain parameters to an existing API? I think we should have a good path forward for how to mark these things as depre

(pinot) branch master updated: Add config for logging queries before processing on broker instances (#16056)

2025-06-11 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new fecaca97d0 Add config for logging queries bef

Re: [PR] Add config for logging queries before processing on broker instances [pinot]

2025-06-11 Thread via GitHub
vvivekiyer merged PR #16056: URL: https://github.com/apache/pinot/pull/16056 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

Re: [PR] Tenant Rebalance API Improvement [pinot]

2025-06-11 Thread via GitHub
somandal commented on code in PR #15891: URL: https://github.com/apache/pinot/pull/15891#discussion_r2141249400 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTenantRestletResource.java: ## @@ -690,9 +685,52 @@ public SuccessResponse deleteTenan

[PR] [MSE] For constant expression query, solve it with a single random server [pinot]

2025-06-11 Thread via GitHub
Jackie-Jiang opened a new pull request, #16083: URL: https://github.com/apache/pinot/pull/16083 Fix #16024 For constant expression query (no table accessed), pick a random enabled server to solve it instead of fanning it out to all servers. -- This is an automated message from the

Re: [PR] Tenant Rebalance API Improvement [pinot]

2025-06-11 Thread via GitHub
somandal commented on code in PR #15891: URL: https://github.com/apache/pinot/pull/15891#discussion_r2141244767 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTenantRestletResource.java: ## @@ -690,9 +685,52 @@ public SuccessResponse deleteTenan

[I] Support for a Native Time Series/Metrics Format [pinot]

2025-06-11 Thread via GitHub
itschrispeck opened a new issue, #16082: URL: https://github.com/apache/pinot/issues/16082 I had worked on a POC earlier this year for storing time series data in pinot - the columnar format has some inefficiencies we were trying to overcome. Wanted to share some of the ideas here, and gaug

Re: [PR] Add validations for MV columns: reject MV primary-keys for upsert/dedup, BIG_DECIMAL, and JSON [pinot]

2025-06-11 Thread via GitHub
somandal merged PR #16079: URL: https://github.com/apache/pinot/pull/16079 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

(pinot) branch master updated: Add validations for MV columns: reject MV primary-keys for upsert/dedup, BIG_DECIMAL, and JSON (#16079)

2025-06-11 Thread somandal
This is an automated email from the ASF dual-hosted git repository. somandal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new ba2013728b Add validations for MV columns: reje

Re: [I] Upgrade Pinot service source version to Java 21 [pinot]

2025-06-11 Thread via GitHub
chenboat commented on issue #15343: URL: https://github.com/apache/pinot/issues/15343#issuecomment-2964488066 Moving to Java 21 will also enable new features like SIMD processing. @gortiz will this java21 upgrade go with 1.4 release? -- This is an automated message from the Apache Git Ser

Re: [PR] [Priority Replica Group Routing] Support orderedPreferredReplicas query option for customizable routing strategy [pinot]

2025-06-11 Thread via GitHub
somandal commented on code in PR #15203: URL: https://github.com/apache/pinot/pull/15203#discussion_r2141196007 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandler.java: ## @@ -758,6 +761,11 @@ protected BrokerResponse doHandl

Re: [PR] [Minor] Fix typo in table config validation [pinot]

2025-06-11 Thread via GitHub
Jackie-Jiang merged PR #16081: URL: https://github.com/apache/pinot/pull/16081 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Add validations for MV columns: reject MV primary-keys for upsert/dedup, BIG_DECIMAL, and JSON [pinot]

2025-06-11 Thread via GitHub
Jackie-Jiang commented on code in PR #16079: URL: https://github.com/apache/pinot/pull/16079#discussion_r2141178674 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/SchemaUtils.java: ## @@ -147,6 +147,9 @@ public static void validate(Schema schema, boole

(pinot) branch master updated: [Minor] Fix typo in table config validation (#16081)

2025-06-11 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new c8c9c8444d [Minor] Fix typo in table config valid

Re: [PR] Add validations for MV columns: reject MV primary-keys for upsert/dedup, BIG_DECIMAL, and JSON [pinot]

2025-06-11 Thread via GitHub
somandal commented on code in PR #16079: URL: https://github.com/apache/pinot/pull/16079#discussion_r2141163073 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/SchemaUtils.java: ## @@ -147,6 +147,9 @@ public static void validate(Schema schema, boolean

Re: [PR] Add validations for MV columns: reject MV primary-keys for upsert/dedup, BIG_DECIMAL, and JSON [pinot]

2025-06-11 Thread via GitHub
Jackie-Jiang commented on code in PR #16079: URL: https://github.com/apache/pinot/pull/16079#discussion_r2141160516 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/SchemaUtils.java: ## @@ -147,6 +147,9 @@ public static void validate(Schema schema, boole

Re: [PR] Add validations for MV columns: reject MV primary-keys for upsert/dedup, BIG_DECIMAL, and JSON [pinot]

2025-06-11 Thread via GitHub
somandal commented on code in PR #16079: URL: https://github.com/apache/pinot/pull/16079#discussion_r2141158012 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/SchemaUtils.java: ## @@ -147,6 +147,9 @@ public static void validate(Schema schema, boolean

Re: [PR] Add validations for MV columns: reject MV primary-keys for upsert/dedup, BIG_DECIMAL, and JSON [pinot]

2025-06-11 Thread via GitHub
somandal commented on code in PR #16079: URL: https://github.com/apache/pinot/pull/16079#discussion_r2141157474 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/TableConfigUtils.java: ## @@ -714,6 +714,12 @@ static void validateUpsertAndDedupConfig(Table

Re: [PR] Add validations for MV columns: reject MV primary-keys for upsert/dedup, BIG_DECIMAL, and JSON [pinot]

2025-06-11 Thread via GitHub
Jackie-Jiang commented on code in PR #16079: URL: https://github.com/apache/pinot/pull/16079#discussion_r2141147412 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/SchemaUtils.java: ## @@ -147,6 +147,9 @@ public static void validate(Schema schema, boole

Re: [PR] Bump nl.jqno.equalsverifier:equalsverifier from 3.19.4 to 4.0.1 [pinot]

2025-06-11 Thread via GitHub
dependabot[bot] commented on PR #16062: URL: https://github.com/apache/pinot/pull/16062#issuecomment-2964327406 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

(pinot) branch dependabot/maven/nl.jqno.equalsverifier-equalsverifier-4.0.1 deleted (was f5fa9e4045)

2025-06-11 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/nl.jqno.equalsverifier-equalsverifier-4.0.1 in repository https://gitbox.apache.org/repos/asf/pinot.git was f5fa9e4045 Bump nl.jqno.equalsverifier:equalsverifier from 3

Re: [PR] Bump nl.jqno.equalsverifier:equalsverifier from 3.19.4 to 4.0.1 [pinot]

2025-06-11 Thread via GitHub
Jackie-Jiang closed pull request #16062: Bump nl.jqno.equalsverifier:equalsverifier from 3.19.4 to 4.0.1 URL: https://github.com/apache/pinot/pull/16062 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

(pinot) branch dependabot/maven/org.webjars-swagger-ui-5.24.1 deleted (was f5ed115cff)

2025-06-11 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/org.webjars-swagger-ui-5.24.1 in repository https://gitbox.apache.org/repos/asf/pinot.git was f5ed115cff Bump org.webjars:swagger-ui from 5.24.0 to 5.24.1 The revisions tha

(pinot) branch master updated (69929e57fd -> b8900834e5)

2025-06-11 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 69929e57fd Bump com.google.cloud:libraries-bom from 26.61.0 to 26.62.0 (#16072) add b8900834e5 Bump org.webjars:s

Re: [PR] Bump org.webjars:swagger-ui from 5.24.0 to 5.24.1 [pinot]

2025-06-11 Thread via GitHub
Jackie-Jiang merged PR #16073: URL: https://github.com/apache/pinot/pull/16073 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch dependabot/maven/com.google.cloud-libraries-bom-26.62.0 deleted (was 630a657150)

2025-06-11 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/com.google.cloud-libraries-bom-26.62.0 in repository https://gitbox.apache.org/repos/asf/pinot.git was 630a657150 Bump com.google.cloud:libraries-bom from 26.61.0 to 26.62.

(pinot) branch master updated (ac32016b35 -> 69929e57fd)

2025-06-11 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from ac32016b35 Low-risk optProgram rule enhancements (#16035) add 69929e57fd Bump com.google.cloud:libraries-bom from

Re: [PR] Bump com.google.cloud:libraries-bom from 26.61.0 to 26.62.0 [pinot]

2025-06-11 Thread via GitHub
Jackie-Jiang merged PR #16072: URL: https://github.com/apache/pinot/pull/16072 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[PR] [Minor] Fix typo in table config validation [pinot]

2025-06-11 Thread via GitHub
Jackie-Jiang opened a new pull request, #16081: URL: https://github.com/apache/pinot/pull/16081 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

(pinot) branch master updated: Low-risk optProgram rule enhancements (#16035)

2025-06-11 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new ac32016b35 Low-risk optProgram rule enhancements

Re: [PR] Low-risk optProgram rule enhancements [pinot]

2025-06-11 Thread via GitHub
Jackie-Jiang merged PR #16035: URL: https://github.com/apache/pinot/pull/16035 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: [upsert] Ensure consistent creation time to prevent data inconsistency across replicas (#16034)

2025-06-11 Thread ankitsultana
This is an automated email from the ASF dual-hosted git repository. ankitsultana pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 12bf942c30 [upsert] Ensure consistent creat

Re: [PR] Add multistage thread limiting configs at the broker and server level [pinot]

2025-06-11 Thread via GitHub
Jackie-Jiang commented on code in PR #16080: URL: https://github.com/apache/pinot/pull/16080#discussion_r2140933749 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/MultiStageQueryThrottler.java: ## @@ -64,18 +66,19 @@ public class MultiStageQueryThrottler im

Re: [PR] [upsert] Ensure consistent creation time to prevent data inconsistency across replicas [pinot]

2025-06-11 Thread via GitHub
ankitsultana merged PR #16034: URL: https://github.com/apache/pinot/pull/16034 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] [Priority Replica Group Routing] Support orderedPreferredReplicas query option for customizable routing strategy [pinot]

2025-06-11 Thread via GitHub
wirybeaver commented on PR #15203: URL: https://github.com/apache/pinot/pull/15203#issuecomment-2963825346 @yashmayya Are you concerning the following scenario that a pinot-server instance's pool has different value for different tenant? ``` "pool": { "foo_REALTIME": "1",

[PR] Add multistage thread limiting configs at the broker and server level [pinot]

2025-06-11 Thread via GitHub
satwik-pachigolla opened a new pull request, #16080: URL: https://github.com/apache/pinot/pull/16080 ### Summary Making existing functionality to control multistage thread limits at the broker and server that is configurable with cluster configs now configurable with broker and server co

Re: [PR] [Priority Replica Group Routing] Support orderedPreferredReplicas query option for customizable routing strategy [pinot]

2025-06-11 Thread via GitHub
yashmayya commented on code in PR #15203: URL: https://github.com/apache/pinot/pull/15203#discussion_r214067 ## pinot-common/src/main/java/org/apache/pinot/common/metrics/BrokerMeter.java: ## @@ -50,6 +50,24 @@ public class BrokerMeter implements AbstractMetrics.Meter {

Re: [PR] [multistage] Enable Tests for New Optimizer + Bug Fixes [pinot]

2025-06-11 Thread via GitHub
ankitsultana commented on code in PR #15958: URL: https://github.com/apache/pinot/pull/15958#discussion_r2140654229 ## pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotRuleUtils.java: ## @@ -122,4 +135,129 @@ public static String extractFunctionName(RexC

[PR] Add validations for MV columns: reject MV primary-keys for upsert/dedup, BIG_DECIMAL, and JSON [pinot]

2025-06-11 Thread via GitHub
somandal opened a new pull request, #16079: URL: https://github.com/apache/pinot/pull/16079 Add validations for MV columns to reject the following unsupported configurations: - Primary key columns for upsert / dedup should not be of type MV - validation added to TableConfigUtils -

(pinot) branch master updated: Enable ZK-based progress tracking for SegmentRelocator rebalances (#16008)

2025-06-11 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 6639f89842 Enable ZK-based progress tracking f

Re: [PR] Add reset segment operation to UI [pinot]

2025-06-11 Thread via GitHub
Copilot commented on code in PR #16078: URL: https://github.com/apache/pinot/pull/16078#discussion_r2140628292 ## pinot-controller/src/main/resources/app/pages/SegmentDetails.tsx: ## @@ -325,6 +325,29 @@ const SegmentDetails = ({ match }: RouteComponentProps) => { setConfi

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.31.61 deleted (was a41d87a975)

2025-06-11 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.31.61 in repository https://gitbox.apache.org/repos/asf/pinot.git was a41d87a975 Bump software.amazon.awssdk:bom from 2.31.60 to 2.31.61 Th

[PR] Add reset segment operation to UI [pinot]

2025-06-11 Thread via GitHub
KKcorps opened a new pull request, #16078: URL: https://github.com/apache/pinot/pull/16078 This pull request introduces a new feature to reset a segment in the Pinot Controller UI. The changes include adding a "Reset Segment" button to the `SegmentDetails` page, implementing the correspondi

Re: [PR] [Priority Replica Group Routing] Support orderedPreferredReplicas query option for customizable routing strategy [pinot]

2025-06-11 Thread via GitHub
yashmayya commented on code in PR #15203: URL: https://github.com/apache/pinot/pull/15203#discussion_r2140631690 ## pinot-core/src/main/java/org/apache/pinot/core/transport/ServerInstance.java: ## @@ -173,4 +189,20 @@ public int hashCode() { public String toString() { re

Re: [I] Pinot and JDBC Connection [pinot]

2025-06-11 Thread via GitHub
ravi-Mode commented on issue #14743: URL: https://github.com/apache/pinot/issues/14743#issuecomment-2963503785 Pinot port 9000 is for controller and you can run query using it. To run the query you need to use broker port which is 8000. Also from the logs it confirms. Now to connect usi

Re: [PR] Enable ZK-based progress tracking for SegmentRelocator rebalances [pinot]

2025-06-11 Thread via GitHub
yashmayya merged PR #16008: URL: https://github.com/apache/pinot/pull/16008 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

Re: [PR] Enable ZK-based progress tracking for SegmentRelocator rebalances [pinot]

2025-06-11 Thread via GitHub
somandal commented on code in PR #16008: URL: https://github.com/apache/pinot/pull/16008#discussion_r2140607980 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceChecker.java: ## @@ -111,18 +111,35 @@ private synchronized int retryRebala

Re: [PR] Bump software.amazon.awssdk:bom from 2.31.60 to 2.31.61 [pinot]

2025-06-11 Thread via GitHub
yashmayya merged PR #16074: URL: https://github.com/apache/pinot/pull/16074 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

Re: [PR] [multistage] Enable Tests for New Optimizer + Bug Fixes [pinot]

2025-06-11 Thread via GitHub
ankitsultana commented on code in PR #15958: URL: https://github.com/apache/pinot/pull/15958#discussion_r2140651234 ## pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotRuleUtils.java: ## @@ -122,4 +135,129 @@ public static String extractFunctionName(RexC

Re: [PR] [multistage] Enable Tests for New Optimizer + Bug Fixes [pinot]

2025-06-11 Thread via GitHub
ankitsultana commented on code in PR #15958: URL: https://github.com/apache/pinot/pull/15958#discussion_r2140659671 ## pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotRuleUtils.java: ## @@ -122,4 +135,129 @@ public static String extractFunctionName(RexC

(pinot) branch master updated (c5f9e44527 -> b5b803e454)

2025-06-11 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from c5f9e44527 [multistage] Fix Handling for Reserved Words in LeafStageToPinotQuery (#16068) add b5b803e454 Bump

Re: [I] Use Correlation ID instead of query id in PerQueryCPUMemResourceUsageAccountant [pinot]

2025-06-11 Thread via GitHub
vrajat commented on issue #16015: URL: https://github.com/apache/pinot/issues/16015#issuecomment-2963484075 Yes. They will be treated as the same query. The degenerate case is a high qps workload with many concurrent queries and same correlation id. IIUC from the docs, the best pract

Re: [PR] Enable ZK-based progress tracking for SegmentRelocator rebalances [pinot]

2025-06-11 Thread via GitHub
yashmayya commented on code in PR #16008: URL: https://github.com/apache/pinot/pull/16008#discussion_r2140531538 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceChecker.java: ## @@ -111,18 +111,35 @@ private synchronized int retryRebal

Re: [PR] Enable ZK-based progress tracking for SegmentRelocator rebalances [pinot]

2025-06-11 Thread via GitHub
yashmayya commented on code in PR #16008: URL: https://github.com/apache/pinot/pull/16008#discussion_r2140531538 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceChecker.java: ## @@ -111,18 +111,35 @@ private synchronized int retryRebal

(pinot) branch master updated: [multistage] Fix Handling for Reserved Words in LeafStageToPinotQuery (#16068)

2025-06-11 Thread ankitsultana
This is an automated email from the ASF dual-hosted git repository. ankitsultana pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new c5f9e44527 [multistage] Fix Handling for Re

Re: [I] [Discussion] Event listener to gather query stats/information [pinot]

2025-06-11 Thread via GitHub
ankitsultana closed issue #7391: [Discussion] Event listener to gather query stats/information URL: https://github.com/apache/pinot/issues/7391 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [I] [Discussion] Event listener to gather query stats/information [pinot]

2025-06-11 Thread via GitHub
ankitsultana commented on issue #7391: URL: https://github.com/apache/pinot/issues/7391#issuecomment-2963305721 Just saw this today. This was closed by #10606. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] [multistage] Enable Tests for New Optimizer + Bug Fixes [pinot]

2025-06-11 Thread via GitHub
wirybeaver commented on code in PR #15958: URL: https://github.com/apache/pinot/pull/15958#discussion_r2140497888 ## pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotRuleUtils.java: ## @@ -122,4 +135,129 @@ public static String extractFunctionName(RexCal

Re: [PR] [multistage] Enable Tests for New Optimizer + Bug Fixes [pinot]

2025-06-11 Thread via GitHub
wirybeaver commented on code in PR #15958: URL: https://github.com/apache/pinot/pull/15958#discussion_r2140492307 ## pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotRuleUtils.java: ## @@ -122,4 +135,129 @@ public static String extractFunctionName(RexCal

Re: [PR] Enable ZK-based progress tracking for SegmentRelocator rebalances [pinot]

2025-06-11 Thread via GitHub
somandal commented on code in PR #16008: URL: https://github.com/apache/pinot/pull/16008#discussion_r2140473728 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceChecker.java: ## @@ -111,18 +111,35 @@ private synchronized int retryRebala

Re: [PR] [multistage] Fix Handling for Reserved Words in LeafStageToPinotQuery [pinot]

2025-06-11 Thread via GitHub
ankitsultana merged PR #16068: URL: https://github.com/apache/pinot/pull/16068 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Enable ZK-based progress tracking for SegmentRelocator rebalances [pinot]

2025-06-11 Thread via GitHub
yashmayya commented on PR #16008: URL: https://github.com/apache/pinot/pull/16008#issuecomment-2963054722 We decided to go ahead with retaining the allow / disallow retry mechanism from this PR (to avoid `SegmentRelocator` and `RebalanceChecker` stepping on each other's toes since they're b

Re: [PR] Adding changes for supporting RLS [pinot]

2025-06-11 Thread via GitHub
yashmayya commented on code in PR #16043: URL: https://github.com/apache/pinot/pull/16043#discussion_r2140198489 ## pinot-common/src/main/java/org/apache/pinot/sql/parsers/rewriter/RlsFiltersRewriter.java: ## @@ -0,0 +1,39 @@ +package org.apache.pinot.sql.parsers.rewriter; + +im

Re: [PR] [multistage] Enable Tests for New Optimizer + Bug Fixes [pinot]

2025-06-11 Thread via GitHub
wirybeaver commented on code in PR #15958: URL: https://github.com/apache/pinot/pull/15958#discussion_r2140170305 ## pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotRuleUtils.java: ## @@ -122,4 +135,129 @@ public static String extractFunctionName(RexCal

Re: [PR] Adding changes for supporting RLS [pinot]

2025-06-11 Thread via GitHub
yashmayya commented on code in PR #16043: URL: https://github.com/apache/pinot/pull/16043#discussion_r2140142627 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/plan/server/ServerPlanRequestUtils.java: ## @@ -114,6 +123,11 @@ public static OpChain compileLeaf

[PR] Add exception for resource limit exceeded in MultiStageOperator [pinot]

2025-06-11 Thread via GitHub
gortiz opened a new pull request, #16077: URL: https://github.com/apache/pinot/pull/16077 While working on https://github.com/apache/pinot/pull/16075, I've reused the `sampleAndCheckInterruption` method to also check for timeouts during execution. When I did that, I found it strange that th

Re: [PR] Improve broker error messaging when broker is the one reporting the failure [pinot]

2025-06-11 Thread via GitHub
Copilot commented on code in PR #16076: URL: https://github.com/apache/pinot/pull/16076#discussion_r2140098486 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/service/dispatch/QueryDispatcher.java: ## @@ -644,21 +666,21 @@ public static QueryResult runReducer(long re

[PR] Improve broker error messaging when broker is the one reporting the failure [pinot]

2025-06-11 Thread via GitHub
gortiz opened a new pull request, #16076: URL: https://github.com/apache/pinot/pull/16076 This is a simple PR that modifies the way logs are reported when the broker is the one that reports it. In previous code, given we didn't initialized the worker thread context, we didn't print the name

Re: [PR] Introduce MSE active and passive timeouts [pinot]

2025-06-11 Thread via GitHub
Copilot commented on code in PR #16075: URL: https://github.com/apache/pinot/pull/16075#discussion_r2140095695 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MultiStageOperator.java: ## @@ -72,9 +73,29 @@ public MultiStageOperator(OpChainExecutionCo

[PR] Introduce MSE active and passive timeouts [pinot]

2025-06-11 Thread via GitHub
gortiz opened a new pull request, #16075: URL: https://github.com/apache/pinot/pull/16075 Given that all workers were using the same deadtime, any worker could fail. Due to the transmission relay, workers running stages closer to the root have a higher chance of producing the message that u

Re: [PR] Make OOM Protection work with GRPC Queries [pinot]

2025-06-11 Thread via GitHub
gortiz merged PR #16004: URL: https://github.com/apache/pinot/pull/16004 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

Re: [PR] Use correlation ID instead of request id in PerQueryCpuMemAccountant [pinot]

2025-06-11 Thread via GitHub
Copilot commented on code in PR #16040: URL: https://github.com/apache/pinot/pull/16040#discussion_r2139908174 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/logicaltable/LogicalTableWithOneOfflineOneRealtimeTableIntegrationTest.java: ## @@ -41,4 +43

Re: [I] Use Correlation ID instead of query id in PerQueryCPUMemResourceUsageAccountant [pinot]

2025-06-11 Thread via GitHub
gortiz commented on issue #16015: URL: https://github.com/apache/pinot/issues/16015#issuecomment-2962343861 Remember that correlation id can be set by customers and therefore we cannot assume correlation ids are unique. Therefore, we need to think what would happen if two queries share the

[PR] Bump software.amazon.awssdk:bom from 2.31.60 to 2.31.61 [pinot]

2025-06-11 Thread via GitHub
dependabot[bot] opened a new pull request, #16074: URL: https://github.com/apache/pinot/pull/16074 Bumps software.amazon.awssdk:bom from 2.31.60 to 2.31.61. Most Recent Ignore Conditions Applied to This Pull Request | Dependency Name | Ignore Conditions | | --- | --- |

(pinot) branch master updated: Make OOM Protection work with GRPC Queries (#16004)

2025-06-11 Thread gortiz
This is an automated email from the ASF dual-hosted git repository. gortiz pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new c8eb53a4d3 Make OOM Protection work with GRPC Que

Re: [I] GRPC Query Tests fail when OOM Protection is enabled [pinot]

2025-06-11 Thread via GitHub
gortiz closed issue #16041: GRPC Query Tests fail when OOM Protection is enabled URL: https://github.com/apache/pinot/issues/16041 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [I] Fix OOM Cancellation in MSE [pinot]

2025-06-11 Thread via GitHub
vrajat commented on issue #16060: URL: https://github.com/apache/pinot/issues/16060#issuecomment-2962302045 Is it OK if that thread is interrupted ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] Make OOM Protection work with GRPC Queries [pinot]

2025-06-11 Thread via GitHub
vrajat commented on code in PR #16004: URL: https://github.com/apache/pinot/pull/16004#discussion_r2139885317 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineGRPCServerMultiStageOOMAccountingIntegrationTest.java: ## @@ -0,0 +1,30 @@ +/** + * Li

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.31.61 created (now a41d87a975)

2025-06-11 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.31.61 in repository https://gitbox.apache.org/repos/asf/pinot.git at a41d87a975 Bump software.amazon.awssdk:bom from 2.31.60 to 2.31.61 N

(pinot) branch dependabot/maven/org.webjars-swagger-ui-5.24.1 created (now f5ed115cff)

2025-06-11 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.webjars-swagger-ui-5.24.1 in repository https://gitbox.apache.org/repos/asf/pinot.git at f5ed115cff Bump org.webjars:swagger-ui from 5.24.0 to 5.24.1 No new revis

[PR] Bump org.webjars:swagger-ui from 5.24.0 to 5.24.1 [pinot]

2025-06-11 Thread via GitHub
dependabot[bot] opened a new pull request, #16073: URL: https://github.com/apache/pinot/pull/16073 Bumps [org.webjars:swagger-ui](https://github.com/swagger-api/swagger-ui) from 5.24.0 to 5.24.1. Release notes Sourced from https://github.com/swagger-api/swagger-ui/releases";>org.we

(pinot) branch dependabot/maven/com.google.cloud-libraries-bom-26.62.0 created (now 630a657150)

2025-06-11 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.google.cloud-libraries-bom-26.62.0 in repository https://gitbox.apache.org/repos/asf/pinot.git at 630a657150 Bump com.google.cloud:libraries-bom from 26.61.0 to 26

[PR] Bump com.google.cloud:libraries-bom from 26.61.0 to 26.62.0 [pinot]

2025-06-11 Thread via GitHub
dependabot[bot] opened a new pull request, #16072: URL: https://github.com/apache/pinot/pull/16072 Bumps [com.google.cloud:libraries-bom](https://github.com/googleapis/java-cloud-bom) from 26.61.0 to 26.62.0. Release notes Sourced from https://github.com/googleapis/java-cloud-bom/

Re: [PR] Return 404 for non-existent GET table requests [pinot]

2025-06-11 Thread via GitHub
yashmayya commented on code in PR #16070: URL: https://github.com/apache/pinot/pull/16070#discussion_r2139658718 ## pinot-controller/src/test/java/org/apache/pinot/controller/api/PinotTableRestletResourceTest.java: ## @@ -960,6 +960,32 @@ public void testTableWithSameNameAsLogi

Re: [PR] MSE error management improvements [pinot]

2025-06-11 Thread via GitHub
gortiz commented on code in PR #15919: URL: https://github.com/apache/pinot/pull/15919#discussion_r2137677116 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/ZeroCopyDataBlockSerde.java: ## @@ -254,21 +265,37 @@ private long calculateEndOffset(DataBuffer buffer,

Re: [PR] Add config to use LegacyMd5Plugin in S3 client, which restores the pre-2.30.0 MD5 checksum behavior [pinot]

2025-06-11 Thread via GitHub
xiangfu0 merged PR #16065: URL: https://github.com/apache/pinot/pull/16065 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

(pinot) branch master updated (423f4e0e0a -> b440fa7488)

2025-06-11 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 423f4e0e0a MSE error management improvements (#15919) add b440fa7488 Add config to use LegacyMd5Plugin in S3 cli

Re: [PR] Database name validation for logical tables [pinot]

2025-06-11 Thread via GitHub
abhishekbafna commented on code in PR #15994: URL: https://github.com/apache/pinot/pull/15994#discussion_r2139756494 ## pinot-common/src/main/java/org/apache/pinot/common/utils/LogicalTableConfigUtils.java: ## @@ -139,13 +139,22 @@ public static void validateLogicalTableConfig(

(pinot) branch master updated (e18845e4b9 -> 423f4e0e0a)

2025-06-11 Thread gortiz
This is an automated email from the ASF dual-hosted git repository. gortiz pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from e18845e4b9 Bump software.amazon.awssdk:bom from 2.31.59 to 2.31.60 (#16063) add 423f4e0e0a MSE error management i

Re: [PR] MSE error management improvements [pinot]

2025-06-11 Thread via GitHub
gortiz merged PR #15919: URL: https://github.com/apache/pinot/pull/15919 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

[PR] Adds BEST_EFFORT disaster recovery mode in dedup [pinot]

2025-06-11 Thread via GitHub
noob-se7en opened a new pull request, #16071: URL: https://github.com/apache/pinot/pull/16071 Instructions: 1. The PR has to be tagged with at least one of the following labels (*): 1. `feature` 2. `bugfix` 3. `performance` 4. `ui` 5. `backward-incompat`

Re: [PR] Adds BEST_EFFORT disaster recovery mode in dedup [pinot]

2025-06-11 Thread via GitHub
codecov-commenter commented on PR #16071: URL: https://github.com/apache/pinot/pull/16071#issuecomment-2961835785 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/16071?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Return 404 for non-existent GET table requests [pinot]

2025-06-11 Thread via GitHub
codecov-commenter commented on PR #16070: URL: https://github.com/apache/pinot/pull/16070#issuecomment-2961683056 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/16070?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Return 404 for non-existent GET table requests [pinot]

2025-06-11 Thread via GitHub
abhishekbafna opened a new pull request, #16070: URL: https://github.com/apache/pinot/pull/16070 Currently, GET table config requests return 200 status and empty response non existent tables. The PR adds a check and return 404 status code. Closes: #16069 Realtime Table:

  1   2   >