Re: [I] Group By Truncates at Limit in Combine Operator [pinot]

2025-06-03 Thread via GitHub
anandkrshaw commented on issue #14956: URL: https://github.com/apache/pinot/issues/14956#issuecomment-2938840041 Agreed—at present, this behavior is safely gated behind a query option, so there’s no immediate impact on existing workloads. Regarding performance, the overhead introduced here

[I] Add debug server API for logical table metadata cache [pinot]

2025-06-03 Thread via GitHub
abhishekbafna opened a new issue, #15989: URL: https://github.com/apache/pinot/issues/15989 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] Cache configs for logical table context in server [pinot]

2025-06-03 Thread via GitHub
abhishekbafna commented on PR #15881: URL: https://github.com/apache/pinot/pull/15881#issuecomment-2938681617 > > > How do we test or debug this production ? > > > > > > > > * For the cache misses a warn message is logged. > > * Should we build REST api(s) for getting the ca

[I] Same database validation for logical table and reference physical tables [pinot]

2025-06-03 Thread via GitHub
abhishekbafna opened a new issue, #15988: URL: https://github.com/apache/pinot/issues/15988 The logical table and all the referenced physical table should be part of the same database. Pinot does not support query across the databases. -- This is an automated message from the Apache Git S

Re: [PR] Cache configs for logical table context in server [pinot]

2025-06-03 Thread via GitHub
vrajat commented on PR #15881: URL: https://github.com/apache/pinot/pull/15881#issuecomment-2938613257 > > How do we test or debug this production ? > > * For the cache misses a warn message is logged. > * Should we build REST api(s) for getting the cache state +1 to these tw

Re: [PR] Allow RealtimeSegmentValidationManager to fix error segments for partial upsert and dedup tables. [pinot]

2025-06-03 Thread via GitHub
codecov-commenter commented on PR #15987: URL: https://github.com/apache/pinot/pull/15987#issuecomment-2938498900 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15987?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] [multistage] Add Support for Broker Server/Segment Pruning [pinot]

2025-06-03 Thread via GitHub
ankitsultana commented on PR #15959: URL: https://github.com/apache/pinot/pull/15959#issuecomment-2938471919 > This is very useful for MSE. Is it possible to extract the code into a common util class? Maybe move the code from SSE to a util? Moved the logic to a util class. -- This

Re: [I] Use body for /zk/put PUT request API in ZK browser UI [pinot]

2025-06-03 Thread via GitHub
frankchih commented on issue #15311: URL: https://github.com/apache/pinot/issues/15311#issuecomment-2938435786 Hi @ishaan1091 and @xiangfu0 , Just checking in, is anyone currently working on this issue? If not, I’d be happy to take it on. I have some availability over the next few

[PR] Allow RealtimeSegmentValidationManager to fix error segments for partial upsert and dedup tables. [pinot]

2025-06-03 Thread via GitHub
9aman opened a new pull request, #15987: URL: https://github.com/apache/pinot/pull/15987 ## Scope of the PR The RealtimeSegmentValidationManager can now take parameters for it's manual runs: https://github.com/apache/pinot/pull/14915 The PR aims to introduce the ability to

Re: [PR] Support for Virtual DataSource [pinot]

2025-06-03 Thread via GitHub
praveenc7 commented on code in PR #15350: URL: https://github.com/apache/pinot/pull/15350#discussion_r2125512834 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/IndexSegment.java: ## @@ -76,10 +76,9 @@ default DataSource getDataSource(String column) { /// Retu

Re: [PR] Support for Virtual DataSource [pinot]

2025-06-03 Thread via GitHub
praveenc7 commented on code in PR #15350: URL: https://github.com/apache/pinot/pull/15350#discussion_r2125267956 ## pinot-core/src/main/java/org/apache/pinot/core/query/pruner/SegmentPrunerService.java: ## @@ -105,7 +106,8 @@ public List prune(List segments, QueryContext query,

Re: [PR] Support for Virtual DataSource [pinot]

2025-06-03 Thread via GitHub
praveenc7 commented on code in PR #15350: URL: https://github.com/apache/pinot/pull/15350#discussion_r2125267956 ## pinot-core/src/main/java/org/apache/pinot/core/query/pruner/SegmentPrunerService.java: ## @@ -105,7 +106,8 @@ public List prune(List segments, QueryContext query,

Re: [PR] Support for Virtual DataSource [pinot]

2025-06-03 Thread via GitHub
praveenc7 commented on code in PR #15350: URL: https://github.com/apache/pinot/pull/15350#discussion_r2125267956 ## pinot-core/src/main/java/org/apache/pinot/core/query/pruner/SegmentPrunerService.java: ## @@ -105,7 +106,8 @@ public List prune(List segments, QueryContext query,

Re: [PR] Support for Virtual DataSource [pinot]

2025-06-03 Thread via GitHub
praveenc7 commented on code in PR #15350: URL: https://github.com/apache/pinot/pull/15350#discussion_r2125267956 ## pinot-core/src/main/java/org/apache/pinot/core/query/pruner/SegmentPrunerService.java: ## @@ -105,7 +106,8 @@ public List prune(List segments, QueryContext query,

Re: [PR] Support for Virtual DataSource [pinot]

2025-06-03 Thread via GitHub
praveenc7 commented on code in PR #15350: URL: https://github.com/apache/pinot/pull/15350#discussion_r2125284372 ## pinot-core/src/main/java/org/apache/pinot/core/query/pruner/BloomFilterSegmentPruner.java: ## @@ -189,26 +190,26 @@ private List prefetch(List segments, QueryCont

Re: [I] Rebalance table status not reporting correctly in Pinot controller UI [pinot]

2025-06-03 Thread via GitHub
himanish-star commented on issue #15939: URL: https://github.com/apache/pinot/issues/15939#issuecomment-2938198193 Will take a look! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Support for Virtual DataSource [pinot]

2025-06-03 Thread via GitHub
praveenc7 commented on code in PR #15350: URL: https://github.com/apache/pinot/pull/15350#discussion_r2125284372 ## pinot-core/src/main/java/org/apache/pinot/core/query/pruner/BloomFilterSegmentPruner.java: ## @@ -189,26 +190,26 @@ private List prefetch(List segments, QueryCont

Re: [PR] Support for Virtual DataSource [pinot]

2025-06-03 Thread via GitHub
praveenc7 commented on code in PR #15350: URL: https://github.com/apache/pinot/pull/15350#discussion_r2125267956 ## pinot-core/src/main/java/org/apache/pinot/core/query/pruner/SegmentPrunerService.java: ## @@ -105,7 +106,8 @@ public List prune(List segments, QueryContext query,

Re: [PR] Honor 'numReplicaGroupsToQuery' when 'useFixedReplica' is set [pinot]

2025-06-03 Thread via GitHub
Jackie-Jiang merged PR #15986: URL: https://github.com/apache/pinot/pull/15986 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: Honor 'numReplicaGroupsToQuery' when 'useFixedReplica' is set (#15986)

2025-06-03 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new c15440466a Honor 'numReplicaGroupsToQuery' when '

Re: [PR] Support for Virtual DataSource [pinot]

2025-06-03 Thread via GitHub
Jackie-Jiang commented on code in PR #15350: URL: https://github.com/apache/pinot/pull/15350#discussion_r2125209005 ## pinot-core/src/main/java/org/apache/pinot/core/query/pruner/BloomFilterSegmentPruner.java: ## @@ -189,26 +190,26 @@ private List prefetch(List segments, QueryC

Re: [PR] Honor 'numReplicaGroupsToQuery' when 'useFixedReplica' is set [pinot]

2025-06-03 Thread via GitHub
codecov-commenter commented on PR #15986: URL: https://github.com/apache/pinot/pull/15986#issuecomment-2937913465 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15986?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Move the SPI check to GHA and deprecate baseline jar [pinot]

2025-06-03 Thread via GitHub
codecov-commenter commented on PR #15982: URL: https://github.com/apache/pinot/pull/15982#issuecomment-2937872594 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15982?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Check `batchSizePerServer` in Rebalance Config Pre-check [pinot]

2025-06-03 Thread via GitHub
somandal merged PR #15984: URL: https://github.com/apache/pinot/pull/15984 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

(pinot) branch master updated: Check `batchSizePerServer` in Rebalance Config Pre-check (#15984)

2025-06-03 Thread somandal
This is an automated email from the ASF dual-hosted git repository. somandal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 6dccd39bc4 Check `batchSizePerServer` in Rebala

[PR] Honor 'numReplicaGroupsToQuery' when 'useFixedReplica' is set [pinot]

2025-06-03 Thread via GitHub
Jackie-Jiang opened a new pull request, #15986: URL: https://github.com/apache/pinot/pull/15986 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Check `batchSizePerServer` in Rebalance Config Pre-check [pinot]

2025-06-03 Thread via GitHub
codecov-commenter commented on PR #15984: URL: https://github.com/apache/pinot/pull/15984#issuecomment-2937806202 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15984?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Move the SPI check to GHA and deprecate baseline jar [pinot]

2025-06-03 Thread via GitHub
UOETianleZhang commented on code in PR #15982: URL: https://github.com/apache/pinot/pull/15982#discussion_r2125141175 ## .github/workflows/pinot_tests.yml: ## @@ -98,8 +101,34 @@ jobs: key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys:

Re: [PR] Move the SPI check to GHA and deprecate baseline jar [pinot]

2025-06-03 Thread via GitHub
UOETianleZhang commented on code in PR #15982: URL: https://github.com/apache/pinot/pull/15982#discussion_r2125141175 ## .github/workflows/pinot_tests.yml: ## @@ -98,8 +101,34 @@ jobs: key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys:

Re: [PR] MQE rule enhancements [pinot]

2025-06-03 Thread via GitHub
songwdfu closed pull request #15983: MQE rule enhancements URL: https://github.com/apache/pinot/pull/15983 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

[PR] MQE rule enhancements [pinot]

2025-06-03 Thread via GitHub
songwdfu opened a new pull request, #15983: URL: https://github.com/apache/pinot/pull/15983 This PR involves several additions / changes to MQE's pinot-query-planner: 1. Changes to Rules. These changes to rules are tested with customized queries on TPC-H, which show major improve

(pinot) branch master updated: Update batchSizePerServer description for rebalance UI to provide recommendations on how to use it (#15981)

2025-06-03 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new d4b0903841 Update batchSizePerServer description

Re: [I] [multistage] Add Support for Multi-Group Queries for Window Functions [pinot]

2025-06-03 Thread via GitHub
wirybeaver commented on issue #15985: URL: https://github.com/apache/pinot/issues/15985#issuecomment-2937728410 @ankitsultana Hi Ankit, can I work on this task as a ramp up? I have deeply reviewed several MSE optimizer PRs. -- This is an automated message from the Apache Git Service. To r

Re: [PR] Update batchSizePerServer description for rebalance UI to provide recommendations on how to use it [pinot]

2025-06-03 Thread via GitHub
Jackie-Jiang merged PR #15981: URL: https://github.com/apache/pinot/pull/15981 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[I] [multistage] Add Support for Multi-Group Queries for Window Functions [pinot]

2025-06-03 Thread via GitHub
ankitsultana opened a new issue, #15985: URL: https://github.com/apache/pinot/issues/15985 Queries with multiple partition-by clauses are not supported yet. Example: ``` SELECT ROW_NUMBER() OVER ( PARTITION BY uuid, path, ORDER BY i

Re: [PR] Move the SPI check to GHA and deprecate baseline jar [pinot]

2025-06-03 Thread via GitHub
UOETianleZhang commented on PR #15982: URL: https://github.com/apache/pinot/pull/15982#issuecomment-2937715628 Thanks for making the change! Could you add your test result of the forked repo into the PR description? -- This is an automated message from the Apache Git Service. To respond t

[PR] Check `batchSizePerServer` in Rebalance Config Pre-check [pinot]

2025-06-03 Thread via GitHub
J-HowHuang opened a new pull request, #15984: URL: https://github.com/apache/pinot/pull/15984 ## Description For the newly added rebalance config `batchSizePerServer` (https://github.com/apache/pinot/pull/15617), this PR adds a new logic in rebalance config pre-check in table rebalan

(pinot) branch master updated: Pinot Server : Refresh message for Table Schema (#15956)

2025-06-03 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 43ee0f3ada Pinot Server : Refresh message for Tab

Re: [PR] Pinot Server : Refresh message for Table Schema [pinot]

2025-06-03 Thread via GitHub
Jackie-Jiang merged PR #15956: URL: https://github.com/apache/pinot/pull/15956 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Pinot Server : Refresh message for Table Schema [pinot]

2025-06-03 Thread via GitHub
praveenc7 commented on code in PR #15956: URL: https://github.com/apache/pinot/pull/15956#discussion_r2125054291 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java: ## @@ -1585,13 +1586,28 @@ public void updateSchema(Schema sc

Re: [PR] Update batchSizePerServer description for rebalance UI to provide recommendations on how to use it [pinot]

2025-06-03 Thread via GitHub
codecov-commenter commented on PR #15981: URL: https://github.com/apache/pinot/pull/15981#issuecomment-2937480854 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15981?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Move the SPI check to GHA and deprecate baseline jar [pinot]

2025-06-03 Thread via GitHub
leujean02 opened a new pull request, #15982: URL: https://github.com/apache/pinot/pull/15982 [PINOT-20995](https://linkedin.atlassian.net/browse/PINOT-20995) Previously, we used Maven japicmp plugin and a checked-in baseline JAR to enforce SPI compatibility. This should fail builds wh

Re: [PR] [multistage] Add Support for Broker Server/Segment Pruning [pinot]

2025-06-03 Thread via GitHub
ankitsultana commented on code in PR #15959: URL: https://github.com/apache/pinot/pull/15959#discussion_r2125031108 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java: ## @@ -634,6 +634,11 @@ public static class QueryOptionKey { // name, when tha

Re: [PR] Pinot Server : Refresh message for Table Schema [pinot]

2025-06-03 Thread via GitHub
Jackie-Jiang commented on code in PR #15956: URL: https://github.com/apache/pinot/pull/15956#discussion_r2125031019 ## pinot-common/src/main/java/org/apache/pinot/common/messages/TableConfigSchemaRefreshMessage.java: ## @@ -0,0 +1,54 @@ +/** + * Licensed to the Apache Software F

Re: [PR] [multistage] Add Support for Broker Server/Segment Pruning [pinot]

2025-06-03 Thread via GitHub
ankitsultana commented on code in PR #15959: URL: https://github.com/apache/pinot/pull/15959#discussion_r2125031108 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java: ## @@ -634,6 +634,11 @@ public static class QueryOptionKey { // name, when tha

Re: [PR] Modify OFFLINE and REALTIME completed segments to calculate default partitionId based on same logic as consuming segments rather than 0 [pinot]

2025-06-03 Thread via GitHub
somandal commented on PR #15817: URL: https://github.com/apache/pinot/pull/15817#issuecomment-2937425229 > Let's merge if all validation passes Merging, i see simplified and consistent behavior in terms of partitionId calculation based on the rebalance tests I ran, and no errors in fe

(pinot) branch master updated (197362465e -> 0710db86f8)

2025-06-03 Thread somandal
This is an automated email from the ASF dual-hosted git repository. somandal pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 197362465e Bump org.checkerframework:checker-qual from 3.49.3 to 3.49.4 (#15979) add 0710db86f8 Modify OFFLINE

Re: [PR] Modify OFFLINE and REALTIME completed segments to calculate default partitionId based on same logic as consuming segments rather than 0 [pinot]

2025-06-03 Thread via GitHub
somandal merged PR #15817: URL: https://github.com/apache/pinot/pull/15817 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] [multistage] Add Support for Broker Server/Segment Pruning [pinot]

2025-06-03 Thread via GitHub
Jackie-Jiang commented on code in PR #15959: URL: https://github.com/apache/pinot/pull/15959#discussion_r2125022096 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java: ## @@ -634,6 +634,11 @@ public static class QueryOptionKey { // name, when tha

[PR] Update batchSizePerServer description for rebalance UI to provide recommendations on how to use it [pinot]

2025-06-03 Thread via GitHub
somandal opened a new pull request, #15981: URL: https://github.com/apache/pinot/pull/15981 Update batchSizePerServer description for rebalance UI to provide recommendations on how to use it -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] [WIP]Offset auto reset [pinot]

2025-06-03 Thread via GitHub
lnbest0707-uber commented on PR #15782: URL: https://github.com/apache/pinot/pull/15782#issuecomment-2937249732 > @lnbest0707-uber can you please add description here. This is because I am also planning to work on offset reset problem. Not sure if you are also trying to achieve the same thi

Re: [I] Group By Truncates at Limit in Combine Operator [pinot]

2025-06-03 Thread via GitHub
ankitsultana commented on issue #14956: URL: https://github.com/apache/pinot/issues/14956#issuecomment-2936209819 The feature that you added is hidden behind a query option right? Enabling that query option at scale is going to be a challenge for us, and I understand we can't enable it by d

(pinot) branch master updated: Bump org.checkerframework:checker-qual from 3.49.3 to 3.49.4 (#15979)

2025-06-03 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 197362465e Bump org.checkerframework:checker-qua

(pinot) branch dependabot/maven/org.checkerframework-checker-qual-3.49.4 deleted (was e6139f4635)

2025-06-03 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch dependabot/maven/org.checkerframework-checker-qual-3.49.4 in repository https://gitbox.apache.org/repos/asf/pinot.git was e6139f4635 Bump org.checkerframework:checker-qual from 3.49.3 to 3

Re: [PR] Bump org.checkerframework:checker-qual from 3.49.3 to 3.49.4 [pinot]

2025-06-03 Thread via GitHub
xiangfu0 merged PR #15979: URL: https://github.com/apache/pinot/pull/15979 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] Bump software.amazon.awssdk:bom from 2.31.54 to 2.31.55 [pinot]

2025-06-03 Thread via GitHub
xiangfu0 merged PR #15980: URL: https://github.com/apache/pinot/pull/15980 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.31.55 deleted (was ceed923412)

2025-06-03 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.31.55 in repository https://gitbox.apache.org/repos/asf/pinot.git was ceed923412 Bump software.amazon.awssdk:bom from 2.31.54 to 2.31.55 The

Re: [PR] Cache configs for logical table context in server [pinot]

2025-06-03 Thread via GitHub
abhishekbafna commented on PR #15881: URL: https://github.com/apache/pinot/pull/15881#issuecomment-2936496783 > How do we test or debug this production ? - Added logs when cache is updated for the resources. - For the cache misses a warn message is logged. - Should we build REST

Re: [PR] [multistage] Add Support for Broker Server/Segment Pruning [pinot]

2025-06-03 Thread via GitHub
ankitsultana commented on code in PR #15959: URL: https://github.com/apache/pinot/pull/15959#discussion_r2124395131 ## pinot-query-planner/src/main/java/org/apache/pinot/query/planner/physical/v2/opt/rules/LeafStageWorkerAssignmentRule.java: ## @@ -400,6 +413,70 @@ static List s

Re: [PR] Tenant Rebalance API Improvement [pinot]

2025-06-03 Thread via GitHub
J-HowHuang commented on code in PR #15891: URL: https://github.com/apache/pinot/pull/15891#discussion_r2124365913 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/tenant/TenantRebalancer.java: ## @@ -18,6 +18,10 @@ */ package org.apache.pinot

Re: [PR] Tenant Rebalance API Improvement [pinot]

2025-06-03 Thread via GitHub
somandal commented on code in PR #15891: URL: https://github.com/apache/pinot/pull/15891#discussion_r2124115039 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/tenant/TenantRebalanceConfig.java: ## @@ -63,6 +79,7 @@ public void setParallelWhite

Re: [PR] Cache configs for logical table context in server [pinot]

2025-06-03 Thread via GitHub
vrajat commented on code in PR #15881: URL: https://github.com/apache/pinot/pull/15881#discussion_r2124004893 ## pinot-server/src/main/java/org/apache/pinot/server/starter/helix/LogicalTableMetadataCache.java: ## @@ -0,0 +1,285 @@ +/** + * Licensed to the Apache Software Foundat

Re: [PR] Schema and table config deletion to validate with logical table ref table names [pinot]

2025-06-03 Thread via GitHub
yashmayya merged PR #15900: URL: https://github.com/apache/pinot/pull/15900 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

(pinot) branch master updated: Schema and table config deletion to validate with logical table ref table names (#15900)

2025-06-03 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new c7ce654d81 Schema and table config deletion to

Re: [I] Schema and table config deletion should check logical table config references [pinot]

2025-06-03 Thread via GitHub
yashmayya closed issue #15882: Schema and table config deletion should check logical table config references URL: https://github.com/apache/pinot/issues/15882 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[PR] Create and remove partitionMetadataProvider atomically [pinot]

2025-06-03 Thread via GitHub
KKcorps opened a new pull request, #15978: URL: https://github.com/apache/pinot/pull/15978 This pull request refactors the handling of the `StreamMetadataProvider` in the `RealtimeSegmentDataManager` class to improve thread safety, reduce redundant object creation, and enhance error handlin

Re: [PR] Reduce ZK access for SendStatsPredicate [pinot]

2025-06-03 Thread via GitHub
gortiz commented on PR #15895: URL: https://github.com/apache/pinot/pull/15895#issuecomment-2934712451 If this is a single unsafe/safe crossover, then let us introduce a config that users can roll once they are in safe version. We can even provide a controller API if needed to assure that t

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.31.55 created (now ceed923412)

2025-06-03 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.31.55 in repository https://gitbox.apache.org/repos/asf/pinot.git at ceed923412 Bump software.amazon.awssdk:bom from 2.31.54 to 2.31.55 N

[PR] Bump software.amazon.awssdk:bom from 2.31.54 to 2.31.55 [pinot]

2025-06-03 Thread via GitHub
dependabot[bot] opened a new pull request, #15980: URL: https://github.com/apache/pinot/pull/15980 Bumps software.amazon.awssdk:bom from 2.31.54 to 2.31.55. Most Recent Ignore Conditions Applied to This Pull Request | Dependency Name | Ignore Conditions | | --- | --- |

(pinot) branch dependabot/maven/org.checkerframework-checker-qual-3.49.4 created (now e6139f4635)

2025-06-03 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.checkerframework-checker-qual-3.49.4 in repository https://gitbox.apache.org/repos/asf/pinot.git at e6139f4635 Bump org.checkerframework:checker-qual from 3.49.3 t

[PR] Bump org.checkerframework:checker-qual from 3.49.3 to 3.49.4 [pinot]

2025-06-03 Thread via GitHub
dependabot[bot] opened a new pull request, #15979: URL: https://github.com/apache/pinot/pull/15979 Bumps [org.checkerframework:checker-qual](https://github.com/typetools/checker-framework) from 3.49.3 to 3.49.4. Release notes Sourced from https://github.com/typetools/checker-frame

Re: [PR] Create and remove partitionMetadataProvider atomically [pinot]

2025-06-03 Thread via GitHub
codecov-commenter commented on PR #15978: URL: https://github.com/apache/pinot/pull/15978#issuecomment-2934707007 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15978?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

(pinot) branch master updated (85236c4393 -> 608f89134e)

2025-06-03 Thread gortiz
This is an automated email from the ASF dual-hosted git repository. gortiz pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 85236c4393 Invalidate from cache the stats of opchains that finish successfully. (#15967) add 608f89134e Apply so

Re: [PR] Apply some o11y: [pinot]

2025-06-03 Thread via GitHub
gortiz merged PR #15975: URL: https://github.com/apache/pinot/pull/15975 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

Re: [PR] Tenant Rebalance API Improvement [pinot]

2025-06-03 Thread via GitHub
shounakmk219 commented on code in PR #15891: URL: https://github.com/apache/pinot/pull/15891#discussion_r2123399794 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/tenant/TenantRebalancer.java: ## @@ -18,6 +18,10 @@ */ package org.apache.pin

Re: [PR] Tenant Rebalance API Improvement [pinot]

2025-06-03 Thread via GitHub
shounakmk219 commented on code in PR #15891: URL: https://github.com/apache/pinot/pull/15891#discussion_r2123399794 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/tenant/TenantRebalancer.java: ## @@ -18,6 +18,10 @@ */ package org.apache.pin

Re: [PR] Add new ImplicitRealtimeTablePartitionSelector strategy for instance assignment [pinot]

2025-06-03 Thread via GitHub
yashmayya commented on code in PR #15930: URL: https://github.com/apache/pinot/pull/15930#discussion_r2123362083 ## pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancerClusterStatelessTest.java: ## @@ -764,6 +768,195 @@ public void test

Re: [PR] Tenant Rebalance API Improvement [pinot]

2025-06-03 Thread via GitHub
shounakmk219 commented on code in PR #15891: URL: https://github.com/apache/pinot/pull/15891#discussion_r2123365388 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/tenant/TenantRebalanceConfig.java: ## @@ -63,6 +79,7 @@ public void setParallelW

Re: [PR] Schema and table config deletion to validate with logical table ref table names [pinot]

2025-06-03 Thread via GitHub
abhishekbafna commented on code in PR #15900: URL: https://github.com/apache/pinot/pull/15900#discussion_r2123212889 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java: ## @@ -466,6 +470,23 @@ private boolean verifyTableTyp

Re: [PR] Apply some o11y: [pinot]

2025-06-03 Thread via GitHub
codecov-commenter commented on PR #15975: URL: https://github.com/apache/pinot/pull/15975#issuecomment-2934267837 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15975?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Modify MSE operators to release memory once computation finished [pinot]

2025-06-03 Thread via GitHub
codecov-commenter commented on PR #15977: URL: https://github.com/apache/pinot/pull/15977#issuecomment-2934164712 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15977?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Use atomic reference for partition metadata provider [pinot]

2025-06-03 Thread via GitHub
codecov-commenter commented on PR #15976: URL: https://github.com/apache/pinot/pull/15976#issuecomment-2934102817 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15976?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Make offset lag metric periodic [pinot]

2025-06-03 Thread via GitHub
KKcorps commented on code in PR #15831: URL: https://github.com/apache/pinot/pull/15831#discussion_r2123098296 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/IngestionDelayTracker.java: ## @@ -416,16 +502,74 @@ public long getPartitionIngestionOffsetLag(

Re: [PR] Use atomic reference for partition metadata provider [pinot]

2025-06-03 Thread via GitHub
KKcorps closed pull request #15976: Use atomic reference for partition metadata provider URL: https://github.com/apache/pinot/pull/15976 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Schema and table config deletion to validate with logical table ref table names [pinot]

2025-06-03 Thread via GitHub
vrajat commented on code in PR #15900: URL: https://github.com/apache/pinot/pull/15900#discussion_r2122874300 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java: ## @@ -466,6 +470,23 @@ private boolean verifyTableType(Strin

Re: [PR] Invalidate from cache the stats of opchains that finish successfully. [pinot]

2025-06-03 Thread via GitHub
gortiz merged PR #15967: URL: https://github.com/apache/pinot/pull/15967 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

(pinot) branch master updated: Invalidate from cache the stats of opchains that finish successfully. (#15967)

2025-06-03 Thread gortiz
This is an automated email from the ASF dual-hosted git repository. gortiz pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 85236c4393 Invalidate from cache the stats of opc

Re: [PR] Modify MSE operators to release memory once computation finished [pinot]

2025-06-03 Thread via GitHub
Copilot commented on code in PR #15977: URL: https://github.com/apache/pinot/pull/15977#discussion_r2123040716 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/NonEquiJoinOperator.java: ## @@ -106,6 +113,7 @@ protected List buildJoinedRows(MseBlock.Da

[PR] Modify MSE operators to release memory once computation finished [pinot]

2025-06-03 Thread via GitHub
gortiz opened a new pull request, #15977: URL: https://github.com/apache/pinot/pull/15977 This PR fixes the performance regression detected on https://github.com/apache/pinot/pull/15609. Contrary to https://github.com/apache/pinot/pull/15967, this change doesn't reduce stats precision and

Re: [PR] [WIP]Offset auto reset [pinot]

2025-06-03 Thread via GitHub
noob-se7en commented on PR #15782: URL: https://github.com/apache/pinot/pull/15782#issuecomment-2933969187 @lnbest0707-uber can you please add description here. This is because I am also planning to work on offset reset problem. Not sure if you are also trying to achieve the same thing.

Re: [PR] Use atomic reference for partition metadata provider [pinot]

2025-06-03 Thread via GitHub
Copilot commented on code in PR #15976: URL: https://github.com/apache/pinot/pull/15976#discussion_r2122914097 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java: ## @@ -1057,10 +1058,13 @@ public Map getConsumerPartitionStat

Re: [PR] Add new ImplicitRealtimeTablePartitionSelector strategy for instance assignment [pinot]

2025-06-03 Thread via GitHub
yashmayya commented on code in PR #15930: URL: https://github.com/apache/pinot/pull/15930#discussion_r2122830984 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/TableRebalanceIntegrationTest.java: ## @@ -88,6 +88,78 @@ private String getRebalanceUrl(R

Re: [PR] Add new ImplicitRealtimeTablePartitionSelector strategy for instance assignment [pinot]

2025-06-03 Thread via GitHub
yashmayya commented on code in PR #15930: URL: https://github.com/apache/pinot/pull/15930#discussion_r2122830984 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/TableRebalanceIntegrationTest.java: ## @@ -88,6 +88,78 @@ private String getRebalanceUrl(R

Re: [I] MSE error code when a table doesn't exist [pinot]

2025-06-03 Thread via GitHub
vrajat commented on issue #15272: URL: https://github.com/apache/pinot/issues/15272#issuecomment-2933896284 On a similar note, MSE tries to run a query on disabled tables and throws an unavailable segments instead of TableIsDisabledError. In SSE, these conditions are checked by updating `Ta

Re: [PR] Use atomic reference for partition metadata provider [pinot]

2025-06-03 Thread via GitHub
Copilot commented on code in PR #15976: URL: https://github.com/apache/pinot/pull/15976#discussion_r2122914097 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java: ## @@ -1057,10 +1058,13 @@ public Map getConsumerPartitionStat

[PR] Use atomic reference for partition metadata provider [pinot]

2025-06-03 Thread via GitHub
KKcorps opened a new pull request, #15976: URL: https://github.com/apache/pinot/pull/15976 This helps solve issues where we were making metadata requests on provider that was already closed -- This is an automated message from the Apache Git Service. To respond to the message, please log