Re: [PR] Add hooks to service starters to apply custom cluster and instance configs [pinot]

2025-04-18 Thread via GitHub
codecov-commenter commented on PR #15595: URL: https://github.com/apache/pinot/pull/15595#issuecomment-2816424396 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15595?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] roundDecimal() fails for null FLOAT values [pinot]

2025-04-18 Thread via GitHub
Jackie-Jiang commented on code in PR #15377: URL: https://github.com/apache/pinot/pull/15377#discussion_r2051295324 ## pinot-core/src/main/java/org/apache/pinot/core/operator/query/SelectionOnlyOperator.java: ## @@ -132,7 +132,11 @@ protected SelectionResultsBlock getNextBlock()

Re: [PR] Fix range index on raw TIMESTAMP column [pinot]

2025-04-18 Thread via GitHub
Jackie-Jiang merged PR #15589: URL: https://github.com/apache/pinot/pull/15589 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: Fix range index on raw TIMESTAMP column (#15589)

2025-04-18 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 4442d4199e Fix range index on raw TIMESTAMP colum

[PR] Add hooks to service starters to apply custom cluster and instance configs [pinot]

2025-04-18 Thread via GitHub
Jackie-Jiang opened a new pull request, #15595: URL: https://github.com/apache/pinot/pull/15595 - Add hook to all components to apply custom instance configs - Add hook to controller to apply default Helix instance configs during controller `init()` - Change worth noting (shouldn't cau

(pinot) branch hotfix-array-agg-task-mananger updated (5ac5156a4d -> e266711f67)

2025-04-18 Thread jtao
This is an automated email from the ASF dual-hosted git repository. jtao pushed a change to branch hotfix-array-agg-task-mananger in repository https://gitbox.apache.org/repos/asf/pinot.git from 5ac5156a4d Add predownload functionality to Pinot (#14686) add 701bba3500 Fix array agg func

(pinot) 01/01: set tables for controller task scheduling (#15594)

2025-04-18 Thread jtao
This is an automated email from the ASF dual-hosted git repository. jtao pushed a commit to branch hotfix-array-agg-task-mananger in repository https://gitbox.apache.org/repos/asf/pinot.git commit e266711f6762c8a8b8b7e955ead86c33bff780bd Author: liranjiao AuthorDate: Fri Apr 18 14:48:29 2025 -07

(pinot) branch hotfix-array-agg-task-mananger created (now 5ac5156a4d)

2025-04-18 Thread jtao
This is an automated email from the ASF dual-hosted git repository. jtao pushed a change to branch hotfix-array-agg-task-mananger in repository https://gitbox.apache.org/repos/asf/pinot.git at 5ac5156a4d Add predownload functionality to Pinot (#14686) No new revisions were added by this u

(pinot) branch hotfix-task-manager updated: set tables for controller task scheduling (#15594)

2025-04-18 Thread jtao
This is an automated email from the ASF dual-hosted git repository. jtao pushed a commit to branch hotfix-task-manager in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/hotfix-task-manager by this push: new dd409d4e60 set tables fo

Re: [PR] Add a New Pre-check Item for Replica Groups Info [pinot]

2025-04-18 Thread via GitHub
J-HowHuang commented on code in PR #15575: URL: https://github.com/apache/pinot/pull/15575#discussion_r2051176216 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/DefaultRebalancePreChecker.java: ## @@ -379,6 +382,58 @@ private RebalancePreCheck

(pinot) branch hotfix-task-manager created (now 5ac5156a4d)

2025-04-18 Thread jtao
This is an automated email from the ASF dual-hosted git repository. jtao pushed a change to branch hotfix-task-manager in repository https://gitbox.apache.org/repos/asf/pinot.git at 5ac5156a4d Add predownload functionality to Pinot (#14686) No new revisions were added by this update. --

Re: [PR] set tables for controller task scheduling [pinot]

2025-04-18 Thread via GitHub
jtao15 merged PR #15594: URL: https://github.com/apache/pinot/pull/15594 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

(pinot) branch master updated: set tables for controller task scheduling (#15594)

2025-04-18 Thread jtao
This is an automated email from the ASF dual-hosted git repository. jtao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 5cf9c3b104 set tables for controller task schedulin

Re: [PR] Enhance ClusterTest.sanitizeResponse() to reduce error [pinot]

2025-04-18 Thread via GitHub
codecov-commenter commented on PR #15593: URL: https://github.com/apache/pinot/pull/15593#issuecomment-2816223952 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15593?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] set tables for controller task scheduling [pinot]

2025-04-18 Thread via GitHub
liranjiao commented on PR #15594: URL: https://github.com/apache/pinot/pull/15594#issuecomment-2816218074 cc @sajjad-moradi @jtao15 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[PR] set tables for controller task scheduling [pinot]

2025-04-18 Thread via GitHub
liranjiao opened a new pull request, #15594: URL: https://github.com/apache/pinot/pull/15594 Bug fix for https://github.com/apache/pinot/issues/15592. If not set `setTablesToSchedule()`, the all controllers will generate tasks for all tables, which causes the minions to do lots of dup

[PR] Enhance ClusterTest.sanitizeResponse() to reduce error [pinot]

2025-04-18 Thread via GitHub
Jackie-Jiang opened a new pull request, #15593: URL: https://github.com/apache/pinot/pull/15593 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [I] Controller schedules tasks for all tables, instead of only the tables for which it is the lead. [pinot]

2025-04-18 Thread via GitHub
jtao15 commented on issue #15592: URL: https://github.com/apache/pinot/issues/15592#issuecomment-2816149583 cc @shounakmk219 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[I] Controller schedules tasks for all tables, instead of only the tables for which it is the lead. [pinot]

2025-04-18 Thread via GitHub
jtao15 opened a new issue, #15592: URL: https://github.com/apache/pinot/issues/15592 There seems to be a regression in #14829 that controller schedules minion tasks for all tables, instead of only the tables for which it is the lead. We should set the `tableNamesWithType` in the context

Re: [PR] Metrics for Segment Upload/Download [pinot]

2025-04-18 Thread via GitHub
praveenc7 commented on code in PR #15585: URL: https://github.com/apache/pinot/pull/15585#discussion_r2050943100 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java: ## @@ -985,6 +1010,11 @@ protected File downloadSegmentFromDeepStore(Segmen

Re: [PR] Add a New Pre-check Item for Replica Groups Info [pinot]

2025-04-18 Thread via GitHub
J-HowHuang commented on code in PR #15575: URL: https://github.com/apache/pinot/pull/15575#discussion_r2050892065 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/DefaultRebalancePreChecker.java: ## @@ -379,6 +382,58 @@ private RebalancePreCheck

Re: [PR] [bugfix]: Fixes rebalance server status rendering [pinot]

2025-04-18 Thread via GitHub
Jackie-Jiang merged PR #15554: URL: https://github.com/apache/pinot/pull/15554 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated (97aa5cacce -> 597534ec7e)

2025-04-18 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 97aa5cacce [multistage] Bug Fixes and Refactorings Based on E2E Physical Optimizer Test (#15561) add 597534ec7e

Re: [PR] Add offset based lag metrics [pinot]

2025-04-18 Thread via GitHub
ankitsultana commented on PR #13298: URL: https://github.com/apache/pinot/pull/13298#issuecomment-2811646926 @KKcorps : this feature is a production risk and has impacted us too (I see Stripe was also affected). Is the fix merged already? If not we'll likely comment this out until the risk

Re: [PR] Preserve the Order of the Pre-check Items [pinot]

2025-04-18 Thread via GitHub
somandal commented on code in PR #15496: URL: https://github.com/apache/pinot/pull/15496#discussion_r2048069236 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/DefaultRebalancePreChecker.java: ## @@ -77,7 +78,9 @@ public Map check(PreCheckCont

Re: [PR] Add button to repair realtime table by triggering RealtimeSegmentValidationManager [pinot]

2025-04-18 Thread via GitHub
jayeshchoudhary commented on code in PR #15584: URL: https://github.com/apache/pinot/pull/15584#discussion_r2049175001 ## pinot-controller/src/main/resources/app/pages/TenantDetails.tsx: ## @@ -558,6 +572,13 @@ const TenantPageDetails = ({ match }: RouteComponentProps) => {

Re: [PR] Set cluster.tenant.isolation.enable to true in ControllerTest's default controller config [pinot]

2025-04-18 Thread via GitHub
xiangfu0 merged PR #15588: URL: https://github.com/apache/pinot/pull/15588 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

(pinot) branch master updated: Enhance and clean up star-tree tests (#15587)

2025-04-18 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 9d9f4fcf44 Enhance and clean up star-tree tests (

Re: [I] Hard Dependency on Groovy [pinot]

2025-04-18 Thread via GitHub
ankitsultana commented on issue #15586: URL: https://github.com/apache/pinot/issues/15586#issuecomment-2814511247 Yeah we exclude it. There are some other dependencies in pinot-spi and some other packages which ideally shouldn't be there either (e.g. controller relies on a pinot-plugin modu

Re: [PR] Add a New Pre-check Item for Replica Groups Info [pinot]

2025-04-18 Thread via GitHub
J-HowHuang commented on code in PR #15575: URL: https://github.com/apache/pinot/pull/15575#discussion_r2050875503 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/DefaultRebalancePreChecker.java: ## @@ -379,6 +382,58 @@ private RebalancePreCheck

Re: [PR] Reduce segment metadata memory overhead. [pinot]

2025-04-18 Thread via GitHub
bziobrowski commented on code in PR #15591: URL: https://github.com/apache/pinot/pull/15591#discussion_r2050690253 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/immutable/ImmutableSegmentLoader.java: ## @@ -256,7 +256,7 @@ public static Immutab

Re: [PR] Prevent NPE in minion tasks [pinot]

2025-04-18 Thread via GitHub
ankitsultana merged PR #15550: URL: https://github.com/apache/pinot/pull/15550 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Reduce segment metadata memory overhead. [pinot]

2025-04-18 Thread via GitHub
bziobrowski commented on code in PR #15591: URL: https://github.com/apache/pinot/pull/15591#discussion_r2050687631 ## pinot-integration-test-base/src/test/java/org/apache/pinot/integration/tests/ClusterIntegrationTestUtils.java: ## @@ -295,7 +295,7 @@ public static void buildSeg

Re: [PR] Bump software.amazon.awssdk:bom from 2.31.22 to 2.31.23 [pinot]

2025-04-18 Thread via GitHub
dependabot[bot] commented on PR #15582: URL: https://github.com/apache/pinot/pull/15582#issuecomment-2815290690 Superseded by #15590. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Reduce segment metadata memory overhead. [pinot]

2025-04-18 Thread via GitHub
codecov-commenter commented on PR #15591: URL: https://github.com/apache/pinot/pull/15591#issuecomment-2815589827 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15591?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Reduce segment metadata memory overhead. [pinot]

2025-04-18 Thread via GitHub
Copilot commented on code in PR #15591: URL: https://github.com/apache/pinot/pull/15591#discussion_r2050819216 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/metadata/ColumnMetadataImpl.java: ## @@ -164,10 +172,40 @@ public Set getPartitions() { retur

[PR] Add button to repair realtime table by triggering RealtimeSegmentValidationManager [pinot]

2025-04-18 Thread via GitHub
KKcorps opened a new pull request, #15584: URL: https://github.com/apache/pinot/pull/15584 ![Screenshot 2025-04-17 at 8 26 32  PM](https://github.com/user-attachments/assets/1aa9716b-5fbe-446d-bd25-036bbb38ec03) ![Screenshot 2025-04-17 at 8 26 40  PM](https://github.com/user-attachments/as

Re: [PR] Bump software.amazon.awssdk:bom from 2.31.22 to 2.31.23 [pinot]

2025-04-18 Thread via GitHub
dependabot[bot] closed pull request #15582: Bump software.amazon.awssdk:bom from 2.31.22 to 2.31.23 URL: https://github.com/apache/pinot/pull/15582 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Reduce segment metadata memory overhead. [pinot]

2025-04-18 Thread via GitHub
bziobrowski commented on code in PR #15591: URL: https://github.com/apache/pinot/pull/15591#discussion_r2050691351 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/column/PhysicalColumnIndexContainer.java: ## @@ -39,7 +39,11 @@ public final clas

Re: [PR] Reduce segment metadata memory overhead. [pinot]

2025-04-18 Thread via GitHub
bziobrowski commented on code in PR #15591: URL: https://github.com/apache/pinot/pull/15591#discussion_r2050690066 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/immutable/ImmutableSegmentImpl.java: ## @@ -88,7 +87,8 @@ public ImmutableSegmentIm

[PR] Reduce segment metadata memory overhead. [pinot]

2025-04-18 Thread via GitHub
bziobrowski opened a new pull request, #15591: URL: https://github.com/apache/pinot/pull/15591 PR changes a few data structures (used related to immutable segment metadata) to reduce memory usage. For wide tables (100+ columns) with many segments (10k+) these changes can reduce heap size

Re: [PR] roundDecimal() fails for null FLOAT values [pinot]

2025-04-18 Thread via GitHub
mghildiy commented on code in PR #15377: URL: https://github.com/apache/pinot/pull/15377#discussion_r2050657284 ## pinot-core/src/main/java/org/apache/pinot/core/operator/query/SelectionOnlyOperator.java: ## @@ -132,7 +132,11 @@ protected SelectionResultsBlock getNextBlock() {

Re: [PR] Bump software.amazon.awssdk:bom from 2.31.22 to 2.31.24 [pinot]

2025-04-18 Thread via GitHub
codecov-commenter commented on PR #15590: URL: https://github.com/apache/pinot/pull/15590#issuecomment-2815338816 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15590?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.31.23 deleted (was c859ac3bc4)

2025-04-18 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.31.23 in repository https://gitbox.apache.org/repos/asf/pinot.git was c859ac3bc4 Bump software.amazon.awssdk:bom from 2.31.22 to 2.31.23 T

[PR] Bump software.amazon.awssdk:bom from 2.31.22 to 2.31.24 [pinot]

2025-04-18 Thread via GitHub
dependabot[bot] opened a new pull request, #15590: URL: https://github.com/apache/pinot/pull/15590 Bumps software.amazon.awssdk:bom from 2.31.22 to 2.31.24. Most Recent Ignore Conditions Applied to This Pull Request | Dependency Name | Ignore Conditions | | --- | --- |

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.31.24 created (now 877203a802)

2025-04-18 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.31.24 in repository https://gitbox.apache.org/repos/asf/pinot.git at 877203a802 Bump software.amazon.awssdk:bom from 2.31.22 to 2.31.24 N

Re: [PR] Add validations for Pauseless Tables [pinot]

2025-04-18 Thread via GitHub
KKcorps commented on code in PR #15567: URL: https://github.com/apache/pinot/pull/15567#discussion_r2050499115 ## pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisStreamMetadataProvider.java: ## @@ -101,7 +100,11 @@ p

Re: [PR] Enhance and clean up star-tree tests [pinot]

2025-04-18 Thread via GitHub
Jackie-Jiang merged PR #15587: URL: https://github.com/apache/pinot/pull/15587 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: Add progress stats rollback support for rebalance progress stats on IdealState update failure (#15510)

2025-04-18 Thread somandal
This is an automated email from the ASF dual-hosted git repository. somandal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 764ae956b3 Add progress stats rollback support