Re: [PR] Fix starvation in consumer lock [pinot]

2025-03-28 Thread via GitHub
KKcorps merged PR #15404: URL: https://github.com/apache/pinot/pull/15404 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

Re: [PR] Consuming Rebalance Summary [pinot]

2025-03-28 Thread via GitHub
somandal commented on code in PR #15368: URL: https://github.com/apache/pinot/pull/15368#discussion_r2019508180 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceSummaryResult.java: ## @@ -240,6 +240,92 @@ public Map getServerSegmentCha

(pinot) branch master updated: Fix starvation in consumer lock (#15404)

2025-03-28 Thread kharekartik
This is an automated email from the ASF dual-hosted git repository. kharekartik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 20ec20da88 Fix starvation in consumer lock (

Re: [PR] Fix starvation in consumer lock [pinot]

2025-03-28 Thread via GitHub
KKcorps commented on code in PR #15404: URL: https://github.com/apache/pinot/pull/15404#discussion_r2019672893 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -557,22 +565,10 @@ private void doAddConsumingSegment(Strin

Re: [PR] Consuming Rebalance Summary [pinot]

2025-03-28 Thread via GitHub
J-HowHuang commented on code in PR #15368: URL: https://github.com/apache/pinot/pull/15368#discussion_r2019273577 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -733,15 +781,148 @@ private RebalanceSummaryResult ca

Re: [PR] Fix starvation in consumer lock [pinot]

2025-03-28 Thread via GitHub
Jackie-Jiang commented on code in PR #15404: URL: https://github.com/apache/pinot/pull/15404#discussion_r2019674182 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -557,22 +565,10 @@ private void doAddConsumingSegment(

Re: [PR] Fix starvation in consumer lock [pinot]

2025-03-28 Thread via GitHub
codecov-commenter commented on PR #15404: URL: https://github.com/apache/pinot/pull/15404#issuecomment-2763022039 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15404?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Adds a new config to create a configurable JFR recording in continuous mode [pinot]

2025-03-28 Thread via GitHub
gortiz commented on code in PR #15364: URL: https://github.com/apache/pinot/pull/15364#discussion_r2011731034 ## pinot-core/src/main/java/org/apache/pinot/core/util/trace/ContinuousJfrStarter.java: ## @@ -0,0 +1,143 @@ +package org.apache.pinot.core.util.trace; + +import java.io

Re: [PR] Consuming Rebalance Summary [pinot]

2025-03-28 Thread via GitHub
J-HowHuang commented on code in PR #15368: URL: https://github.com/apache/pinot/pull/15368#discussion_r2019520767 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceSummaryResult.java: ## @@ -240,6 +240,92 @@ public Map getServerSegmentC

Re: [PR] Fix starvation in consumer lock [pinot]

2025-03-28 Thread via GitHub
Jackie-Jiang commented on PR #15404: URL: https://github.com/apache/pinot/pull/15404#issuecomment-2762987862 cc @noob-se7en -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Fix starvation in consumer lock [pinot]

2025-03-28 Thread via GitHub
Jackie-Jiang commented on code in PR #15404: URL: https://github.com/apache/pinot/pull/15404#discussion_r2019656629 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/ConsumerCoordinator.java: ## @@ -299,26 +227,36 @@ AtomicBoolean getFirstTransitionProcesse

Re: [PR] Fix starvation in consumer lock [pinot]

2025-03-28 Thread via GitHub
Jackie-Jiang commented on code in PR #15404: URL: https://github.com/apache/pinot/pull/15404#discussion_r2019655819 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/ConsumerCoordinator.java: ## @@ -115,177 +109,111 @@ Semaphore getSemaphore() { return

[PR] Fix starvation in consumer lock [pinot]

2025-03-28 Thread via GitHub
Jackie-Jiang opened a new pull request, #15404: URL: https://github.com/apache/pinot/pull/15404 Currently the consumer lock is acquired during the `OFFLINE -> CONSUMING` state transition. If the lock cannot be acquired, helix thread will be blocked, and end up causing starvation. With #1526

Re: [PR] Add an interface for SecretStore [pinot]

2025-03-28 Thread via GitHub
nayanika-u commented on code in PR #15226: URL: https://github.com/apache/pinot/pull/15226#discussion_r2019517125 ## pinot-spi/src/main/java/org/apache/pinot/spi/secretstore/SecretStore.java: ## @@ -0,0 +1,77 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Introducing dynamic broadcast broker level configuration [pinot]

2025-03-28 Thread via GitHub
codecov-commenter commented on PR #15402: URL: https://github.com/apache/pinot/pull/15402#issuecomment-2762947924 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15402?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Consuming Rebalance Summary [pinot]

2025-03-28 Thread via GitHub
somandal commented on code in PR #15368: URL: https://github.com/apache/pinot/pull/15368#discussion_r2019522044 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceSummaryResult.java: ## @@ -240,6 +240,92 @@ public Map getServerSegmentCha

(pinot) branch master updated (f8a92604aa -> c25059447c)

2025-03-28 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from f8a92604aa Disables isSegmentAlreadyConsumed check while waiting on prev segment (#15399) add c25059447c Support

Re: [PR] Consuming Rebalance Summary [pinot]

2025-03-28 Thread via GitHub
J-HowHuang commented on code in PR #15368: URL: https://github.com/apache/pinot/pull/15368#discussion_r2019596710 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceSummaryResult.java: ## @@ -240,6 +240,92 @@ public Map getServerSegmentC

Re: [PR] Support FunnelEventsFunctionEvalAggregationFunction and FunnelStepDurationStatsAggregationFunction functions [pinot]

2025-03-28 Thread via GitHub
xiangfu0 merged PR #14941: URL: https://github.com/apache/pinot/pull/14941 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] Add an interface for SecretStore [pinot]

2025-03-28 Thread via GitHub
nayanika-u commented on code in PR #15226: URL: https://github.com/apache/pinot/pull/15226#discussion_r2019506269 ## pinot-spi/src/main/java/org/apache/pinot/spi/secretstore/SecretStoreException.java: ## @@ -0,0 +1,55 @@ +/** + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Consuming Rebalance Summary [pinot]

2025-03-28 Thread via GitHub
J-HowHuang commented on code in PR #15368: URL: https://github.com/apache/pinot/pull/15368#discussion_r2019489057 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceSummaryResult.java: ## @@ -252,6 +338,7 @@ public static class SegmentInf

Re: [PR] Consuming Rebalance Summary [pinot]

2025-03-28 Thread via GitHub
somandal commented on code in PR #15368: URL: https://github.com/apache/pinot/pull/15368#discussion_r2019509223 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceSummaryResult.java: ## @@ -240,6 +240,92 @@ public Map getServerSegmentCha

Re: [PR] Consuming Rebalance Summary [pinot]

2025-03-28 Thread via GitHub
somandal commented on code in PR #15368: URL: https://github.com/apache/pinot/pull/15368#discussion_r2019528019 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceSummaryResult.java: ## @@ -240,6 +240,92 @@ public Map getServerSegmentCha

Re: [PR] Consuming Rebalance Summary [pinot]

2025-03-28 Thread via GitHub
J-HowHuang commented on code in PR #15368: URL: https://github.com/apache/pinot/pull/15368#discussion_r2019521761 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceSummaryResult.java: ## @@ -240,6 +240,92 @@ public Map getServerSegmentC

Re: [PR] Consuming Rebalance Summary [pinot]

2025-03-28 Thread via GitHub
somandal commented on code in PR #15368: URL: https://github.com/apache/pinot/pull/15368#discussion_r2019506948 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceSummaryResult.java: ## @@ -240,6 +240,92 @@ public Map getServerSegmentCha

Re: [PR] Consuming Rebalance Summary [pinot]

2025-03-28 Thread via GitHub
J-HowHuang commented on code in PR #15368: URL: https://github.com/apache/pinot/pull/15368#discussion_r2019497497 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceSummaryResult.java: ## @@ -240,6 +240,92 @@ public Map getServerSegmentC

Re: [PR] Consuming Rebalance Summary [pinot]

2025-03-28 Thread via GitHub
J-HowHuang commented on code in PR #15368: URL: https://github.com/apache/pinot/pull/15368#discussion_r2019489926 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceSummaryResult.java: ## @@ -240,6 +240,92 @@ public Map getServerSegmentC

Re: [PR] Cleanup controller and MSE broker entry points to keep error codes an… [pinot]

2025-03-28 Thread via GitHub
gortiz commented on code in PR #15277: URL: https://github.com/apache/pinot/pull/15277#discussion_r2005730051 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/MultiStageEngineIntegrationTest.java: ## @@ -703,7 +705,9 @@ public void testBase64Func()

Re: [PR] Add Tenants Info to Rebalance API summary [pinot]

2025-03-28 Thread via GitHub
Jackie-Jiang merged PR #15284: URL: https://github.com/apache/pinot/pull/15284 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] [wip] optimise consumption in order [pinot]

2025-03-28 Thread via GitHub
codecov-commenter commented on PR #15403: URL: https://github.com/apache/pinot/pull/15403#issuecomment-2762665741 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15403?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

(pinot) branch master updated: Add Tenants Info to Rebalance API summary (#15284)

2025-03-28 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 0075ad7d53 Add Tenants Info to Rebalance API summ

[PR] [wip] optimise consumption in order [pinot]

2025-03-28 Thread via GitHub
noob-se7en opened a new pull request, #15403: URL: https://github.com/apache/pinot/pull/15403 Instructions: 1. The PR has to be tagged with at least one of the following labels (*): 1. `feature` 2. `bugfix` 3. `performance` 4. `ui` 5. `backward-incompat`

Re: [PR] Add Tenants Info to Rebalance API summary [pinot]

2025-03-28 Thread via GitHub
Jackie-Jiang commented on code in PR #15284: URL: https://github.com/apache/pinot/pull/15284#discussion_r2019296046 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -662,6 +663,29 @@ private RebalanceSummaryResult ca

Re: [PR] Consuming Rebalance Summary [pinot]

2025-03-28 Thread via GitHub
J-HowHuang commented on code in PR #15368: URL: https://github.com/apache/pinot/pull/15368#discussion_r2019280409 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -733,15 +781,148 @@ private RebalanceSummaryResult ca

Re: [PR] Consuming Rebalance Summary [pinot]

2025-03-28 Thread via GitHub
J-HowHuang commented on code in PR #15368: URL: https://github.com/apache/pinot/pull/15368#discussion_r2019277838 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -711,6 +750,15 @@ private RebalanceSummaryResult calc

Re: [PR] Consuming Rebalance Summary [pinot]

2025-03-28 Thread via GitHub
J-HowHuang commented on code in PR #15368: URL: https://github.com/apache/pinot/pull/15368#discussion_r2019275371 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -733,15 +781,148 @@ private RebalanceSummaryResult ca

Re: [PR] Add `gridDisk` H3 Function [pinot]

2025-03-28 Thread via GitHub
ashishjayamohan commented on code in PR #15349: URL: https://github.com/apache/pinot/pull/15349#discussion_r2019191055 ## pinot-core/src/main/java/org/apache/pinot/core/geospatial/transform/function/ScalarFunctions.java: ## @@ -267,4 +268,18 @@ public static long gridDistance(lo

Re: [PR] Add Tenants Info to Rebalance API summary [pinot]

2025-03-28 Thread via GitHub
J-HowHuang commented on code in PR #15284: URL: https://github.com/apache/pinot/pull/15284#discussion_r2019188598 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -662,6 +663,20 @@ private RebalanceSummaryResult calc

Re: [PR] Fix the completed segment check when trying to add it as consuming segment [pinot]

2025-03-28 Thread via GitHub
codecov-commenter commented on PR #15401: URL: https://github.com/apache/pinot/pull/15401#issuecomment-2762177158 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15401?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Introducing dynamic broadcast broker level configuration [pinot]

2025-03-28 Thread via GitHub
shauryachats opened a new pull request, #15402: URL: https://github.com/apache/pinot/pull/15402 A new configuration, `pinot.broker.enable.dynamic.broadcast,` is introduced to control whether dynamic broadcast should be enabled at a broker level. By default, it is `true` to maintain the cur

Re: [PR] Fix the completed segment check when trying to add it as consuming segment [pinot]

2025-03-28 Thread via GitHub
Jackie-Jiang closed pull request #15401: Fix the completed segment check when trying to add it as consuming segment URL: https://github.com/apache/pinot/pull/15401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[PR] Fix the completed segment check when trying to add it as consuming segment [pinot]

2025-03-28 Thread via GitHub
Jackie-Jiang opened a new pull request, #15401: URL: https://github.com/apache/pinot/pull/15401 When segment is completed, regardless of wither consumption order is enforced, we should skip adding it as consuming segment, and wait for `CONSUMING -> ONLINE` state transition to bring it direc

Re: [PR] Add Tenants Info to Rebalance API summary [pinot]

2025-03-28 Thread via GitHub
J-HowHuang commented on code in PR #15284: URL: https://github.com/apache/pinot/pull/15284#discussion_r2019088428 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -699,6 +723,28 @@ private RebalanceSummaryResult calc

Re: [PR] Add Tenants Info to Rebalance API summary [pinot]

2025-03-28 Thread via GitHub
J-HowHuang commented on code in PR #15284: URL: https://github.com/apache/pinot/pull/15284#discussion_r2019090910 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -662,6 +663,29 @@ private RebalanceSummaryResult calc

(pinot) branch master updated (0ff9959876 -> b952affb21)

2025-03-28 Thread tingchen
This is an automated email from the ASF dual-hosted git repository. tingchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 0ff9959876 Fix needReload rebalance pre-check to only fetch status from currently assigned servers (#15360) add

Re: [PR] Clean the class behavior description for SchemaConformingTransformer for easier adoption. [pinot]

2025-03-28 Thread via GitHub
chenboat merged PR #15390: URL: https://github.com/apache/pinot/pull/15390 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] Add Tenants Info to Rebalance API summary [pinot]

2025-03-28 Thread via GitHub
J-HowHuang commented on code in PR #15284: URL: https://github.com/apache/pinot/pull/15284#discussion_r2019086302 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -662,6 +663,20 @@ private RebalanceSummaryResult calc

Re: [PR] Feature: New UI to monitor Rebalance Server Status [pinot]

2025-03-28 Thread via GitHub
codecov-commenter commented on PR #15400: URL: https://github.com/apache/pinot/pull/15400#issuecomment-2761943377 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15400?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Disables isSegmentAlreadyConsumed check while waiting on prev segment [pinot]

2025-03-28 Thread via GitHub
KKcorps merged PR #15399: URL: https://github.com/apache/pinot/pull/15399 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

(pinot) branch master updated (b952affb21 -> f8a92604aa)

2025-03-28 Thread kharekartik
This is an automated email from the ASF dual-hosted git repository. kharekartik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from b952affb21 Clean up the class comments (#15390) add f8a92604aa Disables isSegmentAlreadyConsumed check while

(pinot) branch master updated (e8ec40677e -> 0ff9959876)

2025-03-28 Thread xbli
This is an automated email from the ASF dual-hosted git repository. xbli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from e8ec40677e Add default impl to read string values in ForwardIndexReader (#15391) add 0ff9959876 Fix needReload reba

Re: [PR] Fix needReload rebalance pre-check to only fetch status from currently assigned servers [pinot]

2025-03-28 Thread via GitHub
klsince merged PR #15360: URL: https://github.com/apache/pinot/pull/15360 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

Re: [PR] Add Tenants Info to Rebalance API summary [pinot]

2025-03-28 Thread via GitHub
somandal commented on code in PR #15284: URL: https://github.com/apache/pinot/pull/15284#discussion_r2019072233 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -662,6 +663,20 @@ private RebalanceSummaryResult calcul

Re: [PR] Support FunnelEventsFunctionEvalAggregationFunction and FunnelStepDurationStatsAggregationFunction functions [pinot]

2025-03-28 Thread via GitHub
Jackie-Jiang commented on code in PR #14941: URL: https://github.com/apache/pinot/pull/14941#discussion_r2019059134 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/AggregationFunctionType.java: ## @@ -179,8 +179,13 @@ public enum AggregationFunctionType { // f

Re: [PR] [Refactor] Add default impl to read string values in ForwardIndexReader [pinot]

2025-03-28 Thread via GitHub
Jackie-Jiang merged PR #15391: URL: https://github.com/apache/pinot/pull/15391 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated (678b6f8dec -> e8ec40677e)

2025-03-28 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 678b6f8dec close grpc connection after query in ClusterTest (#15394) add e8ec40677e Add default impl to read strin

Re: [PR] Feature: New UI to monitor Rebalance Server Status [pinot]

2025-03-28 Thread via GitHub
himanish-star commented on code in PR #15400: URL: https://github.com/apache/pinot/pull/15400#discussion_r2018979032 ## pinot-controller/src/main/resources/app/components/Table.tsx: ## @@ -562,7 +564,7 @@ export default function CustomizedTables({ ) : (

[PR] Feature: New UI to monitor Rebalance Server Status [pinot]

2025-03-28 Thread via GitHub
himanish-star opened a new pull request, #15400: URL: https://github.com/apache/pinot/pull/15400 ### Description Uploading Rebalance Phase 3.mp4… -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] Disables isSegmentAlreadyConsumed check while waiting on prev segment [pinot]

2025-03-28 Thread via GitHub
codecov-commenter commented on PR #15399: URL: https://github.com/apache/pinot/pull/15399#issuecomment-2761605065 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15399?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Disables isSegmentAlreadyConsumed check while waiting on prev segment [pinot]

2025-03-28 Thread via GitHub
noob-se7en opened a new pull request, #15399: URL: https://github.com/apache/pinot/pull/15399 Instructions: 1. The PR has to be tagged with at least one of the following labels (*): 1. `feature` 2. `bugfix` 3. `performance` 4. `ui` 5. `backward-incompat`

Re: [I] Add native memory manager [pinot]

2025-03-28 Thread via GitHub
mghildiy commented on issue #15381: URL: https://github.com/apache/pinot/issues/15381#issuecomment-2761443018 Fomr initial research, I see for ByteBuffer based allocations, we can make use of BufferPoolMXBean to hook into jvm to track the 0ff-heap memory. For Unsafe, we may need to track ma

Re: [PR] Substitute TransferableBlock with MseBlock [pinot]

2025-03-28 Thread via GitHub
yashmayya commented on code in PR #15245: URL: https://github.com/apache/pinot/pull/15245#discussion_r2016415044 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/blocks/RowHeapDataBlock.java: ## @@ -0,0 +1,112 @@ +/** + * Licensed to the Apache Software Founda

Re: [PR] Adds Intgeration tests for Dedup and Pauseless [pinot]

2025-03-28 Thread via GitHub
codecov-commenter commented on PR #15398: URL: https://github.com/apache/pinot/pull/15398#issuecomment-2761279933 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15398?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Bump software.amazon.awssdk:bom from 2.31.9 to 2.31.10 [pinot]

2025-03-28 Thread via GitHub
dependabot[bot] opened a new pull request, #15396: URL: https://github.com/apache/pinot/pull/15396 Bumps software.amazon.awssdk:bom from 2.31.9 to 2.31.10. Most Recent Ignore Conditions Applied to This Pull Request | Dependency Name | Ignore Conditions | | --- | --- |

[PR] Adds Intgeration tests for Dedup and Pauseless [pinot]

2025-03-28 Thread via GitHub
noob-se7en opened a new pull request, #15398: URL: https://github.com/apache/pinot/pull/15398 Adds Intgeration tests for Dedup and Pauseless enforcing consumption in order. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

(pinot) branch master updated: [HELM]: Publish Helm 0.3.2 (#15323)

2025-03-28 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 41e33b8781 [HELM]: Publish Helm 0.3.2 (#15323) 41

(pinot) branch dependabot/maven/org.assertj-assertj-core-3.27.3 created (now 955453368f)

2025-03-28 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.assertj-assertj-core-3.27.3 in repository https://gitbox.apache.org/repos/asf/pinot.git at 955453368f Bump org.assertj:assertj-core from 3.27.2 to 3.27.3 No new r

Re: [PR] Adding encoding type into the BrokerGrpcServer [pinot]

2025-03-28 Thread via GitHub
codecov-commenter commented on PR #15395: URL: https://github.com/apache/pinot/pull/15395#issuecomment-2761082596 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15395?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Add metrics for tracking zookeeper's max buffer size in services [pinot]

2025-03-28 Thread via GitHub
rhodo commented on PR #15380: URL: https://github.com/apache/pinot/pull/15380#issuecomment-2760461666 Address comment: add metrics to other components as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

(pinot) branch dependabot/maven/org.apache.hadoop.thirdparty-hadoop-shaded-protobuf_3_25-1.4.0 created (now ef05a1103a)

2025-03-28 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.apache.hadoop.thirdparty-hadoop-shaded-protobuf_3_25-1.4.0 in repository https://gitbox.apache.org/repos/asf/pinot.git at ef05a1103a Bump org.apache.hadoop.thirdp

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.31.10 created (now 25554d1e60)

2025-03-28 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.31.10 in repository https://gitbox.apache.org/repos/asf/pinot.git at 25554d1e60 Bump software.amazon.awssdk:bom from 2.31.9 to 2.31.10 No

Re: [PR] Emit metrics for each query error code returned by the broker [pinot]

2025-03-28 Thread via GitHub
gortiz merged PR #15387: URL: https://github.com/apache/pinot/pull/15387 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

[PR] Adding encoding type into the BrokerGrpcServer [pinot]

2025-03-28 Thread via GitHub
xiangfu0 opened a new pull request, #15395: URL: https://github.com/apache/pinot/pull/15395 Adding encoding type to BrokerGrpcServer protocol and default to JSON, so we can extend it later on. -- This is an automated message from the Apache Git Service. To respond to the message, please l

(pinot) branch master updated: close grpc connection after query in ClusterTest (#15394)

2025-03-28 Thread kharekartik
This is an automated email from the ASF dual-hosted git repository. kharekartik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 678b6f8dec close grpc connection after query

Re: [PR] Close grpc connection after query in ClusterTest [pinot]

2025-03-28 Thread via GitHub
KKcorps merged PR #15394: URL: https://github.com/apache/pinot/pull/15394 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

Re: [PR] fix numGroups metric and add metric for warnings [pinot]

2025-03-28 Thread via GitHub
gortiz merged PR #15280: URL: https://github.com/apache/pinot/pull/15280 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

(pinot) branch master updated: fix numGroups metric and add metric for warnings (#15280)

2025-03-28 Thread gortiz
This is an automated email from the ASF dual-hosted git repository. gortiz pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 1a29bc fix numGroups metric and add metric fo

Re: [PR] Adding metrics for pauseless observability [pinot]

2025-03-28 Thread via GitHub
KKcorps merged PR #15384: URL: https://github.com/apache/pinot/pull/15384 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

(pinot) branch master updated: Unblocks helix thread for online segment (#15246)

2025-03-28 Thread kharekartik
This is an automated email from the ASF dual-hosted git repository. kharekartik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new e5d9a04650 Unblocks helix thread for online

Re: [PR] Unblocks helix thread for online segment [pinot]

2025-03-28 Thread via GitHub
KKcorps merged PR #15246: URL: https://github.com/apache/pinot/pull/15246 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

(pinot) branch master updated: Adding metrics for pauseless observability (#15384)

2025-03-28 Thread kharekartik
This is an automated email from the ASF dual-hosted git repository. kharekartik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 5dd0c3a06d Adding metrics for pauseless obse

(pinot) branch master updated (5550c24f1e -> 9b96fd4c43)

2025-03-28 Thread gortiz
This is an automated email from the ASF dual-hosted git repository. gortiz pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 5550c24f1e Disable reingestion for Pauseless dedup (#15383) add 9b96fd4c43 Emit metrics for each query error code

Re: [PR] Support FunnelEventsFunctionEvalAggregationFunction and FunnelStepDurationStatsAggregationFunction functions [pinot]

2025-03-28 Thread via GitHub
xiangfu0 commented on code in PR #14941: URL: https://github.com/apache/pinot/pull/14941#discussion_r2018101817 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/funnel/window/FunnelEventsFunctionEvalAggregationFunction.java: ## @@ -0,0 +1,502 @@ +/**

(pinot) branch dependabot/maven/org.apache.kafka-kafka-clients-3.7.1 deleted (was 05d116575e)

2025-03-28 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.apache.kafka-kafka-clients-3.7.1 in repository https://gitbox.apache.org/repos/asf/pinot.git was 05d116575e Bump org.apache.kafka:kafka-clients from 2.8.2 to 3.7.1

Re: [PR] roundDecimal() fails for null FLOAT values [pinot]

2025-03-28 Thread via GitHub
codecov-commenter commented on PR #15377: URL: https://github.com/apache/pinot/pull/15377#issuecomment-2760417546 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15377?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Emit metrics for each query error code returned by the broker [pinot]

2025-03-28 Thread via GitHub
krishan1390 commented on code in PR #15387: URL: https://github.com/apache/pinot/pull/15387#discussion_r2018063221 ## pinot-common/src/main/java/org/apache/pinot/common/metrics/BrokerMetrics.java: ## @@ -84,4 +91,23 @@ protected BrokerMeter[] getMeters() { protected BrokerGau

Re: [PR] Disable reingestion for Pauseless dedup [pinot]

2025-03-28 Thread via GitHub
KKcorps merged PR #15383: URL: https://github.com/apache/pinot/pull/15383 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac