[GitHub] [pinot] codecov-commenter commented on pull request #10136: Decouple dependency on using peer segment download scheme for using split commits in table

codecov-commenter commented on PR #10136: URL: https://github.com/apache/pinot/pull/10136#issuecomment-1386652338 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10136?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] zhtaoxiang commented on a diff in pull request #10132: add API to get progress of subtasks with given state

zhtaoxiang commented on code in PR #10132: URL: https://github.com/apache/pinot/pull/10132#discussion_r1073214804 ## pinot-minion/src/main/java/org/apache/pinot/minion/api/resources/PinotTaskProgressResource.java: ## @@ -84,4 +86,34 @@ public String getSubtaskProgress(

[GitHub] [pinot] zhtaoxiang commented on a diff in pull request #10132: add API to get progress of subtasks with given state

zhtaoxiang commented on code in PR #10132: URL: https://github.com/apache/pinot/pull/10132#discussion_r1073214804 ## pinot-minion/src/main/java/org/apache/pinot/minion/api/resources/PinotTaskProgressResource.java: ## @@ -84,4 +86,34 @@ public String getSubtaskProgress(

[GitHub] [pinot] ankitsultana closed pull request #9388: [Draft] [multistage] Add Partitioning Keys for Table Scan Node

ankitsultana closed pull request #9388: [Draft] [multistage] Add Partitioning Keys for Table Scan Node URL: https://github.com/apache/pinot/pull/9388 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [pinot] ankitsultana opened a new issue, #10137: Determine Number of Rows for a Realtime Segment Based on a Config

ankitsultana opened a new issue, #10137: URL: https://github.com/apache/pinot/issues/10137 At present we have two types of configs: 1. Set threshold based on size 2. Set threshold based on number of rows Regarding the second type, while the value set in those configs is prop

[GitHub] [pinot] Ferix9288 opened a new issue, #10138: JMX To Prom Exporter - More Inclusive Table Naming

Ferix9288 opened a new issue, #10138: URL: https://github.com/apache/pinot/issues/10138 # The Issue For our company usecase, we actually use UUIDs for our table names. Ex: cfec8068-c04e-4131-8c29-1e4850b5c015_OFFLINE However, the current JMX Prom Java agent config looks like b

[GitHub] [pinot] Ferix9288 opened a new pull request, #10139: JMX to Prom Exporter - More Inclusive Table Regex

Ferix9288 opened a new pull request, #10139: URL: https://github.com/apache/pinot/pull/10139 See issue https://github.com/apache/pinot/issues/10138 Effectively, the idea is to allow table names that include characters such as `-` be properly exported. -- This is an automated messag

[GitHub] [pinot] ankitsultana opened a new issue, #10140: [multistage] Expensive RoundRobinScheduler::hasNext Calls

ankitsultana opened a new issue, #10140: URL: https://github.com/apache/pinot/issues/10140 **Context** When the `Monitor` in `RoundRobinScheduler` leaves, it will call `Monitor#signalNextWaiter` which will call `RoundRobinScheduler#hasNext` since the `signalNextWaiter` method iterate

[GitHub] [pinot] ankitsultana opened a new pull request, #10141: [multistage] Optimize hasNext in RoundRobinScheduler

ankitsultana opened a new pull request, #10141: URL: https://github.com/apache/pinot/pull/10141 Details in #10140 Open to alternatives and discussions. cc: @agavra @walterddr -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [pinot] ankitsultana commented on a diff in pull request #10135: [multistage] make QueryServer async process query

ankitsultana commented on code in PR #10135: URL: https://github.com/apache/pinot/pull/10135#discussion_r1073672483 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/service/QueryServer.java: ## @@ -89,25 +88,12 @@ public void submit(Worker.QueryRequest request, Strea

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073676199 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/function/InternalReduceFunctions.java: ## @@ -0,0 +1,45 @@ +/** + * Licensed to the Apache S

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073688833 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/function/InternalReduceFunctions.java: ## @@ -0,0 +1,45 @@ +/** + * Licensed to the Apache S

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073691592 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/function/InternalReduceFunctions.java: ## @@ -0,0 +1,45 @@ +/** + * Licensed to the Apache S

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073691592 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/function/InternalReduceFunctions.java: ## @@ -0,0 +1,45 @@ +/** + * Licensed to the Apache S

[GitHub] [pinot] ankitsultana commented on issue #10140: [multistage] Expensive RoundRobinScheduler::hasNext Calls

ankitsultana commented on issue #10140: URL: https://github.com/apache/pinot/issues/10140#issuecomment-1387268763 @agavra : Also, do you think we can have default scheduler release timeout of a few minutes? The scenario where the stages have been dispatched and one of the stages dies, leadi

[GitHub] [pinot] walterddr commented on a diff in pull request #10135: [multistage] make QueryServer async process query

walterddr commented on code in PR #10135: URL: https://github.com/apache/pinot/pull/10135#discussion_r1073713700 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/service/QueryServer.java: ## @@ -89,25 +88,12 @@ public void submit(Worker.QueryRequest request, StreamOb

[GitHub] [pinot] agavra commented on issue #10140: [multistage] Expensive RoundRobinScheduler::hasNext Calls

agavra commented on issue #10140: URL: https://github.com/apache/pinot/issues/10140#issuecomment-1387317550 Nice catch @ankitsultana - for your second question, we might want to make the default be the default query timeout x2 or something like that. That way it only gets triggered to clean

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073749087 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/AggregationFunctionType.java: ## @@ -65,6 +65,7 @@ public enum AggregationFunctionType { STDD

[GitHub] [pinot] agavra commented on issue #10140: [multistage] Expensive RoundRobinScheduler::hasNext Calls

agavra commented on issue #10140: URL: https://github.com/apache/pinot/issues/10140#issuecomment-1387335390 I think we might want to make `onDataAvailable` non-blocking (e.g. doesn't go through `_monitor`). That makes it much more complicated and error prone for the implementation (e.g. `_s

[GitHub] [pinot] agavra commented on pull request #10141: [multistage] Optimize hasNext in RoundRobinScheduler

agavra commented on PR #10141: URL: https://github.com/apache/pinot/pull/10141#issuecomment-1387336117 Let's continue the conversation for alternatives in the issue ticket (for anyone watching). -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [pinot] walterddr merged pull request #10141: [multistage] Optimize hasNext in RoundRobinScheduler

walterddr merged PR #10141: URL: https://github.com/apache/pinot/pull/10141 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

[pinot] branch master updated: [multistage] Optimize hasNext in RoundRobinScheduler (#10141)

This is an automated email from the ASF dual-hosted git repository. rongr pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new eaa60d93fb [multistage] Optimize hasNext in RoundR

[GitHub] [pinot] walterddr commented on pull request #10135: [multistage] make QueryServer async process query

walterddr commented on PR #10135: URL: https://github.com/apache/pinot/pull/10135#issuecomment-1387353323 CC @agavra @61yao -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073781198 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ +

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073784131 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ +

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073802214 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ +

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073802214 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ +

[GitHub] [pinot] 61yao commented on a diff in pull request #10117: [multistage] adding async submission

61yao commented on code in PR #10117: URL: https://github.com/apache/pinot/pull/10117#discussion_r1072987281 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/service/QueryDispatcher.java: ## @@ -86,19 +90,29 @@ public int submit(long requestId, QueryPlan queryPlan, lo

[GitHub] [pinot] 61yao commented on a diff in pull request #10123: [multistage][refactor] Introduce MultiStageOperator with close() API

61yao commented on code in PR #10123: URL: https://github.com/apache/pinot/pull/10123#discussion_r1073830894 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MultiStageOperator.java: ## @@ -0,0 +1,49 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] [pinot] siddharthteotia merged pull request #10123: [multistage][refactor] Introduce MultiStageOperator with close() API

siddharthteotia merged PR #10123: URL: https://github.com/apache/pinot/pull/10123 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pi

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10123: [multistage][refactor] Introduce MultiStageOperator with close() API

siddharthteotia commented on code in PR #10123: URL: https://github.com/apache/pinot/pull/10123#discussion_r1073837664 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MultiStageOperator.java: ## @@ -0,0 +1,49 @@ +/** + * Licensed to the Apache Softwa

[pinot] branch master updated: [multistage][refactor] Introduce MultiStageOperator with close() API (#10123)

This is an automated email from the ASF dual-hosted git repository. siddteotia pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 58b113225a [multistage][refactor] Introduce M

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10128: Support for DistinctSumMV and DistinctAvgMV aggregation functions

siddharthteotia commented on code in PR #10128: URL: https://github.com/apache/pinot/pull/10128#discussion_r1073857754 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/BaseDistinctAggregateAggregationFunction.java: ## @@ -136,8 +202,84 @@ public void

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10128: Support for DistinctSumMV and DistinctAvgMV aggregation functions

siddharthteotia commented on code in PR #10128: URL: https://github.com/apache/pinot/pull/10128#discussion_r1073858511 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/BaseDistinctAggregateAggregationFunction.java: ## @@ -259,68 +478,90 @@ public void

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10128: Support for DistinctSumMV and DistinctAvgMV aggregation functions

siddharthteotia commented on code in PR #10128: URL: https://github.com/apache/pinot/pull/10128#discussion_r1073859554 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/DistinctSumMVAggregationFunction.java: ## @@ -0,0 +1,73 @@ +/** + * Licensed to the

[GitHub] [pinot] siddharthteotia commented on pull request #10128: Support for DistinctSumMV and DistinctAvgMV aggregation functions

siddharthteotia commented on PR #10128: URL: https://github.com/apache/pinot/pull/10128#issuecomment-1387465400 Mostly code is moved around. Just minor comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [pinot] klsince commented on a diff in pull request #10089: Reload conditional skip

klsince commented on code in PR #10089: URL: https://github.com/apache/pinot/pull/10089#discussion_r1073866912 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java: ## @@ -403,6 +419,16 @@ public void reloadSegment(String segmentName, IndexLo

[GitHub] [pinot] saurabhd336 commented on a diff in pull request #10089: Reload conditional skip

saurabhd336 commented on code in PR #10089: URL: https://github.com/apache/pinot/pull/10089#discussion_r1073882373 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessor.java: ## @@ -207,11 +207,7 @@ private boolean needProce

[GitHub] [pinot] klsince commented on a diff in pull request #10089: Reload conditional skip

klsince commented on code in PR #10089: URL: https://github.com/apache/pinot/pull/10089#discussion_r1073947901 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessor.java: ## @@ -207,11 +207,7 @@ private boolean needProcessSt

[GitHub] [pinot] walterddr commented on a diff in pull request #10122: [multistage] Push Down Project for Aggregation With No Grouping Sets

walterddr commented on code in PR #10122: URL: https://github.com/apache/pinot/pull/10122#discussion_r1073965486 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotRuleUtils.java: ## @@ -23,22 +23,28 @@ import org.apache.calcite.rel.RelNode; import org.apac

[GitHub] [pinot] walterddr commented on a diff in pull request #10122: [multistage] Push Down Project for Aggregation With No Grouping Sets

walterddr commented on code in PR #10122: URL: https://github.com/apache/pinot/pull/10122#discussion_r1073965486 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotRuleUtils.java: ## @@ -23,22 +23,28 @@ import org.apache.calcite.rel.RelNode; import org.apac

[GitHub] [pinot] walterddr commented on a diff in pull request #10122: [multistage] Push Down Project for Aggregation With No Grouping Sets

walterddr commented on code in PR #10122: URL: https://github.com/apache/pinot/pull/10122#discussion_r1073971528 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotQueryRuleSets.java: ## @@ -43,6 +43,8 @@ private PinotQueryRuleSets() { CoreRules.FI

[GitHub] [pinot] walterddr commented on a diff in pull request #10122: [multistage] Push Down Project for Aggregation With No Grouping Sets

walterddr commented on code in PR #10122: URL: https://github.com/apache/pinot/pull/10122#discussion_r1073971528 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotQueryRuleSets.java: ## @@ -43,6 +43,8 @@ private PinotQueryRuleSets() { CoreRules.FI

[GitHub] [pinot] agavra commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

agavra commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073974390 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ + {"name"

[GitHub] [pinot] agavra commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

agavra commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073975015 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ + {"name"

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073980628 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ +

[GitHub] [pinot] agavra commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

agavra commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073981132 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/AggregationFunctionType.java: ## @@ -65,6 +65,7 @@ public enum AggregationFunctionType { STDDEVSAMP("s

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073981667 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ +

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073749087 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/AggregationFunctionType.java: ## @@ -65,6 +65,7 @@ public enum AggregationFunctionType { STDD

[GitHub] [pinot] agavra commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

agavra commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073983925 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/function/InternalReduceFunctions.java: ## @@ -0,0 +1,45 @@ +/** + * Licensed to the Apache Software F

[GitHub] [pinot] walterddr commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

walterddr commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073985700 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ + {"na

[GitHub] [pinot] walterddr commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

walterddr commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073987193 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/AggregationFunctionType.java: ## @@ -65,6 +65,7 @@ public enum AggregationFunctionType { STDDEVSAMP

[GitHub] [pinot] walterddr commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

walterddr commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073987541 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/function/InternalReduceFunctions.java: ## @@ -0,0 +1,45 @@ +/** + * Licensed to the Apache Softwar

[pinot] branch master updated: Add RealtimeIngestionDelayMs metric to JMX exporter (#10134)

This is an automated email from the ASF dual-hosted git repository. snlee pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new df66d83eed Add RealtimeIngestionDelayMs metric to

[GitHub] [pinot] snleee merged pull request #10134: Add RealtimeIngestionDelayMs metric to JMX exporter

snleee merged PR #10134: URL: https://github.com/apache/pinot/pull/10134 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

[GitHub] [pinot] snleee commented on pull request #10134: Add RealtimeIngestionDelayMs metric to JMX exporter

snleee commented on PR #10134: URL: https://github.com/apache/pinot/pull/10134#issuecomment-1387729221 Merging it because the trivy test was failing while the other tests passed. Also, we increased the timeout setting. -- This is an automated message from the Apache Git Service. To respon

[GitHub] [pinot] dependabot[bot] opened a new pull request, #10143: Bump testng from 6.11 to 7.7.0

dependabot[bot] opened a new pull request, #10143: URL: https://github.com/apache/pinot/pull/10143 Bumps [testng](https://github.com/cbeust/testng) from 6.11 to 7.7.0. Release notes Sourced from https://github.com/cbeust/testng/releases";>testng's releases. TestNG v7.7.0 W

[pinot] branch dependabot/maven/org.testng-testng-7.7.0 created (now d5482f5114)

This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.testng-testng-7.7.0 in repository https://gitbox.apache.org/repos/asf/pinot.git at d5482f5114 Bump testng from 6.11 to 7.7.0 No new revisions were added by this up

[GitHub] [pinot] codecov-commenter commented on pull request #10143: Bump testng from 6.11 to 7.7.0

codecov-commenter commented on PR #10143: URL: https://github.com/apache/pinot/pull/10143#issuecomment-1396086138 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10143?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] navina opened a new pull request, #10144: Adding duration unit in trivy timeout

navina opened a new pull request, #10144: URL: https://github.com/apache/pinot/pull/10144 Fixing an error in the previous PR (https://github.com/apache/pinot/pull/10134) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [pinot] agavra commented on a diff in pull request #10094: [multistage] [debuggability] OpChain and operator stats

agavra commented on code in PR #10094: URL: https://github.com/apache/pinot/pull/10094#discussion_r1080594649 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/SortOperator.java: ## @@ -134,8 +150,10 @@ private void consumeInputBlocks() { for

[GitHub] [pinot] agavra commented on a diff in pull request #10094: [multistage] [debuggability] OpChain and operator stats

agavra commented on code in PR #10094: URL: https://github.com/apache/pinot/pull/10094#discussion_r1080595360 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/SortOperator.java: ## @@ -134,8 +150,10 @@ private void consumeInputBlocks() { for

[GitHub] [pinot] agavra commented on a diff in pull request #10094: [multistage] [debuggability] OpChain and operator stats

agavra commented on code in PR #10094: URL: https://github.com/apache/pinot/pull/10094#discussion_r1080596901 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/OperatorStats.java: ## @@ -0,0 +1,77 @@ +/** + * Licensed to the Apache Software Foundation

[GitHub] [pinot] agavra commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

agavra commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1080614690 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/function/InternalReduceFunctions.java: ## @@ -0,0 +1,45 @@ +/** + * Licensed to the Apache Software F

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1080618239 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/AggregationFunctionType.java: ## @@ -65,6 +65,7 @@ public enum AggregationFunctionType { STDD

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1080618390 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ +

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1080620242 ## pinot-common/src/main/java/org/apache/pinot/common/CustomObject.java: ## @@ -0,0 +1,43 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1080622521 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotReduceAggregateFunctionsRule.java: ## @@ -0,0 +1,201 @@ +/** + * Licensed to the Apache S

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1080622521 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotReduceAggregateFunctionsRule.java: ## @@ -0,0 +1,201 @@ +/** + * Licensed to the Apache S

[GitHub] [pinot] codecov-commenter commented on pull request #10144: Adding duration unit in trivy timeout

codecov-commenter commented on PR #10144: URL: https://github.com/apache/pinot/pull/10144#issuecomment-1396178575 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10144?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] Jackie-Jiang commented on issue #10137: Determine Number of Rows for a Realtime Segment Based on a Config

Jackie-Jiang commented on issue #10137: URL: https://github.com/apache/pinot/issues/10137#issuecomment-1396181623 cc @mcvsubbu @siddharthteotia @sajjad-moradi +1 on this. Dividing the number of kafka partitions on a server always confuses me, which makes the stream very hard to confi

[GitHub] [pinot] navina commented on pull request #10144: Adding duration unit in trivy timeout

navina commented on PR #10144: URL: https://github.com/apache/pinot/pull/10144#issuecomment-1396183165 @snleee / @gortiz / @xiangfu0 : are we ok with moving the trivy check to a nightly trigger, rather than on every commit and PR? We can add a badge in our repo to indicate the status of t

[GitHub] [pinot] Jackie-Jiang commented on pull request #10139: JMX to Prom Exporter - More Inclusive Table Regex

Jackie-Jiang commented on PR #10139: URL: https://github.com/apache/pinot/pull/10139#issuecomment-1396204741 Good catch! Table name can actually contain spaces, so we can probably just match any character. There are other regex that do not contain the table type, which should also be

[GitHub] [pinot] Jackie-Jiang merged pull request #10090: fix(ui): update table status to inline with new segment status

Jackie-Jiang merged PR #10090: URL: https://github.com/apache/pinot/pull/10090 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[pinot] branch master updated: fix(ui): update table status to inline with new segment status (#10090)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 93bfc527e9 fix(ui): update table status to inline

[GitHub] [pinot] Jackie-Jiang closed pull request #10143: Bump testng from 6.11 to 7.7.0

Jackie-Jiang closed pull request #10143: Bump testng from 6.11 to 7.7.0 URL: https://github.com/apache/pinot/pull/10143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

[GitHub] [pinot] dependabot[bot] commented on pull request #10143: Bump testng from 6.11 to 7.7.0

dependabot[bot] commented on PR #10143: URL: https://github.com/apache/pinot/pull/10143#issuecomment-1396212857 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

[GitHub] [pinot] mcvsubbu commented on issue #10137: Determine Number of Rows for a Realtime Segment Based on a Config

mcvsubbu commented on issue #10137: URL: https://github.com/apache/pinot/issues/10137#issuecomment-1396217784 @ankitsultana anda @Jackie-Jiang Pinot serves queries while consuming rows. In order to have enough CPU left over to process queries, it is best to limit the consumption rate to so

[GitHub] [pinot] Jackie-Jiang commented on issue #10137: Determine Number of Rows for a Realtime Segment Based on a Config

Jackie-Jiang commented on issue #10137: URL: https://github.com/apache/pinot/issues/10137#issuecomment-1396237583 @mcvsubbu The proposed way is very useful when we want to limit the CPU usage for all the consumers on a given server, but for certain use cases, CPU might not be the bottleneck

[GitHub] [pinot] jtao15 commented on a diff in pull request #10136: Decouple dependency on using peer segment download scheme for doing metadata pushes to controller

jtao15 commented on code in PR #10136: URL: https://github.com/apache/pinot/pull/10136#discussion_r1080680161 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/SegmentCommitterFactory.java: ## @@ -46,25 +46,34 @@ public SegmentCommitterFactory(Logger segmen

[GitHub] [pinot] Jackie-Jiang merged pull request #10087: [Clean up] Remove the support for non-zero padding

Jackie-Jiang merged PR #10087: URL: https://github.com/apache/pinot/pull/10087 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[pinot] branch master updated: [Clean up] Remove the support for non-zero padding (#10087)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 2d43123026 [Clean up] Remove the support for non-

[GitHub] [pinot] snleee commented on pull request #10144: Adding duration unit in trivy timeout

snleee commented on PR #10144: URL: https://github.com/apache/pinot/pull/10144#issuecomment-1396273909 I'm fine with the change but we will need to make one time effort to clean up the test. I think that may involve in bumping up multiple libraries. -- This is an automated message from th

[GitHub] [pinot] walterddr commented on a diff in pull request #10144: Adding duration unit in trivy timeout

walterddr commented on code in PR #10144: URL: https://github.com/apache/pinot/pull/10144#discussion_r1080695552 ## .github/workflows/pinot_vuln_check.yml: ## @@ -19,18 +19,8 @@ name: Pinot Dependencies on: - push: -branches: - - master - pull_request: -branch

[GitHub] [pinot] vvivekiyer commented on pull request #10128: Support for DistinctSumMV and DistinctAvgMV aggregation functions

vvivekiyer commented on PR #10128: URL: https://github.com/apache/pinot/pull/10128#issuecomment-1396304548 Addressed review comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [pinot] 61yao commented on a diff in pull request #10094: [multistage] [debuggability] OpChain and operator stats

61yao commented on code in PR #10094: URL: https://github.com/apache/pinot/pull/10094#discussion_r1080712321 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/SortOperator.java: ## @@ -134,8 +150,10 @@ private void consumeInputBlocks() { for (

[GitHub] [pinot] 61yao commented on a diff in pull request #10094: [multistage] [debuggability] OpChain and operator stats

61yao commented on code in PR #10094: URL: https://github.com/apache/pinot/pull/10094#discussion_r1080713806 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/OperatorStats.java: ## @@ -0,0 +1,77 @@ +/** + * Licensed to the Apache Software Foundation (

[GitHub] [pinot] mcvsubbu commented on issue #10137: Determine Number of Rows for a Realtime Segment Based on a Config

mcvsubbu commented on issue #10137: URL: https://github.com/apache/pinot/issues/10137#issuecomment-1396307559 > @mcvsubbu The proposed way is very useful when we want to limit the CPU usage for all the consumers on a given server, but for certain use cases, CPU might not be the bottleneck (

[GitHub] [pinot] siddharthteotia commented on pull request #10128: Support for DistinctSumMV and DistinctAvgMV aggregation functions

siddharthteotia commented on PR #10128: URL: https://github.com/apache/pinot/pull/10128#issuecomment-1396312237 only text / javadoc changed in the latest commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [pinot] siddharthteotia merged pull request #10128: Support for DistinctSumMV and DistinctAvgMV aggregation functions

siddharthteotia merged PR #10128: URL: https://github.com/apache/pinot/pull/10128 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pi

[pinot] branch master updated: Support for DistinctSumMV and DistinctAvgMV aggregation functions (#10128)

This is an automated email from the ASF dual-hosted git repository. siddteotia pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 5e76b100b9 Support for DistinctSumMV and Dist

[GitHub] [pinot] siddharthteotia closed issue #10109: Support DistinctSum and DistinctAvg aggregation functions for MV columns

siddharthteotia closed issue #10109: Support DistinctSum and DistinctAvg aggregation functions for MV columns URL: https://github.com/apache/pinot/issues/10109 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [pinot] siddharthteotia merged pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia merged PR #10120: URL: https://github.com/apache/pinot/pull/10120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pi

[pinot] branch master updated: [multistage] support aggregations that require intermediate representations (#10120)

This is an automated email from the ASF dual-hosted git repository. siddteotia pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 1231a2c985 [multistage] support aggregations

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1080718243 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/AggregationFunctionType.java: ## @@ -65,6 +65,7 @@ public enum AggregationFunctionType { STDD

[GitHub] [pinot] xiangfu0 opened a new pull request, #10145: Fix docker build script tagging

xiangfu0 opened a new pull request, #10145: URL: https://github.com/apache/pinot/pull/10145 Fix the tagging variables generation from the build code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [pinot] Jackie-Jiang commented on issue #10137: Determine Number of Rows for a Realtime Segment Based on a Config

Jackie-Jiang commented on issue #10137: URL: https://github.com/apache/pinot/issues/10137#issuecomment-1396323369 Most users are not using the recommendation engine though. What I'm trying to convey is that currently there is not a straight forward way to configure the rows per segment. Fix

[GitHub] [pinot] xiangfu0 merged pull request #10145: Fix docker build script tagging

xiangfu0 merged PR #10145: URL: https://github.com/apache/pinot/pull/10145 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[pinot] branch master updated: Fix docker build script tagging (#10145)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 21c6532564 Fix docker build script tagging (#101

[GitHub] [pinot] siddharthteotia commented on issue #9277: [Feature] Support for Correlation Function

siddharthteotia commented on issue #9277: URL: https://github.com/apache/pinot/issues/9277#issuecomment-1396328556 May be you are running into divide by 0 problem ? Did you try step into the code to understand where it is turning into NaN ? cc @jasperjiaguo / @SabrinaZhaozyf -- T

  1   2   >