[GitHub] [pinot] gortiz commented on pull request #10144: Adding duration unit in trivy timeout

2023-01-18 Thread GitBox
gortiz commented on PR #10144: URL: https://github.com/apache/pinot/pull/10144#issuecomment-1396567329 I understand that the initial idea was to increase the timeout but we are using this PR to change the way trivy is triggered. Therefore I would suggest to change the title of the PR --

[GitHub] [pinot] gortiz commented on a diff in pull request #10144: Adding duration unit in trivy timeout

2023-01-18 Thread GitBox
gortiz commented on code in PR #10144: URL: https://github.com/apache/pinot/pull/10144#discussion_r1080906416 ## .github/workflows/pinot_vuln_check.yml: ## @@ -19,18 +19,8 @@ name: Pinot Dependencies on: - push: -branches: - - master - pull_request: -branches:

[GitHub] [pinot] codecov-commenter commented on pull request #10150: Fix json-smart version

2023-01-18 Thread GitBox
codecov-commenter commented on PR #10150: URL: https://github.com/apache/pinot/pull/10150#issuecomment-1396548970 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10150?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] codecov-commenter commented on pull request #10149: Fixing the metadata tar gz file path

2023-01-18 Thread GitBox
codecov-commenter commented on PR #10149: URL: https://github.com/apache/pinot/pull/10149#issuecomment-1396526998 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10149?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] zhtaoxiang commented on a diff in pull request #10106: clean up the logic of reading guage metrics

2023-01-18 Thread GitBox
zhtaoxiang commented on code in PR #10106: URL: https://github.com/apache/pinot/pull/10106#discussion_r1080869461 ## pinot-common/src/test/java/org/apache/pinot/common/metrics/MetricValueUtils.java: ## @@ -0,0 +1,128 @@ +/** + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [pinot] xiangfu0 opened a new pull request, #10150: Fix json-smart version

2023-01-18 Thread GitBox
xiangfu0 opened a new pull request, #10150: URL: https://github.com/apache/pinot/pull/10150 Instructions: 1. The PR has to be tagged with at least one of the following labels (*): 1. `feature` 2. `bugfix` 3. `performance` 4. `ui` 5. `backward-incompat` 6

[GitHub] [pinot] xiangfu0 commented on pull request #10144: Adding duration unit in trivy timeout

2023-01-18 Thread GitBox
xiangfu0 commented on PR #10144: URL: https://github.com/apache/pinot/pull/10144#issuecomment-1396496351 My only concern is that who will be responsible for the trivy check results and version bump. -- This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [pinot] codecov-commenter commented on pull request #10148: [multistage] flag all columns as nullable

2023-01-18 Thread GitBox
codecov-commenter commented on PR #10148: URL: https://github.com/apache/pinot/pull/10148#issuecomment-1396491709 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10148?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] navina commented on a diff in pull request #10144: Adding duration unit in trivy timeout

2023-01-18 Thread GitBox
navina commented on code in PR #10144: URL: https://github.com/apache/pinot/pull/10144#discussion_r1080845287 ## .github/workflows/pinot_vuln_check.yml: ## @@ -19,18 +19,8 @@ name: Pinot Dependencies on: - push: -branches: - - master - pull_request: -branches:

[GitHub] [pinot] navina commented on pull request #10136: Decouple dependency on using peer segment download scheme for doing metadata pushes to controller

2023-01-18 Thread GitBox
navina commented on PR #10136: URL: https://github.com/apache/pinot/pull/10136#issuecomment-1396486769 @Jackie-Jiang I don't think we need 2 new configs, as we discussed. We can make do with 1 new config. Please review! -- This is an automated message from the Apache Git Service. To resp

[GitHub] [pinot] xiangfu0 opened a new pull request, #10149: Fixing the metadata tar gz file path

2023-01-18 Thread GitBox
xiangfu0 opened a new pull request, #10149: URL: https://github.com/apache/pinot/pull/10149 Fixing https://apache-pinot.slack.com/archives/C011C9JHN7R/p1674101384733239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [pinot] walterddr opened a new pull request, #10148: [multistage] flag all columns as nullable

2023-01-18 Thread GitBox
walterddr opened a new pull request, #10148: URL: https://github.com/apache/pinot/pull/10148 currently, there's no way to tell whether a table column is nullable setting the nullability for all columns on v2 engine as default true (previously default false), b/c - having nullabili

[GitHub] [pinot] shwin commented on a diff in pull request #10034: Create metadata only tarball for metadata push job

2023-01-18 Thread GitBox
shwin commented on code in PR #10034: URL: https://github.com/apache/pinot/pull/10034#discussion_r1080812390 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/SegmentPushUtils.java: ## @@ -284,7 +284,22 @@ public static void sendSegmentUriAndMetadata(Seg

[GitHub] [pinot] shwin commented on a diff in pull request #10034: Create metadata only tarball for metadata push job

2023-01-18 Thread GitBox
shwin commented on code in PR #10034: URL: https://github.com/apache/pinot/pull/10034#discussion_r1080811688 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/SegmentPushUtils.java: ## @@ -284,7 +284,22 @@ public static void sendSegmentUriAndMetadata(Seg

[GitHub] [pinot] jasperjiaguo commented on issue #9277: [Feature] Support for Correlation Function

2023-01-18 Thread GitBox
jasperjiaguo commented on issue #9277: URL: https://github.com/apache/pinot/issues/9277#issuecomment-1396440724 NaN could be due to Math.sqrt(negative_number) or 0.0/0.0 We have recently discovered this impl of covariance/correlation has numerical stability issue when E[x^2] ~ E[x]^2 >>

[GitHub] [pinot] sajjad-moradi commented on issue #10147: Pauseless Consumption

2023-01-18 Thread GitBox
sajjad-moradi commented on issue #10147: URL: https://github.com/apache/pinot/issues/10147#issuecomment-1396417093 CC: @mcvsubbu @npawar @Jackie-Jiang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [pinot] sajjad-moradi opened a new issue, #10147: Pauseless Consumption

2023-01-18 Thread GitBox
sajjad-moradi opened a new issue, #10147: URL: https://github.com/apache/pinot/issues/10147 # Background Currently when commit protocol starts for a partition, the realtime server stops consumption and till the new consuming->online helix message for the next consuming segment arrives, m

[GitHub] [pinot] snleee opened a new issue, #10146: Add the support for `STDDEV_POP/SAMP, VAR_POP/DEV` for star tree

2023-01-18 Thread GitBox
snleee opened a new issue, #10146: URL: https://github.com/apache/pinot/issues/10146 We recently added the `STDDEV_POP/SAMP, VAR_POP/DEV` aggregation function. It would be great if we can support this in the star tree index. We need to implement `ValueAggregator` interface for stddev

[GitHub] [pinot] mcvsubbu commented on issue #10137: Determine Number of Rows for a Realtime Segment Based on a Config

2023-01-18 Thread GitBox
mcvsubbu commented on issue #10137: URL: https://github.com/apache/pinot/issues/10137#issuecomment-1396336852 We should really think about configs that add no value. IMO this config adds no value. I think the non-expert users will come back asking us to tune this number because they rebalan

[GitHub] [pinot] siddharthteotia commented on issue #9277: [Feature] Support for Correlation Function

2023-01-18 Thread GitBox
siddharthteotia commented on issue #9277: URL: https://github.com/apache/pinot/issues/9277#issuecomment-1396328556 May be you are running into divide by 0 problem ? Did you try step into the code to understand where it is turning into NaN ? cc @jasperjiaguo / @SabrinaZhaozyf -- T

[pinot] branch master updated: Fix docker build script tagging (#10145)

2023-01-18 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 21c6532564 Fix docker build script tagging (#101

[GitHub] [pinot] xiangfu0 merged pull request #10145: Fix docker build script tagging

2023-01-18 Thread GitBox
xiangfu0 merged PR #10145: URL: https://github.com/apache/pinot/pull/10145 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[GitHub] [pinot] Jackie-Jiang commented on issue #10137: Determine Number of Rows for a Realtime Segment Based on a Config

2023-01-18 Thread GitBox
Jackie-Jiang commented on issue #10137: URL: https://github.com/apache/pinot/issues/10137#issuecomment-1396323369 Most users are not using the recommendation engine though. What I'm trying to convey is that currently there is not a straight forward way to configure the rows per segment. Fix

[GitHub] [pinot] xiangfu0 opened a new pull request, #10145: Fix docker build script tagging

2023-01-18 Thread GitBox
xiangfu0 opened a new pull request, #10145: URL: https://github.com/apache/pinot/pull/10145 Fix the tagging variables generation from the build code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1080718243 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/AggregationFunctionType.java: ## @@ -65,6 +65,7 @@ public enum AggregationFunctionType { STDD

[pinot] branch master updated: [multistage] support aggregations that require intermediate representations (#10120)

2023-01-18 Thread siddteotia
This is an automated email from the ASF dual-hosted git repository. siddteotia pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 1231a2c985 [multistage] support aggregations

[GitHub] [pinot] siddharthteotia merged pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
siddharthteotia merged PR #10120: URL: https://github.com/apache/pinot/pull/10120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pi

[GitHub] [pinot] siddharthteotia closed issue #10109: Support DistinctSum and DistinctAvg aggregation functions for MV columns

2023-01-18 Thread GitBox
siddharthteotia closed issue #10109: Support DistinctSum and DistinctAvg aggregation functions for MV columns URL: https://github.com/apache/pinot/issues/10109 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[pinot] branch master updated: Support for DistinctSumMV and DistinctAvgMV aggregation functions (#10128)

2023-01-18 Thread siddteotia
This is an automated email from the ASF dual-hosted git repository. siddteotia pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 5e76b100b9 Support for DistinctSumMV and Dist

[GitHub] [pinot] siddharthteotia merged pull request #10128: Support for DistinctSumMV and DistinctAvgMV aggregation functions

2023-01-18 Thread GitBox
siddharthteotia merged PR #10128: URL: https://github.com/apache/pinot/pull/10128 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pi

[GitHub] [pinot] siddharthteotia commented on pull request #10128: Support for DistinctSumMV and DistinctAvgMV aggregation functions

2023-01-18 Thread GitBox
siddharthteotia commented on PR #10128: URL: https://github.com/apache/pinot/pull/10128#issuecomment-1396312237 only text / javadoc changed in the latest commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [pinot] mcvsubbu commented on issue #10137: Determine Number of Rows for a Realtime Segment Based on a Config

2023-01-18 Thread GitBox
mcvsubbu commented on issue #10137: URL: https://github.com/apache/pinot/issues/10137#issuecomment-1396307559 > @mcvsubbu The proposed way is very useful when we want to limit the CPU usage for all the consumers on a given server, but for certain use cases, CPU might not be the bottleneck (

[GitHub] [pinot] 61yao commented on a diff in pull request #10094: [multistage] [debuggability] OpChain and operator stats

2023-01-18 Thread GitBox
61yao commented on code in PR #10094: URL: https://github.com/apache/pinot/pull/10094#discussion_r1080713806 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/OperatorStats.java: ## @@ -0,0 +1,77 @@ +/** + * Licensed to the Apache Software Foundation (

[GitHub] [pinot] 61yao commented on a diff in pull request #10094: [multistage] [debuggability] OpChain and operator stats

2023-01-18 Thread GitBox
61yao commented on code in PR #10094: URL: https://github.com/apache/pinot/pull/10094#discussion_r1080712321 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/SortOperator.java: ## @@ -134,8 +150,10 @@ private void consumeInputBlocks() { for (

[GitHub] [pinot] vvivekiyer commented on pull request #10128: Support for DistinctSumMV and DistinctAvgMV aggregation functions

2023-01-18 Thread GitBox
vvivekiyer commented on PR #10128: URL: https://github.com/apache/pinot/pull/10128#issuecomment-1396304548 Addressed review comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [pinot] walterddr commented on a diff in pull request #10144: Adding duration unit in trivy timeout

2023-01-18 Thread GitBox
walterddr commented on code in PR #10144: URL: https://github.com/apache/pinot/pull/10144#discussion_r1080695552 ## .github/workflows/pinot_vuln_check.yml: ## @@ -19,18 +19,8 @@ name: Pinot Dependencies on: - push: -branches: - - master - pull_request: -branch

[GitHub] [pinot] snleee commented on pull request #10144: Adding duration unit in trivy timeout

2023-01-18 Thread GitBox
snleee commented on PR #10144: URL: https://github.com/apache/pinot/pull/10144#issuecomment-1396273909 I'm fine with the change but we will need to make one time effort to clean up the test. I think that may involve in bumping up multiple libraries. -- This is an automated message from th

[pinot] branch master updated: [Clean up] Remove the support for non-zero padding (#10087)

2023-01-18 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 2d43123026 [Clean up] Remove the support for non-

[GitHub] [pinot] Jackie-Jiang merged pull request #10087: [Clean up] Remove the support for non-zero padding

2023-01-18 Thread GitBox
Jackie-Jiang merged PR #10087: URL: https://github.com/apache/pinot/pull/10087 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] jtao15 commented on a diff in pull request #10136: Decouple dependency on using peer segment download scheme for doing metadata pushes to controller

2023-01-18 Thread GitBox
jtao15 commented on code in PR #10136: URL: https://github.com/apache/pinot/pull/10136#discussion_r1080680161 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/SegmentCommitterFactory.java: ## @@ -46,25 +46,34 @@ public SegmentCommitterFactory(Logger segmen

[GitHub] [pinot] Jackie-Jiang commented on issue #10137: Determine Number of Rows for a Realtime Segment Based on a Config

2023-01-18 Thread GitBox
Jackie-Jiang commented on issue #10137: URL: https://github.com/apache/pinot/issues/10137#issuecomment-1396237583 @mcvsubbu The proposed way is very useful when we want to limit the CPU usage for all the consumers on a given server, but for certain use cases, CPU might not be the bottleneck

[GitHub] [pinot] mcvsubbu commented on issue #10137: Determine Number of Rows for a Realtime Segment Based on a Config

2023-01-18 Thread GitBox
mcvsubbu commented on issue #10137: URL: https://github.com/apache/pinot/issues/10137#issuecomment-1396217784 @ankitsultana anda @Jackie-Jiang Pinot serves queries while consuming rows. In order to have enough CPU left over to process queries, it is best to limit the consumption rate to so

[GitHub] [pinot] dependabot[bot] commented on pull request #10143: Bump testng from 6.11 to 7.7.0

2023-01-18 Thread GitBox
dependabot[bot] commented on PR #10143: URL: https://github.com/apache/pinot/pull/10143#issuecomment-1396212857 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

[GitHub] [pinot] Jackie-Jiang closed pull request #10143: Bump testng from 6.11 to 7.7.0

2023-01-18 Thread GitBox
Jackie-Jiang closed pull request #10143: Bump testng from 6.11 to 7.7.0 URL: https://github.com/apache/pinot/pull/10143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

[pinot] branch master updated: fix(ui): update table status to inline with new segment status (#10090)

2023-01-18 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 93bfc527e9 fix(ui): update table status to inline

[GitHub] [pinot] Jackie-Jiang merged pull request #10090: fix(ui): update table status to inline with new segment status

2023-01-18 Thread GitBox
Jackie-Jiang merged PR #10090: URL: https://github.com/apache/pinot/pull/10090 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] Jackie-Jiang commented on pull request #10139: JMX to Prom Exporter - More Inclusive Table Regex

2023-01-18 Thread GitBox
Jackie-Jiang commented on PR #10139: URL: https://github.com/apache/pinot/pull/10139#issuecomment-1396204741 Good catch! Table name can actually contain spaces, so we can probably just match any character. There are other regex that do not contain the table type, which should also be

[GitHub] [pinot] navina commented on pull request #10144: Adding duration unit in trivy timeout

2023-01-18 Thread GitBox
navina commented on PR #10144: URL: https://github.com/apache/pinot/pull/10144#issuecomment-1396183165 @snleee / @gortiz / @xiangfu0 : are we ok with moving the trivy check to a nightly trigger, rather than on every commit and PR? We can add a badge in our repo to indicate the status of t

[GitHub] [pinot] Jackie-Jiang commented on issue #10137: Determine Number of Rows for a Realtime Segment Based on a Config

2023-01-18 Thread GitBox
Jackie-Jiang commented on issue #10137: URL: https://github.com/apache/pinot/issues/10137#issuecomment-1396181623 cc @mcvsubbu @siddharthteotia @sajjad-moradi +1 on this. Dividing the number of kafka partitions on a server always confuses me, which makes the stream very hard to confi

[GitHub] [pinot] codecov-commenter commented on pull request #10144: Adding duration unit in trivy timeout

2023-01-18 Thread GitBox
codecov-commenter commented on PR #10144: URL: https://github.com/apache/pinot/pull/10144#issuecomment-1396178575 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10144?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1080622521 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotReduceAggregateFunctionsRule.java: ## @@ -0,0 +1,201 @@ +/** + * Licensed to the Apache S

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1080622521 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotReduceAggregateFunctionsRule.java: ## @@ -0,0 +1,201 @@ +/** + * Licensed to the Apache S

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1080620242 ## pinot-common/src/main/java/org/apache/pinot/common/CustomObject.java: ## @@ -0,0 +1,43 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1080618390 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ +

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1080618239 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/AggregationFunctionType.java: ## @@ -65,6 +65,7 @@ public enum AggregationFunctionType { STDD

[GitHub] [pinot] agavra commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
agavra commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1080614690 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/function/InternalReduceFunctions.java: ## @@ -0,0 +1,45 @@ +/** + * Licensed to the Apache Software F

[GitHub] [pinot] agavra commented on a diff in pull request #10094: [multistage] [debuggability] OpChain and operator stats

2023-01-18 Thread GitBox
agavra commented on code in PR #10094: URL: https://github.com/apache/pinot/pull/10094#discussion_r1080596901 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/OperatorStats.java: ## @@ -0,0 +1,77 @@ +/** + * Licensed to the Apache Software Foundation

[GitHub] [pinot] agavra commented on a diff in pull request #10094: [multistage] [debuggability] OpChain and operator stats

2023-01-18 Thread GitBox
agavra commented on code in PR #10094: URL: https://github.com/apache/pinot/pull/10094#discussion_r1080595360 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/SortOperator.java: ## @@ -134,8 +150,10 @@ private void consumeInputBlocks() { for

[GitHub] [pinot] agavra commented on a diff in pull request #10094: [multistage] [debuggability] OpChain and operator stats

2023-01-18 Thread GitBox
agavra commented on code in PR #10094: URL: https://github.com/apache/pinot/pull/10094#discussion_r1080594649 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/SortOperator.java: ## @@ -134,8 +150,10 @@ private void consumeInputBlocks() { for

[GitHub] [pinot] navina opened a new pull request, #10144: Adding duration unit in trivy timeout

2023-01-18 Thread GitBox
navina opened a new pull request, #10144: URL: https://github.com/apache/pinot/pull/10144 Fixing an error in the previous PR (https://github.com/apache/pinot/pull/10134) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [pinot] codecov-commenter commented on pull request #10143: Bump testng from 6.11 to 7.7.0

2023-01-18 Thread GitBox
codecov-commenter commented on PR #10143: URL: https://github.com/apache/pinot/pull/10143#issuecomment-1396086138 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10143?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[pinot] branch dependabot/maven/org.testng-testng-7.7.0 created (now d5482f5114)

2023-01-18 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.testng-testng-7.7.0 in repository https://gitbox.apache.org/repos/asf/pinot.git at d5482f5114 Bump testng from 6.11 to 7.7.0 No new revisions were added by this up

[GitHub] [pinot] dependabot[bot] opened a new pull request, #10143: Bump testng from 6.11 to 7.7.0

2023-01-18 Thread GitBox
dependabot[bot] opened a new pull request, #10143: URL: https://github.com/apache/pinot/pull/10143 Bumps [testng](https://github.com/cbeust/testng) from 6.11 to 7.7.0. Release notes Sourced from https://github.com/cbeust/testng/releases";>testng's releases. TestNG v7.7.0 W

[GitHub] [pinot] snleee commented on pull request #10134: Add RealtimeIngestionDelayMs metric to JMX exporter

2023-01-18 Thread GitBox
snleee commented on PR #10134: URL: https://github.com/apache/pinot/pull/10134#issuecomment-1387729221 Merging it because the trivy test was failing while the other tests passed. Also, we increased the timeout setting. -- This is an automated message from the Apache Git Service. To respon

[GitHub] [pinot] snleee merged pull request #10134: Add RealtimeIngestionDelayMs metric to JMX exporter

2023-01-18 Thread GitBox
snleee merged PR #10134: URL: https://github.com/apache/pinot/pull/10134 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

[pinot] branch master updated: Add RealtimeIngestionDelayMs metric to JMX exporter (#10134)

2023-01-18 Thread snlee
This is an automated email from the ASF dual-hosted git repository. snlee pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new df66d83eed Add RealtimeIngestionDelayMs metric to

[GitHub] [pinot] walterddr commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
walterddr commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073987541 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/function/InternalReduceFunctions.java: ## @@ -0,0 +1,45 @@ +/** + * Licensed to the Apache Softwar

[GitHub] [pinot] walterddr commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
walterddr commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073987193 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/AggregationFunctionType.java: ## @@ -65,6 +65,7 @@ public enum AggregationFunctionType { STDDEVSAMP

[GitHub] [pinot] walterddr commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
walterddr commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073985700 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ + {"na

[GitHub] [pinot] agavra commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
agavra commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073983925 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/function/InternalReduceFunctions.java: ## @@ -0,0 +1,45 @@ +/** + * Licensed to the Apache Software F

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073749087 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/AggregationFunctionType.java: ## @@ -65,6 +65,7 @@ public enum AggregationFunctionType { STDD

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073981667 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ +

[GitHub] [pinot] agavra commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
agavra commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073981132 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/AggregationFunctionType.java: ## @@ -65,6 +65,7 @@ public enum AggregationFunctionType { STDDEVSAMP("s

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073980628 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ +

[GitHub] [pinot] agavra commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
agavra commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073975015 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ + {"name"

[GitHub] [pinot] agavra commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
agavra commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073974390 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ + {"name"

[GitHub] [pinot] walterddr commented on a diff in pull request #10122: [multistage] Push Down Project for Aggregation With No Grouping Sets

2023-01-18 Thread GitBox
walterddr commented on code in PR #10122: URL: https://github.com/apache/pinot/pull/10122#discussion_r1073971528 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotQueryRuleSets.java: ## @@ -43,6 +43,8 @@ private PinotQueryRuleSets() { CoreRules.FI

[GitHub] [pinot] walterddr commented on a diff in pull request #10122: [multistage] Push Down Project for Aggregation With No Grouping Sets

2023-01-18 Thread GitBox
walterddr commented on code in PR #10122: URL: https://github.com/apache/pinot/pull/10122#discussion_r1073971528 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotQueryRuleSets.java: ## @@ -43,6 +43,8 @@ private PinotQueryRuleSets() { CoreRules.FI

[GitHub] [pinot] walterddr commented on a diff in pull request #10122: [multistage] Push Down Project for Aggregation With No Grouping Sets

2023-01-18 Thread GitBox
walterddr commented on code in PR #10122: URL: https://github.com/apache/pinot/pull/10122#discussion_r1073965486 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotRuleUtils.java: ## @@ -23,22 +23,28 @@ import org.apache.calcite.rel.RelNode; import org.apac

[GitHub] [pinot] walterddr commented on a diff in pull request #10122: [multistage] Push Down Project for Aggregation With No Grouping Sets

2023-01-18 Thread GitBox
walterddr commented on code in PR #10122: URL: https://github.com/apache/pinot/pull/10122#discussion_r1073965486 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotRuleUtils.java: ## @@ -23,22 +23,28 @@ import org.apache.calcite.rel.RelNode; import org.apac

[GitHub] [pinot] klsince commented on a diff in pull request #10089: Reload conditional skip

2023-01-18 Thread GitBox
klsince commented on code in PR #10089: URL: https://github.com/apache/pinot/pull/10089#discussion_r1073947901 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessor.java: ## @@ -207,11 +207,7 @@ private boolean needProcessSt

[GitHub] [pinot] saurabhd336 commented on a diff in pull request #10089: Reload conditional skip

2023-01-18 Thread GitBox
saurabhd336 commented on code in PR #10089: URL: https://github.com/apache/pinot/pull/10089#discussion_r1073882373 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessor.java: ## @@ -207,11 +207,7 @@ private boolean needProce

[GitHub] [pinot] klsince commented on a diff in pull request #10089: Reload conditional skip

2023-01-18 Thread GitBox
klsince commented on code in PR #10089: URL: https://github.com/apache/pinot/pull/10089#discussion_r1073866912 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java: ## @@ -403,6 +419,16 @@ public void reloadSegment(String segmentName, IndexLo

[GitHub] [pinot] siddharthteotia commented on pull request #10128: Support for DistinctSumMV and DistinctAvgMV aggregation functions

2023-01-18 Thread GitBox
siddharthteotia commented on PR #10128: URL: https://github.com/apache/pinot/pull/10128#issuecomment-1387465400 Mostly code is moved around. Just minor comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10128: Support for DistinctSumMV and DistinctAvgMV aggregation functions

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10128: URL: https://github.com/apache/pinot/pull/10128#discussion_r1073859554 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/DistinctSumMVAggregationFunction.java: ## @@ -0,0 +1,73 @@ +/** + * Licensed to the

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10128: Support for DistinctSumMV and DistinctAvgMV aggregation functions

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10128: URL: https://github.com/apache/pinot/pull/10128#discussion_r1073858511 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/BaseDistinctAggregateAggregationFunction.java: ## @@ -259,68 +478,90 @@ public void

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10128: Support for DistinctSumMV and DistinctAvgMV aggregation functions

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10128: URL: https://github.com/apache/pinot/pull/10128#discussion_r1073857754 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/BaseDistinctAggregateAggregationFunction.java: ## @@ -136,8 +202,84 @@ public void

[pinot] branch master updated: [multistage][refactor] Introduce MultiStageOperator with close() API (#10123)

2023-01-18 Thread siddteotia
This is an automated email from the ASF dual-hosted git repository. siddteotia pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 58b113225a [multistage][refactor] Introduce M

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10123: [multistage][refactor] Introduce MultiStageOperator with close() API

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10123: URL: https://github.com/apache/pinot/pull/10123#discussion_r1073837664 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MultiStageOperator.java: ## @@ -0,0 +1,49 @@ +/** + * Licensed to the Apache Softwa

[GitHub] [pinot] siddharthteotia merged pull request #10123: [multistage][refactor] Introduce MultiStageOperator with close() API

2023-01-18 Thread GitBox
siddharthteotia merged PR #10123: URL: https://github.com/apache/pinot/pull/10123 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pi

[GitHub] [pinot] 61yao commented on a diff in pull request #10123: [multistage][refactor] Introduce MultiStageOperator with close() API

2023-01-18 Thread GitBox
61yao commented on code in PR #10123: URL: https://github.com/apache/pinot/pull/10123#discussion_r1073830894 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MultiStageOperator.java: ## @@ -0,0 +1,49 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] [pinot] 61yao commented on a diff in pull request #10117: [multistage] adding async submission

2023-01-18 Thread GitBox
61yao commented on code in PR #10117: URL: https://github.com/apache/pinot/pull/10117#discussion_r1072987281 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/service/QueryDispatcher.java: ## @@ -86,19 +90,29 @@ public int submit(long requestId, QueryPlan queryPlan, lo

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073802214 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ +

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073802214 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ +

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073784131 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ +

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10120: [multistage] support aggregations that require intermediate representations

2023-01-18 Thread GitBox
siddharthteotia commented on code in PR #10120: URL: https://github.com/apache/pinot/pull/10120#discussion_r1073781198 ## pinot-query-runtime/src/test/resources/queries/Skew.json: ## @@ -0,0 +1,79 @@ +{ + "skew": { +"tables": { + "tbl": { +"schema": [ +

[GitHub] [pinot] walterddr commented on pull request #10135: [multistage] make QueryServer async process query

2023-01-18 Thread GitBox
walterddr commented on PR #10135: URL: https://github.com/apache/pinot/pull/10135#issuecomment-1387353323 CC @agavra @61yao -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[pinot] branch master updated: [multistage] Optimize hasNext in RoundRobinScheduler (#10141)

2023-01-18 Thread rongr
This is an automated email from the ASF dual-hosted git repository. rongr pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new eaa60d93fb [multistage] Optimize hasNext in RoundR

[GitHub] [pinot] walterddr merged pull request #10141: [multistage] Optimize hasNext in RoundRobinScheduler

2023-01-18 Thread GitBox
walterddr merged PR #10141: URL: https://github.com/apache/pinot/pull/10141 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

[GitHub] [pinot] agavra commented on pull request #10141: [multistage] Optimize hasNext in RoundRobinScheduler

2023-01-18 Thread GitBox
agavra commented on PR #10141: URL: https://github.com/apache/pinot/pull/10141#issuecomment-1387336117 Let's continue the conversation for alternatives in the issue ticket (for anyone watching). -- This is an automated message from the Apache Git Service. To respond to the message, please

  1   2   >