[GitHub] [pinot] gortiz commented on a diff in pull request #11496: Close channel on direct oom

2023-09-11 Thread via GitHub
gortiz commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321079385 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under o

[GitHub] [pinot] jackjlli commented on issue #11488: Losing a controller leads to a minimum 5 minute window where segments cannot be committed

2023-09-11 Thread via GitHub
jackjlli commented on issue #11488: URL: https://github.com/apache/pinot/issues/11488#issuecomment-1713288970 The purpose of introducing the 5 min delay is that most of the time controllers encounter restart/deployment other than host replacement. And most of the time Pinot controller can b

[GitHub] [pinot] gortiz commented on a diff in pull request #11496: Close channel on direct oom

2023-09-11 Thread via GitHub
gortiz commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321081408 ## pinot-core/src/main/java/org/apache/pinot/core/transport/ServerChannels.java: ## @@ -167,7 +167,10 @@ protected void initChannel(SocketChannel ch) { //

[GitHub] [pinot] jackjlli commented on pull request #11450: Upgrading Apache Helix to latest version

2023-09-11 Thread via GitHub
jackjlli commented on PR #11450: URL: https://github.com/apache/pinot/pull/11450#issuecomment-1713292079 Thanks @Jackie-Jiang for identifying the issue! I've synced up with Helix team about the bug. I'm good with reverting the version bump for now. -- This is an automated message from the

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321097521 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] eaugene opened a new pull request, #11561: Fix stream ( kafka) consumer metrics

2023-09-11 Thread via GitHub
eaugene opened a new pull request, #11561: URL: https://github.com/apache/pinot/pull/11561 1. This reverts the change made in [#10324 ](https://github.com/apache/pinot/pull/10324) , as that would cause existing metrics to break . This revert is made with the idea to keep metrics key and st

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321100718 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321102291 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,94 @@ +/** + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321100718 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321112263 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321114917 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321116589 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321117892 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321126797 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321129623 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] gortiz commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
gortiz commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321132529 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under o

[GitHub] [pinot] gortiz commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
gortiz commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321133753 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under o

svn commit: r63906 - /dev/pinot/apache-pinot-1.0.0-rc1/

2023-09-11 Thread saurabhd336
Author: saurabhd336 Date: Mon Sep 11 08:08:09 2023 New Revision: 63906 Log: Delete release-1.0.0-rc1 Removed: dev/pinot/apache-pinot-1.0.0-rc1/ - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional c

[pinot] branch release-1.0.0-rc updated (642173f595 -> bc63e05610)

2023-09-11 Thread saurabhd336
This is an automated email from the ASF dual-hosted git repository. saurabhd336 pushed a change to branch release-1.0.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git discard 642173f595 [maven-release-plugin] prepare for next development iteration omit cd14b46219 [maven-rel

[GitHub] [pinot] codecov-commenter commented on pull request #11561: Fix stream ( kafka) consumer metrics & warn in mBean Instance creation

2023-09-11 Thread via GitHub
codecov-commenter commented on PR #11561: URL: https://github.com/apache/pinot/pull/11561#issuecomment-1713401127 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11561?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[pinot] branch release-1.0.0-rc updated: [maven-release-plugin] prepare for next development iteration

2023-09-11 Thread saurabhd336
This is an automated email from the ASF dual-hosted git repository. saurabhd336 pushed a commit to branch release-1.0.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-1.0.0-rc by this push: new d5a7314537 [maven-releas

[pinot] branch release-1.0.0-rc updated: [maven-release-plugin] prepare release release-1.0.0-rc2

2023-09-11 Thread saurabhd336
This is an automated email from the ASF dual-hosted git repository. saurabhd336 pushed a commit to branch release-1.0.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-1.0.0-rc by this push: new c86aaafba4 [maven-releas

[pinot] annotated tag release-1.0.0-rc2 updated (c86aaafba4 -> 81301612a5)

2023-09-11 Thread saurabhd336
This is an automated email from the ASF dual-hosted git repository. saurabhd336 pushed a change to annotated tag release-1.0.0-rc2 in repository https://gitbox.apache.org/repos/asf/pinot.git *** WARNING: tag release-1.0.0-rc2 was modified! *** from c86aaafba4 (commit) to 81301612a5 (t

[GitHub] [pinot] JasonLeeWeiHern opened a new issue, #11562: Add protection to Pinot upon reload of table

2023-09-11 Thread via GitHub
JasonLeeWeiHern opened a new issue, #11562: URL: https://github.com/apache/pinot/issues/11562 I experience this issue upon configuring my star tree index. I moved a high cardinality col from the top of the dimensionSplitOrder to the bottom which 3x my star tree. Upon reload of the table, my

[GitHub] [pinot] eaugene opened a new issue, #11563: Pinot build in local is getting into error

2023-09-11 Thread via GitHub
eaugene opened a new issue, #11563: URL: https://github.com/apache/pinot/issues/11563 ``` [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on project pinot-integration-tests: Compilation failure: Compilation failure:

[GitHub] [pinot] eaugene opened a new pull request, #11564: Use builder method for initalizing immutable map in integration test

2023-09-11 Thread via GitHub
eaugene opened a new pull request, #11564: URL: https://github.com/apache/pinot/pull/11564 Fixes: https://github.com/apache/pinot/issues/11563 cc : @saurabhd336 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

svn commit: r63916 - /dev/pinot/apache-pinot-1.0.0-rc2/

2023-09-11 Thread saurabhd336
Author: saurabhd336 Date: Mon Sep 11 11:06:12 2023 New Revision: 63916 Log: Update apache-pinot-1.0.0-rc2 Added: dev/pinot/apache-pinot-1.0.0-rc2/ dev/pinot/apache-pinot-1.0.0-rc2/apache-pinot-1.0.0-bin.tar.gz (with props) dev/pinot/apache-pinot-1.0.0-rc2/apache-pinot-1.0.0-bin.tar

[GitHub] [pinot] gortiz commented on a diff in pull request #11545: [multistage] (improve #11518) disable calcite implicit float to double cast

2023-09-11 Thread via GitHub
gortiz commented on code in PR #11545: URL: https://github.com/apache/pinot/pull/11545#discussion_r1321385894 ## pinot-query-planner/src/main/java/org/apache/calcite/rex/PinotRexBuilder.java: ## @@ -0,0 +1,64 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [pinot] codecov-commenter commented on pull request #11564: Use builder method for initalizing immutable map in integration test

2023-09-11 Thread via GitHub
codecov-commenter commented on PR #11564: URL: https://github.com/apache/pinot/pull/11564#issuecomment-1713709032 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11564?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] saurabhd336 commented on issue #11563: Pinot build in local is getting into error

2023-09-11 Thread via GitHub
saurabhd336 commented on issue #11563: URL: https://github.com/apache/pinot/issues/11563#issuecomment-1713714782 @eaugene Can you share the build command you're using? I'm unable to repro this. The method used does exist in the guava lib https://guava.dev/releases/32.0.1-jre/api/docs/com/go

[GitHub] [pinot] sylph-eu opened a new pull request, #11565: [bugfix] Reduce noise level due to slow topics

2023-09-11 Thread via GitHub
sylph-eu opened a new pull request, #11565: URL: https://github.com/apache/pinot/pull/11565 Our deployment of Pinot has plenty of "slow" REALTIME table for which no new data are produced for several minutes in a row. As the result, stream consumers are regularly recreated with the following

[GitHub] [pinot] saurabhd336 commented on issue #11563: Pinot build in local is getting into error

2023-09-11 Thread via GitHub
saurabhd336 commented on issue #11563: URL: https://github.com/apache/pinot/issues/11563#issuecomment-1713717562 The dependency tree ``` $ mvn clean dependency:tree -Dincludes=com.google.guava:guava [INFO] org.apache.pinot:pinot-integration-tests:jar:0.13.0-SNAPSHOT [INFO]

[GitHub] [pinot] saurabhd336 commented on pull request #11564: Use builder method for initalizing immutable map in integration test

2023-09-11 Thread via GitHub
saurabhd336 commented on PR #11564: URL: https://github.com/apache/pinot/pull/11564#issuecomment-1713726958 While this change can be merged regardless, I'm unable to repro the issue https://github.com/apache/pinot/issues/11563. Could you please add more details on steps to repro? Thanks -

[GitHub] [pinot] eaugene commented on issue #11563: Pinot build in local is getting into error

2023-09-11 Thread via GitHub
eaugene commented on issue #11563: URL: https://github.com/apache/pinot/issues/11563#issuecomment-1713769725 @saurabhd336 I was using ` mvn install package -Pbin-dist -DdownloadSources -DdownloadJavadocs -Drat.numUnapprovedLicenses=1 -DskipTests` to build -- This is an automated mes

[GitHub] [pinot] codecov-commenter commented on pull request #11565: [bugfix] Reduce noise level due to slow topics

2023-09-11 Thread via GitHub
codecov-commenter commented on PR #11565: URL: https://github.com/apache/pinot/pull/11565#issuecomment-1713777826 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11565?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[pinot] branch master updated: [feature] Reduce noise level due to slow topics (#11565)

2023-09-11 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 722f093ccd [feature] Reduce noise level due to s

[GitHub] [pinot] xiangfu0 merged pull request #11565: [logging] Reduce noise level due to slow topics

2023-09-11 Thread via GitHub
xiangfu0 merged PR #11565: URL: https://github.com/apache/pinot/pull/11565 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #11556: Add argument check for aggregation functions with single argument

2023-09-11 Thread via GitHub
xiangfu0 commented on code in PR #11556: URL: https://github.com/apache/pinot/pull/11556#discussion_r1321641330 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/AvgAggregationFunction.java: ## @@ -37,11 +39,16 @@ public class AvgAggregationFunction ex

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #11557: Optimize in clause evaluation

2023-09-11 Thread via GitHub
xiangfu0 commented on code in PR #11557: URL: https://github.com/apache/pinot/pull/11557#discussion_r1321664612 ## pinot-core/src/main/java/org/apache/pinot/core/operator/filter/predicate/PredicateUtils.java: ## @@ -145,23 +145,26 @@ public static IntSet getDictIdSet(BaseInPredi

[GitHub] [pinot] ankitsultana commented on a diff in pull request #11561: Fix stream ( kafka) consumer metrics & warn in mBean Instance creation

2023-09-11 Thread via GitHub
ankitsultana commented on code in PR #11561: URL: https://github.com/apache/pinot/pull/11561#discussion_r1321678344 ## pinot-spi/src/main/java/org/apache/pinot/spi/stream/PartitionGroupMetadataFetcher.java: ## @@ -63,8 +65,8 @@ public Exception getException() { @Override p

[GitHub] [pinot] cyrilou242 commented on issue #11503: COUNT(DISTINCT col1, col2) does not throw a syntax error and returns an incorrect result

2023-09-11 Thread via GitHub
cyrilou242 commented on issue #11503: URL: https://github.com/apache/pinot/issues/11503#issuecomment-1714068485 Thanks a lot @Jackie-Jiang! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [pinot] jadami10 commented on issue #11488: Losing a controller leads to a minimum 5 minute window where segments cannot be committed

2023-09-11 Thread via GitHub
jadami10 commented on issue #11488: URL: https://github.com/apache/pinot/issues/11488#issuecomment-1714088033 thanks for the comment jack. I think our issue ended up being a combination of things: - our clusters are large enough (in number of tables + segments) where there's effectively

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #11555: scalar func

2023-09-11 Thread via GitHub
xiangfu0 commented on code in PR #11555: URL: https://github.com/apache/pinot/pull/11555#discussion_r1321727245 ## pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArrayFunctions.java: ## @@ -74,6 +76,50 @@ public static int arrayIndexOfString(String[] values,

[GitHub] [pinot] ege-st commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
ege-st commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321730234 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under o

[GitHub] [pinot] wirybeaver commented on a diff in pull request #11555: scalar func

2023-09-11 Thread via GitHub
wirybeaver commented on code in PR #11555: URL: https://github.com/apache/pinot/pull/11555#discussion_r1321734059 ## pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArrayFunctions.java: ## @@ -74,6 +76,50 @@ public static int arrayIndexOfString(String[] values

[GitHub] [pinot] ankitsultana commented on a diff in pull request #11558: Handle interrupt gracefully for Lucene mutable index

2023-09-11 Thread via GitHub
ankitsultana commented on code in PR #11558: URL: https://github.com/apache/pinot/pull/11558#discussion_r1321683982 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneTextIndex.java: ## @@ -109,27 +112,46 @@ public Immut

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #11555: scalar func

2023-09-11 Thread via GitHub
xiangfu0 commented on code in PR #11555: URL: https://github.com/apache/pinot/pull/11555#discussion_r1321737874 ## pinot-core/src/test/java/org/apache/pinot/core/operator/transform/function/ScalarTransformFunctionWrapperTest.java: ## @@ -958,6 +958,46 @@ public void testArrayEle

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321741619 ## pinot-core/src/main/java/org/apache/pinot/core/transport/InstanceRequestHandler.java: ## @@ -321,6 +322,10 @@ private void sendResponse(ChannelHandlerContext

[GitHub] [pinot] ege-st commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
ege-st commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321746926 ## pinot-core/src/main/java/org/apache/pinot/core/transport/InstanceRequestHandler.java: ## @@ -321,6 +322,10 @@ private void sendResponse(ChannelHandlerContext ctx, St

[GitHub] [pinot] gortiz commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
gortiz commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321747015 ## pinot-core/src/main/java/org/apache/pinot/core/transport/InstanceRequestHandler.java: ## @@ -321,6 +322,10 @@ private void sendResponse(ChannelHandlerContext ctx, St

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321750404 ## pinot-core/src/main/java/org/apache/pinot/core/transport/InstanceRequestHandler.java: ## @@ -321,6 +322,10 @@ private void sendResponse(ChannelHandlerContext

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321753014 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] gortiz commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
gortiz commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321758217 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under o

[GitHub] [pinot] eaugene commented on a diff in pull request #11561: Fix stream ( kafka) consumer metrics & warn in mBean Instance creation

2023-09-11 Thread via GitHub
eaugene commented on code in PR #11561: URL: https://github.com/apache/pinot/pull/11561#discussion_r1321780374 ## pinot-spi/src/main/java/org/apache/pinot/spi/stream/PartitionGroupMetadataFetcher.java: ## @@ -63,8 +65,8 @@ public Exception getException() { @Override public

[GitHub] [pinot] PeterCorless commented on issue #11209: Filtering NULL support

2023-09-11 Thread via GitHub
PeterCorless commented on issue #11209: URL: https://github.com/apache/pinot/issues/11209#issuecomment-1714202363 I know this is closed, but can there be any better comments made for this feature? -- This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [pinot] snleee opened a new pull request, #11566: Support up to 10 boolean expressions for casewhen scalar function

2023-09-11 Thread via GitHub
snleee opened a new pull request, #11566: URL: https://github.com/apache/pinot/pull/11566 Current implementation supports up to 5 boolean expressions. This improves to support up to 10 expressions. -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11557: Optimize in clause evaluation

2023-09-11 Thread via GitHub
Jackie-Jiang commented on code in PR #11557: URL: https://github.com/apache/pinot/pull/11557#discussion_r1321836605 ## pinot-core/src/main/java/org/apache/pinot/core/operator/filter/predicate/PredicateUtils.java: ## @@ -145,23 +145,26 @@ public static IntSet getDictIdSet(BaseInP

[GitHub] [pinot] Jackie-Jiang commented on pull request #11557: Optimize in clause evaluation

2023-09-11 Thread via GitHub
Jackie-Jiang commented on PR #11557: URL: https://github.com/apache/pinot/pull/11557#issuecomment-1714284092 > can we comment on the performance of this depending on the number of hits > > for example sorted values length (m) = 1000 dictionary length (n) = 1M > > What is the per

[GitHub] [pinot] ankitsultana commented on a diff in pull request #11561: Fix stream ( kafka) consumer metrics & warn in mBean Instance creation

2023-09-11 Thread via GitHub
ankitsultana commented on code in PR #11561: URL: https://github.com/apache/pinot/pull/11561#discussion_r1321849215 ## pinot-spi/src/main/java/org/apache/pinot/spi/stream/PartitionGroupMetadataFetcher.java: ## @@ -63,8 +65,8 @@ public Exception getException() { @Override p

[GitHub] [pinot] soumitra-st commented on pull request #11501: Change the Forbidden error to Unauthorized

2023-09-11 Thread via GitHub
soumitra-st commented on PR #11501: URL: https://github.com/apache/pinot/pull/11501#issuecomment-1714295784 @Jackie-Jiang Pinot has [Broker](https://github.com/apache/pinot/blob/61dcea6b71a99746805bc7f322daed67f9bce265/pinot-broker/src/main/java/org/apache/pinot/broker/broker/AuthenticationF

[GitHub] [pinot] navina commented on issue #9869: Pinot stop consuming from Kinesis Data Stream

2023-09-11 Thread via GitHub
navina commented on issue #9869: URL: https://github.com/apache/pinot/issues/9869#issuecomment-1714330297 @FranMorilloAWS I wonder if setting completion mode to `DOWNLOAD` is causing this issue. any particular reason why you are using this config? ``` "completionConfig": { "

[GitHub] [pinot] vvivekiyer commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
vvivekiyer commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321874182 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] [pinot] FranMorilloAWS commented on issue #9869: Pinot stop consuming from Kinesis Data Stream

2023-09-11 Thread via GitHub
FranMorilloAWS commented on issue #9869: URL: https://github.com/apache/pinot/issues/9869#issuecomment-1714346496 Because at some point it was not downloading the segments to s3, i will eliminate the completion config and try again. Thanks Ill update here -- This is an automated mess

[GitHub] [pinot] codecov-commenter commented on pull request #11566: Support up to 10 boolean expressions for casewhen scalar function

2023-09-11 Thread via GitHub
codecov-commenter commented on PR #11566: URL: https://github.com/apache/pinot/pull/11566#issuecomment-1714350759 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11566?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] Jackie-Jiang opened a new pull request, #11567: Fix the wrong reference comparison in PinotEvaluateLiteralRule

2023-09-11 Thread via GitHub
Jackie-Jiang opened a new pull request, #11567: URL: https://github.com/apache/pinot/pull/11567 Avoid unnecessary casting caused by different reference but same type -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
jasperjiaguo commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1321978236 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [pinot] ankitsultana commented on pull request #11557: Optimize in clause evaluation

2023-09-11 Thread via GitHub
ankitsultana commented on PR #11557: URL: https://github.com/apache/pinot/pull/11557#issuecomment-1714451494 @Jackie-Jiang : are there any latency numbers for comparison? (this PR's approach vs no optimization vs sort then merge-sort) -- This is an automated message from the Apache Git Se

[GitHub] [pinot] codecov-commenter commented on pull request #11567: Fix the wrong reference comparison in PinotEvaluateLiteralRule

2023-09-11 Thread via GitHub
codecov-commenter commented on PR #11567: URL: https://github.com/apache/pinot/pull/11567#issuecomment-1714458716 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11567?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] Jackie-Jiang commented on issue #11485: Enable Compression on BYTES star tree index

2023-09-11 Thread via GitHub
Jackie-Jiang commented on issue #11485: URL: https://github.com/apache/pinot/issues/11485#issuecomment-1714483527 Let's use the original issue ticket to track this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [pinot] Jackie-Jiang closed issue #11485: Enable Compression on BYTES star tree index

2023-09-11 Thread via GitHub
Jackie-Jiang closed issue #11485: Enable Compression on BYTES star tree index URL: https://github.com/apache/pinot/issues/11485 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [pinot] Jackie-Jiang commented on issue #9694: Can we add compression configuration for functionColumnPairs to StartreeIndexConfigs?

2023-09-11 Thread via GitHub
Jackie-Jiang commented on issue #9694: URL: https://github.com/apache/pinot/issues/9694#issuecomment-1714484728 @priyen Have you started working on this? @t0mpere is volunteering to work on this -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [pinot] KKcorps opened a new pull request, #11568: Add integration test for rebalance in upsert tables

2023-09-11 Thread via GitHub
KKcorps opened a new pull request, #11568: URL: https://github.com/apache/pinot/pull/11568 Mostly tests that partitions move as a whole unit to new servers -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [pinot] Jackie-Jiang merged pull request #11567: Fix the wrong reference comparison in PinotEvaluateLiteralRule

2023-09-11 Thread via GitHub
Jackie-Jiang merged PR #11567: URL: https://github.com/apache/pinot/pull/11567 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[pinot] branch master updated (722f093ccd -> 032a5cce47)

2023-09-11 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 722f093ccd [feature] Reduce noise level due to slow topics (#11565) add 032a5cce47 Fix the wrong reference compari

[GitHub] [pinot] vvivekiyer commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
vvivekiyer commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1322014368 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] [pinot] snleee commented on pull request #11566: Support up to 10 boolean expressions for casewhen scalar function

2023-09-11 Thread via GitHub
snleee commented on PR #11566: URL: https://github.com/apache/pinot/pull/11566#issuecomment-1714512635 @walterddr Added the test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [pinot] siddharthteotia commented on pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
siddharthteotia commented on PR #11496: URL: https://github.com/apache/pinot/pull/11496#issuecomment-1714516642 @jasperjiaguo - you may also want to share numbers from the internal production test we did using this, before v/s after numbers / observations from heap dump etc - Direct

[GitHub] [pinot] jackjlli commented on a diff in pull request #11509: make lead controller resource rebalance delay configurable

2023-09-11 Thread via GitHub
jackjlli commented on code in PR #11509: URL: https://github.com/apache/pinot/pull/11509#discussion_r1322027323 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/util/HelixSetupUtils.java: ## @@ -226,4 +193,77 @@ private static void createLeadControllerRe

[GitHub] [pinot] jadami10 commented on issue #9694: Can we add compression configuration for functionColumnPairs to StartreeIndexConfigs?

2023-09-11 Thread via GitHub
jadami10 commented on issue #9694: URL: https://github.com/apache/pinot/issues/9694#issuecomment-1714540708 we haven't prioritized work on this. Internally, we've changed this to always lz4 compress any byte columns. @t0mpere, feel free to take! -- This is an automated message from the Ap

[GitHub] [pinot] abhioncbr commented on pull request #11501: Change the Forbidden error to Unauthorized

2023-09-11 Thread via GitHub
abhioncbr commented on PR #11501: URL: https://github.com/apache/pinot/pull/11501#issuecomment-1714629766 > @Jackie-Jiang , Pinot has [Broker](https://github.com/apache/pinot/blob/61dcea6b71a99746805bc7f322daed67f9bce265/pinot-broker/src/main/java/org/apache/pinot/broker/broker/Authenticatio

[GitHub] [pinot] deemoliu commented on pull request #11346: Add hyperLogLogPlus aggregation function for distinct count

2023-09-11 Thread via GitHub
deemoliu commented on PR #11346: URL: https://github.com/apache/pinot/pull/11346#issuecomment-1714683427 @Jackie-Jiang @chenboat @cbalci can you please review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
jasperjiaguo commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1322161854 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
jasperjiaguo commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1322162691 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
jasperjiaguo commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1322163247 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [pinot] ege-st commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
ege-st commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1322163994 ## pinot-core/src/main/java/org/apache/pinot/core/transport/InstanceRequestHandler.java: ## @@ -321,6 +322,10 @@ private void sendResponse(ChannelHandlerContext ctx, St

[GitHub] [pinot] ege-st commented on pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
ege-st commented on PR #11496: URL: https://github.com/apache/pinot/pull/11496#issuecomment-1714703586 > @jasperjiaguo - you may also want to share numbers from the internal production test we did using this, before v/s after numbers / observations from heap dump etc > > * Direct

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
jasperjiaguo commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1322164451 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [pinot] ege-st commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
ege-st commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1322165856 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under o

[GitHub] [pinot] Jackie-Jiang commented on issue #11522: New segments for consuming Kinesis Data Streams not being created

2023-09-11 Thread via GitHub
Jackie-Jiang commented on issue #11522: URL: https://github.com/apache/pinot/issues/11522#issuecomment-1714738059 When using the segment flush threshold, segment will start from a smaller size, then slowly grow to the desired size (there is no direct translation from row count to size). The

[GitHub] [pinot] ege-st commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
ege-st commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1322186391 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under o

[GitHub] [pinot] snleee opened a new issue, #11569: Add support as statement for `CASE WHEN` group by

2023-09-11 Thread via GitHub
snleee opened a new issue, #11569: URL: https://github.com/apache/pinot/issues/11569 Working properly: ``` SELECT CASE WHEN O_CUSTKEY > 1 THEN 1 WHEN O_CUSTKEY > 2 THEN 2 ELSE 3 END AS cust, COUNT(*) as count FROM

[GitHub] [pinot] snleee commented on pull request #11566: Support up to 15 boolean expressions for casewhen scalar function

2023-09-11 Thread via GitHub
snleee commented on PR #11566: URL: https://github.com/apache/pinot/pull/11566#issuecomment-1714770865 The use case that I faced needs the else statement. If we don't cover else, we need to tweak by setting `default value` on the schema to cover else statement for that column. This will mak

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
jasperjiaguo commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1322211634 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [pinot] snleee merged pull request #11566: Support up to 15 boolean expressions for casewhen scalar function

2023-09-11 Thread via GitHub
snleee merged PR #11566: URL: https://github.com/apache/pinot/pull/11566 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

[pinot] branch master updated (032a5cce47 -> b297245f1c)

2023-09-11 Thread snlee
This is an automated email from the ASF dual-hosted git repository. snlee pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 032a5cce47 Fix the wrong reference comparison in PinotEvaluateLiteralRule (#11567) add b297245f1c Support up to 15

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
jasperjiaguo commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r131624 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [pinot] snleee commented on issue #11569: `as` statement for group by columns

2023-09-11 Thread via GitHub
snleee commented on issue #11569: URL: https://github.com/apache/pinot/issues/11569#issuecomment-171406 Checked this on MySql, PostgreSQL. Adding `as` to group by is not valid SQL syntax. closing the ticket. -- This is an automated message from the Apache Git Service. To respon

[GitHub] [pinot] snleee closed issue #11569: `as` statement for group by columns

2023-09-11 Thread via GitHub
snleee closed issue #11569: `as` statement for group by columns URL: https://github.com/apache/pinot/issues/11569 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
jasperjiaguo commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r137562 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-11 Thread via GitHub
jasperjiaguo commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1322239091 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) u

  1   2   >