[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9278: Optimize ser/de to avoid using output stream

siddharthteotia commented on code in PR #9278: URL: https://github.com/apache/pinot/pull/9278#discussion_r954569284 ## pinot-core/src/main/java/org/apache/pinot/core/common/ObjectSerDeUtils.java: ## @@ -540,39 +538,38 @@ public byte[] serialize(Map map) { return new byt

[pinot] branch master updated: add @ManualAuthorization annotation for non-standard endpoints (#9252)

This is an automated email from the ASF dual-hosted git repository. apucher pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new f6e26c2350 add @ManualAuthorization annotation f

[GitHub] [pinot] apucher merged pull request #9252: add @ManualAuthorization annotation for non-standard endpoints

apucher merged PR #9252: URL: https://github.com/apache/pinot/pull/9252 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache

[GitHub] [pinot] walterddr commented on a diff in pull request #9236: Add Support for Covariance Function

walterddr commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954440149 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,237 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] walterddr commented on a diff in pull request #9236: Add Support for Covariance Function

walterddr commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954440149 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,237 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] walterddr commented on a diff in pull request #9274: [multistage] support HAVING clause

walterddr commented on code in PR #9274: URL: https://github.com/apache/pinot/pull/9274#discussion_r954439695 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/FilterOperator.java: ## @@ -0,0 +1,91 @@ +/** + * Licensed to the Apache Software Foundation

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9274: [multistage] support HAVING clause

siddharthteotia commented on code in PR #9274: URL: https://github.com/apache/pinot/pull/9274#discussion_r954414615 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/FilterOperator.java: ## @@ -0,0 +1,91 @@ +/** + * Licensed to the Apache Software Foun

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9236: Add Support for Covariance Function

siddharthteotia commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954414122 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,237 @@ +/** + * Licensed to the Apac

[GitHub] [pinot] siddharthteotia commented on issue #9273: [multistage] performance and stability tracker

siddharthteotia commented on issue #9273: URL: https://github.com/apache/pinot/issues/9273#issuecomment-1226636894 Should we separately track resource / thread pool / execution scheduling issue or add it here ? Without that, this is unlikely to work for a mixed workload deployment wh

[GitHub] [pinot] jackjlli closed issue #8306: Upgrade Helix to 1.0+ in Pinot

jackjlli closed issue #8306: Upgrade Helix to 1.0+ in Pinot URL: https://github.com/apache/pinot/issues/8306 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

[GitHub] [pinot] codecov-commenter commented on pull request #9278: Optimize ser/de to avoid using output stream

codecov-commenter commented on PR #9278: URL: https://github.com/apache/pinot/pull/9278#issuecomment-1226620646 # [Codecov](https://codecov.io/gh/apache/pinot/pull/9278?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9274: [multistage] support HAVING clause

siddharthteotia commented on code in PR #9274: URL: https://github.com/apache/pinot/pull/9274#discussion_r954390590 ## pinot-query-planner/src/test/java/org/apache/pinot/query/QueryEnvironmentTestBase.java: ## @@ -57,6 +57,8 @@ protected Object[][] provideQueries() {

[GitHub] [pinot] Jackie-Jiang opened a new pull request, #9278: Optimize ser/de to avoid using output stream

Jackie-Jiang opened a new pull request, #9278: URL: https://github.com/apache/pinot/pull/9278 Avoid using `ByteArrayOutputStream` which will potentially copy bytes multiple times (during expand and final `toByteArray()`). Pre-calculate the buffer size required, and then create the buffer on

[GitHub] [pinot] walterddr commented on a diff in pull request #9236: Add Support for Covariance Function

walterddr commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954370975 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,237 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] walterddr commented on a diff in pull request #9236: Add Support for Covariance Function

walterddr commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954370975 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,237 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] SabrinaZhaozyf commented on a diff in pull request #9236: Add Support for Covariance Function

SabrinaZhaozyf commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954368611 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,225 @@ +/** + * Licensed to the Apach

[GitHub] [pinot] SabrinaZhaozyf commented on a diff in pull request #9236: Add Support for Covariance Function

SabrinaZhaozyf commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r953186422 ## pinot-core/src/test/java/org/apache/pinot/queries/BaseQueriesTest.java: ## @@ -197,4 +197,70 @@ protected BrokerResponseNative getBrokerResponseForOptimizedQuer

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #9236: Add Support for Covariance Function

jasperjiaguo commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954356571 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,237 @@ +/** + * Licensed to the Apache

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #9236: Add Support for Covariance Function

jasperjiaguo commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954356571 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,237 @@ +/** + * Licensed to the Apache

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #9236: Add Support for Covariance Function

jasperjiaguo commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954356571 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,237 @@ +/** + * Licensed to the Apache

[GitHub] [pinot] SabrinaZhaozyf opened a new issue, #9277: [Feature] Support for Correlation Function

SabrinaZhaozyf opened a new issue, #9277: URL: https://github.com/apache/pinot/issues/9277 Add support for calculating Pearson's coefficient `corr(x,y)` as part of the effort in #8493. Can leverage implementation for COVAR_SAMP and COVAR_POP as mentioned in #9236. -- This is an autom

[pinot] branch master updated: [Feature] Add an option to search input files recursively in ingestion job. The default is set to true to be backward compatible. (#9265)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new ae8781959c [Feature] Add an option to search inpu

[GitHub] [pinot] Jackie-Jiang merged pull request #9265: [Feature] Add an option to search input files recursively in ingestion job. The default is set to true to be backward compatible.

Jackie-Jiang merged PR #9265: URL: https://github.com/apache/pinot/pull/9265 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

[pinot] branch master updated: add query cancel APIs on controller to easily show running queries across the cluster and cancel them (#9276)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 7d9dcdfd83 add query cancel APIs on controller to

[GitHub] [pinot] Jackie-Jiang merged pull request #9276: add query cancel APIs on controller backed by those on brokers

Jackie-Jiang merged PR #9276: URL: https://github.com/apache/pinot/pull/9276 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

[pinot] branch manual-authorization-annotation updated (890aa3cb8c -> 8c1aa68df1)

This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch manual-authorization-annotation in repository https://gitbox.apache.org/repos/asf/pinot.git from 890aa3cb8c switch null tableName in access control utils add 8bf94e7252 refactor query a

[GitHub] [pinot] walterddr commented on a diff in pull request #9236: Add Support for Covariance Function

walterddr commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954320248 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,237 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] walterddr commented on a diff in pull request #9236: Add Support for Covariance Function

walterddr commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954320248 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,237 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] walterddr commented on a diff in pull request #9236: Add Support for Covariance Function

walterddr commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954319642 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,237 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] walterddr commented on a diff in pull request #9236: Add Support for Covariance Function

walterddr commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954314178 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,237 @@ +/** + * Licensed to the Apache Sof

[pinot] branch master updated: refactor query and option compilation (#9239)

This is an automated email from the ASF dual-hosted git repository. siddteotia pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 8bf94e7252 refactor query and option compilat

[GitHub] [pinot] siddharthteotia merged pull request #9239: refactor query and option compilation

siddharthteotia merged PR #9239: URL: https://github.com/apache/pinot/pull/9239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pino

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #9236: Add Support for Covariance Function

jasperjiaguo commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954297069 ## pinot-core/src/test/java/org/apache/pinot/queries/BaseQueriesTest.java: ## @@ -91,7 +95,9 @@ protected T getOperatorWithFilter(String query) { /** * Run q

[GitHub] [pinot] tanmesh commented on a diff in pull request #9184: Add support for EXTRACT syntax and converts it to appropriate Pinot expression

tanmesh commented on code in PR #9184: URL: https://github.com/apache/pinot/pull/9184#discussion_r954281226 ## pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/ExtractTransformFunction.java: ## @@ -0,0 +1,97 @@ +package org.apache.pinot.core.operator.tr

[GitHub] [pinot] amrishlal commented on a diff in pull request #9236: Add Support for Covariance Function

amrishlal commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954095008 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,225 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] amrishlal commented on a diff in pull request #9236: Add Support for Covariance Function

amrishlal commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954253485 ## pinot-core/src/test/java/org/apache/pinot/queries/BaseQueriesTest.java: ## @@ -146,14 +160,21 @@ private BrokerResponseNative getBrokerResponse(String query, PlanMak

[GitHub] [pinot] jadami10 commented on a diff in pull request #9244: add a freshness based consumption status checker

jadami10 commented on code in PR #9244: URL: https://github.com/apache/pinot/pull/9244#discussion_r954230103 ## pinot-server/src/main/java/org/apache/pinot/server/starter/helix/FreshnessBasedConsumptionStatusChecker.java: ## @@ -0,0 +1,102 @@ +/** + * Licensed to the Apache Soft

[GitHub] [pinot] tanmesh commented on a diff in pull request #9184: Add support for EXTRACT syntax and converts it to appropriate Pinot expression

tanmesh commented on code in PR #9184: URL: https://github.com/apache/pinot/pull/9184#discussion_r954213735 ## pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/ExtractTransformFunction.java: ## @@ -0,0 +1,97 @@ +package org.apache.pinot.core.operator.tr

[GitHub] [pinot] tanmesh commented on a diff in pull request #9184: Add support for EXTRACT syntax and converts it to appropriate Pinot expression

tanmesh commented on code in PR #9184: URL: https://github.com/apache/pinot/pull/9184#discussion_r954213735 ## pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/ExtractTransformFunction.java: ## @@ -0,0 +1,97 @@ +package org.apache.pinot.core.operator.tr

[GitHub] [pinot] klsince opened a new pull request, #9276: add query cancel APIs on controller backed by those on brokers

klsince opened a new pull request, #9276: URL: https://github.com/apache/pinot/pull/9276 Add query cancel APIs on controller to easily show running queries across the cluster and cancel them -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #9236: Add Support for Covariance Function

jasperjiaguo commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954154389 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,225 @@ +/** + * Licensed to the Apache

[GitHub] [pinot] codecov-commenter commented on pull request #9274: [multistage] support having

codecov-commenter commented on PR #9274: URL: https://github.com/apache/pinot/pull/9274#issuecomment-1226074305 # [Codecov](https://codecov.io/gh/apache/pinot/pull/9274?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #9236: Add Support for Covariance Function

jasperjiaguo commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954131730 ## pinot-core/src/test/java/org/apache/pinot/queries/BaseQueriesTest.java: ## @@ -146,14 +160,21 @@ private BrokerResponseNative getBrokerResponse(String query, Plan

[GitHub] [pinot] Jackie-Jiang commented on issue #9266: [Question] whether refreshing/ingesting segments will cause it to rebalance?

Jackie-Jiang commented on issue #9266: URL: https://github.com/apache/pinot/issues/9266#issuecomment-1226066957 That is fine. The ingestion (refresh) won't change the segment assignment, only the rebalance can. Conceptually you can run both of them at the same time (not recommended, but sho

[GitHub] [pinot] ddcprg opened a new issue, #9275: New metrics registry for micrometer

ddcprg opened a new issue, #9275: URL: https://github.com/apache/pinot/issues/9275 At the moment only Dropwizard/Yammer metrics are provided. A new plugin for Micrometer metrics would be a nice to have. However, I think this should reside in a separate project. Not sure if there is an

[GitHub] [pinot] amrishlal commented on a diff in pull request #9236: Add Support for Covariance Function

amrishlal commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954116254 ## pinot-core/src/test/java/org/apache/pinot/queries/BaseQueriesTest.java: ## @@ -146,14 +160,21 @@ private BrokerResponseNative getBrokerResponse(String query, PlanMak

[GitHub] [pinot] amrishlal commented on a diff in pull request #9236: Add Support for Covariance Function

amrishlal commented on code in PR #9236: URL: https://github.com/apache/pinot/pull/9236#discussion_r954095008 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/CovarianceAggregationFunction.java: ## @@ -0,0 +1,225 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] walterddr commented on issue #9273: [multistage] performance and stability tracker

walterddr commented on issue #9273: URL: https://github.com/apache/pinot/issues/9273#issuecomment-1225794909 CC @siddharthteotia ^ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [pinot] walterddr opened a new issue, #9273: [multistage] performance and stability tracker

walterddr opened a new issue, #9273: URL: https://github.com/apache/pinot/issues/9273 We've received many feedback from multi-stage engine beta test. This issue tracks all the reported and fixed performance and stability issues - [ ] large data block cause grpc failure: https://github

[GitHub] [pinot] walterddr commented on issue #9209: [multistage] Support limit properly

walterddr commented on issue #9209: URL: https://github.com/apache/pinot/issues/9209#issuecomment-1225793237 fetch unorder is supported but we should still limit the number of rows returned at final stage. -- This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [pinot] walterddr closed issue #9127: [flaky test] HelixZNodeSizeLimitTest generates huge log

walterddr closed issue #9127: [flaky test] HelixZNodeSizeLimitTest generates huge log URL: https://github.com/apache/pinot/issues/9127 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [pinot] navina commented on pull request #9260: added a default groupid in kafka consumer

navina commented on PR #9260: URL: https://github.com/apache/pinot/pull/9260#issuecomment-1225738499 > Please reformat the changes with [Pinot Style](https://docs.pinot.apache.org/developers/developers-and-contributors/code-setup?q=pinot+style#intellij) > > @navina Can you please help

[GitHub] [pinot] navina commented on a diff in pull request #9260: added a default groupid in kafka consumer

navina commented on code in PR #9260: URL: https://github.com/apache/pinot/pull/9260#discussion_r953800486 ## pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/src/main/java/org/apache/pinot/plugin/stream/kafka20/KafkaStreamLevelStreamConfig.java: ## @@ -53,21 +53,24 @@ publi

[GitHub] [pinot] xuhui-stripe commented on issue #9266: [Question] whether refreshing/ingesting segments will cause it to rebalance?

xuhui-stripe commented on issue #9266: URL: https://github.com/apache/pinot/issues/9266#issuecomment-1225445547 Thanks for answering, does it mean it is okay to have for example 1 or 2 segment ingestions/refreshes before actually rebalancing the table? I did a quick test in our QA cluste

[GitHub] [pinot] INNOCENT-BOY commented on pull request #9251: use default authorization behavior for zk user config

INNOCENT-BOY commented on PR #9251: URL: https://github.com/apache/pinot/pull/9251#issuecomment-1225320113 > That's great work! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi