[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7075: [WIP] Startable from config

codecov-commenter edited a comment on pull request #7075: URL: https://github.com/apache/incubator-pinot/pull/7075#issuecomment-864348974 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7075?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7075: [WIP] Startable from config

codecov-commenter edited a comment on pull request #7075: URL: https://github.com/apache/incubator-pinot/pull/7075#issuecomment-864348974 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7075?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7075: [WIP] Startable from config

codecov-commenter edited a comment on pull request #7075: URL: https://github.com/apache/incubator-pinot/pull/7075#issuecomment-864348974 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7075?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7075: [WIP] Startable from config

codecov-commenter commented on pull request #7075: URL: https://github.com/apache/incubator-pinot/pull/7075#issuecomment-864348974 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7075?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #7074: Adding support for config files in compat tests

mcvsubbu commented on a change in pull request #7074: URL: https://github.com/apache/incubator-pinot/pull/7074#discussion_r654741407 ## File path: compatibility-verifier/sample-test-suite/config/ControllerConfig.properties ## @@ -0,0 +1,24 @@ +# +# Licensed to the Apache Softw

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #7074: Adding support for config files in compat tests

mcvsubbu commented on a change in pull request #7074: URL: https://github.com/apache/incubator-pinot/pull/7074#discussion_r654741380 ## File path: compatibility-verifier/sample-test-suite/config/ControllerConfig.properties ## @@ -0,0 +1,24 @@ +# +# Licensed to the Apache Softw

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #7074: Adding support for config files in compat tests

mcvsubbu commented on a change in pull request #7074: URL: https://github.com/apache/incubator-pinot/pull/7074#discussion_r654741303 ## File path: pinot-integration-tests/src/test/java/org/apache/pinot/compat/tests/CompatibilityOpsRunner.java ## @@ -65,6 +65,10 @@ public stati

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #7074: Adding support for config files in compat tests

mcvsubbu commented on a change in pull request #7074: URL: https://github.com/apache/incubator-pinot/pull/7074#discussion_r654741293 ## File path: compatibility-verifier/sample-test-suite/config/ServerConfig.properties ## @@ -0,0 +1,25 @@ +# +# Licensed to the Apache Software

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #7074: Adding support for config files in compat tests

mcvsubbu commented on a change in pull request #7074: URL: https://github.com/apache/incubator-pinot/pull/7074#discussion_r654741286 ## File path: compatibility-verifier/sample-test-suite/config/ControllerConfig.properties ## @@ -0,0 +1,24 @@ +# +# Licensed to the Apache Softw

[GitHub] [incubator-pinot] npawar opened a new pull request #7075: Startable from config

npawar opened a new pull request #7075: URL: https://github.com/apache/incubator-pinot/pull/7075 ## Description ## Upgrade Notes Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion) * [ ] Yes (Please label as **backward-incom

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7074: Adding support for config files in compat tests

codecov-commenter edited a comment on pull request #7074: URL: https://github.com/apache/incubator-pinot/pull/7074#issuecomment-864335905 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7074?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7074: Adding support for config files in compat tests

codecov-commenter commented on pull request #7074: URL: https://github.com/apache/incubator-pinot/pull/7074#issuecomment-864335905 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7074?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #7074: Adding support for config files in compat tests

jackjlli commented on a change in pull request #7074: URL: https://github.com/apache/incubator-pinot/pull/7074#discussion_r654722587 ## File path: compatibility-verifier/sample-test-suite/config/ControllerConfig.properties ## @@ -0,0 +1,24 @@ +# +# Licensed to the Apache Softw

[GitHub] [incubator-pinot] mqliang commented on a change in pull request #7074: Adding support for config files in compat tests

mqliang commented on a change in pull request #7074: URL: https://github.com/apache/incubator-pinot/pull/7074#discussion_r654721734 ## File path: pinot-integration-tests/src/test/java/org/apache/pinot/compat/tests/CompatibilityOpsRunner.java ## @@ -65,6 +65,10 @@ public static

[GitHub] [incubator-pinot] mcvsubbu opened a new pull request #7074: Adding support for config files in compat tests

mcvsubbu opened a new pull request #7074: URL: https://github.com/apache/incubator-pinot/pull/7074 Pinot installation administrators can now specify configuration files to run the tests as per their environment. Changed the java files to get the ports as declared in the config fil

[incubator-pinot] branch add-config-file-support created (now 6eba9df)

This is an automated email from the ASF dual-hosted git repository. mcvsubbu pushed a change to branch add-config-file-support in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at 6eba9df Adding support for config files in compat tests This branch includes the follow

[incubator-pinot] 01/01: Adding support for config files in compat tests

This is an automated email from the ASF dual-hosted git repository. mcvsubbu pushed a commit to branch add-config-file-support in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 6eba9df5b3271aee885ddf7bac6fb32b55e164a7 Author: Subbu Subramaniam AuthorDate: Fri Jun 18 15

[GitHub] [incubator-pinot] dongxiaoman closed pull request #7073: Update hostnames in Helix for controller and broker

dongxiaoman closed pull request #7073: URL: https://github.com/apache/incubator-pinot/pull/7073 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this ser

[GitHub] [incubator-pinot] dongxiaoman commented on pull request #7073: Update hostnames in Helix for controller and broker

dongxiaoman commented on pull request #7073: URL: https://github.com/apache/incubator-pinot/pull/7073#issuecomment-864313697 testing branch now working. Duplicate of https://github.com/apache/incubator-pinot/pull/7064 -- This is an automated message from the Apache Git Service. To respon

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7064: Allow updating controller and broker helix hostname

codecov-commenter edited a comment on pull request #7064: URL: https://github.com/apache/incubator-pinot/pull/7064#issuecomment-861995023 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7064?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7073: Update hostnames in Helix for controller and broker

codecov-commenter edited a comment on pull request #7073: URL: https://github.com/apache/incubator-pinot/pull/7073#issuecomment-864258870 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7073?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #7063: Add support for range index rule recommendation(#7034)

jackjlli commented on a change in pull request #7063: URL: https://github.com/apache/incubator-pinot/pull/7063#discussion_r654704283 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/recommender/rules/impl/RangeIndexRule.java ## @@ -0,0 +1,117 @@ +/** +

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7064: Allow updating controller and broker helix hostname

codecov-commenter edited a comment on pull request #7064: URL: https://github.com/apache/incubator-pinot/pull/7064#issuecomment-861995023 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7064?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7073: Update hostnames in Helix for controller and broker

codecov-commenter edited a comment on pull request #7073: URL: https://github.com/apache/incubator-pinot/pull/7073#issuecomment-864258870 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7073?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] tdunning edited a comment on pull request #7069: Revert "Upgrade t-digest to latest release (#7029)"

tdunning edited a comment on pull request #7069: URL: https://github.com/apache/incubator-pinot/pull/7069#issuecomment-864294396 Errors in estimation with the t-digest are usually phrased in terms of quantile error (that is, q, not the samples x). Errors in the sample space are notionally

[GitHub] [incubator-pinot] tdunning edited a comment on pull request #7069: Revert "Upgrade t-digest to latest release (#7029)"

tdunning edited a comment on pull request #7069: URL: https://github.com/apache/incubator-pinot/pull/7069#issuecomment-864294396 Errors in estimation with the t-digest are usually phrased in terms of quantile error (that is, q, not the samples x). Errors in the sample space are notionally

[GitHub] [incubator-pinot] tdunning commented on pull request #7069: Revert "Upgrade t-digest to latest release (#7029)"

tdunning commented on pull request #7069: URL: https://github.com/apache/incubator-pinot/pull/7069#issuecomment-864294396 Errors in estimation with the t-digest are usually phrased in terms of quantile error (that is, q, not the samples x). Errors in the sample space are notionally unbound

[GitHub] [incubator-pinot] mayankshriv opened a new pull request #7071: Clear legacy configs when converting to new TableConfig.

mayankshriv opened a new pull request #7071: URL: https://github.com/apache/incubator-pinot/pull/7071 ## Description We moved to StreamIngestionConfig and BatchIngestionConfig recently. In this PR, we clear the legacy configs after converting to the new TableConfig in the utility `c

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7068: Make BitmapDocIdSetOperator a top level operator

Jackie-Jiang commented on a change in pull request #7068: URL: https://github.com/apache/incubator-pinot/pull/7068#discussion_r654077802 ## File path: pinot-core/src/main/java/org/apache/pinot/core/operator/BitmapDocIdSetOperator.java ## @@ -0,0 +1,71 @@ +/** + * Licensed to t

[GitHub] [incubator-pinot] mayankshriv merged pull request #7071: Clear legacy configs when converting to new TableConfig.

mayankshriv merged pull request #7071: URL: https://github.com/apache/incubator-pinot/pull/7071 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this ser

[GitHub] [incubator-pinot] dongxiaoman commented on a change in pull request #7064: Allow updating controller and broker helix hostname

dongxiaoman commented on a change in pull request #7064: URL: https://github.com/apache/incubator-pinot/pull/7064#discussion_r654065894 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/broker/helix/HelixBrokerStarter.java ## @@ -330,6 +334,27 @@ public void sta

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7052: Add Query Options for GroupBy In-Segment Trim

codecov-commenter edited a comment on pull request #7052: URL: https://github.com/apache/incubator-pinot/pull/7052#issuecomment-859828970 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [incubator-pinot] tdunning commented on pull request #7069: Revert "Upgrade t-digest to latest release (#7029)"

tdunning commented on pull request #7069: URL: https://github.com/apache/incubator-pinot/pull/7069#issuecomment-863828785 I have done an analysis here. I believe that the root of the issue is that the t-digest takes the compression factor more seriously now than it did before. That means t

[GitHub] [incubator-pinot] deemoliu commented on a change in pull request #6899: Add partial upsert config and mergers

deemoliu commented on a change in pull request #6899: URL: https://github.com/apache/incubator-pinot/pull/6899#discussion_r654042298 ## File path: pinot-segment-local/src/test/java/org/apache/pinot/segment/local/upsert/PartialUpsertHandlerTest.java ## @@ -0,0 +1,81 @@ +/** + *

[GitHub] [incubator-pinot] chenboat commented on a change in pull request #6899: Add partial upsert config and mergers

chenboat commented on a change in pull request #6899: URL: https://github.com/apache/incubator-pinot/pull/6899#discussion_r654067697 ## File path: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/PartitionUpsertMetadataManager.java ## @@ -60,128 +63,170

[GitHub] [incubator-pinot-site] mayankshriv opened a new pull request #50: Fix footer to say `Integrations` instead of `Components`.

mayankshriv opened a new pull request #50: URL: https://github.com/apache/incubator-pinot-site/pull/50 The section that listed Presto/TE was titled `Components`. Fixed it to say `Integrations` instead. -- This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7073: Update hostnames in Helix for controller and broker

codecov-commenter commented on pull request #7073: URL: https://github.com/apache/incubator-pinot/pull/7073#issuecomment-864258870 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7073?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[GitHub] [incubator-pinot-site] mayankshriv merged pull request #50: Fix footer to say `Integrations` instead of `Components`.

mayankshriv merged pull request #50: URL: https://github.com/apache/incubator-pinot-site/pull/50 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this se

[GitHub] [incubator-pinot] mayankshriv commented on pull request #7069: Revert "Upgrade t-digest to latest release (#7029)"

mayankshriv commented on pull request #7069: URL: https://github.com/apache/incubator-pinot/pull/7069#issuecomment-864158645 Thanks @tdunning for the detailed analysis. @Jackie-Jiang could you revisit the test with suggestions from Ted? -- This is an automated message from the Apac

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #7026: Add Apache Pulsar low level and high level connector

mcvsubbu commented on a change in pull request #7026: URL: https://github.com/apache/incubator-pinot/pull/7026#discussion_r653794441 ## File path: pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarPartitionLevelConsumer.java ##

[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #7071: Clear legacy configs when converting to new TableConfig.

mayankshriv commented on a change in pull request #7071: URL: https://github.com/apache/incubator-pinot/pull/7071#discussion_r654105233 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/config/TableConfigUtils.java ## @@ -235,34 +233,56 @@ public static vo

[GitHub] [incubator-pinot] mayankshriv commented on pull request #7071: Clear legacy configs when converting to new TableConfig.

mayankshriv commented on pull request #7071: URL: https://github.com/apache/incubator-pinot/pull/7071#issuecomment-863612278 > We should convert the config based on the table type. For OFFLINE table, convert the batch config; for REALTIME table, convert the stream config. > Or only conve

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7071: Clear legacy configs when converting to new TableConfig.

codecov-commenter commented on pull request #7071: URL: https://github.com/apache/incubator-pinot/pull/7071#issuecomment-863596801 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7071?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7052: Add Query Options for GroupBy In-Segment Trim

Jackie-Jiang commented on a change in pull request #7052: URL: https://github.com/apache/incubator-pinot/pull/7052#discussion_r653803756 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/QueryOptions.java ## @@ -41,15 +43,39 @@ public QueryOptions(@Nullable Map

[GitHub] [incubator-pinot] Jackie-Jiang commented on pull request #7069: Revert "Upgrade t-digest to latest release (#7029)"

Jackie-Jiang commented on pull request #7069: URL: https://github.com/apache/incubator-pinot/pull/7069#issuecomment-864235351 @tdunning Thanks for the explanation. It there a formula to calculate the max error rate from the compression factor (or reversed) so that I can modify the test a

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7068: Make BitmapDocIdSetOperator a top level operator

codecov-commenter edited a comment on pull request #7068: URL: https://github.com/apache/incubator-pinot/pull/7068#issuecomment-862795753 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #7052: Add Query Options for GroupBy In-Segment Trim

Jackie-Jiang merged pull request #7052: URL: https://github.com/apache/incubator-pinot/pull/7052 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this serv

[GitHub] [incubator-pinot] dongxiaoman opened a new pull request #7073: Update hostnames in Helix for controller and broker

dongxiaoman opened a new pull request #7073: URL: https://github.com/apache/incubator-pinot/pull/7073 ## Description ## Upgrade Notes Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion) * [ ] Yes (Please label as **backward-

[GitHub] [incubator-pinot] syed72 opened a new issue #7072: Support database concept to create like dbname.tablename

syed72 opened a new issue #7072: URL: https://github.com/apache/incubator-pinot/issues/7072 Hi, Need support something like Mysql where we can have multiple databases and same table inside it... i.e., dbName.TableName { "tableName": "myDb.myTable", "table

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7064: Allow updating controller and broker helix hostname

codecov-commenter edited a comment on pull request #7064: URL: https://github.com/apache/incubator-pinot/pull/7064#issuecomment-861995023 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [incubator-pinot] mayankshriv edited a comment on pull request #7069: Revert "Upgrade t-digest to latest release (#7029)"

mayankshriv edited a comment on pull request #7069: URL: https://github.com/apache/incubator-pinot/pull/7069#issuecomment-864158645 Thanks @tdunning for the detailed analysis, much appreciated. @Jackie-Jiang could you revisit the test with suggestions from Ted? -- This is an automa

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7071: Clear legacy configs when converting to new TableConfig.

Jackie-Jiang commented on a change in pull request #7071: URL: https://github.com/apache/incubator-pinot/pull/7071#discussion_r653959152 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/config/TableConfigUtils.java ## @@ -263,6 +262,14 @@ public static vo

[GitHub] [incubator-pinot] tdunning edited a comment on pull request #7069: Revert "Upgrade t-digest to latest release (#7029)"

tdunning edited a comment on pull request #7069: URL: https://github.com/apache/incubator-pinot/pull/7069#issuecomment-863828785 I have done an analysis here. I believe that the root of the issue is that the t-digest takes the compression factor more seriously now than it did before. That

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #7068: Make BitmapDocIdSetOperator a top level operator

Jackie-Jiang merged pull request #7068: URL: https://github.com/apache/incubator-pinot/pull/7068 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this se

[GitHub] [incubator-pinot] wuwenw commented on a change in pull request #7052: Add Query Options for GroupBy In-Segment Trim

wuwenw commented on a change in pull request #7052: URL: https://github.com/apache/incubator-pinot/pull/7052#discussion_r654124536 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/QueryOptions.java ## @@ -82,4 +98,24 @@ public static Long getTimeoutMs(Map quer

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #7062: Support data preprocessing for AVRO and ORC formats

mcvsubbu commented on a change in pull request #7062: URL: https://github.com/apache/incubator-pinot/pull/7062#discussion_r654042893 ## File path: pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop/src/main/java/org/apache/pinot/hadoop/job/HadoopSegmentPreprocessingJob.jav

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7071: Clear legacy configs when converting to new TableConfig.

codecov-commenter edited a comment on pull request #7071: URL: https://github.com/apache/incubator-pinot/pull/7071#issuecomment-863596801 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[incubator-pinot] branch master updated: Add Query Options for GroupBy In-Segment Trim (#7052)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new f773208 Add Query Options for GroupBy

[incubator-pinot] branch master updated (18051eb -> 849b4c1)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 18051eb Clear legacy configs when converting to new TableConfig. (#7071) add 849b4c1 Make BitmapDocI