[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7099: Add minion metrics of task queueing time and task numbers

2021-06-28 Thread GitBox
codecov-commenter edited a comment on pull request #7099: URL: https://github.com/apache/incubator-pinot/pull/7099#issuecomment-870103030 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7099?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7099: Add minion metrics of task queueing time and task numbers

2021-06-28 Thread GitBox
codecov-commenter edited a comment on pull request #7099: URL: https://github.com/apache/incubator-pinot/pull/7099#issuecomment-870103030 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7099?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7102: Add controller/server API to fetch aggregated segment metadata

2021-06-28 Thread GitBox
codecov-commenter edited a comment on pull request #7102: URL: https://github.com/apache/incubator-pinot/pull/7102#issuecomment-870233688 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7102?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7101: Add `PinotHelixResourceManager` to `TableConfigTuner.init()` method.

2021-06-28 Thread GitBox
codecov-commenter edited a comment on pull request #7101: URL: https://github.com/apache/incubator-pinot/pull/7101#issuecomment-870205551 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7101?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7101: Add `PinotHelixResourceManager` to `TableConfigTuner.init()` method.

2021-06-28 Thread GitBox
codecov-commenter edited a comment on pull request #7101: URL: https://github.com/apache/incubator-pinot/pull/7101#issuecomment-870205551 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7101?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7102: Add controller/server API to fetch aggregated segment metadata

2021-06-28 Thread GitBox
codecov-commenter edited a comment on pull request #7102: URL: https://github.com/apache/incubator-pinot/pull/7102#issuecomment-870233688 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7102?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7102: Add controller/server API to fetch aggregated segment metadata

2021-06-28 Thread GitBox
codecov-commenter commented on pull request #7102: URL: https://github.com/apache/incubator-pinot/pull/7102#issuecomment-870233688 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7102?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #7101: Add `PinotHelixResourceManager` to `TableConfigTuner.init()` method.

2021-06-28 Thread GitBox
mayankshriv commented on a change in pull request #7101: URL: https://github.com/apache/incubator-pinot/pull/7101#discussion_r660275816 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/tuner/TableConfigTuner.java ## @@ -27,11 +29,18 @@ * Interface for

[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #7101: Add `PinotHelixResourceManager` to `TableConfigTuner.init()` method.

2021-06-28 Thread GitBox
mayankshriv commented on a change in pull request #7101: URL: https://github.com/apache/incubator-pinot/pull/7101#discussion_r660274576 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/tuner/TableConfigTuner.java ## @@ -27,11 +29,18 @@ * Interface for

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7101: Add `PinotHelixResourceManager` to `TableConfigTuner.init()` method.

2021-06-28 Thread GitBox
Jackie-Jiang commented on a change in pull request #7101: URL: https://github.com/apache/incubator-pinot/pull/7101#discussion_r660263784 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/tuner/TableConfigTuner.java ## @@ -27,11 +29,18 @@ * Interface fo

[GitHub] [incubator-pinot] minwoo-jung edited a comment on issue #7100: [question] Even if I recreate the kafka topic or modify the topic properties, I wonder how consuming can continue to do it.

2021-06-28 Thread GitBox
minwoo-jung edited a comment on issue #7100: URL: https://github.com/apache/incubator-pinot/issues/7100#issuecomment-870210632 @mcvsubbu Is there any way(workaround) to lose some data in the table and recover consuming **without recreating the table**? For example, Data stored in the

[GitHub] [incubator-pinot] minwoo-jung commented on issue #7100: [question] Even if I recreate the kafka topic or modify the topic properties, I wonder how consuming can continue to do it.

2021-06-28 Thread GitBox
minwoo-jung commented on issue #7100: URL: https://github.com/apache/incubator-pinot/issues/7100#issuecomment-870210632 Is there any way(workaround) to lose some data in the table and recover consuming **without recreating the table**? For example, Data stored in the consuming segment i

[GitHub] [incubator-pinot] minwoo-jung edited a comment on issue #7100: [question] Even if I recreate the kafka topic or modify the topic properties, I wonder how consuming can continue to do it.

2021-06-28 Thread GitBox
minwoo-jung edited a comment on issue #7100: URL: https://github.com/apache/incubator-pinot/issues/7100#issuecomment-870208681 @mcvsubbu First of all, thanks for the quick reply. :) This means that the data in the previously saved table is maintained, and Messages in Kafka can be los

[GitHub] [incubator-pinot] minwoo-jung commented on issue #7100: [question] Even if I recreate the kafka topic or modify the topic properties, I wonder how consuming can continue to do it.

2021-06-28 Thread GitBox
minwoo-jung commented on issue #7100: URL: https://github.com/apache/incubator-pinot/issues/7100#issuecomment-870208681 @mcvsubbu First of all, thanks for the quick reply. :) This means that the data in the previously saved table is maintained, and the data in kafka is irrelevant eve

[GitHub] [incubator-pinot] siddharthteotia commented on pull request #7102: Add controller/server API to fetch aggregated segment metadata

2021-06-28 Thread GitBox
siddharthteotia commented on pull request #7102: URL: https://github.com/apache/incubator-pinot/pull/7102#issuecomment-870207105 @mqliang , it will be great if you can update the PR with outputs from local runs for various scenarios just like you did in the previous PR. Also, we should add

[GitHub] [incubator-pinot] mqliang opened a new pull request #7102: Add controller/server API to fetch aggregated segment metadata

2021-06-28 Thread GitBox
mqliang opened a new pull request #7102: URL: https://github.com/apache/incubator-pinot/pull/7102 ## Description ## Upgrade Notes Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion) * [ ] Yes (Please label as **backward-inco

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7101: Add `PinotHelixResourceManager` to `TableConfigTuner.init()` method.

2021-06-28 Thread GitBox
codecov-commenter edited a comment on pull request #7101: URL: https://github.com/apache/incubator-pinot/pull/7101#issuecomment-870205551 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7101?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7101: Add `PinotHelixResourceManager` to `TableConfigTuner.init()` method.

2021-06-28 Thread GitBox
codecov-commenter commented on pull request #7101: URL: https://github.com/apache/incubator-pinot/pull/7101#issuecomment-870205551 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7101?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[GitHub] [incubator-pinot] mcvsubbu commented on issue #7100: [question] Even if I recreate the kafka topic or modify the topic properties, I wonder how consuming can continue to do it.

2021-06-28 Thread GitBox
mcvsubbu commented on issue #7100: URL: https://github.com/apache/incubator-pinot/issues/7100#issuecomment-870200636 If you don't care about data loss, then you can do the following: - Drop your table - Recreate the table with the correct topic. -- This is an automated message from

[GitHub] [incubator-pinot] minwoo-jung commented on issue #7100: [question] Even if I recreate the kafka topic or modify the topic properties, I wonder how consuming can continue to do it.

2021-06-28 Thread GitBox
minwoo-jung commented on issue #7100: URL: https://github.com/apache/incubator-pinot/issues/7100#issuecomment-870197371 It doesn't matter how much data is lost. Only comsuming should be restored and operated normally.:) -- This is an automated message from the Apache Git Service. To res

[GitHub] [incubator-pinot] mayankshriv opened a new pull request #7101: Add `PinotHelixResourceManager` to `TableConfigTuner.init()` method.

2021-06-28 Thread GitBox
mayankshriv opened a new pull request #7101: URL: https://github.com/apache/incubator-pinot/pull/7101 ## Description The current tuner interface only has access to table config, schema, and tuner specific properties. Most tuners would also need more information from the system, suc

[GitHub] [incubator-pinot] minwoo-jung opened a new issue #7100: [question] **Even if I recreate the kafka topic or modify the topic properties, I wonder how consuming can continue to do it.**

2021-06-28 Thread GitBox
minwoo-jung opened a new issue #7100: URL: https://github.com/apache/incubator-pinot/issues/7100 Hello~ I am using pinot well. Thank you for making a great product. :) I have a question because it didn't work as I thought while using it. **Even if I recreate the kafka to

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7064: Allow updating controller and broker helix hostname

2021-06-28 Thread GitBox
codecov-commenter edited a comment on pull request #7064: URL: https://github.com/apache/incubator-pinot/pull/7064#issuecomment-861995023 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7064?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[incubator-pinot] branch master updated (8283d39 -> 3503074)

2021-06-28 Thread mayanks
This is an automated email from the ASF dual-hosted git repository. mayanks pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 8283d39 Enhance PinotHelixResourceManager.updateInstance() to preserve custom fields (#7098) add 350

[GitHub] [incubator-pinot] mayankshriv merged pull request #7093: Add system info on instance config for servers.

2021-06-28 Thread GitBox
mayankshriv merged pull request #7093: URL: https://github.com/apache/incubator-pinot/pull/7093 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: co

[GitHub] [incubator-pinot] dongxiaoman commented on a change in pull request #7064: Allow updating controller and broker helix hostname

2021-06-28 Thread GitBox
dongxiaoman commented on a change in pull request #7064: URL: https://github.com/apache/incubator-pinot/pull/7064#discussion_r660200155 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java ## @@ -495,4 +499,104 @@ public IdealState appl

[GitHub] [incubator-pinot] dongxiaoman commented on a change in pull request #7064: Allow updating controller and broker helix hostname

2021-06-28 Thread GitBox
dongxiaoman commented on a change in pull request #7064: URL: https://github.com/apache/incubator-pinot/pull/7064#discussion_r660199802 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java ## @@ -495,4 +499,104 @@ public IdealState appl

[GitHub] [incubator-pinot] dongxiaoman commented on a change in pull request #7064: Allow updating controller and broker helix hostname

2021-06-28 Thread GitBox
dongxiaoman commented on a change in pull request #7064: URL: https://github.com/apache/incubator-pinot/pull/7064#discussion_r660199645 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java ## @@ -495,4 +499,104 @@ public IdealState appl

[GitHub] [incubator-pinot] dongxiaoman commented on a change in pull request #7064: Allow updating controller and broker helix hostname

2021-06-28 Thread GitBox
dongxiaoman commented on a change in pull request #7064: URL: https://github.com/apache/incubator-pinot/pull/7064#discussion_r660199512 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java ## @@ -495,4 +499,104 @@ public IdealState appl

[GitHub] [incubator-pinot] dongxiaoman commented on a change in pull request #7064: Allow updating controller and broker helix hostname

2021-06-28 Thread GitBox
dongxiaoman commented on a change in pull request #7064: URL: https://github.com/apache/incubator-pinot/pull/7064#discussion_r660199258 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java ## @@ -495,4 +499,104 @@ public IdealState appl

[GitHub] [incubator-pinot] dongxiaoman commented on a change in pull request #7064: Allow updating controller and broker helix hostname

2021-06-28 Thread GitBox
dongxiaoman commented on a change in pull request #7064: URL: https://github.com/apache/incubator-pinot/pull/7064#discussion_r660198790 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java ## @@ -495,4 +499,104 @@ public IdealState appl

[GitHub] [incubator-pinot] dongxiaoman commented on a change in pull request #7064: Allow updating controller and broker helix hostname

2021-06-28 Thread GitBox
dongxiaoman commented on a change in pull request #7064: URL: https://github.com/apache/incubator-pinot/pull/7064#discussion_r660197626 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java ## @@ -495,4 +499,104 @@ public IdealState appl

[incubator-pinot] branch master updated (43a67b8 -> 8283d39)

2021-06-28 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 43a67b8 Fix the deleted segments directory not exist warning (#7097) add 8283d39 Enhance PinotHelixRe

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #7098: Enhance PinotHelixResourceManager.updateInstance() to preserve custom fields

2021-06-28 Thread GitBox
Jackie-Jiang merged pull request #7098: URL: https://github.com/apache/incubator-pinot/pull/7098 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: c

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7098: Enhance PinotHelixResourceManager.updateInstance() to preserve custom fields

2021-06-28 Thread GitBox
codecov-commenter edited a comment on pull request #7098: URL: https://github.com/apache/incubator-pinot/pull/7098#issuecomment-870062808 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7098?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7098: Enhance PinotHelixResourceManager.updateInstance() to preserve custom fields

2021-06-28 Thread GitBox
codecov-commenter edited a comment on pull request #7098: URL: https://github.com/apache/incubator-pinot/pull/7098#issuecomment-870062808 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7098?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7087: Implement Append merger for partial upsert

2021-06-28 Thread GitBox
codecov-commenter edited a comment on pull request #7087: URL: https://github.com/apache/incubator-pinot/pull/7087#issuecomment-867251765 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7087?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7099: Add minion metrics of task queueing time and task numbers

2021-06-28 Thread GitBox
codecov-commenter edited a comment on pull request #7099: URL: https://github.com/apache/incubator-pinot/pull/7099#issuecomment-870103030 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7099?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7099: Add minion metrics of task queueing time and task numbers

2021-06-28 Thread GitBox
codecov-commenter commented on pull request #7099: URL: https://github.com/apache/incubator-pinot/pull/7099#issuecomment-870103030 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7099?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7064: Allow updating controller and broker helix hostname

2021-06-28 Thread GitBox
Jackie-Jiang commented on a change in pull request #7064: URL: https://github.com/apache/incubator-pinot/pull/7064#discussion_r660158809 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java ## @@ -495,4 +499,104 @@ public IdealState app

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7098: Enhance PinotHelixResourceManager.updateInstance() to preserve custom fields

2021-06-28 Thread GitBox
codecov-commenter edited a comment on pull request #7098: URL: https://github.com/apache/incubator-pinot/pull/7098#issuecomment-870062808 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7098?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #7099: Add minion metrics of task queueing time and task numbers

2021-06-28 Thread GitBox
mcvsubbu commented on a change in pull request #7099: URL: https://github.com/apache/incubator-pinot/pull/7099#discussion_r660152383 ## File path: pinot-minion/src/main/java/org/apache/pinot/minion/taskfactory/TaskFactoryRegistry.java ## @@ -108,6 +120,8 @@ public TaskResult r

[GitHub] [incubator-pinot] Jackie-Jiang closed issue #7082: Fix handling of missing Deleted_Segments directory in SegmentDeletionManager

2021-06-28 Thread GitBox
Jackie-Jiang closed issue #7082: URL: https://github.com/apache/incubator-pinot/issues/7082 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commit

[incubator-pinot] branch master updated (c950926 -> 43a67b8)

2021-06-28 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from c950926 Added TaskMetricsEmitted periodic controller job (#7091) add 43a67b8 Fix the deleted segments

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #7097: Fix the deleted segments directory not exist warning

2021-06-28 Thread GitBox
Jackie-Jiang merged pull request #7097: URL: https://github.com/apache/incubator-pinot/pull/7097 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: c

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7085: modify segment generator to not return illegal filenames

2021-06-28 Thread GitBox
Jackie-Jiang commented on a change in pull request #7085: URL: https://github.com/apache/incubator-pinot/pull/7085#discussion_r660143563 ## File path: pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/creator/name/NormalizedDateSegmentNameGenerator.java ## @@ -90,10

[GitHub] [incubator-pinot] mqliang closed issue #7088: Add CRC validation on pinot-minion to drop duplicate tasks

2021-06-28 Thread GitBox
mqliang closed issue #7088: URL: https://github.com/apache/incubator-pinot/issues/7088 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-uns

[GitHub] [incubator-pinot] mqliang commented on issue #7088: Add CRC validation on pinot-minion to drop duplicate tasks

2021-06-28 Thread GitBox
mqliang commented on issue #7088: URL: https://github.com/apache/incubator-pinot/issues/7088#issuecomment-870074544 close this as the PR has been merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] mqliang opened a new pull request #7099: Add minion metrics of task queueing time and task numbers

2021-06-28 Thread GitBox
mqliang opened a new pull request #7099: URL: https://github.com/apache/incubator-pinot/pull/7099 ## Description This PR add minion metric/Gauge of: * task queuing time: task_dequeue_time - task_inqueue_time (assume the time drift between helix controller and pinot minion is n

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7087: Implement Append merger for partial upsert

2021-06-28 Thread GitBox
Jackie-Jiang commented on a change in pull request #7087: URL: https://github.com/apache/incubator-pinot/pull/7087#discussion_r660140790 ## File path: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/merger/UnionMerger.java ## @@ -0,0 +1,47 @@ +/** + * L

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7098: Enhance PinotHelixResourceManager.updateInstance() to preserve custom fields

2021-06-28 Thread GitBox
codecov-commenter commented on pull request #7098: URL: https://github.com/apache/incubator-pinot/pull/7098#issuecomment-870062808 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7098?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[GitHub] [incubator-pinot] mcvsubbu commented on issue #7084: Need more metrics on minion task status

2021-06-28 Thread GitBox
mcvsubbu commented on issue #7084: URL: https://github.com/apache/incubator-pinot/issues/7084#issuecomment-870039473 PR #7091 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[incubator-pinot] branch master updated: Added TaskMetricsEmitted periodic controller job (#7091)

2021-06-28 Thread mcvsubbu
This is an automated email from the ASF dual-hosted git repository. mcvsubbu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new c950926 Added TaskMetricsEmitted per

[GitHub] [incubator-pinot] mcvsubbu merged pull request #7091: Added TaskMetricsEmitted periodic controller job

2021-06-28 Thread GitBox
mcvsubbu merged pull request #7091: URL: https://github.com/apache/incubator-pinot/pull/7091 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commi

[GitHub] [incubator-pinot] Jackie-Jiang opened a new pull request #7098: Enhance PinotHelixResourceManager.updateInstance() to preserve custom fields

2021-06-28 Thread GitBox
Jackie-Jiang opened a new pull request #7098: URL: https://github.com/apache/incubator-pinot/pull/7098 When using rest API to update the instance config, preserve the custom fields. Some example custom fields are: fault domain, resource info etc. Also add support to custom broker hostnam

[GitHub] [incubator-pinot] Jackie-Jiang commented on pull request #7097: Fix the deleted segments directory not exist warning

2021-06-28 Thread GitBox
Jackie-Jiang commented on pull request #7097: URL: https://github.com/apache/incubator-pinot/pull/7097#issuecomment-870018268 @kkrugler I thought about that solution, and decide to do the current solution of skipping the processing if deleted segments dir does not exist. The deleted segmen

[GitHub] [incubator-pinot] kkrugler commented on pull request #7097: Fix the deleted segments directory not exist warning

2021-06-28 Thread GitBox
kkrugler commented on pull request #7097: URL: https://github.com/apache/incubator-pinot/pull/7097#issuecomment-869965636 Hi @Jackie-Jiang - I thought we were also going to create the deleted segment directory in the `SegmentDeletionManager` constructor. And if so, then log a warning in `r

[incubator-pinot] branch master updated (1561414 -> 0c813bd)

2021-06-28 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 1561414 Fixing docker build scripts by providing JDK_VERSION as parameter (#7095) add 0c813bd Optimize

[GitHub] [incubator-pinot] jackjlli merged pull request #7086: Optimize BaseSingleSegmentConversionExecutor by skipping task if CRC not match

2021-06-28 Thread GitBox
jackjlli merged pull request #7086: URL: https://github.com/apache/incubator-pinot/pull/7086 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commi

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7091: Added TaskMetricsEmitted periodic controller job

2021-06-28 Thread GitBox
codecov-commenter edited a comment on pull request #7091: URL: https://github.com/apache/incubator-pinot/pull/7091#issuecomment-869089682 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7091?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7091: Added TaskMetricsEmitted periodic controller job

2021-06-28 Thread GitBox
codecov-commenter edited a comment on pull request #7091: URL: https://github.com/apache/incubator-pinot/pull/7091#issuecomment-869089682 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7091?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #7082: Fix handling of missing Deleted_Segments directory in SegmentDeletionManager

2021-06-28 Thread GitBox
Jackie-Jiang commented on issue #7082: URL: https://github.com/apache/incubator-pinot/issues/7082#issuecomment-869916753 @kkrugler @mcvsubbu Fix for the issue, please take a look: #7097 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [incubator-pinot] Jackie-Jiang opened a new pull request #7097: Fix the deleted segments directory not exist warning

2021-06-28 Thread GitBox
Jackie-Jiang opened a new pull request #7097: URL: https://github.com/apache/incubator-pinot/pull/7097 Fix the warning log in `SegmentDeletionManager` when the deleted segments directory does not exist (new cluster without any segment removed yet) -- This is an automated message from the

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #7086: Optimize BaseSingleSegmentConversionExecutor by skipping task if CRC not match

2021-06-28 Thread GitBox
mcvsubbu commented on a change in pull request #7086: URL: https://github.com/apache/incubator-pinot/pull/7086#discussion_r659980113 ## File path: pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/BaseSingleSegmentConversion

[GitHub] [incubator-pinot] mqliang commented on a change in pull request #7086: Optimize BaseSingleSegmentConversionExecutor by skipping task if CRC not match

2021-06-28 Thread GitBox
mqliang commented on a change in pull request #7086: URL: https://github.com/apache/incubator-pinot/pull/7086#discussion_r659971793 ## File path: pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/BaseSingleSegmentConversionE

[GitHub] [incubator-pinot] mqliang commented on a change in pull request #7086: Optimize BaseSingleSegmentConversionExecutor by skipping task if CRC not match

2021-06-28 Thread GitBox
mqliang commented on a change in pull request #7086: URL: https://github.com/apache/incubator-pinot/pull/7086#discussion_r659970454 ## File path: pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/BaseSingleSegmentConversionE

[incubator-pinot] 01/01: Fixed a comment

2021-06-28 Thread mcvsubbu
This is an automated email from the ASF dual-hosted git repository. mcvsubbu pushed a commit to branch minion-improvements in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit a9f9e8aab23f5ff9ecbda91b9db5d723e2c6d912 Author: Subbu Subramaniam AuthorDate: Mon Jun 28 10:05:

[incubator-pinot] branch minion-improvements created (now a9f9e8a)

2021-06-28 Thread mcvsubbu
This is an automated email from the ASF dual-hosted git repository. mcvsubbu pushed a change to branch minion-improvements in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at a9f9e8a Fixed a comment This branch includes the following new commits: new a9f9e8a

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #7091: Added TaskMetricsEmitted periodic controller job

2021-06-28 Thread GitBox
mcvsubbu commented on a change in pull request #7091: URL: https://github.com/apache/incubator-pinot/pull/7091#discussion_r659966207 ## File path: pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/SimpleMinionClusterIntegrationTest.java ## @@ -85,20 +85,

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #7091: Added TaskMetricsEmitted periodic controller job

2021-06-28 Thread GitBox
jackjlli commented on a change in pull request #7091: URL: https://github.com/apache/incubator-pinot/pull/7091#discussion_r659952371 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotHelixTaskResourceManager.java ## @@ -249,15 +253

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #7091: Added TaskMetricsEmitted periodic controller job

2021-06-28 Thread GitBox
mcvsubbu commented on a change in pull request #7091: URL: https://github.com/apache/incubator-pinot/pull/7091#discussion_r659947333 ## File path: pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/SimpleMinionClusterIntegrationTest.java ## @@ -85,18 +85,

[GitHub] [incubator-pinot] atris commented on issue #7060: Optimize DISTINCT + ORDER BY

2021-06-28 Thread GitBox
atris commented on issue #7060: URL: https://github.com/apache/incubator-pinot/issues/7060#issuecomment-869822727 Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[GitHub] [incubator-pinot] dongxiaoman commented on a change in pull request #7064: Allow updating controller and broker helix hostname

2021-06-28 Thread GitBox
dongxiaoman commented on a change in pull request #7064: URL: https://github.com/apache/incubator-pinot/pull/7064#discussion_r659925421 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java ## @@ -495,4 +498,104 @@ public IdealState appl

[GitHub] [incubator-pinot] kishoreg commented on issue #7060: Optimize DISTINCT + ORDER BY

2021-06-28 Thread GitBox
kishoreg commented on issue #7060: URL: https://github.com/apache/incubator-pinot/issues/7060#issuecomment-869807440 Done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [incubator-pinot] dongxiaoman commented on a change in pull request #7064: Allow updating controller and broker helix hostname

2021-06-28 Thread GitBox
dongxiaoman commented on a change in pull request #7064: URL: https://github.com/apache/incubator-pinot/pull/7064#discussion_r659915596 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/broker/helix/HelixBrokerStarter.java ## @@ -53,9 +54,7 @@ public HelixBroker

[GitHub] [incubator-pinot] atris commented on issue #7060: Optimize DISTINCT + ORDER BY

2021-06-28 Thread GitBox
atris commented on issue #7060: URL: https://github.com/apache/incubator-pinot/issues/7060#issuecomment-869801003 I would like to take this up please. Please. assign this to me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub