[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6640: Make Pinot metrics pluggable

2021-03-05 Thread GitBox
codecov-io edited a comment on pull request #6640: URL: https://github.com/apache/incubator-pinot/pull/6640#issuecomment-790374407 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6640?src=pr&el=h1) Report > Merging [#6640](https://codecov.io/gh/apache/incubator-pinot/pull

svn commit: r46483 - /release/incubator/pinot/KEYS

2021-03-05 Thread yupeng
Author: yupeng Date: Sat Mar 6 05:24:19 2021 New Revision: 46483 Log: Update Pinot KEYS file for F41AB2B856C6C29D Modified: release/incubator/pinot/KEYS Modified: release/incubator/pinot/KEYS == --- release/incubat

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6653: add uploadLLCSegment endpoint in TableResource

2021-03-05 Thread GitBox
codecov-io edited a comment on pull request #6653: URL: https://github.com/apache/incubator-pinot/pull/6653#issuecomment-791825604 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6653?src=pr&el=h1) Report > Merging [#6653](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] liuchang0520 commented on pull request #6653: add uploadLLCSegment endpoint in TableResource

2021-03-05 Thread GitBox
liuchang0520 commented on pull request #6653: URL: https://github.com/apache/incubator-pinot/pull/6653#issuecomment-791844297 > Hi @liuchang0520, this feature is already being worked on at #6567. > > Are you aware of this? Hi @snleee , my understanding is this change covers a d

[GitHub] [incubator-pinot] liuchang0520 removed a comment on pull request #6653: add uploadLLCSegment endpoint in TableResource

2021-03-05 Thread GitBox
liuchang0520 removed a comment on pull request #6653: URL: https://github.com/apache/incubator-pinot/pull/6653#issuecomment-791840974 > Hi @liuchang0520, this feature is already being worked on at #6567. > > Are you aware of this?

[GitHub] [incubator-pinot] liuchang0520 commented on pull request #6653: add uploadLLCSegment endpoint in TableResource

2021-03-05 Thread GitBox
liuchang0520 commented on pull request #6653: URL: https://github.com/apache/incubator-pinot/pull/6653#issuecomment-791840974 > Hi @liuchang0520, this feature is already being worked on at #6567. > > Are you aware of this?

[GitHub] [incubator-pinot] liuchang0520 closed pull request #6653: add uploadLLCSegment endpoint in TableResource

2021-03-05 Thread GitBox
liuchang0520 closed pull request #6653: URL: https://github.com/apache/incubator-pinot/pull/6653 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [incubator-pinot] snleee commented on pull request #6649: Skip loading columns not in the schema

2021-03-05 Thread GitBox
snleee commented on pull request #6649: URL: https://github.com/apache/incubator-pinot/pull/6649#issuecomment-791834095 Does this imply that we now support dropping columns? This is an automated message from the Apache Git Se

[GitHub] [incubator-pinot] snleee edited a comment on pull request #6653: add uploadLLCSegment endpoint in TableResource

2021-03-05 Thread GitBox
snleee edited a comment on pull request #6653: URL: https://github.com/apache/incubator-pinot/pull/6653#issuecomment-791833134 Hi @liuchang0520, this feature is already being worked on at https://github.com/apache/incubator-pinot/pull/6567. Are you aware of this?

[GitHub] [incubator-pinot] snleee commented on pull request #6653: add uploadLLCSegment endpoint in TableResource

2021-03-05 Thread GitBox
snleee commented on pull request #6653: URL: https://github.com/apache/incubator-pinot/pull/6653#issuecomment-791833134 Hi @liuchang0520, similar work is currently worked from https://github.com/apache/incubator-pinot/pull/6567. Are you aware of this?

[GitHub] [incubator-pinot] codecov-io commented on pull request #6653: add uploadLLCSegment endpoint in TableResource

2021-03-05 Thread GitBox
codecov-io commented on pull request #6653: URL: https://github.com/apache/incubator-pinot/pull/6653#issuecomment-791825604 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6653?src=pr&el=h1) Report > Merging [#6653](https://codecov.io/gh/apache/incubator-pinot/pull/6653?s

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #6650: complete compatibility regression testing

2021-03-05 Thread GitBox
mcvsubbu commented on a change in pull request #6650: URL: https://github.com/apache/incubator-pinot/pull/6650#discussion_r588800523 ## File path: compatibility-verifier/compCheck.sh ## @@ -34,61 +34,45 @@ # and run all the scripts in the directory in alpha order, one script

[GitHub] [incubator-pinot] liuchang0520 opened a new pull request #6653: add uploadLLCSegment endpoint in TableResource

2021-03-05 Thread GitBox
liuchang0520 opened a new pull request #6653: URL: https://github.com/apache/incubator-pinot/pull/6653 ## Description To complete [deep store bypass for realtime segment completion](https://cwiki.apache.org/confluence/display/PINOT/By-passing+deep-store+requirement+for+Realtime+segment+c

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6650: complete compatibility regression testing

2021-03-05 Thread GitBox
codecov-io edited a comment on pull request #6650: URL: https://github.com/apache/incubator-pinot/pull/6650#issuecomment-791657823 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6650?src=pr&el=h1) Report > Merging [#6650](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] mqliang closed pull request #6650: complete compatibility regression testing

2021-03-05 Thread GitBox
mqliang closed pull request #6650: URL: https://github.com/apache/incubator-pinot/pull/6650 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [incubator-pinot] liuchang0520 opened a new issue #6652: add new endpoint to upload low level consumer segment to segment store

2021-03-05 Thread GitBox
liuchang0520 opened a new issue #6652: URL: https://github.com/apache/incubator-pinot/issues/6652 To complete [deep store bypass for realtime segment completion](https://cwiki.apache.org/confluence/display/PINOT/By-passing+deep-store+requirement+for+Realtime+segment+completion), we need to

[incubator-pinot] branch make-metrics-pluggable updated (1cbf29c -> 5a3c787)

2021-03-05 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch make-metrics-pluggable in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard 1cbf29c Address PR comment discard dfa6ed2 Address PR comments discard b190a45 Mark pinot

[incubator-pinot] branch master updated: Remove methods in PinotMetricsRegistryListener (#6630)

2021-03-05 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new 5137025 Remove methods in PinotMetricsRe

[GitHub] [incubator-pinot] jackjlli merged pull request #6630: Remove methods in PinotMetricsRegistryListener

2021-03-05 Thread GitBox
jackjlli merged pull request #6630: URL: https://github.com/apache/incubator-pinot/pull/6630 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] codecov-io commented on pull request #6651: Don't throw exception in json_extract_scalar when default value is available.

2021-03-05 Thread GitBox
codecov-io commented on pull request #6651: URL: https://github.com/apache/incubator-pinot/pull/6651#issuecomment-791742340 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6651?src=pr&el=h1) Report > Merging [#6651](https://codecov.io/gh/apache/incubator-pinot/pull/6651?s

[incubator-pinot] branch make-metrics-pluggable updated (dfa6ed2 -> 1cbf29c)

2021-03-05 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch make-metrics-pluggable in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from dfa6ed2 Address PR comments add 1cbf29c Address PR comment No new revisions were added

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #6640: Make Pinot metrics pluggable

2021-03-05 Thread GitBox
jackjlli commented on a change in pull request #6640: URL: https://github.com/apache/incubator-pinot/pull/6640#discussion_r588716590 ## File path: pinot-common/src/main/java/org/apache/pinot/common/metrics/PinotMetricUtils.java ## @@ -18,82 +18,186 @@ */ package org.apache.

[GitHub] [incubator-pinot] amrishlal opened a new pull request #6651: Don't throw exception in json_extract_scalar when default value is available.

2021-03-05 Thread GitBox
amrishlal opened a new pull request #6651: URL: https://github.com/apache/incubator-pinot/pull/6651 ## Description JSON data is stored in columns of type STRING, so there is nothing preventing the column from storing bad json string. Bad JSON string in columns will cause json_extract_sc

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6640: Make Pinot metrics pluggable

2021-03-05 Thread GitBox
fx19880617 commented on a change in pull request #6640: URL: https://github.com/apache/incubator-pinot/pull/6640#discussion_r588708121 ## File path: pinot-common/src/main/java/org/apache/pinot/common/metrics/PinotMetricUtils.java ## @@ -18,82 +18,186 @@ */ package org.apach

[GitHub] [incubator-pinot] npawar commented on a change in pull request #6518: Kinesis Connector

2021-03-05 Thread GitBox
npawar commented on a change in pull request #6518: URL: https://github.com/apache/incubator-pinot/pull/6518#discussion_r588703998 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/stream/Checkpoint.java ## @@ -0,0 +1,29 @@ +/** + * Licensed to the Apache Software Foun

[GitHub] [incubator-pinot] npawar closed pull request #5455: Use java.io.tmpdir in KafkaStarterUtils

2021-03-05 Thread GitBox
npawar closed pull request #5455: URL: https://github.com/apache/incubator-pinot/pull/5455 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [incubator-pinot] jackjlli commented on issue #6453: Handle creation of empty segment

2021-03-05 Thread GitBox
jackjlli commented on issue #6453: URL: https://github.com/apache/incubator-pinot/issues/6453#issuecomment-791675203 Hi, what's the pros and cons between creating an empty segment v.s. not creating a segment, e.g. in Kinesis connector? Failing the segment creation could be helpful t

[GitHub] [incubator-pinot] mcvsubbu commented on issue #6645: Support multiple levels of config validation

2021-03-05 Thread GitBox
mcvsubbu commented on issue #6645: URL: https://github.com/apache/incubator-pinot/issues/6645#issuecomment-791665873 Perhaps we should list all the checks that we WANT to perform, and then make sure that they get done in one go, so that we can have one (maybe two) levels of validation. Th

[GitHub] [incubator-pinot] mcvsubbu commented on issue #6645: Support multiple levels of config validation

2021-03-05 Thread GitBox
mcvsubbu commented on issue #6645: URL: https://github.com/apache/incubator-pinot/issues/6645#issuecomment-791664266 > Shall we list down all the validations we have now? I feel all (or at least most) of them are **must have** for pinot to properly work Lol! Pinot has been working f

[incubator-pinot] branch master updated (36803c0 -> 779c29c)

2021-03-05 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 36803c0 Recover the segment from controller when LLC table cannot load it (#6647) add 779c29c Do not

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #6632: Do not log warning when gRPC or admin port is not configured for an instance

2021-03-05 Thread GitBox
Jackie-Jiang merged pull request #6632: URL: https://github.com/apache/incubator-pinot/pull/6632 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [incubator-pinot] codecov-io commented on pull request #6650: complete compatibility regression testing

2021-03-05 Thread GitBox
codecov-io commented on pull request #6650: URL: https://github.com/apache/incubator-pinot/pull/6650#issuecomment-791657823 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6650?src=pr&el=h1) Report > Merging [#6650](https://codecov.io/gh/apache/incubator-pinot/pull/6650?s

[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6645: Support multiple levels of config validation

2021-03-05 Thread GitBox
Jackie-Jiang commented on issue #6645: URL: https://github.com/apache/incubator-pinot/issues/6645#issuecomment-791641953 Shall we list down all the validations we have now? I feel all (or at least most) of them are **must have** for pinot to properly work

[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6648: Support nested json schema in realtime ingest

2021-03-05 Thread GitBox
Jackie-Jiang commented on issue #6648: URL: https://github.com/apache/incubator-pinot/issues/6648#issuecomment-791640826 For the json field, you can configure it as string field in the schema, and enable json index for it (Check https://docs.pinot.apache.org/basics/indexing/json-index for

[incubator-pinot] branch master updated (b2d716d -> 36803c0)

2021-03-05 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from b2d716d Add generation number to data and queries (#6638) add 36803c0 Recover the segment from contro

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #6647: Recover the segment from controller when LLC table cannot load it

2021-03-05 Thread GitBox
Jackie-Jiang merged pull request #6647: URL: https://github.com/apache/incubator-pinot/pull/6647 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [incubator-pinot] mqliang opened a new pull request #6650: complete compatibility regression testing

2021-03-05 Thread GitBox
mqliang opened a new pull request #6650: URL: https://github.com/apache/incubator-pinot/pull/6650 ## Description * split compCheck.sh as two script so we can build and test separately * redirect services' logs to a file for easy of debug * complete and enable test during rolling upg

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #6640: Make Pinot metrics pluggable

2021-03-05 Thread GitBox
jackjlli commented on a change in pull request #6640: URL: https://github.com/apache/incubator-pinot/pull/6640#discussion_r588542959 ## File path: pinot-distribution/pinot-assembly.xml ## @@ -123,11 +123,17 @@ + Review comment: Good catch! 👍 Updat

[incubator-pinot] branch make-metrics-pluggable updated (b190a45 -> dfa6ed2)

2021-03-05 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch make-metrics-pluggable in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from b190a45 Mark pinot-yammer as test scope add dfa6ed2 Address PR comments No new revisio

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #6640: Make Pinot metrics pluggable

2021-03-05 Thread GitBox
jackjlli commented on a change in pull request #6640: URL: https://github.com/apache/incubator-pinot/pull/6640#discussion_r588538093 ## File path: pinot-common/src/main/java/org/apache/pinot/common/metrics/PinotMetricUtils.java ## @@ -18,82 +18,186 @@ */ package org.apache.

[GitHub] [incubator-pinot] fx19880617 commented on pull request #6640: Make Pinot metrics pluggable

2021-03-05 Thread GitBox
fx19880617 commented on pull request #6640: URL: https://github.com/apache/incubator-pinot/pull/6640#issuecomment-791305421 @chenboat This is an important step towards pluggable metrics. I think you can create pinot-dropwizard very soon. ---

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6640: Make Pinot metrics pluggable

2021-03-05 Thread GitBox
fx19880617 commented on a change in pull request #6640: URL: https://github.com/apache/incubator-pinot/pull/6640#discussion_r588164301 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/annotations/metrics/MetricsFactory.java ## @@ -0,0 +1,39 @@ +/** + * Licensed to th

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6640: Make Pinot metrics pluggable

2021-03-05 Thread GitBox
fx19880617 commented on a change in pull request #6640: URL: https://github.com/apache/incubator-pinot/pull/6640#discussion_r588161265 ## File path: pinot-distribution/pinot-assembly.xml ## @@ -123,11 +123,17 @@ + Review comment: Switched the comme

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6640: Make Pinot metrics pluggable

2021-03-05 Thread GitBox
fx19880617 commented on a change in pull request #6640: URL: https://github.com/apache/incubator-pinot/pull/6640#discussion_r588158268 ## File path: pinot-common/src/main/java/org/apache/pinot/common/metrics/PinotMetricUtils.java ## @@ -18,82 +18,186 @@ */ package org.apach

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6640: Make Pinot metrics pluggable

2021-03-05 Thread GitBox
fx19880617 commented on a change in pull request #6640: URL: https://github.com/apache/incubator-pinot/pull/6640#discussion_r588158268 ## File path: pinot-common/src/main/java/org/apache/pinot/common/metrics/PinotMetricUtils.java ## @@ -18,82 +18,186 @@ */ package org.apach

svn commit: r46475 - /dev/incubator/pinot/apache-pinot-incubating-0.7.0-rc0/

2021-03-05 Thread yupeng
Author: yupeng Date: Fri Mar 5 09:07:32 2021 New Revision: 46475 Log: Update apache-pinot-incubating-0.7.0-rc0 Added: dev/incubator/pinot/apache-pinot-incubating-0.7.0-rc0/ dev/incubator/pinot/apache-pinot-incubating-0.7.0-rc0/apache-pinot-incubating-0.7.0-bin.tar.gz (with props)

[GitHub] [incubator-pinot] dubin555 commented on issue #6648: Support nested json schema in realtime ingest

2021-03-05 Thread GitBox
dubin555 commented on issue #6648: URL: https://github.com/apache/incubator-pinot/issues/6648#issuecomment-791256094 Also does the kafka realtime ingest part have resource control? e.g. how many consumer thread This is an a