[incubator-pinot] branch fixing_spark_path_validation created (now a039de3)

2021-02-02 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch fixing_spark_path_validation in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at a039de3 Fix the issue in Pinot Spark ingestion job to handle listing input files wi

[incubator-pinot] 01/01: Fix the issue in Pinot Spark ingestion job to handle listing input files with correct scheme in URI

2021-02-02 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch fixing_spark_path_validation in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit a039de3055d7ecf831a84312f4eb84a97e41fc57 Author: Xiang Fu AuthorDate: Tue Feb 2 23:53:48

[GitHub] [incubator-pinot] codecov-io commented on pull request #6536: Default to enable controller task scheduler for minion tasks in Pinot Helm

2021-02-02 Thread GitBox
codecov-io commented on pull request #6536: URL: https://github.com/apache/incubator-pinot/pull/6536#issuecomment-772309634 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6536?src=pr&el=h1) Report > Merging [#6536](https://codecov.io/gh/apache/incubator-pinot/pull/6536?s

[incubator-pinot] branch master updated (a2f0dbd -> b51518f)

2021-02-02 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from a2f0dbd update superset k8s example to latest image (#6534) add b51518f Fix hadoop batch ingest (#65

[GitHub] [incubator-pinot] fx19880617 commented on pull request #6506: Fix hadoop batch ingest

2021-02-02 Thread GitBox
fx19880617 commented on pull request #6506: URL: https://github.com/apache/incubator-pinot/pull/6506#issuecomment-772300880 Thanks for fixing this! This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [incubator-pinot] fx19880617 merged pull request #6506: Fix hadoop batch ingest

2021-02-02 Thread GitBox
fx19880617 merged pull request #6506: URL: https://github.com/apache/incubator-pinot/pull/6506 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-pinot] fx19880617 opened a new pull request #6536: Default to enable controller task scheduler for minion tasks in Pinot Helm

2021-02-02 Thread GitBox
fx19880617 opened a new pull request #6536: URL: https://github.com/apache/incubator-pinot/pull/6536 ## Description Default to enable controller task scheduler for minion tasks in Pinot Helm ## Upgrade Notes Does this PR prevent a zero down-time upgrade? (Assume upgrade orde

[incubator-pinot] 01/01: Default to enable controller task scheduler for minion tasks

2021-02-02 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch default_cron_scheduler_for_k8s_helm in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit df0616538f7eeec169dfd6bcfbc0e86936946a3b Author: Xiang Fu AuthorDate: Tue Feb 2 2

[incubator-pinot] branch default_cron_scheduler_for_k8s_helm created (now df06165)

2021-02-02 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch default_cron_scheduler_for_k8s_helm in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at df06165 Default to enable controller task scheduler for minion tasks This br

[GitHub] [incubator-pinot] siddharthteotia commented on pull request #6530: Update DimTableSegmentAssignment to include both OFFLINE and REALTIME servers

2021-02-02 Thread GitBox
siddharthteotia commented on pull request #6530: URL: https://github.com/apache/incubator-pinot/pull/6530#issuecomment-772237440 > LGTM. > (Not related to this PR) For easier management of the cluster, I would recommend having a separate tenant for the dimension table so that dimension

[GitHub] [incubator-pinot] yupeng9 commented on pull request #6530: Update DimTableSegmentAssignment to include both OFFLINE and REALTIME servers

2021-02-02 Thread GitBox
yupeng9 commented on pull request #6530: URL: https://github.com/apache/incubator-pinot/pull/6530#issuecomment-772225363 > > > LGTM. > > > (Not related to this PR) For easier management of the cluster, I would recommend having a separate tenant for the dimension table so that dimension

[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #6535: JSON_MATCH predicate bug fix and test cases.

2021-02-02 Thread GitBox
siddharthteotia commented on a change in pull request #6535: URL: https://github.com/apache/incubator-pinot/pull/6535#discussion_r569129249 ## File path: pinot-core/src/test/java/org/apache/pinot/queries/JsonMatchPredicateTest.java ## @@ -0,0 +1,291 @@ +/** + * Licensed to the

[GitHub] [incubator-pinot] yupeng9 commented on pull request #6530: Update DimTableSegmentAssignment to include both OFFLINE and REALTIME servers

2021-02-02 Thread GitBox
yupeng9 commented on pull request #6530: URL: https://github.com/apache/incubator-pinot/pull/6530#issuecomment-772224451 > Thanks for the reviews @Jackie-Jiang , @yupeng9 . > > > Since this is the first time we have queries across tables, I think it's a good time to discuss the polic

[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #6535: JSON_MATCH predicate bug fix and test cases.

2021-02-02 Thread GitBox
siddharthteotia commented on a change in pull request #6535: URL: https://github.com/apache/incubator-pinot/pull/6535#discussion_r569127429 ## File path: pinot-core/src/test/java/org/apache/pinot/queries/JsonMatchPredicateTest.java ## @@ -0,0 +1,291 @@ +/** + * Licensed to the

[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #6535: JSON_MATCH predicate bug fix and test cases.

2021-02-02 Thread GitBox
siddharthteotia commented on a change in pull request #6535: URL: https://github.com/apache/incubator-pinot/pull/6535#discussion_r569123015 ## File path: pinot-core/src/test/java/org/apache/pinot/queries/JsonMatchPredicateTest.java ## @@ -0,0 +1,291 @@ +/** + * Licensed to the

[GitHub] [incubator-pinot] amrishlal opened a new pull request #6535: JSON_MATCH predicate bug fix and test cases.

2021-02-02 Thread GitBox
amrishlal opened a new pull request #6535: URL: https://github.com/apache/incubator-pinot/pull/6535 This PR fixes a bug to allow using JSON_MATCH predicate in SQL queries and adds SQL level unit test cases for JSON_MATCH predicate. -`QueryContextConverterUtils.java` was modified to f

[incubator-pinot] branch master updated (d1a0688 -> a2f0dbd)

2021-02-02 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from d1a0688 [THIRDEYE-3799](fix): the alert item style fix (#6504) add a2f0dbd update superset k8s examp

[GitHub] [incubator-pinot] cbalci commented on pull request #6530: Update DimTableSegmentAssignment to include both OFFLINE and REALTIME servers

2021-02-02 Thread GitBox
cbalci commented on pull request #6530: URL: https://github.com/apache/incubator-pinot/pull/6530#issuecomment-772211536 Thanks for the reviews @Jackie-Jiang , @yupeng9 . > Since this is the first time we have queries across tables, I think it's a good time to discuss the policy. The

[GitHub] [incubator-pinot] fx19880617 merged pull request #6534: update superset k8s example to the latest image

2021-02-02 Thread GitBox
fx19880617 merged pull request #6534: URL: https://github.com/apache/incubator-pinot/pull/6534 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-pinot] cbalci commented on a change in pull request #6530: Update DimTableSegmentAssignment to include both OFFLINE and REALTIME servers

2021-02-02 Thread GitBox
cbalci commented on a change in pull request #6530: URL: https://github.com/apache/incubator-pinot/pull/6530#discussion_r569116464 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/OfflineDimTableSegmentAssignment.java ## @@

[GitHub] [incubator-pinot] siddharthteotia commented on pull request #6530: Update DimTableSegmentAssignment to include both OFFLINE and REALTIME servers

2021-02-02 Thread GitBox
siddharthteotia commented on pull request #6530: URL: https://github.com/apache/incubator-pinot/pull/6530#issuecomment-772196569 > > LGTM. > > (Not related to this PR) For easier management of the cluster, I would recommend having a separate tenant for the dimension table so that dimensi

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6534: update superset k8s example to the latest image

2021-02-02 Thread GitBox
codecov-io edited a comment on pull request #6534: URL: https://github.com/apache/incubator-pinot/pull/6534#issuecomment-772177566 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6534?src=pr&el=h1) Report > Merging [#6534](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] codecov-io commented on pull request #6534: update superset k8s example to the latest image

2021-02-02 Thread GitBox
codecov-io commented on pull request #6534: URL: https://github.com/apache/incubator-pinot/pull/6534#issuecomment-772177566 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6534?src=pr&el=h1) Report > Merging [#6534](https://codecov.io/gh/apache/incubator-pinot/pull/6534?s

[GitHub] [incubator-pinot] yupeng9 commented on a change in pull request #6530: Update DimTableSegmentAssignment to include both OFFLINE and REALTIME servers

2021-02-02 Thread GitBox
yupeng9 commented on a change in pull request #6530: URL: https://github.com/apache/incubator-pinot/pull/6530#discussion_r569089286 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/OfflineDimTableSegmentAssignment.java ## @

[GitHub] [incubator-pinot] yupeng9 commented on pull request #6530: Update DimTableSegmentAssignment to include both OFFLINE and REALTIME servers

2021-02-02 Thread GitBox
yupeng9 commented on pull request #6530: URL: https://github.com/apache/incubator-pinot/pull/6530#issuecomment-772170902 > LGTM. > (Not related to this PR) For easier management of the cluster, I would recommend having a separate tenant for the dimension table so that dimension table ca

[GitHub] [incubator-pinot] rkanumul edited a comment on issue #6512: Support Auto creation of containers in ADLSGen2PinotFS on initial run

2021-02-02 Thread GitBox
rkanumul edited a comment on issue #6512: URL: https://github.com/apache/incubator-pinot/issues/6512#issuecomment-772160570 Pull request at : https://github.com/apache/incubator-pinot/pull/6531 This is an automated messag

[GitHub] [incubator-pinot] daniellavoie commented on issue #6524: Provide table ingestion status through API

2021-02-02 Thread GitBox
daniellavoie commented on issue #6524: URL: https://github.com/apache/incubator-pinot/issues/6524#issuecomment-772159802 Counters, metrics and gages does not tell the story of what is wrong. Health check observability is not the problem I am trying to solve here. Client needs to understan

[GitHub] [incubator-pinot] rkanumul commented on issue #6512: Support Auto creation of containers in ADLSGen2PinotFS on initial run

2021-02-02 Thread GitBox
rkanumul commented on issue #6512: URL: https://github.com/apache/incubator-pinot/issues/6512#issuecomment-772160570 https://github.com/apache/incubator-pinot/pull/6531 This is an automated message from the Apache Git Servic

[GitHub] [incubator-pinot] rkanumul commented on issue #6510: Add unit tests for ADLSGen2PinotFS

2021-02-02 Thread GitBox
rkanumul commented on issue #6510: URL: https://github.com/apache/incubator-pinot/issues/6510#issuecomment-772160672 https://github.com/apache/incubator-pinot/pull/6531 This is an automated message from the Apache Git Servic

[GitHub] [incubator-pinot] rkanumul edited a comment on issue #6511: Support Service principal based ADLSGen2PinotFS authentication

2021-02-02 Thread GitBox
rkanumul edited a comment on issue #6511: URL: https://github.com/apache/incubator-pinot/issues/6511#issuecomment-772160605 Pull request at : https://github.com/apache/incubator-pinot/pull/6531 This is an automated messag

[GitHub] [incubator-pinot] rkanumul edited a comment on issue #6510: Add unit tests for ADLSGen2PinotFS

2021-02-02 Thread GitBox
rkanumul edited a comment on issue #6510: URL: https://github.com/apache/incubator-pinot/issues/6510#issuecomment-772160672 Pull request at : https://github.com/apache/incubator-pinot/pull/6531 This is an automated messag

[GitHub] [incubator-pinot] rkanumul commented on issue #6511: Support Service principal based ADLSGen2PinotFS authentication

2021-02-02 Thread GitBox
rkanumul commented on issue #6511: URL: https://github.com/apache/incubator-pinot/issues/6511#issuecomment-772160605 https://github.com/apache/incubator-pinot/pull/6531 This is an automated message from the Apache Git Servic

[GitHub] [incubator-pinot] mcvsubbu commented on issue #6524: Provide table ingestion status through API

2021-02-02 Thread GitBox
mcvsubbu commented on issue #6524: URL: https://github.com/apache/incubator-pinot/issues/6524#issuecomment-772158231 We also return metadata to the broker as to the newest timestamp we have consumed. It is set in the BrokerNativeResponse as the minConsumingFreshnessTimeMs, indicating the

[incubator-pinot] branch update_superset_image_version_in_k8s_example updated (c36297e -> d90968e)

2021-02-02 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch update_superset_image_version_in_k8s_example in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard c36297e udpate superset k8s example to latest image add d90968e

[GitHub] [incubator-pinot] fx19880617 opened a new pull request #6534: update superset k8s example to the latest image

2021-02-02 Thread GitBox
fx19880617 opened a new pull request #6534: URL: https://github.com/apache/incubator-pinot/pull/6534 ## Description Update superset k8s example to the latest image ## Upgrade Notes Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server,

[incubator-pinot] branch update_superset_image_version_in_k8s_example created (now c36297e)

2021-02-02 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch update_superset_image_version_in_k8s_example in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at c36297e udpate superset k8s example to latest image This branch inc

[incubator-pinot] 01/01: udpate superset k8s example to latest image

2021-02-02 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch update_superset_image_version_in_k8s_example in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit c36297e403b84de176ba44529b0fe94c3688bab3 Author: Xiang Fu AuthorDate: Tu

[GitHub] [incubator-pinot] codecov-io commented on pull request #6533: Allow creating a TAR file that contains an entry that is >8 GB in size.

2021-02-02 Thread GitBox
codecov-io commented on pull request #6533: URL: https://github.com/apache/incubator-pinot/pull/6533#issuecomment-772146836 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6533?src=pr&el=h1) Report > Merging [#6533](https://codecov.io/gh/apache/incubator-pinot/pull/6533?s

[GitHub] [incubator-pinot] fx19880617 opened a new pull request #6533: Allow creating a TAR file that contains an entry that is >8 GB in size.

2021-02-02 Thread GitBox
fx19880617 opened a new pull request #6533: URL: https://github.com/apache/incubator-pinot/pull/6533 ## Description Allow creating a TAR file that contains an entry that is >8 GB in size. ## Upgrade Notes Does this PR prevent a zero down-time upgrade? (Assume upgrade order:

[GitHub] [incubator-pinot] jihaozh merged pull request #6504: [TE]frontend - alert detail item UI fix

2021-02-02 Thread GitBox
jihaozh merged pull request #6504: URL: https://github.com/apache/incubator-pinot/pull/6504 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [incubator-pinot] daniellavoie commented on issue #6524: Provide table ingestion status through API

2021-02-02 Thread GitBox
daniellavoie commented on issue #6524: URL: https://github.com/apache/incubator-pinot/issues/6524#issuecomment-772102874 Thanks for these details @mcvsubbu. My use case is more about providing feedback to the user who created the table config from the rest API. Metrics are intended to mon

[GitHub] [incubator-pinot] codecov-io commented on pull request #6525: Adding native parquet record reader support

2021-02-02 Thread GitBox
codecov-io commented on pull request #6525: URL: https://github.com/apache/incubator-pinot/pull/6525#issuecomment-771975222 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6525?src=pr&el=h1) Report > Merging [#6525](https://codecov.io/gh/apache/incubator-pinot/pull/6525?s

[GitHub] [incubator-pinot] mcvsubbu commented on issue #6516: Set error messages in helix for state transition failues

2021-02-02 Thread GitBox
mcvsubbu commented on issue #6516: URL: https://github.com/apache/incubator-pinot/issues/6516#issuecomment-771818395 I think Helix preserves the error stack/messages for a few days? State stored in segmentDataManager can go away on restart. I think there is value in retrieving helix messa

[GitHub] [incubator-pinot] daniellavoie edited a comment on issue #6520: Expose segment reload status as JMX Metric

2021-02-02 Thread GitBox
daniellavoie edited a comment on issue #6520: URL: https://github.com/apache/incubator-pinot/issues/6520#issuecomment-772061131 Yes aggregated would be fine. Mostly, this metric is to answer `Is it done yet?` This is an aut

[GitHub] [incubator-pinot] rkanumul closed pull request #6529: Update ADLSGen2PinotFS auth; Introduce unit tests

2021-02-02 Thread GitBox
rkanumul closed pull request #6529: URL: https://github.com/apache/incubator-pinot/pull/6529 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] pabrahamusa closed issue #6522: Failed to send lease extension for table - Pinot Stopped

2021-02-02 Thread GitBox
pabrahamusa closed issue #6522: URL: https://github.com/apache/incubator-pinot/issues/6522 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [incubator-pinot] akshayrai commented on a change in pull request #6528: [TE] SQL migration - escape the keywords

2021-02-02 Thread GitBox
akshayrai commented on a change in pull request #6528: URL: https://github.com/apache/incubator-pinot/pull/6528#discussion_r568961845 ## File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/datasource/pinot/SqlUtils.java ## @@ -189,9 +195,13 @@ public sta

[GitHub] [incubator-pinot] fx19880617 edited a comment on issue #6520: Expose segment reload status as JMX Metric

2021-02-02 Thread GitBox
fx19880617 edited a comment on issue #6520: URL: https://github.com/apache/incubator-pinot/issues/6520#issuecomment-772069028 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [incubator-pinot] HyunjoonCho edited a comment on issue #6519: Update document for JDBC client

2021-02-02 Thread GitBox
HyunjoonCho edited a comment on issue #6519: URL: https://github.com/apache/incubator-pinot/issues/6519#issuecomment-771631182 Hi, the page I mentioned said WIP 8 hours ago but seems someone fixed it today! Thanks ![image](https://user-images.githubusercontent.com/59246310/106606701-c7

[GitHub] [incubator-pinot] fx19880617 commented on issue #6520: Expose segment reload status as JMX Metric

2021-02-02 Thread GitBox
fx19880617 commented on issue #6520: URL: https://github.com/apache/incubator-pinot/issues/6520#issuecomment-772059592 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [incubator-pinot] HyunjoonCho commented on issue #6519: Update document for JDBC client

2021-02-02 Thread GitBox
HyunjoonCho commented on issue #6519: URL: https://github.com/apache/incubator-pinot/issues/6519#issuecomment-771631182 Hi, the page I mentioned said WIP 8 hours ago but seems someone fixed it! Thanks This is an automated m

[GitHub] [incubator-pinot] mcvsubbu commented on issue #6524: Provide table ingestion status through API

2021-02-02 Thread GitBox
mcvsubbu commented on issue #6524: URL: https://github.com/apache/incubator-pinot/issues/6524#issuecomment-772095947 We have a metric that is set to 1 if a stream partition is consuming correctly, 0 otherwise. In LinkedIn, we set an alert if the consumption falls to 0 for more than some p

[GitHub] [incubator-pinot] daniellavoie commented on issue #6520: Expose segment reload status as JMX Metric

2021-02-02 Thread GitBox
daniellavoie commented on issue #6520: URL: https://github.com/apache/incubator-pinot/issues/6520#issuecomment-772061131 Yes aggregated would be fine. Mostly, this metric is to answer `Is it down yet?` This is an automated

[GitHub] [incubator-pinot] npawar commented on issue #6277: Creating segment larger than 8GB will fail due to gzip limitation

2021-02-02 Thread GitBox
npawar commented on issue #6277: URL: https://github.com/apache/incubator-pinot/issues/6277#issuecomment-772105123 The segment size threshold config will kick in only when rows threshold is set to 0. Prolly not working for you because the flush.threshold.rows is set to something in your e

[GitHub] [incubator-pinot] fx19880617 commented on issue #6277: Creating segment larger than 8GB will fail due to gzip limitation

2021-02-02 Thread GitBox
fx19880617 commented on issue #6277: URL: https://github.com/apache/incubator-pinot/issues/6277#issuecomment-771916100 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [incubator-pinot] KKcorps commented on issue #6519: Update document for JDBC client

2021-02-02 Thread GitBox
KKcorps commented on issue #6519: URL: https://github.com/apache/incubator-pinot/issues/6519#issuecomment-771605142 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [incubator-pinot] jihaozh commented on a change in pull request #6528: [TE] SQL migration - escape the keywords

2021-02-02 Thread GitBox
jihaozh commented on a change in pull request #6528: URL: https://github.com/apache/incubator-pinot/pull/6528#discussion_r568990857 ## File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/datasource/pinot/SqlUtils.java ## @@ -189,9 +195,13 @@ public stati

[GitHub] [incubator-pinot] pabrahamusa commented on issue #6522: Failed to send lease extension for table - Pinot Stopped

2021-02-02 Thread GitBox
pabrahamusa commented on issue #6522: URL: https://github.com/apache/incubator-pinot/issues/6522#issuecomment-771689647 This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [incubator-pinot] pabrahamusa edited a comment on issue #6277: Creating segment larger than 8GB will fail due to gzip limitation

2021-02-02 Thread GitBox
pabrahamusa edited a comment on issue #6277: URL: https://github.com/apache/incubator-pinot/issues/6277#issuecomment-771973867 @fx19880617 Here is the stack trace , I tried these ``` # realtime.segment.flush.desired.size="400M" -- not sure this is working

[GitHub] [incubator-pinot] codecov-io commented on pull request #6518: Kinesis Connector

2021-02-02 Thread GitBox
codecov-io commented on pull request #6518: URL: https://github.com/apache/incubator-pinot/pull/6518#issuecomment-772093618 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6518?src=pr&el=h1) Report > Merging [#6518](https://codecov.io/gh/apache/incubator-pinot/pull/6518?s

[GitHub] [incubator-pinot] jihaozh merged pull request #6528: [TE] SQL migration - escape the keywords

2021-02-02 Thread GitBox
jihaozh merged pull request #6528: URL: https://github.com/apache/incubator-pinot/pull/6528 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [incubator-pinot] pabrahamusa commented on issue #6277: Creating segment larger than 8GB will fail due to gzip limitation

2021-02-02 Thread GitBox
pabrahamusa commented on issue #6277: URL: https://github.com/apache/incubator-pinot/issues/6277#issuecomment-771881448 This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6506: Fix hadoop batch ingest

2021-02-02 Thread GitBox
fx19880617 commented on a change in pull request #6506: URL: https://github.com/apache/incubator-pinot/pull/6506#discussion_r568261109 ## File path: pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-hadoop/src/main/java/org/apache/pinot/plugin/ingestion/batch/hadoop/HadoopSegm

[GitHub] [incubator-pinot] tannghia025 commented on a change in pull request #5787: [Feature] - Spark Pinot Connector

2021-02-02 Thread GitBox
tannghia025 commented on a change in pull request #5787: URL: https://github.com/apache/incubator-pinot/pull/5787#discussion_r568417075 ## File path: pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/datasource/PinotDataSourceReader.scala #

[GitHub] [incubator-pinot] kishoreg commented on issue #6519: Update document for JDBC client

2021-02-02 Thread GitBox
kishoreg commented on issue #6519: URL: https://github.com/apache/incubator-pinot/issues/6519#issuecomment-771365730 @KKcorps This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [incubator-pinot] tannghia025 edited a comment on issue #5598: Spark-Pinot connector to read and write data from/to Pinot directly

2021-02-02 Thread GitBox
tannghia025 edited a comment on issue #5598: URL: https://github.com/apache/incubator-pinot/issues/5598#issuecomment-771462472 có ví dụ write xuống pinot đâu nói như thiệt cay vl This is an automated message from the Apache

[GitHub] [incubator-pinot] kishoreg commented on issue #6516: Set error messages in helix for state transition failues

2021-02-02 Thread GitBox
kishoreg commented on issue #6516: URL: https://github.com/apache/incubator-pinot/issues/6516#issuecomment-771387950 Got it. One thing to keep in mind is that state transition error messages get cleaned up automatically by Helix. we should probably do something like this - each server s

[GitHub] [incubator-pinot] apucher commented on a change in pull request #6507: Add Access Control for REST endpoints of Controller - Declarative Approach

2021-02-02 Thread GitBox
apucher commented on a change in pull request #6507: URL: https://github.com/apache/incubator-pinot/pull/6507#discussion_r568197551 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/api/access/AccessControl.java ## @@ -30,9 +30,38 @@ /** * Return

[GitHub] [incubator-pinot] subourbonite edited a comment on issue #6517: Time column transform with same name

2021-02-02 Thread GitBox
subourbonite edited a comment on issue #6517: URL: https://github.com/apache/incubator-pinot/issues/6517#issuecomment-771281912 I actually think that having the same column name spread around multiple systems with a different grain / format / definition can become a serious data usability

[GitHub] [incubator-pinot] apucher commented on a change in pull request #6515: use default broker and controller ports if not defined

2021-02-02 Thread GitBox
apucher commented on a change in pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#discussion_r568224380 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/broker/helix/HelixBrokerStarter.java ## @@ -133,9 +133,7 @@ public HelixBrokerSt

[GitHub] [incubator-pinot] jihaozh merged pull request #6514: [TE]frontend - Add filtering support for Entity Monitoring tables

2021-02-02 Thread GitBox
jihaozh merged pull request #6514: URL: https://github.com/apache/incubator-pinot/pull/6514 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [incubator-pinot] icefury71 commented on a change in pull request #6485: [WIP] add schema validation mechanism for table config

2021-02-02 Thread GitBox
icefury71 commented on a change in pull request #6485: URL: https://github.com/apache/incubator-pinot/pull/6485#discussion_r568227370 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/utils/validator/BaseJsonValidator.java ## @@ -0,0 +1,56 @@ +package org.apache.pinot

[GitHub] [incubator-pinot] kkrugler commented on a change in pull request #6506: Fix hadoop batch ingest

2021-02-02 Thread GitBox
kkrugler commented on a change in pull request #6506: URL: https://github.com/apache/incubator-pinot/pull/6506#discussion_r568254617 ## File path: pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-hadoop/src/main/java/org/apache/pinot/plugin/ingestion/batch/hadoop/HadoopSegmen

[GitHub] [incubator-pinot] fx19880617 commented on issue #6509: Mmap lib issue after upgrading from Ubuntu 16.0 to 20.0

2021-02-02 Thread GitBox
fx19880617 commented on issue #6509: URL: https://github.com/apache/incubator-pinot/issues/6509#issuecomment-771222590 hmm, does it work with java 11? This is an automated message from the Apache Git Service. To respond to t

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #6414: Increase code coverage for pinot-java-client

2021-02-02 Thread GitBox
jackjlli commented on a change in pull request #6414: URL: https://github.com/apache/incubator-pinot/pull/6414#discussion_r568214842 ## File path: pinot-clients/pinot-java-client/src/test/java/org/apache/pinot/client/DynamicBrokerSelectorTest.java ## @@ -0,0 +1,109 @@ +/** + *

[incubator-pinot] 01/01: Allow tar gz with > 8gb size

2021-02-02 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch fixing_big_tar_gz_file_issue in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 9fa79be8e6d68ce91f2561dc8f8130fe34fa53ed Author: Xiang Fu AuthorDate: Tue Feb 2 16:59:31

[incubator-pinot] branch fixing_big_tar_gz_file_issue created (now 9fa79be)

2021-02-02 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch fixing_big_tar_gz_file_issue in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at 9fa79be Allow tar gz with > 8gb size This branch includes the following new commits

[GitHub] [incubator-pinot] longtrinhvan removed a comment on issue #5598: Spark-Pinot connector to read and write data from/to Pinot directly

2021-02-02 Thread GitBox
longtrinhvan removed a comment on issue #5598: URL: https://github.com/apache/incubator-pinot/issues/5598#issuecomment-771458430 không làm mà đòi có ăn thì ăn đầu buồi, ăn cứt This is an automated message from the Apache Git

[GitHub] [incubator-pinot] subourbonite commented on issue #6517: Time column transform with same name

2021-02-02 Thread GitBox
subourbonite commented on issue #6517: URL: https://github.com/apache/incubator-pinot/issues/6517#issuecomment-771281912 I actually think that having the same column name spread around multiple systems with a different grain / format / definition can become a serious data usability proble

[GitHub] [incubator-pinot] longtrinhvan commented on issue #5598: Spark-Pinot connector to read and write data from/to Pinot directly

2021-02-02 Thread GitBox
longtrinhvan commented on issue #5598: URL: https://github.com/apache/incubator-pinot/issues/5598#issuecomment-771458430 không làm mà đòi có ăn thì ăn đầu buồi, ăn cứt This is an automated message from the Apache Git Service

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6515: use default broker and controller ports if not defined

2021-02-02 Thread GitBox
codecov-io edited a comment on pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#issuecomment-771241792 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6515?src=pr&el=h1) Report > Merging [#6515](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] tannghia025 commented on issue #5598: Spark-Pinot connector to read and write data from/to Pinot directly

2021-02-02 Thread GitBox
tannghia025 commented on issue #5598: URL: https://github.com/apache/incubator-pinot/issues/5598#issuecomment-771462472 có ví dụ write xuống pinot đâu nó như thiệt cay vl This is an automated message from the Apache Git Serv

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #6515: use default broker and controller ports if not defined

2021-02-02 Thread GitBox
Jackie-Jiang merged pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [incubator-pinot] codecov-io commented on pull request #6515: use default broker and controller ports if not defined

2021-02-02 Thread GitBox
codecov-io commented on pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#issuecomment-771241792 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6515?src=pr&el=h1) Report > Merging [#6515](https://codecov.io/gh/apache/incubator-pinot/pull/6515?s

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6515: use default broker and controller ports if not defined

2021-02-02 Thread GitBox
Jackie-Jiang commented on a change in pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#discussion_r568220638 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/broker/helix/HelixBrokerStarter.java ## @@ -133,9 +133,7 @@ public HelixBro

[GitHub] [incubator-pinot] xulinjintu opened a new issue #6532: Server Rack Metadata Retrieval and Persistence on Azure Environment

2021-02-02 Thread GitBox
xulinjintu opened a new issue #6532: URL: https://github.com/apache/incubator-pinot/issues/6532 In order to move Pinot to cloud, cloud VM rack metadata awareness is needed for maintenance purpose and disaster recovery. In Azure use cases, fault domain (FD) metadata is needed in the Pinot e

[GitHub] [incubator-pinot] npawar commented on issue #6277: Creating segment larger than 8GB will fail due to gzip limitation

2021-02-02 Thread GitBox
npawar commented on issue #6277: URL: https://github.com/apache/incubator-pinot/issues/6277#issuecomment-772105123 The segment size threshold config will kick in only when rows threshold is set to 0. Prolly not working for you because the flush.threshold.rows is set to something in your e

[incubator-pinot] branch master updated: [THIRDEYE-3799](fix): the alert item style fix (#6504)

2021-02-02 Thread jihao
This is an automated email from the ASF dual-hosted git repository. jihao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new d1a0688 [THIRDEYE-3799](fix): the alert

[GitHub] [incubator-pinot] jihaozh merged pull request #6504: [TE]frontend - alert detail item UI fix

2021-02-02 Thread GitBox
jihaozh merged pull request #6504: URL: https://github.com/apache/incubator-pinot/pull/6504 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [incubator-pinot] daniellavoie commented on issue #6524: Provide table ingestion status through API

2021-02-02 Thread GitBox
daniellavoie commented on issue #6524: URL: https://github.com/apache/incubator-pinot/issues/6524#issuecomment-772102874 Thanks for these details @mcvsubbu. My use case is more about providing feedback to the user who created the table config from the rest API. Metrics are intended to mon

[GitHub] [incubator-pinot] fx19880617 commented on issue #6277: Creating segment larger than 8GB will fail due to gzip limitation

2021-02-02 Thread GitBox
fx19880617 commented on issue #6277: URL: https://github.com/apache/incubator-pinot/issues/6277#issuecomment-772100779 Meanwhile we will fix the tar gz issue This is an automated message from the Apache Git Service. To respo

[GitHub] [incubator-pinot] fx19880617 commented on issue #6277: Creating segment larger than 8GB will fail due to gzip limitation

2021-02-02 Thread GitBox
fx19880617 commented on issue #6277: URL: https://github.com/apache/incubator-pinot/issues/6277#issuecomment-772100661 Right, `column.psf` is the single file that merges all the columns' data. Do you have the index_map file or segment metadata file, which will tell the size for each colum

[GitHub] [incubator-pinot] rkanumul opened a new pull request #6531: Update ADLSGen2PinotFS auth; Introduce unit tests

2021-02-02 Thread GitBox
rkanumul opened a new pull request #6531: URL: https://github.com/apache/incubator-pinot/pull/6531 ## Description Linkedin plans to use ADLSGen2PinotFS but requires 1) Service principal based authentication to do that for better ACL control. The current accesskey based auth would be h

[GitHub] [incubator-pinot] rkanumul closed pull request #6529: Update ADLSGen2PinotFS auth; Introduce unit tests

2021-02-02 Thread GitBox
rkanumul closed pull request #6529: URL: https://github.com/apache/incubator-pinot/pull/6529 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] mcvsubbu commented on issue #6524: Provide table ingestion status through API

2021-02-02 Thread GitBox
mcvsubbu commented on issue #6524: URL: https://github.com/apache/incubator-pinot/issues/6524#issuecomment-772095947 We have a metric that is set to 1 if a stream partition is consuming correctly, 0 otherwise. In LinkedIn, we set an alert if the consumption falls to 0 for more than some p

[GitHub] [incubator-pinot] codecov-io commented on pull request #6518: Kinesis Connector

2021-02-02 Thread GitBox
codecov-io commented on pull request #6518: URL: https://github.com/apache/incubator-pinot/pull/6518#issuecomment-772093618 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6518?src=pr&el=h1) Report > Merging [#6518](https://codecov.io/gh/apache/incubator-pinot/pull/6518?s

[GitHub] [incubator-pinot] pabrahamusa commented on issue #6277: Creating segment larger than 8GB will fail due to gzip limitation

2021-02-02 Thread GitBox
pabrahamusa commented on issue #6277: URL: https://github.com/apache/incubator-pinot/issues/6277#issuecomment-772084940 @fx19880617 further analysis it is clear that the size is coming from columns.psf ``` root@my-pinot-server-0:/var/pinot/server/data/index/mylog_REALTIME/mylog_

[incubator-pinot] branch master updated: [TE] SQL migration - escape the keywords (#6528)

2021-02-02 Thread jihao
This is an automated email from the ASF dual-hosted git repository. jihao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new 84ae4f1 [TE] SQL migration - escape the

[GitHub] [incubator-pinot] jihaozh merged pull request #6528: [TE] SQL migration - escape the keywords

2021-02-02 Thread GitBox
jihaozh merged pull request #6528: URL: https://github.com/apache/incubator-pinot/pull/6528 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [incubator-pinot] jihaozh commented on a change in pull request #6528: [TE] SQL migration - escape the keywords

2021-02-02 Thread GitBox
jihaozh commented on a change in pull request #6528: URL: https://github.com/apache/incubator-pinot/pull/6528#discussion_r568990857 ## File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/datasource/pinot/SqlUtils.java ## @@ -189,9 +195,13 @@ public stati

  1   2   >