[GitHub] [incubator-pinot] kishoreg commented on issue #6516: Set error messages in helix for state transition failues

2021-02-01 Thread GitBox
kishoreg commented on issue #6516: URL: https://github.com/apache/incubator-pinot/issues/6516#issuecomment-771387950 Got it. One thing to keep in mind is that state transition error messages get cleaned up automatically by Helix. we should probably do something like this - each server s

[GitHub] [incubator-pinot] kishoreg commented on issue #6519: Update document for JDBC client

2021-02-01 Thread GitBox
kishoreg commented on issue #6519: URL: https://github.com/apache/incubator-pinot/issues/6519#issuecomment-771365730 @KKcorps This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [incubator-pinot] HyunjoonCho opened a new issue #6519: Update document for JDBC client

2021-02-01 Thread GitBox
HyunjoonCho opened a new issue #6519: URL: https://github.com/apache/incubator-pinot/issues/6519 Hello, I updated JDBC client from 0.5.0 to 0.6.0 and it did not work with previous setting. Then found out that it only requires controller url now, which is referred as WIP in document.

[incubator-pinot] branch master updated (209f57c -> 12ee45c)

2021-02-01 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 209f57c [TE]frontend - Add filtering support for Entity Monitoring tables (#6514) add 12ee45c use de

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #6515: use default broker and controller ports if not defined

2021-02-01 Thread GitBox
Jackie-Jiang merged pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [incubator-pinot] npawar opened a new pull request #6518: Kinesis

2021-02-01 Thread GitBox
npawar opened a new pull request #6518: URL: https://github.com/apache/incubator-pinot/pull/6518 ## Description Adding Kinesis Connector for Pinot realtime ingestion Added a `KinesisConsumerFactory` to provide the Kinesis implementation. Each kinesis consumer consumes from a

[incubator-pinot] 47/47: Avoid writing 'stream' and also 'stream.kinesis.topic.name'

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit db2136ae4a7f20c3ed9a6c282c4d6325207fe877 Author: Neha Pawar AuthorDate:

[incubator-pinot] 45/47: Add tests for end-of-life cases

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit dcb2ee1354ff86417d28276ef03fbd0eaa758236 Author: Neha Pawar AuthorDate:

[incubator-pinot] 44/47: End-of-shard as end criteria AND consume shards in order

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 751e21205fa53b8c6db01c8ba26aa8b3d5ace424 Author: Neha Pawar AuthorDate:

[incubator-pinot] 46/47: Add unit tests in Kinesis consumer (#6410)

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit ab9655b8d42bd1a41920260ff3080070c6863cca Author: Kartik Khare AuthorDate

[incubator-pinot] 41/47: Remove new partition groups creation in commit

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 40b6dbe760e7ceb9cde8bad1af7180446b51bb85 Author: Neha Pawar AuthorDate:

[incubator-pinot] 43/47: LIcense headers

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 7d1f7a17dbd1fa0076829e66739e19b7490210bc Author: Neha Pawar AuthorDate:

[incubator-pinot] 42/47: Cleanup, javadocs, comments

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit ce1a6462084dfa05d9b8c2b57a23a9c8274725e4 Author: Neha Pawar AuthorDate:

[incubator-pinot] 34/47: Implementation fixes

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit d079c81802adf8354523c3d6bd0b192c10e4d483 Author: Neha Pawar AuthorDate:

[incubator-pinot] 38/47: Remove unused classes and changes

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 570a95a2e3a7189615433bbab23c962f6957805c Author: Neha Pawar AuthorDate:

[incubator-pinot] 36/47: Fix offsets in StreamMetadataProvider impl

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 9c33895266f71512c4e8a0858d8f6eaa783faf44 Author: Neha Pawar AuthorDate:

[incubator-pinot] 40/47: Dont create new CONSUMING segment if shard has reached end of life

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 214c007c2915c8aa149e1e06689e66abaa85b083 Author: Neha Pawar AuthorDate:

[incubator-pinot] 39/47: Use shardId's last digits as partitionGroupId

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit d7d04081131363e582340dfd8dc11fc1a92f3e5a Author: Neha Pawar AuthorDate:

[incubator-pinot] 32/47: Return message batch instead of list in the fetch result

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit c5c42d497320a3e7aedca4a7e1c43808e69222f9 Author: KKcorps AuthorDate: Thu

[incubator-pinot] 37/47: Consumer tweaks to get it working

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 211620640d70377b4169b3cec47258501410af86 Author: Neha Pawar AuthorDate:

[incubator-pinot] 31/47: Add isEndOfPartition check in checkpoints

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit eb428cded05ff23b587fe0ed61b89a3b0ec9dd8e Author: KKcorps AuthorDate: Thu

[incubator-pinot] 24/47: Handle exceptions

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 760ba067ab25a3dfaefc6a0534c53ea0f5d62672 Author: KKcorps AuthorDate: Sun

[incubator-pinot] 26/47: Handle closed connections

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 0b8bb670bfd91e3a455a989a584388f15065b6a3 Author: KKcorps AuthorDate: Mon

[incubator-pinot] 28/47: Change shard metadata logic

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 26085a88ac34d6b00737bbd68b8ff409ad281467 Author: KKcorps AuthorDate: Tue

[incubator-pinot] 33/47: fixing compilation

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 6cb0ebb1b775959c166cbcdeadec74ae3349e4ad Author: Xiang Fu AuthorDate: Sa

[incubator-pinot] 35/47: Add support for stream partition offsets (#6402)

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit a3deab3ea5154f03debf10751e273dc1a82ec919 Author: Kartik Khare AuthorDate

[incubator-pinot] 30/47: Handle timeout exception in consumer and make shard iterator type configurable

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit cf23ee3a83a0ea22d0dde57e306ccadf30db4d1c Author: KKcorps AuthorDate: Thu

[incubator-pinot] 17/47: Fix consumer code

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit b0d8c1b422e58013c48e07c5469698229936a621 Author: KKcorps AuthorDate: Sun

[incubator-pinot] 22/47: fetch records with timeout

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 262d326be4dc8ee8b1a77bc65668ef3fc12c4aea Author: KKcorps AuthorDate: Sun

[incubator-pinot] 19/47: Reformat code

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 6c8af2b54c72407bfa8f91b2403dc317b01cc8cb Author: KKcorps AuthorDate: Sun

[incubator-pinot] 29/47: Add test code for kinesis

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 12880126a7a0c5a8b2e1ac3656e64fce7ebe47d5 Author: KKcorps AuthorDate: Tue

[incubator-pinot] 27/47: Refactor: get shard iterator methods

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit be19cf6866235d7bf4ce0a74424ae2378f40c8bc Author: KKcorps AuthorDate: Mon

[incubator-pinot] 25/47: Refactor code

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 71ac64d04defd07f4a26d53f1d3cbb48d3a7a781 Author: KKcorps AuthorDate: Sun

[incubator-pinot] 23/47: Add license header

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 29068ca2ded7066d4dda1438199718c96b2702ad Author: KKcorps AuthorDate: Sun

[incubator-pinot] 13/47: Add PartitionGroupMetdataMap interface

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 2546098f2ae43f4ff39acd0a48cb42b047dc1fc4 Author: KKcorps AuthorDate: Fri

[incubator-pinot] 18/47: Move shardId out of checkpoint to partition group metadata

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 7a4fccc3ad68f72f363f1663f6956c4b2aa6cc78 Author: KKcorps AuthorDate: Sun

[incubator-pinot] 20/47: Add license headers

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit b05ad0813351a7ecf5f2f2e059e0399859aa2f29 Author: KKcorps AuthorDate: Sun

[incubator-pinot] 12/47: Add initial implementation of Kinesis consumer

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 72a77c61226ce5cf3abd781916f7797a3fa5a230 Author: KKcorps AuthorDate: Thu

[incubator-pinot] 21/47: Add Kinesis config wrapper

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 96c32c461b006a15cdf34170a40f076e09be73cc Author: KKcorps AuthorDate: Sun

[incubator-pinot] 14/47: Add kinesis code to handle offsets

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 04919035cfa9cd9e2f8d554dfd608c643b70d30e Author: KKcorps AuthorDate: Fri

[incubator-pinot] 03/47: Rename partitionId to partitionGroupId

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 3892fc417c2f7d07e15b78eae1e1b3dd09e60090 Author: Neha Pawar AuthorDate:

[incubator-pinot] 15/47: Refactor PartitionGroupMetadataMap interface

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 5ed893eef9f92df42e8997e0213027c05ca91e10 Author: KKcorps AuthorDate: Fri

[incubator-pinot] 11/47: Add interfaces for V2 consumers

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 3f14cf086d2117af2d24d73714e79e08d64c461d Author: KKcorps AuthorDate: Thu

[incubator-pinot] 16/47: Refactor kinesis shard metadata interface and add shardId to the metadata

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 92ddaab79edd23e232e2b2fd8fcc187830c66d40 Author: KKcorps AuthorDate: Fri

[incubator-pinot] 04/47: Controller side code

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 96621f415385649b4f2ec5ea1828723eed4baa7b Author: Neha Pawar AuthorDate:

[incubator-pinot] 07/47: Checnges in test to make it complie

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 8afc48f8752c9044af791279e54568f4a124759e Author: Neha Pawar AuthorDate:

[incubator-pinot] 06/47: More controller side changes

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 75547ec213f4466f0d2f7b9b45e1234641343ac2 Author: Neha Pawar AuthorDate:

[incubator-pinot] 08/47: An attempt at server-side changes

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 31c64a0cc138146dc59c1ce665f3ca72fd7b52f9 Author: Neha Pawar AuthorDate:

[incubator-pinot] 01/47: Controller side changes pseudo code

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 6502a7d29214cc1ad3bdf9feba169192d1c50ff7 Author: Neha Pawar AuthorDate:

[incubator-pinot] 09/47: Server side changes and some fixes

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit ae863a1087a370ed388a37c91cbd46acdece23b5 Author: Neha Pawar AuthorDate:

[incubator-pinot] 05/47: Separate PartitionGroupInfo and PartitionGroupMetadata

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit a7fba5a7ffc843ea576d23e330cd2fd8441ee5fb Author: Neha Pawar AuthorDate:

[incubator-pinot] 10/47: default methods to avoid interface changes

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 396dae01f0804bc916bfba1bf2d0c52a374b767e Author: Neha Pawar AuthorDate:

[incubator-pinot] 02/47: StreamPartitionOffset to implement Checkpoint

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 7504c3148fe86496d1d85add9cb2452ab9f62769 Author: Neha Pawar AuthorDate:

[incubator-pinot] branch sharded_consumer_type_support_with_kinesis updated (f9f2419 -> db2136a)

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a change to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard f9f2419 Avoid writing 'stream' and also 'stream.kinesis.topic.name'

[incubator-pinot] branch sharded_consumer_type_support_with_kinesis updated: Avoid writing 'stream' and also 'stream.kinesis.topic.name'

2021-02-01 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a commit to branch sharded_consumer_type_support_with_kinesis in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/sharded_consumer_type_support

[GitHub] [incubator-pinot] kkrugler commented on a change in pull request #6506: Fix hadoop batch ingest

2021-02-01 Thread GitBox
kkrugler commented on a change in pull request #6506: URL: https://github.com/apache/incubator-pinot/pull/6506#discussion_r568261628 ## File path: pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-hadoop/src/main/java/org/apache/pinot/plugin/ingestion/batch/hadoop/HadoopSegmen

[GitHub] [incubator-pinot] subourbonite edited a comment on issue #6517: Time column transform with same name

2021-02-01 Thread GitBox
subourbonite edited a comment on issue #6517: URL: https://github.com/apache/incubator-pinot/issues/6517#issuecomment-771281912 I actually think that having the same column name spread around multiple systems with a different grain / format / definition can become a serious data usability

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6506: Fix hadoop batch ingest

2021-02-01 Thread GitBox
fx19880617 commented on a change in pull request #6506: URL: https://github.com/apache/incubator-pinot/pull/6506#discussion_r568261109 ## File path: pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-hadoop/src/main/java/org/apache/pinot/plugin/ingestion/batch/hadoop/HadoopSegm

[GitHub] [incubator-pinot] subourbonite commented on issue #6517: Time column transform with same name

2021-02-01 Thread GitBox
subourbonite commented on issue #6517: URL: https://github.com/apache/incubator-pinot/issues/6517#issuecomment-771281912 I actually think that having the same column name spread around multiple systems with a different grain / format / definition can become a serious data usability proble

[GitHub] [incubator-pinot] kkrugler commented on a change in pull request #6506: Fix hadoop batch ingest

2021-02-01 Thread GitBox
kkrugler commented on a change in pull request #6506: URL: https://github.com/apache/incubator-pinot/pull/6506#discussion_r568254949 ## File path: pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-hadoop/src/main/java/org/apache/pinot/plugin/ingestion/batch/hadoop/HadoopSegmen

[GitHub] [incubator-pinot] kkrugler commented on a change in pull request #6506: Fix hadoop batch ingest

2021-02-01 Thread GitBox
kkrugler commented on a change in pull request #6506: URL: https://github.com/apache/incubator-pinot/pull/6506#discussion_r568254617 ## File path: pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-hadoop/src/main/java/org/apache/pinot/plugin/ingestion/batch/hadoop/HadoopSegmen

[GitHub] [incubator-pinot] mcvsubbu opened a new issue #6517: Time column transform with same name

2021-02-01 Thread GitBox
mcvsubbu opened a new issue #6517: URL: https://github.com/apache/incubator-pinot/issues/6517 It will be useful to have a transform function in the ingestion path that transforms the time column into different units (e.g. mills to hours since epoch, or millis to minutes since epoch, etc.)

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6515: use default broker and controller ports if not defined

2021-02-01 Thread GitBox
codecov-io edited a comment on pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#issuecomment-771241792 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6515?src=pr&el=h1) Report > Merging [#6515](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] icefury71 commented on a change in pull request #6485: [WIP] add schema validation mechanism for table config

2021-02-01 Thread GitBox
icefury71 commented on a change in pull request #6485: URL: https://github.com/apache/incubator-pinot/pull/6485#discussion_r568227370 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/utils/validator/BaseJsonValidator.java ## @@ -0,0 +1,56 @@ +package org.apache.pinot

[incubator-pinot] branch master updated (0f398a7 -> 209f57c)

2021-02-01 Thread jihao
This is an automated email from the ASF dual-hosted git repository. jihao pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 0f398a7 Fixing quickstart launcher from IDE (#6508) add 209f57c [TE]frontend - Add filtering support f

[GitHub] [incubator-pinot] jihaozh merged pull request #6514: [TE]frontend - Add filtering support for Entity Monitoring tables

2021-02-01 Thread GitBox
jihaozh merged pull request #6514: URL: https://github.com/apache/incubator-pinot/pull/6514 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [incubator-pinot] apucher commented on a change in pull request #6515: use default broker and controller ports if not defined

2021-02-01 Thread GitBox
apucher commented on a change in pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#discussion_r568224380 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/broker/helix/HelixBrokerStarter.java ## @@ -133,9 +133,7 @@ public HelixBrokerSt

[incubator-pinot] branch use-default-broker-controller-ports-when-undefined updated (d6fb218 -> 96bc4b7)

2021-02-01 Thread apucher
This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch use-default-broker-controller-ports-when-undefined in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from d6fb218 review fixes add 96bc4b7 review fixes 2 No new

[GitHub] [incubator-pinot] codecov-io commented on pull request #6515: use default broker and controller ports if not defined

2021-02-01 Thread GitBox
codecov-io commented on pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#issuecomment-771241792 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6515?src=pr&el=h1) Report > Merging [#6515](https://codecov.io/gh/apache/incubator-pinot/pull/6515?s

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6515: use default broker and controller ports if not defined

2021-02-01 Thread GitBox
Jackie-Jiang commented on a change in pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#discussion_r568220638 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/broker/helix/HelixBrokerStarter.java ## @@ -133,9 +133,7 @@ public HelixBro

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #6414: Increase code coverage for pinot-java-client

2021-02-01 Thread GitBox
jackjlli commented on a change in pull request #6414: URL: https://github.com/apache/incubator-pinot/pull/6414#discussion_r568214842 ## File path: pinot-clients/pinot-java-client/src/test/java/org/apache/pinot/client/DynamicBrokerSelectorTest.java ## @@ -0,0 +1,109 @@ +/** + *

[GitHub] [incubator-pinot] apucher commented on a change in pull request #6507: Add Access Control for REST endpoints of Controller - Declarative Approach

2021-02-01 Thread GitBox
apucher commented on a change in pull request #6507: URL: https://github.com/apache/incubator-pinot/pull/6507#discussion_r568197551 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/api/access/AccessControl.java ## @@ -30,9 +30,38 @@ /** * Return

[GitHub] [incubator-pinot] fx19880617 commented on issue #6509: Mmap lib issue after upgrading from Ubuntu 16.0 to 20.0

2021-02-01 Thread GitBox
fx19880617 commented on issue #6509: URL: https://github.com/apache/incubator-pinot/issues/6509#issuecomment-771222590 hmm, does it work with java 11? This is an automated message from the Apache Git Service. To respond to t

[GitHub] [incubator-pinot] apucher edited a comment on pull request #6515: use default broker and controller ports if not defined

2021-02-01 Thread GitBox
apucher edited a comment on pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#issuecomment-771204637 re high-level question: Pinot mingles two aspects - naming and discovery - and the helix InstanceConfig only supports a single port per instance. I worked aroun

[incubator-pinot] branch use-default-broker-controller-ports-when-undefined updated (9f427eb -> d6fb218)

2021-02-01 Thread apucher
This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch use-default-broker-controller-ports-when-undefined in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 9f427eb more legacy support changes add d6fb218 review

[GitHub] [incubator-pinot] apucher commented on a change in pull request #6515: use default broker and controller ports if not defined

2021-02-01 Thread GitBox
apucher commented on a change in pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#discussion_r568187478 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/ListenerConfigUtil.java ## @@ -112,6 +112,12 @@ private ListenerConfigUtil() {

[GitHub] [incubator-pinot] fx19880617 commented on issue #6516: Set error messages in helix for state transition failues

2021-02-01 Thread GitBox
fx19880617 commented on issue #6516: URL: https://github.com/apache/incubator-pinot/issues/6516#issuecomment-771206290 Yes, right now we only know that segment in error state from external view. The goal is to also set the stacktrace for segment loading and easy for users to retrieve it.

[GitHub] [incubator-pinot] apucher commented on a change in pull request #6515: use default broker and controller ports if not defined

2021-02-01 Thread GitBox
apucher commented on a change in pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#discussion_r568186467 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/ControllerStarter.java ## @@ -197,7 +197,7 @@ private int inferPort() {

[GitHub] [incubator-pinot] apucher commented on a change in pull request #6515: use default broker and controller ports if not defined

2021-02-01 Thread GitBox
apucher commented on a change in pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#discussion_r568186410 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/broker/helix/HelixBrokerStarter.java ## @@ -133,9 +133,8 @@ public HelixBrokerSt

[GitHub] [incubator-pinot] apucher commented on pull request #6515: use default broker and controller ports if not defined

2021-02-01 Thread GitBox
apucher commented on pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#issuecomment-771204637 re high-level question: Pinot mingles two aspects - naming and discovery - and the helix InstanceConfig only supports a single port per instance. I worked around this

[GitHub] [incubator-pinot] zhangloo333 commented on a change in pull request #6504: [TE]frontend - alert detail item UI fix

2021-02-01 Thread GitBox
zhangloo333 commented on a change in pull request #6504: URL: https://github.com/apache/incubator-pinot/pull/6504#discussion_r568180778 ## File path: thirdeye/thirdeye-frontend/app/pods/components/self-serve-alert-yaml-details/template.hbs ## @@ -1,80 +1,104 @@ -

[GitHub] [incubator-pinot] apucher commented on a change in pull request #6515: use default broker and controller ports if not defined

2021-02-01 Thread GitBox
apucher commented on a change in pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#discussion_r568180523 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/ListenerConfigUtil.java ## @@ -112,6 +112,12 @@ private ListenerConfigUtil() {

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6515: use default broker and controller ports if not defined

2021-02-01 Thread GitBox
Jackie-Jiang commented on a change in pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#discussion_r568179873 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/ListenerConfigUtil.java ## @@ -112,6 +112,12 @@ private ListenerConfigUtil

[GitHub] [incubator-pinot] JoeFuentes-WebDev commented on a change in pull request #6504: [TE]frontend - alert detail item UI fix

2021-02-01 Thread GitBox
JoeFuentes-WebDev commented on a change in pull request #6504: URL: https://github.com/apache/incubator-pinot/pull/6504#discussion_r568177195 ## File path: thirdeye/thirdeye-frontend/app/pods/components/self-serve-alert-yaml-details/template.hbs ## @@ -1,80 +1,104 @@

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6515: use default broker and controller ports if not defined

2021-02-01 Thread GitBox
Jackie-Jiang commented on a change in pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515#discussion_r568175015 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/ListenerConfigUtil.java ## @@ -112,6 +112,12 @@ private ListenerConfigUtil

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #6361: Detect invalid column names from query in Pinot server

2021-02-01 Thread GitBox
jackjlli commented on a change in pull request #6361: URL: https://github.com/apache/incubator-pinot/pull/6361#discussion_r568165047 ## File path: pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/BaseClusterIntegrationTestSet.java ## @@ -623,8 +623,10 @

[GitHub] [incubator-pinot] mcvsubbu commented on issue #6516: Set error messages in helix for state transition failues

2021-02-01 Thread GitBox
mcvsubbu commented on issue #6516: URL: https://github.com/apache/incubator-pinot/issues/6516#issuecomment-771178201 If I understand what @fx19880617 is saying, it is to provide an API in the controller to retrieve Helix state transition error messages and perhaps history as well. I conc

[incubator-pinot] branch use-default-broker-controller-ports-when-undefined updated (21fa2b1 -> 9f427eb)

2021-02-01 Thread apucher
This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch use-default-broker-controller-ports-when-undefined in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 21fa2b1 use default broker and controller ports if not define

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #6361: Detect invalid column names from query in Pinot server

2021-02-01 Thread GitBox
jackjlli commented on a change in pull request #6361: URL: https://github.com/apache/incubator-pinot/pull/6361#discussion_r568138689 ## File path: pinot-server/src/main/java/org/apache/pinot/server/starter/helix/HelixInstanceDataManager.java ## @@ -326,6 +339,11 @@ public Segm

[GitHub] [incubator-pinot] kishoreg commented on issue #6516: Set error messages in helix for state transition failues

2021-02-01 Thread GitBox
kishoreg commented on issue #6516: URL: https://github.com/apache/incubator-pinot/issues/6516#issuecomment-771145481 can you please elaborate "set error messages in helix" This is an automated message from the Apache Git Ser

[GitHub] [incubator-pinot] fx19880617 opened a new issue #6516: Set error messages in helix for state transition failues

2021-02-01 Thread GitBox
fx19880617 opened a new issue #6516: URL: https://github.com/apache/incubator-pinot/issues/6516 It can help access the errors from the controller UI without searching server logs for segment errors or broker/server instances go into error status. --

[GitHub] [incubator-pinot] tejasajmera commented on a change in pull request #6514: [TE]frontend - Add filtering support for Entity Monitoring tables

2021-02-01 Thread GitBox
tejasajmera commented on a change in pull request #6514: URL: https://github.com/apache/incubator-pinot/pull/6514#discussion_r568099786 ## File path: thirdeye/thirdeye-frontend/app/utils/utils.js ## @@ -275,11 +277,68 @@ export function buildBounds(series, baseline, timeseries,

[GitHub] [incubator-pinot] zhangloo333 commented on a change in pull request #6514: [TE]frontend - Add filtering support for Entity Monitoring tables

2021-02-01 Thread GitBox
zhangloo333 commented on a change in pull request #6514: URL: https://github.com/apache/incubator-pinot/pull/6514#discussion_r568086902 ## File path: thirdeye/thirdeye-frontend/app/utils/utils.js ## @@ -275,11 +277,68 @@ export function buildBounds(series, baseline, timeseries,

[GitHub] [incubator-pinot] Jackie-Jiang closed pull request #6513: Use default query port for broker when not configured

2021-02-01 Thread GitBox
Jackie-Jiang closed pull request #6513: URL: https://github.com/apache/incubator-pinot/pull/6513 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [incubator-pinot] Jackie-Jiang commented on pull request #6513: Use default query port for broker when not configured

2021-02-01 Thread GitBox
Jackie-Jiang commented on pull request #6513: URL: https://github.com/apache/incubator-pinot/pull/6513#issuecomment-771096497 Replaced by #6515 This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [incubator-pinot] apucher opened a new pull request #6515: use default broker and controller ports if not defined

2021-02-01 Thread GitBox
apucher opened a new pull request #6515: URL: https://github.com/apache/incubator-pinot/pull/6515 ## Description Reverts to legacy behavior on port defaults for broker and controller if listener specs aren't provided. Expands on #6513 . ## Upgrade Notes Does this PR prevent a z

[incubator-pinot] branch use-default-broker-controller-ports-when-undefined created (now 21fa2b1)

2021-02-01 Thread apucher
This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch use-default-broker-controller-ports-when-undefined in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at 21fa2b1 use default broker and controller ports if not define

[incubator-pinot] 01/01: use default broker and controller ports if not defined

2021-02-01 Thread apucher
This is an automated email from the ASF dual-hosted git repository. apucher pushed a commit to branch use-default-broker-controller-ports-when-undefined in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 21fa2b1cce700375871b9eb3004e2c8a9d1366c5 Author: Alexander Pucher

[GitHub] [incubator-pinot] tejasajmera opened a new pull request #6514: [TE]frontend - Add filtering support for Entity Monitoring tables

2021-02-01 Thread GitBox
tejasajmera opened a new pull request #6514: URL: https://github.com/apache/incubator-pinot/pull/6514 Filtering is supposed to work on column level with all possible data types in the table. It will do a substring match of the string passed in with each of the column entries; a return valu

[GitHub] [incubator-pinot] jackjlli commented on pull request #6479: Support data ingestion for generating offline segment in one pass

2021-02-01 Thread GitBox
jackjlli commented on pull request #6479: URL: https://github.com/apache/incubator-pinot/pull/6479#issuecomment-771083123 > > Thanks for the details @jackjlli, could you also is IntermediateSegment better than existing MutableSegment? For example, you could stream input data to MutableSegm

  1   2   >