[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #6424: Use java11

2021-06-08 Thread GitBox
codecov-commenter edited a comment on pull request #6424: URL: https://github.com/apache/incubator-pinot/pull/6424#issuecomment-828596486 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6424?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #6424: Use java11

2021-06-08 Thread GitBox
codecov-commenter edited a comment on pull request #6424: URL: https://github.com/apache/incubator-pinot/pull/6424#issuecomment-828596486 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6424?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] amrishlal commented on pull request #7033: Include null check for setting minion tag.

2021-06-08 Thread GitBox
amrishlal commented on pull request #7033: URL: https://github.com/apache/incubator-pinot/pull/7033#issuecomment-857416829 > we can remove the null check for the returned reference of getTags, while a null check is needed for instanceConfig. Please include that into this PR. Fixed.

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #6991: Introduce in-Segment Trim for GroupBy OrderBy Query

2021-06-08 Thread GitBox
codecov-commenter edited a comment on pull request #6991: URL: https://github.com/apache/incubator-pinot/pull/6991#issuecomment-849847415 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6991?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #6991: Introduce in-Segment Trim for GroupBy OrderBy Query

2021-06-08 Thread GitBox
codecov-commenter edited a comment on pull request #6991: URL: https://github.com/apache/incubator-pinot/pull/6991#issuecomment-849847415 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6991?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] yupeng9 opened a new issue #7036: Segment reader on the deep store for compute frameworks like Presto and Spark

2021-06-08 Thread GitBox
yupeng9 opened a new issue #7036: URL: https://github.com/apache/incubator-pinot/issues/7036 There are senairos that users need to run complex adhoc queries (e.g. multi-way join with other datasets) or ETL jobs that scan all Pinot segments. Pinot servers are not designed this pattern of wo

[GitHub] [incubator-pinot] chenboat commented on pull request #7032: mark PinotFS.move as not final

2021-06-08 Thread GitBox
chenboat commented on pull request #7032: URL: https://github.com/apache/incubator-pinot/pull/7032#issuecomment-857320273 We can contribute the fix. In a high level, we find today's default implementation before `doMove()` has a lot of assumptions on src and dest directory structures. It

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #6899: Add partial upsert config and mergers

2021-06-08 Thread GitBox
codecov-commenter edited a comment on pull request #6899: URL: https://github.com/apache/incubator-pinot/pull/6899#issuecomment-853568285 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6899?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #6899: Add partial upsert config and mergers

2021-06-08 Thread GitBox
codecov-commenter edited a comment on pull request #6899: URL: https://github.com/apache/incubator-pinot/pull/6899#issuecomment-853568285 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6899?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] chenboat edited a comment on pull request #7032: mark PinotFS.move as not final

2021-06-08 Thread GitBox
chenboat edited a comment on pull request #7032: URL: https://github.com/apache/incubator-pinot/pull/7032#issuecomment-857288816 > `move()` is intentional made `final` to force the behavior. To optimize the file move, you can override `doMove()` What is the intention behind add `fina

[GitHub] [incubator-pinot] Jackie-Jiang commented on pull request #7032: mark PinotFS.move as not final

2021-06-08 Thread GitBox
Jackie-Jiang commented on pull request #7032: URL: https://github.com/apache/incubator-pinot/pull/7032#issuecomment-857295337 @chenboat If you find bug in the `move()`, then feel free to fix it. Basically we want to keep the override logic within the `doMove()`, the use `move()` to unify t

[GitHub] [incubator-pinot] chenboat edited a comment on pull request #7032: mark PinotFS.move as not final

2021-06-08 Thread GitBox
chenboat edited a comment on pull request #7032: URL: https://github.com/apache/incubator-pinot/pull/7032#issuecomment-857288816 > `move()` is intentional made `final` to force the behavior. To optimize the file move, you can override `doMove()` What the intention behind add `final`

[GitHub] [incubator-pinot] chenboat commented on pull request #7032: mark PinotFS.move as not final

2021-06-08 Thread GitBox
chenboat commented on pull request #7032: URL: https://github.com/apache/incubator-pinot/pull/7032#issuecomment-857288816 > `move()` is intentional made `final` to force the behavior. To optimize the file move, you can override `doMove()` What the intention behind add `final` to `mov

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #6899: Add partial upsert config and mergers

2021-06-08 Thread GitBox
codecov-commenter edited a comment on pull request #6899: URL: https://github.com/apache/incubator-pinot/pull/6899#issuecomment-853568285 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6899?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[incubator-pinot] branch master updated (d2bfde5 -> 6aac7dd)

2021-06-08 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from d2bfde5 Fixing HadoopPinotFS listFiles method to always contain scheme (#7027) add 6aac7dd fixing p

[incubator-pinot] branch master updated (5237e67 -> d2bfde5)

2021-06-08 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 5237e67 Block filter clause not supported during query parsing (#7023) add d2bfde5 Fixing HadoopPino

[GitHub] [incubator-pinot] xiangfu0 merged pull request #7030: fixing v0_deprecated/pinot-hadoop and v0_deprecated/pinot-spark test

2021-06-08 Thread GitBox
xiangfu0 merged pull request #7030: URL: https://github.com/apache/incubator-pinot/pull/7030 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this servic

[GitHub] [incubator-pinot] xiangfu0 merged pull request #7027: Fixing HadoopPinotFS listFiles method to contain scheme

2021-06-08 Thread GitBox
xiangfu0 merged pull request #7027: URL: https://github.com/apache/incubator-pinot/pull/7027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this servic

[GitHub] [incubator-pinot] jackjlli commented on pull request #7033: Include null check for setting minion tag.

2021-06-08 Thread GitBox
jackjlli commented on pull request #7033: URL: https://github.com/apache/incubator-pinot/pull/7033#issuecomment-857268994 > `InstanceConfig.getTags()` should never return `null`. Are you using a different Helix version? > Here's the Helix code: > > ``` > public List getTags()

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #7033: Include null check for setting minion tag.

2021-06-08 Thread GitBox
jackjlli commented on a change in pull request #7033: URL: https://github.com/apache/incubator-pinot/pull/7033#discussion_r647868700 ## File path: pinot-minion/src/main/java/org/apache/pinot/minion/MinionStarter.java ## @@ -284,7 +284,8 @@ private void addInstanceTagIfNeeded()

[GitHub] [incubator-pinot] Jackie-Jiang commented on pull request #7033: Include null check for setting minion tag.

2021-06-08 Thread GitBox
Jackie-Jiang commented on pull request #7033: URL: https://github.com/apache/incubator-pinot/pull/7033#issuecomment-857255635 `InstanceConfig.getTags()` should never return `null`. Are you using a different Helix version? Here's the Helix code: ``` public List getTags() {

[GitHub] [incubator-pinot] Jackie-Jiang closed pull request #6874: Init mutableSegmentRecordReader to handle partial upsert record lookup

2021-06-08 Thread GitBox
Jackie-Jiang closed pull request #6874: URL: https://github.com/apache/incubator-pinot/pull/6874 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this se

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7035: Support LZ4 compression codec for raw index (#6804)

2021-06-08 Thread GitBox
codecov-commenter edited a comment on pull request #7035: URL: https://github.com/apache/incubator-pinot/pull/7035#issuecomment-857236216 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7035?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7030: fixing v0_deprecated/pinot-hadoop and v0_deprecated/pinot-spark test

2021-06-08 Thread GitBox
codecov-commenter edited a comment on pull request #7030: URL: https://github.com/apache/incubator-pinot/pull/7030#issuecomment-857235402 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7030?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7035: Support LZ4 compression codec for raw index (#6804)

2021-06-08 Thread GitBox
codecov-commenter commented on pull request #7035: URL: https://github.com/apache/incubator-pinot/pull/7035#issuecomment-857236216 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7035?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7030: fixing v0_deprecated/pinot-hadoop and v0_deprecated/pinot-spark test

2021-06-08 Thread GitBox
codecov-commenter commented on pull request #7030: URL: https://github.com/apache/incubator-pinot/pull/7030#issuecomment-857235402 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7030?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[incubator-pinot] branch master updated (88843ee -> 5237e67)

2021-06-08 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 88843ee Fix bug in fetching subset of tier config props (#7025) add 5237e67 Block filter clause not s

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #7023: Block filter clause not supported during query parsing

2021-06-08 Thread GitBox
Jackie-Jiang merged pull request #7023: URL: https://github.com/apache/incubator-pinot/pull/7023 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this se

[GitHub] [incubator-pinot] GSharayu opened a new pull request #7035: Support LZ4 compression codec for raw index (#6804)

2021-06-08 Thread GitBox
GSharayu opened a new pull request #7035: URL: https://github.com/apache/incubator-pinot/pull/7035 This ticket is extension of adding new compression codec LZ4 after ZSTD(#6876) for Issue (#6804). When the forward index is not dictionary encoded, we have 2 choices: store the data

[incubator-pinot] branch master updated (e91f426 -> 88843ee)

2021-06-08 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from e91f426 Improvements in compat-tester (#7024) add 88843ee Fix bug in fetching subset of tier config p

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #7025: Fix bug in fetching subset of tier config props

2021-06-08 Thread GitBox
Jackie-Jiang merged pull request #7025: URL: https://github.com/apache/incubator-pinot/pull/7025 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this se

[GitHub] [incubator-pinot] snleee commented on a change in pull request #7026: Add Apache Pulsar low level and high level connector

2021-06-08 Thread GitBox
snleee commented on a change in pull request #7026: URL: https://github.com/apache/incubator-pinot/pull/7026#discussion_r647769770 ## File path: pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarConfig.java ## @@ -0,0 +

[GitHub] [incubator-pinot] xiangfu0 commented on pull request #7030: fixing v0_deprecated/pinot-hadoop and v0_deprecated/pinot-spark test

2021-06-08 Thread GitBox
xiangfu0 commented on pull request #7030: URL: https://github.com/apache/incubator-pinot/pull/7030#issuecomment-857139059 > I noticed the same thing happening to the pinot-spark module. We need to do the same thing there. done -- This is an automated message from the Apache Git Se

[incubator-pinot] branch fixing_pinot-hadoop-test updated (0b87114 -> 00dcf0c)

2021-06-08 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch fixing_pinot-hadoop-test in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard 0b87114 fixing pinot-hadoop test add 00dcf0c fixing pinot-hadoop and pinot-spark t

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7033: Include null check for setting minion tag.

2021-06-08 Thread GitBox
codecov-commenter commented on pull request #7033: URL: https://github.com/apache/incubator-pinot/pull/7033#issuecomment-857129444 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7033?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[GitHub] [incubator-pinot] GSharayu opened a new issue #7034: Support range index rule recommendation

2021-06-08 Thread GitBox
GSharayu opened a new issue #7034: URL: https://github.com/apache/incubator-pinot/issues/7034 Currently, As part of pinot automated onboarding, we support automated inverted index and sorted index recommendations. Wiki Doc: https://cwiki.apache.org/confluence/display/PINOT/Autom

[GitHub] [incubator-pinot] Jackie-Jiang commented on pull request #7032: mark PinotFS.move as not final

2021-06-08 Thread GitBox
Jackie-Jiang commented on pull request #7032: URL: https://github.com/apache/incubator-pinot/pull/7032#issuecomment-857072025 `move()` is intentional made `final` to force the behavior. To optimize the file move, you can override `doMove()` -- This is an automated message from the Apache

[GitHub] [incubator-pinot] amrishlal commented on pull request #7033: Include null check for setting minion tag.

2021-06-08 Thread GitBox
amrishlal commented on pull request #7033: URL: https://github.com/apache/incubator-pinot/pull/7033#issuecomment-857068913 @jackjlli Please review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [incubator-pinot] amrishlal opened a new pull request #7033: Include null check for setting minion tag.

2021-06-08 Thread GitBox
amrishlal opened a new pull request #7033: URL: https://github.com/apache/incubator-pinot/pull/7033 ## Description Null check to ensure that a default tag will be added for Pinot Minion instance even when `instanceTags` are null. Broker and Server already carry out similar null check.

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7032: mark PinotFS.move as not final

2021-06-08 Thread GitBox
codecov-commenter commented on pull request #7032: URL: https://github.com/apache/incubator-pinot/pull/7032#issuecomment-857065048 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7032?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[GitHub] [incubator-pinot] chenboat commented on pull request #7032: mark PinotFS.move as not final

2021-06-08 Thread GitBox
chenboat commented on pull request #7032: URL: https://github.com/apache/incubator-pinot/pull/7032#issuecomment-857001559 lgtm. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [incubator-pinot] chenboat closed pull request #7032: mark PinotFS.move as not final

2021-06-08 Thread GitBox
chenboat closed pull request #7032: URL: https://github.com/apache/incubator-pinot/pull/7032 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this servic

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7031: Changes in QueryInvertedSortedIndexRecommender to handle non-dimension columns for sorted and inverted index(#7018)

2021-06-08 Thread GitBox
codecov-commenter commented on pull request #7031: URL: https://github.com/apache/incubator-pinot/pull/7031#issuecomment-856998244 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7031?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #6991: Introduce in-Segment Trim for GroupBy OrderBy Query

2021-06-08 Thread GitBox
codecov-commenter edited a comment on pull request #6991: URL: https://github.com/apache/incubator-pinot/pull/6991#issuecomment-849847415 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6991?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] yupeng9 commented on pull request #7032: mark PinotFS.move as not final

2021-06-08 Thread GitBox
yupeng9 commented on pull request #7032: URL: https://github.com/apache/incubator-pinot/pull/7032#issuecomment-856975758 @elonazoulay fyi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7023: Block filter clause not supported during query parsing

2021-06-08 Thread GitBox
Jackie-Jiang commented on a change in pull request #7023: URL: https://github.com/apache/incubator-pinot/pull/7023#discussion_r647670938 ## File path: pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java ## @@ -179,11 +179,6 @@ public void testFi

[GitHub] [incubator-pinot] yupeng9 opened a new pull request #7032: mark PinotFS.move as not final

2021-06-08 Thread GitBox
yupeng9 opened a new pull request #7032: URL: https://github.com/apache/incubator-pinot/pull/7032 Recently `PinotFS.move` was marked as final in this [PR](https://github.com/apache/incubator-pinot/pull/6841). However, it's possible for the subclass to have different implementations for oth

[GitHub] [incubator-pinot] GSharayu opened a new pull request #7031: Changes in QueryInvertedSortedIndexRecommender to handle non-dimension columns for sorted and inverted index(#7018)

2021-06-08 Thread GitBox
GSharayu opened a new pull request #7031: URL: https://github.com/apache/incubator-pinot/pull/7031 Typically the dimension columns are used in filter and group by. However, technically nothing prevents user from doing a filter on a non-dimension column. So, we need to update the query inve

[GitHub] [incubator-pinot] mayankshriv commented on pull request #7029: Upgrade t-digest to latest release

2021-06-08 Thread GitBox
mayankshriv commented on pull request #7029: URL: https://github.com/apache/incubator-pinot/pull/7029#issuecomment-856919437 > OK. I understand this now. The problem is that I did an incompetent job of breaking t-digest into multiple modules. This meant that I needed to publish the parent

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7026: Add Apache Pulsar low level and high level connector

2021-06-08 Thread GitBox
codecov-commenter edited a comment on pull request #7026: URL: https://github.com/apache/incubator-pinot/pull/7026#issuecomment-855347878 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7026?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] KKcorps commented on a change in pull request #7026: Add Apache Pulsar low level and high level connector

2021-06-08 Thread GitBox
KKcorps commented on a change in pull request #7026: URL: https://github.com/apache/incubator-pinot/pull/7026#discussion_r647547669 ## File path: pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarPartitionLevelConsumer.java ###

[GitHub] [incubator-pinot] npawar commented on a change in pull request #7026: Add Apache Pulsar low level and high level connector

2021-06-08 Thread GitBox
npawar commented on a change in pull request #7026: URL: https://github.com/apache/incubator-pinot/pull/7026#discussion_r647530043 ## File path: pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarConsumerFactory.java ##

[GitHub] [incubator-pinot] KKcorps commented on a change in pull request #7026: Add Apache Pulsar low level and high level connector

2021-06-08 Thread GitBox
KKcorps commented on a change in pull request #7026: URL: https://github.com/apache/incubator-pinot/pull/7026#discussion_r647520268 ## File path: pinot-plugins/pinot-stream-ingestion/pinot-pulsar/pom.xml ## @@ -0,0 +1,31 @@ + +http://maven.apache.org/POM/4.0.0"; Review comment

[GitHub] [incubator-pinot] KKcorps commented on a change in pull request #7026: Add Apache Pulsar low level and high level connector

2021-06-08 Thread GitBox
KKcorps commented on a change in pull request #7026: URL: https://github.com/apache/incubator-pinot/pull/7026#discussion_r647312517 ## File path: pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/test/java/org/apache/pinot/plugin/stream/pulsar/PulsarStandaloneCluster.java

[GitHub] [incubator-pinot] KKcorps commented on a change in pull request #7026: Add Apache Pulsar low level and high level connector

2021-06-08 Thread GitBox
KKcorps commented on a change in pull request #7026: URL: https://github.com/apache/incubator-pinot/pull/7026#discussion_r647311963 ## File path: pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarConsumerFactory.java ##

[GitHub] [incubator-pinot] KKcorps commented on a change in pull request #7026: Add Apache Pulsar low level and high level connector

2021-06-08 Thread GitBox
KKcorps commented on a change in pull request #7026: URL: https://github.com/apache/incubator-pinot/pull/7026#discussion_r647310962 ## File path: pinot-plugins/pinot-stream-ingestion/pinot-pulsar/pom.xml ## @@ -0,0 +1,31 @@ + +http://maven.apache.org/POM/4.0.0"; Review comment