[pinot] branch master updated: Wire EmptySegmentPruner to routing config (#8067)

2022-01-31 Thread snlee
This is an automated email from the ASF dual-hosted git repository. snlee pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 71e28a2 Wire EmptySegmentPruner to routing config

[GitHub] [pinot] snleee merged pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
snleee merged pull request #8067: URL: https://github.com/apache/pinot/pull/8067 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr.

[GitHub] [pinot] richardstartin commented on pull request #8055: Support BOOLEAN type in recommender engine

2022-01-31 Thread GitBox
richardstartin commented on pull request #8055: URL: https://github.com/apache/pinot/pull/8055#issuecomment-1026547479 @lksvenoy-r7 I’m not sure why the integration tests are failing. Recommender support for TIMESTAMP was just added, can you rebase please? -- This is an automated messag

[pinot] branch master updated: downgrade verbose log message to DEBUG (#8096)

2022-01-31 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 6eb71f6 downgrade verbose log message to

[GitHub] [pinot] richardstartin merged pull request #8096: downgrade verbose log message to DEBUG

2022-01-31 Thread GitBox
richardstartin merged pull request #8096: URL: https://github.com/apache/pinot/pull/8096 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-u

[pinot] branch master updated (a22cc00 -> bd42dc3)

2022-01-31 Thread siddteotia
This is an automated email from the ASF dual-hosted git repository. siddteotia pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git. from a22cc00 Broker Side validation for the query with aggregation and col but without group by (#7972) add bd42

[pinot] branch master updated: Broker Side validation for the query with aggregation and col but without group by (#7972)

2022-01-31 Thread siddteotia
This is an automated email from the ASF dual-hosted git repository. siddteotia pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new a22cc00 Broker Side validation for the query

[GitHub] [pinot] siddharthteotia merged pull request #7972: Broker Side validation for the query with aggregation and col but without group by

2022-01-31 Thread GitBox
siddharthteotia merged pull request #7972: URL: https://github.com/apache/pinot/pull/7972 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7959: [7867] Handle Select * with Extra Columns

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #7959: URL: https://github.com/apache/pinot/pull/7959#issuecomment-1019871784 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7959?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7959: [7867] Handle Select * with Extra Columns

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #7959: URL: https://github.com/apache/pinot/pull/7959#issuecomment-1019871784 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7959?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7959: [7867] Handle Select * with Extra Columns

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #7959: URL: https://github.com/apache/pinot/pull/7959#issuecomment-1019871784 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7959?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] suddendust commented on pull request #7959: [7867] Handle Select * with Extra Columns

2022-01-31 Thread GitBox
suddendust commented on pull request #7959: URL: https://github.com/apache/pinot/pull/7959#issuecomment-1026483620 @Jackie-Jiang Have addressed the comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [pinot] suddendust commented on a change in pull request #7959: [7867] Handle Select * with Extra Columns

2022-01-31 Thread GitBox
suddendust commented on a change in pull request #7959: URL: https://github.com/apache/pinot/pull/7959#discussion_r796266819 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java ## @@ -1487,6 +1498,33 @@ static void upda

[GitHub] [pinot] suddendust commented on a change in pull request #7959: [7867] Handle Select * with Extra Columns

2022-01-31 Thread GitBox
suddendust commented on a change in pull request #7959: URL: https://github.com/apache/pinot/pull/7959#discussion_r796266529 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java ## @@ -1459,8 +1460,18 @@ static void upda

[GitHub] [pinot] PrachiPrakash commented on a change in pull request #7972: Broker Side validation for the query with aggregation and col but without group by

2022-01-31 Thread GitBox
PrachiPrakash commented on a change in pull request #7972: URL: https://github.com/apache/pinot/pull/7972#discussion_r796242482 ## File path: pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java ## @@ -146,23 +146,30 @@ static void validate(PinotQuery

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #8067: URL: https://github.com/apache/pinot/pull/8067#issuecomment-1020719849 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8067?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #8067: URL: https://github.com/apache/pinot/pull/8067#issuecomment-1020719849 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8067?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] jackjlli edited a comment on pull request #8098: Validate the numbers of input and output files in HadoopSegmentCreationJob

2022-01-31 Thread GitBox
jackjlli edited a comment on pull request #8098: URL: https://github.com/apache/pinot/pull/8098#issuecomment-1026332309 > What if user wants to replace an existing segment with a new generated one? We should allow overriding existing segment, but not the newly pushed ones. If user wa

[GitHub] [pinot] jackjlli commented on pull request #8098: Validate the numbers of input and output files in HadoopSegmentCreationJob

2022-01-31 Thread GitBox
jackjlli commented on pull request #8098: URL: https://github.com/apache/pinot/pull/8098#issuecomment-1026415697 @sajjad-moradi Yes, that's tested in the Hadoop env. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [pinot] jackjlli edited a comment on pull request #8098: Validate the numbers of input and output files in HadoopSegmentCreationJob

2022-01-31 Thread GitBox
jackjlli edited a comment on pull request #8098: URL: https://github.com/apache/pinot/pull/8098#issuecomment-1026332309 > What if user wants to replace an existing segment with a new generated one? We should allow overriding existing segment, but not the newly pushed ones. If user wa

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #8067: URL: https://github.com/apache/pinot/pull/8067#issuecomment-1020719849 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8067?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #8067: URL: https://github.com/apache/pinot/pull/8067#issuecomment-1020719849 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8067?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7819: Upgrade to JsonPath 2.7.0

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #7819: URL: https://github.com/apache/pinot/pull/7819#issuecomment-976515875 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7819?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7819: Upgrade to JsonPath 2.7.0

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #7819: URL: https://github.com/apache/pinot/pull/7819#issuecomment-976515875 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7819?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] snleee commented on a change in pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
snleee commented on a change in pull request #8067: URL: https://github.com/apache/pinot/pull/8067#discussion_r796189579 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/routing/segmentpruner/SegmentPrunerFactory.java ## @@ -45,11 +46,14 @@ private SegmentPrune

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #8067: URL: https://github.com/apache/pinot/pull/8067#issuecomment-1020719849 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8067?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8083: Timeout if waiting server channel lock takes a long time

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #8083: URL: https://github.com/apache/pinot/pull/8083#issuecomment-1023942899 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8083?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8083: Timeout if waiting server channel lock takes a long time

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #8083: URL: https://github.com/apache/pinot/pull/8083#issuecomment-1023942899 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8083?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8083: Timeout if waiting server channel lock takes a long time

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #8083: URL: https://github.com/apache/pinot/pull/8083#issuecomment-1023942899 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8083?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #8067: URL: https://github.com/apache/pinot/pull/8067#issuecomment-1020719849 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8067?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8083: Timeout if waiting server channel lock takes a long time

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #8083: URL: https://github.com/apache/pinot/pull/8083#issuecomment-1023942899 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8083?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7819: Upgrade to JsonPath 2.7.0

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #7819: URL: https://github.com/apache/pinot/pull/7819#issuecomment-976515875 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7819?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] mqliang opened a new issue #8099: Add Built-In plugin related info into pinot-common?

2022-01-31 Thread GitBox
mqliang opened a new issue #8099: URL: https://github.com/apache/pinot/issues/8099 While it's true that any pinot package should has NO dependency on pinot plugins, we have some ***build-in*** plugin, which is likely will never be removed from pinot repo, kinesis is one of the example.

[GitHub] [pinot] jackjlli commented on pull request #8098: Validate the numbers of input and output files in HadoopSegmentCreationJob

2022-01-31 Thread GitBox
jackjlli commented on pull request #8098: URL: https://github.com/apache/pinot/pull/8098#issuecomment-1026332309 > What if user wants to replace an existing segment with a new generated one? We should allow overriding existing segment, but not the newly pushed ones. If user wants to

[pinot] branch disable-overwrite-if-sequence-id-is-excluded updated (82cd206 -> 5f1f61a)

2022-01-31 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch disable-overwrite-if-sequence-id-is-excluded in repository https://gitbox.apache.org/repos/asf/pinot.git. discard 82cd206 Set overwrite to false if exclude.sequence.id is enabled add 5f1f61a

[GitHub] [pinot] mqliang commented on a change in pull request #8083: Timeout if waiting server channel lock takes a long time

2022-01-31 Thread GitBox
mqliang commented on a change in pull request #8083: URL: https://github.com/apache/pinot/pull/8083#discussion_r796157511 ## File path: pinot-core/src/main/java/org/apache/pinot/core/transport/ServerChannels.java ## @@ -139,22 +143,32 @@ private void attachSSLHandler(SocketCha

[GitHub] [pinot] richardstartin commented on a change in pull request #8083: Timeout if waiting server channel lock takes a long time

2022-01-31 Thread GitBox
richardstartin commented on a change in pull request #8083: URL: https://github.com/apache/pinot/pull/8083#discussion_r796152572 ## File path: pinot-core/src/main/java/org/apache/pinot/core/transport/ServerChannels.java ## @@ -139,22 +143,32 @@ private void attachSSLHandler(So

[GitHub] [pinot] richardstartin commented on a change in pull request #8083: Timeout if waiting server channel lock takes a long time

2022-01-31 Thread GitBox
richardstartin commented on a change in pull request #8083: URL: https://github.com/apache/pinot/pull/8083#discussion_r796150428 ## File path: pinot-core/src/main/java/org/apache/pinot/core/transport/ServerChannels.java ## @@ -81,10 +83,11 @@ public ServerChannels(QueryRouter

[GitHub] [pinot] mqliang commented on a change in pull request #8083: Timeout if waiting server channel lock takes a long time

2022-01-31 Thread GitBox
mqliang commented on a change in pull request #8083: URL: https://github.com/apache/pinot/pull/8083#discussion_r796146817 ## File path: pinot-core/src/main/java/org/apache/pinot/core/transport/ServerChannels.java ## @@ -139,22 +143,32 @@ private void attachSSLHandler(SocketCha

[GitHub] [pinot] mqliang commented on a change in pull request #8083: Timeout if waiting server channel lock takes a long time

2022-01-31 Thread GitBox
mqliang commented on a change in pull request #8083: URL: https://github.com/apache/pinot/pull/8083#discussion_r796145253 ## File path: pinot-core/src/main/java/org/apache/pinot/core/transport/ServerChannels.java ## @@ -139,22 +143,32 @@ private void attachSSLHandler(SocketCha

[GitHub] [pinot] mqliang closed pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
mqliang closed pull request #8067: URL: https://github.com/apache/pinot/pull/8067 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[GitHub] [pinot] snleee commented on a change in pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
snleee commented on a change in pull request #8067: URL: https://github.com/apache/pinot/pull/8067#discussion_r796136114 ## File path: pinot-broker/src/test/java/org/apache/pinot/broker/routing/segmentpruner/SegmentPrunerTest.java ## @@ -39,6 +39,7 @@ import org.apache.pinot.

[GitHub] [pinot] Jackie-Jiang commented on pull request #8096: downgrade verbose log message to DEBUG

2022-01-31 Thread GitBox
Jackie-Jiang commented on pull request #8096: URL: https://github.com/apache/pinot/pull/8096#issuecomment-1026295369 The purpose of this warning is to let the admin know we suggest using the table name as the schema name. I agree logging a warning for each fetch is too much, so I'm suggest

[GitHub] [pinot] snleee commented on a change in pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
snleee commented on a change in pull request #8067: URL: https://github.com/apache/pinot/pull/8067#discussion_r796117810 ## File path: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/TableConfigUtils.java ## @@ -857,4 +860,53 @@ public static void verify

[GitHub] [pinot] Jackie-Jiang commented on pull request #8098: Set overwrite to false if exclude.sequence.id is enabled

2022-01-31 Thread GitBox
Jackie-Jiang commented on pull request #8098: URL: https://github.com/apache/pinot/pull/8098#issuecomment-1026281135 What if user wants to replace an existing segment with a new generated one? We should allow overriding existing segment, but not the newly pushed ones. -- This is an aut

[GitHub] [pinot] Jackie-Jiang commented on issue #8085: Add `least` and `greatest` functions

2022-01-31 Thread GitBox
Jackie-Jiang commented on issue #8085: URL: https://github.com/apache/pinot/issues/8085#issuecomment-1026278418 We might want both because transform function cannot be applied to ingestion transforms and post-aggregation transforms. Performance wise, transform function version is definitel

[GitHub] [pinot] snleee commented on a change in pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
snleee commented on a change in pull request #8067: URL: https://github.com/apache/pinot/pull/8067#discussion_r796117810 ## File path: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/TableConfigUtils.java ## @@ -857,4 +860,53 @@ public static void verify

[GitHub] [pinot] richardstartin commented on pull request #8096: downgrade verbose log message to DEBUG

2022-01-31 Thread GitBox
richardstartin commented on pull request #8096: URL: https://github.com/apache/pinot/pull/8096#issuecomment-1026275833 > We can move this check to `TableConfigUtils.validate()` and keep it as warning (debug will mostly be ignored). Then the warning will be logged only when the table config

[pinot] branch master updated (0009877 -> 34f371d)

2022-01-31 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git. from 0009877 Listener TLS customization (#8082) add 34f371d benchmark with polluted readers (#8065) No new

[GitHub] [pinot] richardstartin merged pull request #8065: benchmark with polluted readers

2022-01-31 Thread GitBox
richardstartin merged pull request #8065: URL: https://github.com/apache/pinot/pull/8065 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-u

[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7959: [7867] Handle Select * with Extra Columns

2022-01-31 Thread GitBox
Jackie-Jiang commented on a change in pull request #7959: URL: https://github.com/apache/pinot/pull/7959#discussion_r796107897 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java ## @@ -1487,6 +1498,33 @@ static void up

[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8083: Timeout if waiting server channel lock takes a long time

2022-01-31 Thread GitBox
Jackie-Jiang commented on a change in pull request #8083: URL: https://github.com/apache/pinot/pull/8083#discussion_r796100819 ## File path: pinot-core/src/main/java/org/apache/pinot/core/transport/ServerChannels.java ## @@ -139,22 +143,32 @@ private void attachSSLHandler(Sock

[GitHub] [pinot] apucher commented on a change in pull request #8082: Listener TLS customization

2022-01-31 Thread GitBox
apucher commented on a change in pull request #8082: URL: https://github.com/apache/pinot/pull/8082#discussion_r796096008 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/ListenerConfigUtil.java ## @@ -167,24 +172,38 @@ private ListenerConfigUtil() { retu

[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7972: Broker Side validation for the query with aggregation and col but without group by

2022-01-31 Thread GitBox
Jackie-Jiang commented on a change in pull request #7972: URL: https://github.com/apache/pinot/pull/7972#discussion_r796095223 ## File path: pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java ## @@ -146,23 +146,30 @@ static void validate(PinotQuery p

[pinot] branch master updated (2a36d4c -> 0009877)

2022-01-31 Thread apucher
This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git. from 2a36d4c Add error message if unable to create index / tmp resource dir (#8076) add 0009877 Listener TLS custo

[GitHub] [pinot] daniellavoie commented on a change in pull request #8082: Listener TLS customization

2022-01-31 Thread GitBox
daniellavoie commented on a change in pull request #8082: URL: https://github.com/apache/pinot/pull/8082#discussion_r796093723 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/ListenerConfigUtil.java ## @@ -167,24 +172,38 @@ private ListenerConfigUtil() {

[GitHub] [pinot] apucher merged pull request #8082: Listener TLS customization

2022-01-31 Thread GitBox
apucher merged pull request #8082: URL: https://github.com/apache/pinot/pull/8082 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[GitHub] [pinot] apucher commented on a change in pull request #8082: Listener TLS customization

2022-01-31 Thread GitBox
apucher commented on a change in pull request #8082: URL: https://github.com/apache/pinot/pull/8082#discussion_r796092701 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/ListenerConfigUtil.java ## @@ -167,24 +172,38 @@ private ListenerConfigUtil() { retu

[GitHub] [pinot] walterddr commented on issue #8092: should SegmentDeletionManager expose the 2 public APIs?

2022-01-31 Thread GitBox
walterddr commented on issue #8092: URL: https://github.com/apache/pinot/issues/8092#issuecomment-1026240627 Exactly what I thought as well @Jackie-Jiang ^ implementing a fix -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [pinot] Jackie-Jiang commented on issue #8092: should SegmentDeletionManager expose the 2 public APIs?

2022-01-31 Thread GitBox
Jackie-Jiang commented on issue #8092: URL: https://github.com/apache/pinot/issues/8092#issuecomment-1026236951 Good point. The retention manager doesn't really check the deep store file, but scan the ZK metadata, so I think we should first delete from deep store, then remove the ZK met

[GitHub] [pinot] npawar commented on a change in pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
npawar commented on a change in pull request #8067: URL: https://github.com/apache/pinot/pull/8067#discussion_r796081940 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/routing/segmentpruner/SegmentPrunerFactory.java ## @@ -45,30 +50,41 @@ private SegmentPrune

[GitHub] [pinot] npawar commented on a change in pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
npawar commented on a change in pull request #8067: URL: https://github.com/apache/pinot/pull/8067#discussion_r796081663 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/routing/segmentpruner/SegmentPrunerFactory.java ## @@ -45,30 +50,41 @@ private SegmentPrune

[GitHub] [pinot] richardstartin commented on pull request #7819: Upgrade to JsonPath 2.7.0

2022-01-31 Thread GitBox
richardstartin commented on pull request #7819: URL: https://github.com/apache/pinot/pull/7819#issuecomment-1026223193 > Thanks for contributing the fix to the json-path library! > In `JsonFunctions.java` I can still see some method throwing `JsonProcessingException`. Should it be elimin

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7819: Upgrade to JsonPath 2.7.0

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #7819: URL: https://github.com/apache/pinot/pull/7819#issuecomment-976515875 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7819?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8097: fix FST Like query benchmark to remove SQL parsing from the measurement

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #8097: URL: https://github.com/apache/pinot/pull/8097#issuecomment-1026215800 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8097?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #8098: Set overwrite to false if exclude.sequence.id is enabled

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #8098: URL: https://github.com/apache/pinot/pull/8098#issuecomment-1026208105 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8098?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter commented on pull request #8097: fix FST Like query benchmark to remove SQL parsing from the measurement

2022-01-31 Thread GitBox
codecov-commenter commented on pull request #8097: URL: https://github.com/apache/pinot/pull/8097#issuecomment-1026215800 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8097?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apa

[GitHub] [pinot] mqliang commented on a change in pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
mqliang commented on a change in pull request #8067: URL: https://github.com/apache/pinot/pull/8067#discussion_r796067442 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/routing/segmentpruner/SegmentPrunerFactory.java ## @@ -45,30 +50,41 @@ private SegmentPrun

[GitHub] [pinot] codecov-commenter commented on pull request #8098: Set overwrite to false if exclude.sequence.id is enabled

2022-01-31 Thread GitBox
codecov-commenter commented on pull request #8098: URL: https://github.com/apache/pinot/pull/8098#issuecomment-1026208105 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8098?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apa

[GitHub] [pinot] richardstartin commented on pull request #8097: fix FST Like query benchmark to remove SQL parsing from the measurement

2022-01-31 Thread GitBox
richardstartin commented on pull request #8097: URL: https://github.com/apache/pinot/pull/8097#issuecomment-1026188530 Added a query to match a prefix - Lucene winning for prefix queries too: ``` Benchmark(_fstType) (_intBaseValue) (_numBlocks) (

[GitHub] [pinot] jackjlli opened a new pull request #8098: Set overwrite to false if exclude.sequence.id is enabled

2022-01-31 Thread GitBox
jackjlli opened a new pull request #8098: URL: https://github.com/apache/pinot/pull/8098 ## Description If the `exclude.sequence.id` property is set to `true`, segment name will not have the sequence id as the suffix. If there are multiple input files for the same day within the batch j

[GitHub] [pinot] richardstartin commented on a change in pull request #7819: Upgrade to JsonPath 2.7.0

2022-01-31 Thread GitBox
richardstartin commented on a change in pull request #7819: URL: https://github.com/apache/pinot/pull/7819#discussion_r796037551 ## File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/JsonFunctions.java ## @@ -93,8 +95,7 @@ public static Object jsonPa

[pinot] branch disable-overwrite-if-sequence-id-is-excluded created (now 82cd206)

2022-01-31 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch disable-overwrite-if-sequence-id-is-excluded in repository https://gitbox.apache.org/repos/asf/pinot.git. at 82cd206 Set overwrite to false if exclude.sequence.id is enabled This branch inc

[pinot] 01/01: Set overwrite to false if exclude.sequence.id is enabled

2022-01-31 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch disable-overwrite-if-sequence-id-is-excluded in repository https://gitbox.apache.org/repos/asf/pinot.git commit 82cd206285d049d63e7900e4c23b5c1ef37225a1 Author: Jack Li(Analytics Engineering) Autho

[GitHub] [pinot] jackjlli commented on a change in pull request #8067: Wire EmptySegmentPruner to routing config

2022-01-31 Thread GitBox
jackjlli commented on a change in pull request #8067: URL: https://github.com/apache/pinot/pull/8067#discussion_r796016680 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/routing/segmentpruner/SegmentPrunerFactory.java ## @@ -45,11 +46,14 @@ private SegmentPru

[GitHub] [pinot] mcvsubbu commented on a change in pull request #8078: Segment retention for table

2022-01-31 Thread GitBox
mcvsubbu commented on a change in pull request #8078: URL: https://github.com/apache/pinot/pull/8078#discussion_r796012927 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -703,7 +703,14 @@ public synchron

[GitHub] [pinot] richardstartin commented on a change in pull request #7819: Upgrade to JsonPath 2.7.0

2022-01-31 Thread GitBox
richardstartin commented on a change in pull request #7819: URL: https://github.com/apache/pinot/pull/7819#discussion_r796005683 ## File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/JsonFunctions.java ## @@ -93,8 +95,7 @@ public static Object jsonPa

[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8082: Listener TLS customization

2022-01-31 Thread GitBox
Jackie-Jiang commented on a change in pull request #8082: URL: https://github.com/apache/pinot/pull/8082#discussion_r796004459 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/ListenerConfigUtil.java ## @@ -167,24 +172,38 @@ private ListenerConfigUtil() {

[GitHub] [pinot] atris commented on pull request #8097: fix FST Like query benchmark to remove SQL parsing from the measurement

2022-01-31 Thread GitBox
atris commented on pull request #8097: URL: https://github.com/apache/pinot/pull/8097#issuecomment-1026136438 Thanks for the tag, I will take a look tomorrow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [pinot] Jackie-Jiang commented on pull request #8097: fix FST Like query benchmark to remove SQL parsing from the measurement

2022-01-31 Thread GitBox
Jackie-Jiang commented on pull request #8097: URL: https://github.com/apache/pinot/pull/8097#issuecomment-1026135964 @atris Can you please take a look at this PR, and see if the benchmark result is valid? If so, we should revisit the native FST solution and make its performance at least on

[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7819: Upgrade to JsonPath 2.7.0

2022-01-31 Thread GitBox
Jackie-Jiang commented on a change in pull request #7819: URL: https://github.com/apache/pinot/pull/7819#discussion_r795992164 ## File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/JsonFunctions.java ## @@ -103,11 +104,13 @@ public static Object json

[GitHub] [pinot] walterddr commented on issue #8091: table creation shouldn't be allowed if table delete is happening async in background

2022-01-31 Thread GitBox
walterddr commented on issue #8091: URL: https://github.com/apache/pinot/issues/8091#issuecomment-1026127214 I have the same observation as @Jackie-Jiang and also lean towards the 1st solution. -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [pinot] richardstartin commented on issue #8085: Add `least` and `greatest` functions

2022-01-31 Thread GitBox
richardstartin commented on issue #8085: URL: https://github.com/apache/pinot/issues/8085#issuecomment-1026115986 I would prefer to implement this as a transform function so we can operate on arbitrary tuples. I already experimented with adding `least` and `greatest` to `ArithmeticFunction

[GitHub] [pinot] Jackie-Jiang commented on issue #8085: Add `least` and `greatest` functions

2022-01-31 Thread GitBox
Jackie-Jiang commented on issue #8085: URL: https://github.com/apache/pinot/issues/8085#issuecomment-1026109773 We should replace the scalar function `min` with `least` and `max` with `greatest` (in `ArithmeticFunctions.java`). Currently we don't support arbitrary number of arguments in sc

[GitHub] [pinot] Jackie-Jiang commented on issue #8079: Calcite parsing error on column contains keywords

2022-01-31 Thread GitBox
Jackie-Jiang commented on issue #8079: URL: https://github.com/apache/pinot/issues/8079#issuecomment-1026104938 Is this query supported in presto? This query syntax is not supported by Calcite (at least for the lex and conformance we use), so I think this query cannot be supported by other

[GitHub] [pinot] Jackie-Jiang merged pull request #8076: Add error message if unable to create index / tmp resource dir

2022-01-31 Thread GitBox
Jackie-Jiang merged pull request #8076: URL: https://github.com/apache/pinot/pull/8076 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-uns

[pinot] branch master updated: Add error message if unable to create index / tmp resource dir (#8076)

2022-01-31 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 2a36d4c Add error message if unable to create in

[GitHub] [pinot] Jackie-Jiang commented on issue #8091: table creation shouldn't be allowed if table delete is happening async in background

2022-01-31 Thread GitBox
Jackie-Jiang commented on issue #8091: URL: https://github.com/apache/pinot/issues/8091#issuecomment-1026099184 When creating the table, we should check all the related entries for the table is already removed. Per the sequence of entry deletion, we should check table config, instance part

[GitHub] [pinot] richardstartin commented on pull request #8097: fix FST Like query benchmark to remove SQL parsing from the measurement

2022-01-31 Thread GitBox
richardstartin commented on pull request #8097: URL: https://github.com/apache/pinot/pull/8097#issuecomment-1026096193 At 100 blocks planning time still dominates this benchmark, but the cost of block iteration can be seen to increase for Lucene: Lucene: https://user-images.github

[GitHub] [pinot] walterddr commented on pull request #8078: Segment retention for table

2022-01-31 Thread GitBox
walterddr commented on pull request #8078: URL: https://github.com/apache/pinot/pull/8078#issuecomment-1026090439 > This does not work if the retention from table config is not 0. the `SegmentDeletionManager.removeAgedDeletedSegments()` does not take the config from the table config

[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8078: Segment retention for table

2022-01-31 Thread GitBox
Jackie-Jiang commented on a change in pull request #8078: URL: https://github.com/apache/pinot/pull/8078#discussion_r795947645 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/config/table/SegmentsValidationAndRetentionConfig.java ## @@ -29,6 +29,7 @@ public class S

[pinot] branch master updated (4e595a2 -> 513a7b4)

2022-01-31 Thread sajjad
This is an automated email from the ASF dual-hosted git repository. sajjad pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git. from 4e595a2 Implement Real Time Mutable FST (#8016) add 513a7b4 Add consumption rate limiter for LLConsumer (#6291)

[GitHub] [pinot] richardstartin commented on pull request #8097: fix FST Like query benchmark to remove SQL parsing from the measurement

2022-01-31 Thread GitBox
richardstartin commented on pull request #8097: URL: https://github.com/apache/pinot/pull/8097#issuecomment-1026076363 I've added a parameter to vary the number of blocks iterated over so this benchmark doesn't just measure planning. -- This is an automated message from the Apache Git Se

[GitHub] [pinot] sajjad-moradi merged pull request #6291: Add consumption rate limiter for LLConsumer

2022-01-31 Thread GitBox
sajjad-moradi merged pull request #6291: URL: https://github.com/apache/pinot/pull/6291 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-un

[GitHub] [pinot] codecov-commenter edited a comment on pull request #6291: Add consumption rate limiter for LLConsumer

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #6291: URL: https://github.com/apache/pinot/pull/6291#issuecomment-1021719883 # [Codecov](https://codecov.io/gh/apache/pinot/pull/6291?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] codecov-commenter edited a comment on pull request #6291: Add consumption rate limiter for LLConsumer

2022-01-31 Thread GitBox
codecov-commenter edited a comment on pull request #6291: URL: https://github.com/apache/pinot/pull/6291#issuecomment-1021719883 # [Codecov](https://codecov.io/gh/apache/pinot/pull/6291?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=

[GitHub] [pinot] richardstartin opened a new pull request #8097: fix FST Like query benchmark to remove SQL parsing from the measurement

2022-01-31 Thread GitBox
richardstartin opened a new pull request #8097: URL: https://github.com/apache/pinot/pull/8097 I noticed that the FST Like benchmark results vary a lot: ``` Benchmark Mode Cnt Score Error Units BenchmarkNativeAndLuceneBasedLi

[GitHub] [pinot] walterddr commented on issue #8091: table creation shouldn't be allowed if table delete is happening async in background

2022-01-31 Thread GitBox
walterddr commented on issue #8091: URL: https://github.com/apache/pinot/issues/8091#issuecomment-1026054424 Observation === The issue is whether EXTERNAL VIEW for a table goes away before or after the PropertyStore remove entries for the associated segments. Upon checking

[GitHub] [pinot] sajjad-moradi commented on a change in pull request #6291: Add consumption rate limiter for LLConsumer

2022-01-31 Thread GitBox
sajjad-moradi commented on a change in pull request #6291: URL: https://github.com/apache/pinot/pull/6291#discussion_r795905532 ## File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeConsumptionRateManager.java ## @@ -42,25 +43,35 @@ */ p

[GitHub] [pinot] suddendust commented on pull request #7959: [7867] Handle Select * with Extra Columns

2022-01-31 Thread GitBox
suddendust commented on pull request #7959: URL: https://github.com/apache/pinot/pull/7959#issuecomment-1025987716 @Jackie-Jiang Requesting review, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

  1   2   >