[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7145: Fixing pinot-admin script to parse env var PLUGINS_INCLUDE

2021-07-08 Thread GitBox
codecov-commenter edited a comment on pull request #7145: URL: https://github.com/apache/incubator-pinot/pull/7145#issuecomment-876808361 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7145?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] atris commented on a change in pull request #7141: Support Dictionary Based Plan For DISTINCT

2021-07-08 Thread GitBox
atris commented on a change in pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141#discussion_r85741 ## File path: pinot-core/src/test/java/org/apache/pinot/queries/DistinctQueriesTest.java ## @@ -245,12 +245,14 @@ public void testSingleColumnDisti

[GitHub] [incubator-pinot] atris commented on a change in pull request #7141: Support Dictionary Based Plan For DISTINCT

2021-07-08 Thread GitBox
atris commented on a change in pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141#discussion_r85531 ## File path: pinot-core/src/main/java/org/apache/pinot/core/plan/DictionaryBasedDistinctPlanNode.java ## @@ -0,0 +1,64 @@ +/** + * Licensed to the

[GitHub] [incubator-pinot] atris commented on a change in pull request #7141: Support Dictionary Based Plan For DISTINCT

2021-07-08 Thread GitBox
atris commented on a change in pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141#discussion_r84255 ## File path: pinot-core/src/main/java/org/apache/pinot/core/operator/query/DictionaryBasedDistinctOperator.java ## @@ -0,0 +1,176 @@ +/** + * Lice

[GitHub] [incubator-pinot] yashrsharma44 commented on a change in pull request #7142: Added a query param to /tables for getting sorted table names based on time metadata

2021-07-08 Thread GitBox
yashrsharma44 commented on a change in pull request #7142: URL: https://github.com/apache/incubator-pinot/pull/7142#discussion_r78584 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java ## @@ -210,15 +213,31

[GitHub] [incubator-pinot] yashrsharma44 commented on a change in pull request #7142: Added a query param to /tables for getting sorted table names based on time metadata

2021-07-08 Thread GitBox
yashrsharma44 commented on a change in pull request #7142: URL: https://github.com/apache/incubator-pinot/pull/7142#discussion_r72860 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java ## @@ -210,15 +213,31

[GitHub] [incubator-pinot] bilalmunir commented on pull request #7146: Update ExecutionStats.java

2021-07-08 Thread GitBox
bilalmunir commented on pull request #7146: URL: https://github.com/apache/incubator-pinot/pull/7146#issuecomment-876886081 Thank you @Jackie-Jiang for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7142: Added a query param to /tables for getting sorted table names based on time metadata

2021-07-08 Thread GitBox
codecov-commenter commented on pull request #7142: URL: https://github.com/apache/incubator-pinot/pull/7142#issuecomment-876860754 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7142?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[incubator-pinot] branch master updated: Switch to using 'authority' instead of 'host' when building parent URI in PinotFS::move (#7037)

2021-07-08 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new f0675af Switch to using 'authority' in

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #7037: Switch to using 'authority' instead of 'host' when building parent URI in PinotFS::move

2021-07-08 Thread GitBox
Jackie-Jiang merged pull request #7037: URL: https://github.com/apache/incubator-pinot/pull/7037 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: c

[incubator-pinot] branch master updated: Update ExecutionStats.java (#7146)

2021-07-08 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new 7fb65f0 Update ExecutionStats.java (#7

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #7146: Update ExecutionStats.java

2021-07-08 Thread GitBox
Jackie-Jiang merged pull request #7146: URL: https://github.com/apache/incubator-pinot/pull/7146 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: c

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7142: Added a query param to /tables for getting sorted table names based on time metadata

2021-07-08 Thread GitBox
Jackie-Jiang commented on a change in pull request #7142: URL: https://github.com/apache/incubator-pinot/pull/7142#discussion_r04034 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java ## @@ -210,15 +213,31

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7141: Support Dictionary Based Plan For DISTINCT

2021-07-08 Thread GitBox
Jackie-Jiang commented on a change in pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141#discussion_r01239 ## File path: pinot-core/src/main/java/org/apache/pinot/core/operator/query/DictionaryBasedDistinctOperator.java ## @@ -0,0 +1,163 @@ +/** +

[GitHub] [incubator-pinot] cbalci commented on pull request #7037: Switch to using 'authority' instead of 'host' when building parent URI in PinotFS::move

2021-07-08 Thread GitBox
cbalci commented on pull request #7037: URL: https://github.com/apache/incubator-pinot/pull/7037#issuecomment-876825062 @Jackie-Jiang @chenboat thanks for your reviews. Haven't been able to reach @snleee and @elonazoulay. Can we move forward merging this? -- This is an automated message

[GitHub] [incubator-pinot] mcvsubbu commented on issue #6679: Admin command to recover from deleted CONSUMING segment case

2021-07-08 Thread GitBox
mcvsubbu commented on issue #6679: URL: https://github.com/apache/incubator-pinot/issues/6679#issuecomment-876815501 @npawar , @Jackie-Jiang so far what I have heard is that people do things to Kafka, and then want to recover on the pinot side -- they don't care about missed or duplicate

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7145: Fixing pinot-admin script to parse env var PLUGINS_INCLUDE

2021-07-08 Thread GitBox
codecov-commenter edited a comment on pull request #7145: URL: https://github.com/apache/incubator-pinot/pull/7145#issuecomment-876808361 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7145?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7145: Fixing pinot-admin script to parse env var PLUGINS_INCLUDE

2021-07-08 Thread GitBox
codecov-commenter commented on pull request #7145: URL: https://github.com/apache/incubator-pinot/pull/7145#issuecomment-876808361 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7145?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[GitHub] [incubator-pinot] bilalmunir opened a new pull request #7146: Update ExecutionStats.java

2021-07-08 Thread GitBox
bilalmunir opened a new pull request #7146: URL: https://github.com/apache/incubator-pinot/pull/7146 Make ExecutionStats public to that stats can be used for post-processing. ## Description Make class `ExecutionStats` class in java client public, so that it can used by clients

[GitHub] [incubator-pinot] xiangfu0 removed a comment on issue #7124: pinot-admin.sh has bash error processing IFS and PLUGINS_INCLUDE env

2021-07-08 Thread GitBox
xiangfu0 removed a comment on issue #7124: URL: https://github.com/apache/incubator-pinot/issues/7124#issuecomment-876776106 Can you paste the errors? I feel it could be the issue of sh vs bash. the first line of pinot-admin.sh mentioned sh. ``` ➜ cat bin/pinot-admin.sh #!/bi

[GitHub] [incubator-pinot] xiangfu0 commented on pull request #7145: Fixing pinot-admin script to parse env var PLUGINS_INCLUDE

2021-07-08 Thread GitBox
xiangfu0 commented on pull request #7145: URL: https://github.com/apache/incubator-pinot/pull/7145#issuecomment-876783825 @dongxiaoman Can you please try and confirm this one-line fix on your side? -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [incubator-pinot] xiangfu0 opened a new pull request #7145: Fixing pinot-admin script to parse env var PLUGINS_INCLUDE

2021-07-08 Thread GitBox
xiangfu0 opened a new pull request #7145: URL: https://github.com/apache/incubator-pinot/pull/7145 ## Description https://github.com/apache/incubator-pinot/issues/7124 After the fix, it shows the correct parsing: ``` ➜ JAVA_OPTS="-Xms1G -Xmx4G -Dplugins.dir=/Users/xiangfu/wo

[GitHub] [incubator-pinot] Jackie-Jiang commented on pull request #7142: Added a query param to /tables for getting sorted table names based on time metadata

2021-07-08 Thread GitBox
Jackie-Jiang commented on pull request #7142: URL: https://github.com/apache/incubator-pinot/pull/7142#issuecomment-876781392 > I have a java specific question 😅 > > * I am trying to import `import org.apache.pinot.common.utils.config.TableConfigUtils;`, and this package with the sa

[GitHub] [incubator-pinot] xiangfu0 commented on issue #7124: pinot-admin.sh has bash error processing IFS and PLUGINS_INCLUDE env

2021-07-08 Thread GitBox
xiangfu0 commented on issue #7124: URL: https://github.com/apache/incubator-pinot/issues/7124#issuecomment-876776106 Can you paste the errors? I feel it could be the issue of sh vs bash. the first line of pinot-admin.sh mentioned sh. ``` ➜ cat bin/pinot-admin.sh #!/bin/sh

[incubator-pinot] branch master updated: Move minion metrics definition to pinot-common package (#7107)

2021-07-08 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new 1a12510 Move minion metrics definitio

[GitHub] [incubator-pinot] xiangfu0 closed issue #7106: Move minion metrics into pinot-common

2021-07-08 Thread GitBox
xiangfu0 closed issue #7106: URL: https://github.com/apache/incubator-pinot/issues/7106 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-un

[GitHub] [incubator-pinot] xiangfu0 merged pull request #7107: Move minion metrics definition to pinot-common package

2021-07-08 Thread GitBox
xiangfu0 merged pull request #7107: URL: https://github.com/apache/incubator-pinot/pull/7107 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commi

[GitHub] [incubator-pinot] xiangfu0 closed issue #7139: kinesis test should handle missing docker gracefully

2021-07-08 Thread GitBox
xiangfu0 closed issue #7139: URL: https://github.com/apache/incubator-pinot/issues/7139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-un

[GitHub] [incubator-pinot] xiangfu0 edited a comment on issue #7134: Pinot Is Not Able to Read JAAS File Property (java.security.auth.login.config) While Doing Integration With Kerberos Kafka.

2021-07-08 Thread GitBox
xiangfu0 edited a comment on issue #7134: URL: https://github.com/apache/incubator-pinot/issues/7134#issuecomment-876772227 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [incubator-pinot] xiangfu0 commented on issue #7134: Pinot Is Not Able to Read JAAS File Property (java.security.auth.login.config) While Doing Integration With Kerberos Kafka.

2021-07-08 Thread GitBox
xiangfu0 commented on issue #7134: URL: https://github.com/apache/incubator-pinot/issues/7134#issuecomment-876772227 In your case, please make sure all controllers and servers have this -Djava.security.auth.login.config=/home/dev/client_jaas.conf` -- This is an automated message from th

[GitHub] [incubator-pinot] xiangfu0 edited a comment on issue #7134: Pinot Is Not Able to Read JAAS File Property (java.security.auth.login.config) While Doing Integration With Kerberos Kafka.

2021-07-08 Thread GitBox
xiangfu0 edited a comment on issue #7134: URL: https://github.com/apache/incubator-pinot/issues/7134#issuecomment-876771684 Have you tried to add `-Djava.security.auth.login.config="/home/dev/client_jaas.conf"` into `JAVA_OPTS`? Something like: ``` JAVA_OPTS="-Xms1G -Xmx4G -Dplug

[GitHub] [incubator-pinot] xiangfu0 commented on issue #7134: Pinot Is Not Able to Read JAAS File Property (java.security.auth.login.config) While Doing Integration With Kerberos Kafka.

2021-07-08 Thread GitBox
xiangfu0 commented on issue #7134: URL: https://github.com/apache/incubator-pinot/issues/7134#issuecomment-876771684 Have you tried to add `-Djava.security.auth.login.config="/home/dev/client_jaas.conf"` into `JAVA_OPTS`? Something like: ``` JAVA_OPTS="-Xms1G -Xmx4G -Dplugins.dir

[GitHub] [incubator-pinot] xiangfu0 commented on issue #7124: pinot-admin.sh has bash error processing IFS and PLUGINS_INCLUDE env

2021-07-08 Thread GitBox
xiangfu0 commented on issue #7124: URL: https://github.com/apache/incubator-pinot/issues/7124#issuecomment-876769016 ic, can you make the workaround with java option`-Dplugins.include`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [incubator-pinot] xiangfu0 closed issue #7110: Pinot broker query limit throws 500 error

2021-07-08 Thread GitBox
xiangfu0 closed issue #7110: URL: https://github.com/apache/incubator-pinot/issues/7110 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-un

[GitHub] [incubator-pinot] yashrsharma44 edited a comment on pull request #7142: Added a query param to /tables for getting sorted table names based on time metadata

2021-07-08 Thread GitBox
yashrsharma44 edited a comment on pull request #7142: URL: https://github.com/apache/incubator-pinot/pull/7142#issuecomment-876709565 I have added some boilerplate code for the sorting feature required, wanted to make sure if the tests are not failing. Will add the sorting param after t

[GitHub] [incubator-pinot] Jackie-Jiang opened a new issue #7144: Add `numConsumingSegmentsProcessed` and `numConsumingSegmentsMatched` into the query response metadata

2021-07-08 Thread GitBox
Jackie-Jiang opened a new issue #7144: URL: https://github.com/apache/incubator-pinot/issues/7144 Currently we have the following metadata in the query response: - numSegmentsQueried: number of segments queried by the broker - numSegmentsProcessed: number of segments processed after th

[GitHub] [incubator-pinot] yashrsharma44 commented on a change in pull request #7142: Added a query param to /tables for getting sorted table names based on time metadata

2021-07-08 Thread GitBox
yashrsharma44 commented on a change in pull request #7142: URL: https://github.com/apache/incubator-pinot/pull/7142#discussion_r666503887 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java ## @@ -200,7 +202,8 @

[GitHub] [incubator-pinot] xiangfu0 commented on issue #7143: Remove unapproved company Icon from apache pinot website

2021-07-08 Thread GitBox
xiangfu0 commented on issue #7143: URL: https://github.com/apache/incubator-pinot/issues/7143#issuecomment-876729516 Thanks for bringing this up. Removed the logo! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [incubator-pinot] xiangfu0 closed issue #7143: Remove unapproved company Icon from apache pinot website

2021-07-08 Thread GitBox
xiangfu0 closed issue #7143: URL: https://github.com/apache/incubator-pinot/issues/7143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-un

[GitHub] [incubator-pinot] atris commented on pull request #7141: Support Dictionary Based Plan For DISTINCT

2021-07-08 Thread GitBox
atris commented on pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141#issuecomment-876728697 @Jackie-Jiang Updated, please see and let me know your thoughts -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7141: Support Dictionary Based Plan For DISTINCT

2021-07-08 Thread GitBox
Jackie-Jiang commented on a change in pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141#discussion_r666499698 ## File path: pinot-core/src/main/java/org/apache/pinot/core/operator/query/DictionaryBasedDistinctOperator.java ## @@ -0,0 +1,163 @@ +/** +

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7141: Support Dictionary Based Plan For DISTINCT

2021-07-08 Thread GitBox
Jackie-Jiang commented on a change in pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141#discussion_r666499698 ## File path: pinot-core/src/main/java/org/apache/pinot/core/operator/query/DictionaryBasedDistinctOperator.java ## @@ -0,0 +1,163 @@ +/** +

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7141: Support Dictionary Based Plan For DISTINCT

2021-07-08 Thread GitBox
Jackie-Jiang commented on a change in pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141#discussion_r666498797 ## File path: pinot-core/src/main/java/org/apache/pinot/core/operator/query/DictionaryBasedDistinctOperator.java ## @@ -0,0 +1,163 @@ +/** +

[GitHub] [incubator-pinot] yashrsharma44 commented on a change in pull request #7142: Added a query param to /tables for getting sorted table names based on time metadata

2021-07-08 Thread GitBox
yashrsharma44 commented on a change in pull request #7142: URL: https://github.com/apache/incubator-pinot/pull/7142#discussion_r666497311 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java ## @@ -210,15 +213,36

[incubator-pinot-site] branch dev updated: Adding StarTree logo

2021-07-08 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/incubator-pinot-site.git The following commit(s) were added to refs/heads/dev by this push: new d51b709 Adding StarTree logo d51b709 i

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7142: Added a query param to /tables for getting sorted table names based on time metadata

2021-07-08 Thread GitBox
Jackie-Jiang commented on a change in pull request #7142: URL: https://github.com/apache/incubator-pinot/pull/7142#discussion_r666495789 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java ## @@ -200,7 +202,8 @@

[GitHub] [incubator-pinot] pabrahamasurion opened a new issue #7143: Remove unapproved company Icon from apache pinot website

2021-07-08 Thread GitBox
pabrahamasurion opened a new issue #7143: URL: https://github.com/apache/incubator-pinot/issues/7143 Hello, I can see Asurion logo listed in Apache Pinot Website. As per corporate policy this is not allowed until there is an approval. I did not provided any approval to anyone as we

[GitHub] [incubator-pinot] yashrsharma44 commented on pull request #7142: Added a query param to /tables for getting sorted table names based on time metadata

2021-07-08 Thread GitBox
yashrsharma44 commented on pull request #7142: URL: https://github.com/apache/incubator-pinot/pull/7142#issuecomment-876723057 I have a java specific question 😅 - I am trying to import `import org.apache.pinot.common.utils.config.TableConfigUtils;`, and this package with the same

[GitHub] [incubator-pinot] atris commented on a change in pull request #7141: Support Dictionary Based Plan For DISTINCT

2021-07-08 Thread GitBox
atris commented on a change in pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141#discussion_r666492908 ## File path: pinot-core/src/main/java/org/apache/pinot/core/operator/query/DictionaryBasedDistinctOperator.java ## @@ -0,0 +1,163 @@ +/** + * Lice

[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6679: Admin command to recover from deleted CONSUMING segment case

2021-07-08 Thread GitBox
Jackie-Jiang commented on issue #6679: URL: https://github.com/apache/incubator-pinot/issues/6679#issuecomment-876720534 @npawar Skipping offset can be useful when certain bad records can hang the consumption. Ideally we should automatically skip such records, but this is not always handl

[GitHub] [incubator-pinot-site] xiangfu0 edited a comment on pull request #48: Add disclaimer that Presto and TE are not part of ASF.

2021-07-08 Thread GitBox
xiangfu0 edited a comment on pull request #48: URL: https://github.com/apache/incubator-pinot-site/pull/48#issuecomment-876715042 Can you make this PR against dev branch. This PR is not reviewable :P -- This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [incubator-pinot-site] xiangfu0 commented on pull request #48: Add disclaimer that Presto and TE are not part of ASF.

2021-07-08 Thread GitBox
xiangfu0 commented on pull request #48: URL: https://github.com/apache/incubator-pinot-site/pull/48#issuecomment-876715042 Can you make this PR against dev branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [incubator-pinot-site] xiangfu0 closed pull request #48: Add disclaimer that Presto and TE are not part of ASF.

2021-07-08 Thread GitBox
xiangfu0 closed pull request #48: URL: https://github.com/apache/incubator-pinot-site/pull/48 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: comm

[incubator-pinot-site] branch dev updated: Remove Asurion logo per asks

2021-07-08 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/incubator-pinot-site.git The following commit(s) were added to refs/heads/dev by this push: new d75b6fa Remove Asurion logo per asks d

[GitHub] [incubator-pinot] yashrsharma44 commented on pull request #7142: Added a query param to /tables for getting sorted table names based on time metadata

2021-07-08 Thread GitBox
yashrsharma44 commented on pull request #7142: URL: https://github.com/apache/incubator-pinot/pull/7142#issuecomment-876709565 I have added some boilerplate code for the sorting feature required, wanted to make sure if the tests are not failing. Will add the sorting param after tests pa

[GitHub] [incubator-pinot] yashrsharma44 opened a new pull request #7142: Added a query param to /tables for getting sorted table names based on time metadata

2021-07-08 Thread GitBox
yashrsharma44 opened a new pull request #7142: URL: https://github.com/apache/incubator-pinot/pull/7142 Signed-off-by: Yash Sharma ## Description Adds a query parameter - `sortByTime` - for sorting based on creation/modification time of table. ## Upgrade Notes Does this

[GitHub] [incubator-pinot] atris commented on a change in pull request #7141: Support Dictionary Based Plan For DISTINCT

2021-07-08 Thread GitBox
atris commented on a change in pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141#discussion_r666472342 ## File path: pinot-core/src/main/java/org/apache/pinot/core/operator/query/DictionaryBasedDistinctOperator.java ## @@ -0,0 +1,163 @@ +/** + * Lice

[incubator-pinot] branch master updated: Add support to skip kinesis test if docker is not available (#7140)

2021-07-08 Thread apucher
This is an automated email from the ASF dual-hosted git repository. apucher pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new 88bff60 Add support to skip kinesis t

[GitHub] [incubator-pinot] npawar commented on issue #6679: Admin command to recover from deleted CONSUMING segment case

2021-07-08 Thread GitBox
npawar commented on issue #6679: URL: https://github.com/apache/incubator-pinot/issues/6679#issuecomment-876693015 +1 to this `automatic fix always pick the latest offset if there are completed segments, or earliest available offset if not`. One thing to watch out for here is in Kinesis,

[GitHub] [incubator-pinot] apucher merged pull request #7140: Add support to skip tests if docker is not available

2021-07-08 Thread GitBox
apucher merged pull request #7140: URL: https://github.com/apache/incubator-pinot/pull/7140 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commit

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7140: Add support to skip tests if docker is not available

2021-07-08 Thread GitBox
codecov-commenter edited a comment on pull request #7140: URL: https://github.com/apache/incubator-pinot/pull/7140#issuecomment-876253770 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7140?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] atris commented on a change in pull request #7141: Support Dictionary Based Plan For DISTINCT

2021-07-08 Thread GitBox
atris commented on a change in pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141#discussion_r666455606 ## File path: pinot-core/src/main/java/org/apache/pinot/core/plan/maker/InstancePlanMakerImplV2.java ## @@ -264,4 +267,31 @@ static boolean isFitFo

[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6865: Add controller rest API to list recently created/modified tables

2021-07-08 Thread GitBox
Jackie-Jiang commented on issue #6865: URL: https://github.com/apache/incubator-pinot/issues/6865#issuecomment-876664247 @yashrsharma44 `TableConfig` is stored as a `ZNRecord` in the Zookeeper. We can use the `ZNRecord` metadata for the create/last modify time (`ZNRecord.getCreationTime()

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7141: Support Dictionary Based Plan For DISTINCT

2021-07-08 Thread GitBox
Jackie-Jiang commented on a change in pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141#discussion_r666387959 ## File path: pinot-core/src/main/java/org/apache/pinot/core/operator/query/DictionaryBasedDistinctOperator.java ## @@ -0,0 +1,163 @@ +/** +

[GitHub] [incubator-pinot] yashrsharma44 commented on issue #6865: Add controller rest API to list recently created/modified tables

2021-07-08 Thread GitBox
yashrsharma44 commented on issue #6865: URL: https://github.com/apache/incubator-pinot/issues/6865#issuecomment-876656732 I checked out the `TableConfig` - https://github.com/apache/incubator-pinot/blob/master/pinot-common/src/main/java/org/apache/pinot/common/utils/config/TableConfigUtils

[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6679: Admin command to recover from deleted CONSUMING segment case

2021-07-08 Thread GitBox
Jackie-Jiang commented on issue #6679: URL: https://github.com/apache/incubator-pinot/issues/6679#issuecomment-876650388 I feel this is way too complicated, and we should break it down to several primitive operations. Since the consuming segment can only be removed manually, and the

[GitHub] [incubator-pinot] yupeng9 commented on issue #7138: Support for Tenant data isolation

2021-07-08 Thread GitBox
yupeng9 commented on issue #7138: URL: https://github.com/apache/incubator-pinot/issues/7138#issuecomment-876614275 +1 to this, and we see similar asks at Uber. Also, we could start with removing the constraint that a table name is unique in a cluster, but allow the tenant name as the nam

[GitHub] [incubator-pinot] johighley commented on issue #7138: Support for Tenant data isolation

2021-07-08 Thread GitBox
johighley commented on issue #7138: URL: https://github.com/apache/incubator-pinot/issues/7138#issuecomment-876601274 I _think_ it would be a separate physical table. Table configuration could be the same; basically, the table's broker and server tenant tags would allow a list of tenant

[GitHub] [incubator-pinot] kishoreg commented on issue #7138: Support for Tenant data isolation

2021-07-08 Thread GitBox
kishoreg commented on issue #7138: URL: https://github.com/apache/incubator-pinot/issues/7138#issuecomment-876597332 > We need the ability for each tenant to have the same set of tables without requiring unique table names. what about the data across tenants? will there be a sep

[GitHub] [incubator-pinot] johighley edited a comment on issue #7138: Support for Tenant data isolation

2021-07-08 Thread GitBox
johighley edited a comment on issue #7138: URL: https://github.com/apache/incubator-pinot/issues/7138#issuecomment-876556994 Mayank created this request after our Slack conversation, so to expand the use case: We have a lot of clients (tenants) and require data separation. Each client wi

[GitHub] [incubator-pinot] mcvsubbu commented on issue #6679: Admin command to recover from deleted CONSUMING segment case

2021-07-08 Thread GitBox
mcvsubbu commented on issue #6679: URL: https://github.com/apache/incubator-pinot/issues/6679#issuecomment-876584254 One of the problems faced is that kafka topic (whether new topic on the same table, or some unerlying kafka change) causes offsets to go lower. So, the option to re-

[GitHub] [incubator-pinot] johighley commented on issue #7138: Support for Tenant data isolation

2021-07-08 Thread GitBox
johighley commented on issue #7138: URL: https://github.com/apache/incubator-pinot/issues/7138#issuecomment-876556994 Mayank created this request after our Slack conversation, so to expand the use case: We have a lot of clients (tenants) and require data separation. Each client will have

[GitHub] [incubator-pinot] LiranBri commented on issue #7138: Support for Tenant data isolation

2021-07-08 Thread GitBox
LiranBri commented on issue #7138: URL: https://github.com/apache/incubator-pinot/issues/7138#issuecomment-876457359 We would highly appreciate such an ability! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [incubator-pinot] LiranBri commented on issue #5762: Data Encryption at rest

2021-07-08 Thread GitBox
LiranBri commented on issue #5762: URL: https://github.com/apache/incubator-pinot/issues/5762#issuecomment-876349399 +1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [incubator-pinot] codecov-commenter edited a comment on pull request #7141: Support Dictionary Based Plan For DISTINCT

2021-07-08 Thread GitBox
codecov-commenter edited a comment on pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141#issuecomment-876284165 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7141?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7141: Support Dictionary Based Plan For DISTINCT

2021-07-08 Thread GitBox
codecov-commenter commented on pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141#issuecomment-876284165 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7141?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #7140: Add support to skip tests if docker is not available

2021-07-08 Thread GitBox
codecov-commenter commented on pull request #7140: URL: https://github.com/apache/incubator-pinot/pull/7140#issuecomment-876253770 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/7140?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+commen

[GitHub] [incubator-pinot] atris commented on pull request #7141: Support Dictionary Based Plans For DISTINCT

2021-07-08 Thread GitBox
atris commented on pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141#issuecomment-876233535 @mayankshriv @Jackie-Jiang Please review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [incubator-pinot] atris closed pull request #7129: Support Dictionary Based Plan for DISTINCT Aggregates

2021-07-08 Thread GitBox
atris closed pull request #7129: URL: https://github.com/apache/incubator-pinot/pull/7129 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-

[GitHub] [incubator-pinot] atris commented on pull request #7129: Support Dictionary Based Plan for DISTINCT Aggregates

2021-07-08 Thread GitBox
atris commented on pull request #7129: URL: https://github.com/apache/incubator-pinot/pull/7129#issuecomment-876233250 Superseded by https://github.com/apache/incubator-pinot/pull/7141 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-pinot] atris opened a new pull request #7141: Support Dictionary Based Plans For DISTINCT

2021-07-08 Thread GitBox
atris opened a new pull request #7141: URL: https://github.com/apache/incubator-pinot/pull/7141 ## Description This PR introduces the ability to execute DISTINCT using a dictionary based plan when the DISTINCT query has a single column, no filters and the column is dictionary encoded.

[GitHub] [incubator-pinot] KKcorps commented on a change in pull request #7140: Add support to skip tests if docker is not available

2021-07-08 Thread GitBox
KKcorps commented on a change in pull request #7140: URL: https://github.com/apache/incubator-pinot/pull/7140#discussion_r665948443 ## File path: pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/RealtimeKinesisIntegrationTest.java ## @@ -208,15 +221,11

[GitHub] [incubator-pinot] apucher commented on a change in pull request #7140: Add support to skip tests if docker is not available

2021-07-08 Thread GitBox
apucher commented on a change in pull request #7140: URL: https://github.com/apache/incubator-pinot/pull/7140#discussion_r665945189 ## File path: pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/RealtimeKinesisIntegrationTest.java ## @@ -208,15 +221,11

[GitHub] [incubator-pinot] apucher commented on pull request #7140: Add support to skip tests if docker is not available

2021-07-08 Thread GitBox
apucher commented on pull request #7140: URL: https://github.com/apache/incubator-pinot/pull/7140#issuecomment-876204233 #7139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [incubator-pinot] KKcorps opened a new pull request #7140: Add support to skip tests if docker is not available

2021-07-08 Thread GitBox
KKcorps opened a new pull request #7140: URL: https://github.com/apache/incubator-pinot/pull/7140 ## Description Currently kinesis integration test fails if docker is not installed or is not running in the environment. This creates issues for users who want to build pinot locally or use

[GitHub] [incubator-pinot] apucher opened a new issue #7139: kinesis test should handle missing docker gracefully

2021-07-08 Thread GitBox
apucher opened a new issue #7139: URL: https://github.com/apache/incubator-pinot/issues/7139 just for tracking for now. best case, skip test with notification and document docker dependency -- This is an automated message from the Apache Git Service. To respond to the message, plea