[GitHub] [pinot] codecov-commenter edited a comment on pull request #7225: Update license, notice for 0.8.0 release

2021-07-28 Thread GitBox
codecov-commenter edited a comment on pull request #7225: URL: https://github.com/apache/pinot/pull/7225#issuecomment-44533 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7225?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter commented on pull request #7225: Update license, notice for 0.8.0 release

2021-07-28 Thread GitBox
codecov-commenter commented on pull request #7225: URL: https://github.com/apache/pinot/pull/7225#issuecomment-44533 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7225?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] suddendust commented on issue #7187: Issue with CSV ingestion with non-default delimiter

2021-07-28 Thread GitBox
suddendust commented on issue #7187: URL: https://github.com/apache/pinot/issues/7187#issuecomment-33059 That will be a good starting step for sure. However, can we also have an optional config that the user can set explicitly if the header delimiter is different than the CSV delimiter

[GitHub] [pinot] snleee opened a new pull request #7225: Update license, notice for 0.8.0 release

2021-07-28 Thread GitBox
snleee opened a new pull request #7225: URL: https://github.com/apache/pinot/pull/7225 Updated LICENCE-binary, NOTICE-binary. Added extra license files under `/licenses-binary`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7178: Merge/Rollup task scheduler

2021-07-28 Thread GitBox
codecov-commenter edited a comment on pull request #7178: URL: https://github.com/apache/pinot/pull/7178#issuecomment-882751873 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7178?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7178: Merge/Rollup task scheduler

2021-07-28 Thread GitBox
codecov-commenter edited a comment on pull request #7178: URL: https://github.com/apache/pinot/pull/7178#issuecomment-882751873 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7178?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] kkrugler edited a comment on pull request #7222: 7090 segmentnamegenerator accept input file parameter

2021-07-28 Thread GitBox
kkrugler edited a comment on pull request #7222: URL: https://github.com/apache/pinot/pull/7222#issuecomment-888686908 Hi @Jackie-Jiang - a few things about this WIP... 1. The inputFilePath that I'm getting passed inside the `SegmentGeneratorConfig`, which is passed to `SegmentIndexC

[GitHub] [pinot] kkrugler edited a comment on pull request #7222: 7090 segmentnamegenerator accept input file parameter

2021-07-28 Thread GitBox
kkrugler edited a comment on pull request #7222: URL: https://github.com/apache/pinot/pull/7222#issuecomment-888686908 Hi @Jackie-Jiang - a few things about this WIP... 1. The inputFilePath that I'm getting passed inside the `SegmentGeneratorConfig`, which is passed to `SegmentIndexC

[pinot] branch master updated: Fix the shaded issue with Apache Pulsar (#7223)

2021-07-28 Thread snlee
This is an automated email from the ASF dual-hosted git repository. snlee pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 5af6078 Fix the shaded issue with Apache Pulsar (

[GitHub] [pinot] snleee merged pull request #7223: Fix the shaded issue with Apache Pulsar

2021-07-28 Thread GitBox
snleee merged pull request #7223: URL: https://github.com/apache/pinot/pull/7223 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr.

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7223: Fix the shaded issue with Apache Pulsar

2021-07-28 Thread GitBox
codecov-commenter edited a comment on pull request #7223: URL: https://github.com/apache/pinot/pull/7223#issuecomment-888733361 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7223?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] jtao15 commented on a change in pull request #7178: Merge/Rollup task scheduler which supports multi-level CONCAT tasks f…

2021-07-28 Thread GitBox
jtao15 commented on a change in pull request #7178: URL: https://github.com/apache/pinot/pull/7178#discussion_r678762208 ## File path: pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/merge_rollup/MergeRollupTaskGenerator.j

[GitHub] [pinot] jtao15 commented on a change in pull request #7178: Merge/Rollup task scheduler which supports multi-level CONCAT tasks f…

2021-07-28 Thread GitBox
jtao15 commented on a change in pull request #7178: URL: https://github.com/apache/pinot/pull/7178#discussion_r678761759 ## File path: pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/merge_rollup/MergeRollupTaskGenerator.j

[GitHub] [pinot] codecov-commenter commented on pull request #7223: Fix the shaded issue with Apache Pulsar

2021-07-28 Thread GitBox
codecov-commenter commented on pull request #7223: URL: https://github.com/apache/pinot/pull/7223#issuecomment-888733361 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7223?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7224: cast to Number to convert value between numeric types

2021-07-28 Thread GitBox
codecov-commenter edited a comment on pull request #7224: URL: https://github.com/apache/pinot/pull/7224#issuecomment-888711169 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7224?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter commented on pull request #7224: cast to Number to convert value between numeric types

2021-07-28 Thread GitBox
codecov-commenter commented on pull request #7224: URL: https://github.com/apache/pinot/pull/7224#issuecomment-888711169 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7224?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7222: 7090 segmentnamegenerator accept input file parameter

2021-07-28 Thread GitBox
codecov-commenter edited a comment on pull request #7222: URL: https://github.com/apache/pinot/pull/7222#issuecomment-888708262 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7222?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter commented on pull request #7222: 7090 segmentnamegenerator accept input file parameter

2021-07-28 Thread GitBox
codecov-commenter commented on pull request #7222: URL: https://github.com/apache/pinot/pull/7222#issuecomment-888708262 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7222?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7224: cast to Number to convert value between numeric types

2021-07-28 Thread GitBox
Jackie-Jiang commented on a change in pull request #7224: URL: https://github.com/apache/pinot/pull/7224#discussion_r678726827 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/PinotDataType.java ## @@ -201,27 +201,27 @@ public String toString(Object value

[GitHub] [pinot] klsince commented on pull request #7224: cast to Number to convert value between numeric types

2021-07-28 Thread GitBox
klsince commented on pull request #7224: URL: https://github.com/apache/pinot/pull/7224#issuecomment-888690584 Came into this issue while using jsonPathArray() during ingestion to extract the `id` field from `labels` JSONArray into a Pinot MV column. The output array from jsonPathArray() h

[GitHub] [pinot] klsince opened a new pull request #7224: cast to Number to convert value between numeric types

2021-07-28 Thread GitBox
klsince opened a new pull request #7224: URL: https://github.com/apache/pinot/pull/7224 During ingestion, the array to put in MV column can contain mixing types. This fix would allow mixing numeric types in an array to be put in a numeric MV column. ## Description ## Upgrade

[pinot] branch release-0.8.0-rc updated (798190a -> fa7d7f3)

2021-07-28 Thread snlee
This is an automated email from the ASF dual-hosted git repository. snlee pushed a change to branch release-0.8.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git. discard 798190a Fix the shaded issue with Apache Pulsar new fa7d7f3 Fix the shaded issue with Apache Pulsar T

[pinot] 01/01: Fix the shaded issue with Apache Pulsar

2021-07-28 Thread snlee
This is an automated email from the ASF dual-hosted git repository. snlee pushed a commit to branch release-0.8.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git commit fa7d7f3fbff73300f0f38aad37491bfefad6aacc Author: Seunghyun Lee AuthorDate: Wed Jul 28 16:18:35 2021 -0700 F

[GitHub] [pinot] kkrugler commented on pull request #7222: 7090 segmentnamegenerator accept input file parameter

2021-07-28 Thread GitBox
kkrugler commented on pull request #7222: URL: https://github.com/apache/pinot/pull/7222#issuecomment-888686908 Hi @Jackie-Jiang - a few things about this WIP... 1. The inputFilePath that I'm getting passed inside the `SegmentGeneratorConfig`, which is passed to `SegmentIndexCreation

[GitHub] [pinot] snleee opened a new pull request #7223: Fix the shaded issue with Apache Pulsar

2021-07-28 Thread GitBox
snleee opened a new pull request #7223: URL: https://github.com/apache/pinot/pull/7223 As mentioned in #7221, Apache Pulsar plugin did not create the shaded jar. This PR addresses it. Moreover, we also fix the javadoc lint issue when building the apache release. -- This is an automa

[pinot] branch master updated (c2c887a -> 069bae0)

2021-07-28 Thread snlee
This is an automated email from the ASF dual-hosted git repository. snlee pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git. from c2c887a allow to extract values from array of objects with jsonPathArray (#7208) add 069bae0 Bump up the "grpc-

[GitHub] [pinot] snleee merged pull request #7220: Bump up the "grpc-protobuf-lite" version to 1.19.0 in Apache Pulsar

2021-07-28 Thread GitBox
snleee merged pull request #7220: URL: https://github.com/apache/pinot/pull/7220 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr.

[pinot] branch release-0.8.0-rc updated: Fix the shaded issue with Apache Pulsar

2021-07-28 Thread snlee
This is an automated email from the ASF dual-hosted git repository. snlee pushed a commit to branch release-0.8.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-0.8.0-rc by this push: new 798190a Fix the shaded issue

[GitHub] [pinot] kkrugler opened a new pull request #7222: 7090 segmentnamegenerator accept input file parameter

2021-07-28 Thread GitBox
kkrugler opened a new pull request #7222: URL: https://github.com/apache/pinot/pull/7222 ## Description First cut at resolving #7090, to support generating segment names based on the input file path and a specified pattern/template. ## Upgrade Notes Does this PR prevent a zero down

[GitHub] [pinot] snleee opened a new issue #7221: Update pom files to create shaded jar for Apache Pulsar plugin

2021-07-28 Thread GitBox
snleee opened a new issue #7221: URL: https://github.com/apache/pinot/issues/7221 Currently, Apache Pulsar plugin doesn't create the shaded jar. ``` ~/workspace/pinot/pinot-plugins/pinot-stream-ingestion/pinot-pulsar/target release-0.8.0-rc* ❯ ll -rw-r--r-- 1 snlee staff

[GitHub] [pinot] mayankshriv commented on issue #7187: Issue with CSV ingestion with non-default delimiter

2021-07-28 Thread GitBox
mayankshriv commented on issue #7187: URL: https://github.com/apache/pinot/issues/7187#issuecomment-888576408 I think the code and user point-of-view are contradicting here. It is very natural for users to provide a comma separated header string. May be we should gracefully error out right

[pinot] branch master updated: allow to extract values from array of objects with jsonPathArray (#7208)

2021-07-28 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new c2c887a allow to extract values from array of ob

[GitHub] [pinot] Jackie-Jiang merged pull request #7208: allow to extract values from array of objects with jsonPathArray

2021-07-28 Thread GitBox
Jackie-Jiang merged pull request #7208: URL: https://github.com/apache/pinot/pull/7208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-uns

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7220: Bump up the "grpc-protobuf-lite" version to 1.19.0 in Apache Pulsar

2021-07-28 Thread GitBox
codecov-commenter edited a comment on pull request #7220: URL: https://github.com/apache/pinot/pull/7220#issuecomment-888549770 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7220?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter commented on pull request #7220: Bump up the "grpc-protobuf-lite" version to 1.19.0 in Apache Pulsar

2021-07-28 Thread GitBox
codecov-commenter commented on pull request #7220: URL: https://github.com/apache/pinot/pull/7220#issuecomment-888549770 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7220?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7208: allow to extract values from array of objects with jsonPathArray

2021-07-28 Thread GitBox
codecov-commenter edited a comment on pull request #7208: URL: https://github.com/apache/pinot/pull/7208#issuecomment-887128823 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7208?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7208: allow to extract values from array of objects with jsonPathArray

2021-07-28 Thread GitBox
codecov-commenter edited a comment on pull request #7208: URL: https://github.com/apache/pinot/pull/7208#issuecomment-887128823 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7208?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] suddendust commented on issue #7187: Issue with CSV ingestion with non-default delimiter

2021-07-28 Thread GitBox
suddendust commented on issue #7187: URL: https://github.com/apache/pinot/issues/7187#issuecomment-888514315 @mayankshriv let me know what you think of this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [pinot] suddendust edited a comment on issue #7212: Support raw methods for PercentileTDigest and PercentileEst

2021-07-28 Thread GitBox
suddendust edited a comment on issue #7212: URL: https://github.com/apache/pinot/issues/7212#issuecomment-888014290 @Jackie-Jiang I can pick this up if it's not urgent (I will need to read the theory behind these). -- This is an automated message from the Apache Git Service. To respond

[pinot] branch release-0.8.0-rc updated: Bump up the "grpc-protobuf-lite" version to 1.19.0 in Apache Pulsar plugin

2021-07-28 Thread snlee
This is an automated email from the ASF dual-hosted git repository. snlee pushed a commit to branch release-0.8.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-0.8.0-rc by this push: new ca99cfe Bump up the "grpc-pro

[GitHub] [pinot] snleee opened a new pull request #7220: Bump up the "grpc-protobuf-lite" version to 1.19.0 in Apache Pulsar

2021-07-28 Thread GitBox
snleee opened a new pull request #7220: URL: https://github.com/apache/pinot/pull/7220 gprc-protobuf-lite:1.18.0 pulls the "org.checkerframework:checker-compat-qual" library which is based on GPL v2. GPL v2 is one of category X licenses that are not allowed to include in Apache releas

[GitHub] [pinot] jackjlli commented on a change in pull request #7210: Support EXPLAIN PLAN

2021-07-28 Thread GitBox
jackjlli commented on a change in pull request #7210: URL: https://github.com/apache/pinot/pull/7210#discussion_r678507134 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java ## @@ -498,6 +517,36 @@ private BrokerRespon

[GitHub] [pinot] yupeng9 commented on pull request #7200: Allow any column as the upsert comparison column

2021-07-28 Thread GitBox
yupeng9 commented on pull request #7200: URL: https://github.com/apache/pinot/pull/7200#issuecomment-888469546 @Jackie-Jiang makes sense. updated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [pinot] codecov-commenter commented on pull request #7219: GroupBy Optimization: Trim on the Fly

2021-07-28 Thread GitBox
codecov-commenter commented on pull request #7219: URL: https://github.com/apache/pinot/pull/7219#issuecomment-888427732 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7219?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] wuwenw opened a new pull request #7219: GroupBy Optimization: Trim on the Fly

2021-07-28 Thread GitBox
wuwenw opened a new pull request #7219: URL: https://github.com/apache/pinot/pull/7219 ## Description Group-by issue: If there are too many groups, then it takes up too much memory and the garbage collection process becomes a major performance bottleneck. Current Approach: When t

[GitHub] [pinot] MrNeocore commented on issue #7090: SegmentNameGenerator: Extend interface to accept input file name

2021-07-28 Thread GitBox
MrNeocore commented on issue #7090: URL: https://github.com/apache/pinot/issues/7090#issuecomment-888197579 @kkrugler I see, thanks for the clarification ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [pinot] atris opened a new issue #7218: Expand Full Text Search Capabilities

2021-07-28 Thread GitBox
atris opened a new issue #7218: URL: https://github.com/apache/pinot/issues/7218 Our FTS capabilities are pretty limited right now -- REGEXP_MATCH does a primitive regex match and TEXT_MATCHES is simply a wrapper around basic Lucene level searching (does not expose fuzzy queries or the opt

[GitHub] [pinot] atris opened a new issue #7217: Using DocValues Instead of Stored Fields

2021-07-28 Thread GitBox
atris opened a new issue #7217: URL: https://github.com/apache/pinot/issues/7217 When creating text indices, we use Lucene's stored fields to store the DocID field inside the index. This is inefficient due stored fields' representation within Lucene. A more efficient way would be us

[GitHub] [pinot] atris opened a new issue #7216: Stop Words Should Not Be Hard Coded

2021-07-28 Thread GitBox
atris opened a new issue #7216: URL: https://github.com/apache/pinot/issues/7216 During text index creation, we pass in a hard coded list of stop words which limits the vocabulary that we can use to filter words before indexing. We should use Lucene's StopFilter and read the list of

[GitHub] [pinot] atris opened a new issue #7215: Text Indices Are Force Merged to Single Lucene Segment

2021-07-28 Thread GitBox
atris opened a new issue #7215: URL: https://github.com/apache/pinot/issues/7215 We force merge all segments of a single Lucene index to a single segment for text indices today, thus potentially creating a single large segment and restricting the possibility of searching multiple segments

[GitHub] [pinot] atris closed issue #7194: StarTreeFilterOperator Does Not Maintain Predicate Hiearchy

2021-07-28 Thread GitBox
atris closed issue #7194: URL: https://github.com/apache/pinot/issues/7194 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pin

[GitHub] [pinot] atris commented on issue #7190: Complex Geospatial Query Excessive Time

2021-07-28 Thread GitBox
atris commented on issue #7190: URL: https://github.com/apache/pinot/issues/7190#issuecomment-888119012 Why do we not use Lucene's geo functions to achieve some of these (such as contains)? -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [pinot] atris commented on issue #7194: StarTreeFilterOperator Does Not Maintain Predicate Hiearchy

2021-07-28 Thread GitBox
atris commented on issue #7194: URL: https://github.com/apache/pinot/issues/7194#issuecomment-888118482 This has been resolved with the new CompositePredicateEvaluator -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t