[pinot] branch master updated: convert types per value for array with mixing types (#7234)

2021-07-30 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 78b08e4 convert types per value for array with m

[GitHub] [pinot] Jackie-Jiang merged pull request #7234: convert types per value for array with mixing types

2021-07-30 Thread GitBox
Jackie-Jiang merged pull request #7234: URL: https://github.com/apache/pinot/pull/7234 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-uns

[GitHub] [pinot] xiangfu0 commented on issue #7191: Cross-account AWS Kinesis ingestion not supported

2021-07-30 Thread GitBox
xiangfu0 commented on issue #7191: URL: https://github.com/apache/pinot/issues/7191#issuecomment-890253969 Copying over the slack discussion here: We may need to add a few more params(credentialProvider, roleArn and roleSessionName) to make it happen, according to the example code:

[GitHub] [pinot] xiangfu0 closed issue #7055: ThirdEye not supported anymore?

2021-07-30 Thread GitBox
xiangfu0 closed issue #7055: URL: https://github.com/apache/pinot/issues/7055 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@

[GitHub] [pinot] codecov-commenter commented on pull request #7236: Enable hit counter and max hit rate tracker in broker

2021-07-30 Thread GitBox
codecov-commenter commented on pull request #7236: URL: https://github.com/apache/pinot/pull/7236#issuecomment-890183832 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7236?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] jackjlli opened a new pull request #7236: Enable hit counter and max hit rate tracker in broker

2021-07-30 Thread GitBox
jackjlli opened a new pull request #7236: URL: https://github.com/apache/pinot/pull/7236 ## Description This PR enables hit counter and max hit rate tracker. These two hit counters are useful to track the actual qps in the system, regardless of whether the query quota is set or not.

[pinot] 01/01: Enable hit counter and max hit rate tracker

2021-07-30 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch enable-hit-counter-and-max-hit-rate-tracker in repository https://gitbox.apache.org/repos/asf/pinot.git commit 5d3c2950fe172b0ddee9f64ed44262ab004ecce0 Author: Jack Li(Analytics Engineering) Author

[pinot] branch enable-hit-counter-and-max-hit-rate-tracker created (now 5d3c295)

2021-07-30 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch enable-hit-counter-and-max-hit-rate-tracker in repository https://gitbox.apache.org/repos/asf/pinot.git. at 5d3c295 Enable hit counter and max hit rate tracker This branch includes the foll

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7234: convert types per value for array with mixing types

2021-07-30 Thread GitBox
codecov-commenter edited a comment on pull request #7234: URL: https://github.com/apache/pinot/pull/7234#issuecomment-890137990 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7234?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter commented on pull request #7234: convert types per value for array with mixing types

2021-07-30 Thread GitBox
codecov-commenter commented on pull request #7234: URL: https://github.com/apache/pinot/pull/7234#issuecomment-890137990 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7234?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] jasperjiaguo opened a new issue #7235: Adding testing data for better coverage in compatibility-verifier

2021-07-30 Thread GitBox
jasperjiaguo opened a new issue #7235: URL: https://github.com/apache/pinot/issues/7235 As suggested by @mcvsubbu we would like to test these features in compatibility-verifier - No dictionary column - Offheap memory for consuming segments in server - Bloom filter (not sure if low

[GitHub] [pinot] klsince opened a new pull request #7234: convert types per value for array with mixing types

2021-07-30 Thread GitBox
klsince opened a new pull request #7234: URL: https://github.com/apache/pinot/pull/7234 1. convert types for individual value in array, handling cases where array contains mixing types 2. for safety, stop skipping type conversion based on the speculated type for values in array 3. ca

[pinot] branch master updated (df9f5ef -> 95a6ae8)

2021-07-30 Thread apucher
This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git. from df9f5ef moving the map of pinot data type and java type to common util (#7231) add 95a6ae8 add cli args for a

[GitHub] [pinot] apucher merged pull request #7233: add cli args for auth tokens and user/password to ingestion job command

2021-07-30 Thread GitBox
apucher merged pull request #7233: URL: https://github.com/apache/pinot/pull/7233 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[GitHub] [pinot] kkrugler commented on a change in pull request #7222: 7090 segmentnamegenerator accept input file parameter

2021-07-30 Thread GitBox
kkrugler commented on a change in pull request #7222: URL: https://github.com/apache/pinot/pull/7222#discussion_r680164442 ## File path: pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-common/src/main/java/org/apache/pinot/plugin/ingestion/batch/common/SegmentGenerationTaskR

[pinot] branch master updated (5e0dcff -> df9f5ef)

2021-07-30 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git. from 5e0dcff 7228 /sql REST call now compatible with auth (#7230) add df9f5ef moving the map of pinot data type and

[GitHub] [pinot] Jackie-Jiang merged pull request #7231: moving the map of pinot data type and java type to common util

2021-07-30 Thread GitBox
Jackie-Jiang merged pull request #7231: URL: https://github.com/apache/pinot/pull/7231 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-uns

[GitHub] [pinot] mcvsubbu commented on issue #7229: Pinot Long Term Data Store

2021-07-30 Thread GitBox
mcvsubbu commented on issue #7229: URL: https://github.com/apache/pinot/issues/7229#issuecomment-890055483 I would vote for lazy load in pinot as a first step. I will be glad to review the design and PRs if you can contribute. -- This is an automated message from the Apache Git Service

[GitHub] [pinot] kishoreg commented on issue #7229: Pinot Long Term Data Store

2021-07-30 Thread GitBox
kishoreg commented on issue #7229: URL: https://github.com/apache/pinot/issues/7229#issuecomment-889963936 possible but it will be a huge undertaking. we can probably get there in small steps. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [pinot] suddendust commented on issue #7187: Issue with CSV ingestion with non-default delimiter

2021-07-30 Thread GitBox
suddendust commented on issue #7187: URL: https://github.com/apache/pinot/issues/7187#issuecomment-889960588 @mayankshriv I think gracefully erroring out if they don't match is the least we can do here. Shall we start with that? -- This is an automated message from the Apache Git Service

[GitHub] [pinot] suddendust edited a comment on issue #7229: Pinot Long Term Data Store

2021-07-30 Thread GitBox
suddendust edited a comment on issue #7229: URL: https://github.com/apache/pinot/issues/7229#issuecomment-889956449 @kishoreg just a thought - Can we do something that doesn't require us to download the complete segment from the S3? Some way of looking "inside" a segment and download only

[GitHub] [pinot] suddendust commented on issue #7229: Pinot Long Term Data Store

2021-07-30 Thread GitBox
suddendust commented on issue #7229: URL: https://github.com/apache/pinot/issues/7229#issuecomment-889956449 @kishoreg just a thought - Can we do something that doesn't require us to download the complete segment from the S3? Some way of looking "inside" a segment and download only a part

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7233: add cli args for auth tokens and user/password to ingestion job command

2021-07-30 Thread GitBox
codecov-commenter edited a comment on pull request #7233: URL: https://github.com/apache/pinot/pull/7233#issuecomment-889749493 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7233?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter commented on pull request #7233: add cli args for auth tokens and user/password to ingestion job command

2021-07-30 Thread GitBox
codecov-commenter commented on pull request #7233: URL: https://github.com/apache/pinot/pull/7233#issuecomment-889749493 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7233?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] apucher opened a new pull request #7233: add cli args for auth tokens and user/password to ingestion job command

2021-07-30 Thread GitBox
apucher opened a new pull request #7233: URL: https://github.com/apache/pinot/pull/7233 ## Description adds support for `-authToken`, `-user`, and `-password` cli args to the `LaunchDataIngestionJobCommand` to bring it into sync with the pinot docs and other cli commands. This is more o

[pinot] branch ingestion-job-cli-auth-tokens created (now c34582b)

2021-07-30 Thread apucher
This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch ingestion-job-cli-auth-tokens in repository https://gitbox.apache.org/repos/asf/pinot.git. at c34582b add cli args for auth tokens and user/password to ingestion job command This branch

[pinot] 01/01: add cli args for auth tokens and user/password to ingestion job command

2021-07-30 Thread apucher
This is an automated email from the ASF dual-hosted git repository. apucher pushed a commit to branch ingestion-job-cli-auth-tokens in repository https://gitbox.apache.org/repos/asf/pinot.git commit c34582bf9eebf800dafd4a6272ce0146d56668ea Author: Alexander Pucher AuthorDate: Fri Jul 30 00:54:12

[GitHub] [pinot] kishoreg commented on issue #7229: Pinot Long Term Data Store

2021-07-30 Thread GitBox
kishoreg commented on issue #7229: URL: https://github.com/apache/pinot/issues/7229#issuecomment-889696518 Whether we do it from connector or lazy load in Pinot the latency will be the same. Lazy load might be better if there is ssd one so let of no PoC as l cache on Pinot servers. --

[GitHub] [pinot] kishoreg commented on issue #7229: Pinot Long Term Data Store

2021-07-30 Thread GitBox
kishoreg commented on issue #7229: URL: https://github.com/apache/pinot/issues/7229#issuecomment-889695413 This makes sense and something worth doing. We need two things - enhance tiered storage concept to move segments older than X to S3. - either enhance presto Pinot connector t

[GitHub] [pinot] apucher opened a new issue #7232: obfuscate authToken in ingestion job (and other?) console output

2021-07-30 Thread GitBox
apucher opened a new issue #7232: URL: https://github.com/apache/pinot/issues/7232 console logs of ingestion jobs currently display the full job spec, including auth token in plain text. credentials should be obfuscated eventually. -- This is an automated message from the Apache Git Serv