[GitHub] [pinot] ckharide edited a comment on issue #7404: startServiceManager() not used in StartServiceManagerCommand

2021-09-07 Thread GitBox
ckharide edited a comment on issue #7404: URL: https://github.com/apache/pinot/issues/7404#issuecomment-914927759 Ah thank you . No i didnt run into any issue but wondering whats the thought in returning instanceID , prior to service getting started . -- This is an automated message fro

[GitHub] [pinot] ckharide commented on issue #7404: startServiceManager() not used in StartServiceManagerCommand

2021-09-07 Thread GitBox
ckharide commented on issue #7404: URL: https://github.com/apache/pinot/issues/7404#issuecomment-914927759 Ah thank you . No i didnt run into any issue but wondering whats the point in returning instanceID . -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [pinot] mayankshriv commented on a change in pull request #7407: Obfuscator base implementation

2021-09-07 Thread GitBox
mayankshriv commented on a change in pull request #7407: URL: https://github.com/apache/pinot/pull/7407#discussion_r703929631 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/utils/Obfuscator.java ## @@ -0,0 +1,114 @@ +/** + * Licensed to the Apache Software Foundatio

[pinot] branch config-obfuscator updated (fc7af0c -> 1f996ba)

2021-09-07 Thread apucher
This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch config-obfuscator in repository https://gitbox.apache.org/repos/asf/pinot.git. from fc7af0c basic obfuscator config add 1f996ba license headers No new revisions were added by this up

[GitHub] [pinot] apucher commented on pull request #7081: [#6823] Obfuscation of sensitive config

2021-09-07 Thread GitBox
apucher commented on pull request #7081: URL: https://github.com/apache/pinot/pull/7081#issuecomment-914709248 Hi folks, reviving this thread. I've pushed a PR proposal for a generic obfuscator in #7407. That one might be useful here as well. -- This is an automated message from the

[GitHub] [pinot] apucher commented on pull request #7407: Obfuscator base implementation

2021-09-07 Thread GitBox
apucher commented on pull request #7407: URL: https://github.com/apache/pinot/pull/7407#issuecomment-914707920 #7232 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

[GitHub] [pinot] apucher opened a new pull request #7407: Obfuscator base implementation

2021-09-07 Thread GitBox
apucher opened a new pull request #7407: URL: https://github.com/apache/pinot/pull/7407 ## Description This PR adds a basic `Obfucator` for object trees and config values. It can integrated further with table configs and debug endpoints as needed. ## Upgrade Notes Does this PR

[pinot] branch config-obfuscator created (now fc7af0c)

2021-09-07 Thread apucher
This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch config-obfuscator in repository https://gitbox.apache.org/repos/asf/pinot.git. at fc7af0c basic obfuscator config This branch includes the following new commits: new fc7af0c basic

[pinot] 01/01: basic obfuscator config

2021-09-07 Thread apucher
This is an automated email from the ASF dual-hosted git repository. apucher pushed a commit to branch config-obfuscator in repository https://gitbox.apache.org/repos/asf/pinot.git commit fc7af0c8ea94d0aee364e85815b597c6be3dc01e Author: Alexander Pucher AuthorDate: Tue Sep 7 16:49:43 2021 -0700

[GitHub] [pinot] richardstartin commented on pull request #7402: Faster association between Java Class and PinotDataType

2021-09-07 Thread GitBox
richardstartin commented on pull request #7402: URL: https://github.com/apache/pinot/pull/7402#issuecomment-914695077 The ordering makes some difference but the point was we're not entirely in control, so shouldn't over-optimise. I created a worst case scenario, where there's one string, p

[GitHub] [pinot] xiangfu0 commented on a change in pull request #7399: create pinot-integration-test-base module to release test-jar

2021-09-07 Thread GitBox
xiangfu0 commented on a change in pull request #7399: URL: https://github.com/apache/pinot/pull/7399#discussion_r703915674 ## File path: pinot-integration-test-base/pom.xml ## @@ -0,0 +1,302 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchem

[pinot] branch master updated: Missing Prometheus metric name (#7406)

2021-09-07 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new e7901ee Missing Prometheus metric name (#7406)

[GitHub] [pinot] xiangfu0 merged pull request #7406: Missing Prometheus metric name

2021-09-07 Thread GitBox
xiangfu0 merged pull request #7406: URL: https://github.com/apache/pinot/pull/7406 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7399: create pinot-integration-test-base module to release test-jar

2021-09-07 Thread GitBox
codecov-commenter edited a comment on pull request #7399: URL: https://github.com/apache/pinot/pull/7399#issuecomment-914679147 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7399?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7399: create pinot-integration-test-base module to release test-jar

2021-09-07 Thread GitBox
codecov-commenter edited a comment on pull request #7399: URL: https://github.com/apache/pinot/pull/7399#issuecomment-914679147 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7399?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter commented on pull request #7399: create pinot-integration-test-base module to release test-jar

2021-09-07 Thread GitBox
codecov-commenter commented on pull request #7399: URL: https://github.com/apache/pinot/pull/7399#issuecomment-914679147 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7399?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] Jackie-Jiang commented on pull request #7402: Faster association between Java Class and PinotDataType

2021-09-07 Thread GitBox
Jackie-Jiang commented on pull request #7402: URL: https://github.com/apache/pinot/pull/7402#issuecomment-914665876 I thought the speedup comes from re-ordering the if checks? If JIT can perform runtime optimization, then the order should not matter? One example we can try is to use the

[GitHub] [pinot] richardstartin commented on pull request #7402: Faster association between Java Class and PinotDataType

2021-09-07 Thread GitBox
richardstartin commented on pull request #7402: URL: https://github.com/apache/pinot/pull/7402#issuecomment-914659218 > Are we concluding that plain if statements has the best performance? One fact is that we don't really know the frequency of the input data type, so plain if statements

[GitHub] [pinot] ramabme opened a new pull request #7406: Missing Prometheus metric name

2021-09-07 Thread GitBox
ramabme opened a new pull request #7406: URL: https://github.com/apache/pinot/pull/7406 ## Description Include missing metric BROKER_RESPONSES_WITH_PROCESSING_EXCEPTIONS . ## Upgrade Notes Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker,

[GitHub] [pinot] Jackie-Jiang commented on pull request #7402: Faster association between Java Class and PinotDataType

2021-09-07 Thread GitBox
Jackie-Jiang commented on pull request #7402: URL: https://github.com/apache/pinot/pull/7402#issuecomment-914651122 Are we concluding that plain if statements has the best performance? One fact is that we don't really know the frequency of the input data type, so plain if statements migh

[GitHub] [pinot] walterddr commented on pull request #7399: create pinot-test-utils to share test-jar

2021-09-07 Thread GitBox
walterddr commented on pull request #7399: URL: https://github.com/apache/pinot/pull/7399#issuecomment-914649272 > Might be cleaner if we name the new package `pinot-integration-test-base`? yes. that sounds like a better name, making the change when resolving the merge conflicts --

[GitHub] [pinot] richardstartin commented on pull request #7402: Faster association between Java Class and PinotDataType

2021-09-07 Thread GitBox
richardstartin commented on pull request #7402: URL: https://github.com/apache/pinot/pull/7402#issuecomment-914648851 Unit Test Set 2 timed out again it seems :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [pinot] Jackie-Jiang commented on issue #7404: startServiceManager() not used in StartServiceManagerCommand

2021-09-07 Thread GitBox
Jackie-Jiang commented on issue #7404: URL: https://github.com/apache/pinot/issues/7404#issuecomment-914643229 It is called in the `execute()` method. Did you run into issues with `StartServiceManagerCommand`? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [pinot] Jackie-Jiang commented on pull request #7399: create pinot-test-utils to share test-jar

2021-09-07 Thread GitBox
Jackie-Jiang commented on pull request #7399: URL: https://github.com/apache/pinot/pull/7399#issuecomment-914640777 Might be cleaner if we name the new package `pinot-integration-test-base`? -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7397: [7264] Add Exception to Broker Response When Not All Segments Are Available (Partial Response)

2021-09-07 Thread GitBox
Jackie-Jiang commented on a change in pull request #7397: URL: https://github.com/apache/pinot/pull/7397#discussion_r703824504 ## File path: pinot-common/src/main/java/org/apache/pinot/common/exception/QueryException.java ## @@ -63,19 +63,27 @@ public static void setMaxLinesOf

[GitHub] [pinot] klsince commented on a change in pull request #7402: Faster association between Java Class and PinotDataType

2021-09-07 Thread GitBox
klsince commented on a change in pull request #7402: URL: https://github.com/apache/pinot/pull/7402#discussion_r703830331 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/PinotDataType.java ## @@ -1119,13 +1090,68 @@ public PinotDataType getSingleValueTyp

[pinot] branch master updated (fe14d60 -> 6294d1b)

2021-09-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git. from fe14d60 fixing pinot-presto docker image build script (#7401) add 6294d1b Remove unused constant FIELD_VALUE (#

[GitHub] [pinot] Jackie-Jiang merged pull request #7396: Remove unused constant FIELD_VALUE

2021-09-07 Thread GitBox
Jackie-Jiang merged pull request #7396: URL: https://github.com/apache/pinot/pull/7396 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-uns

[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7394: Replace MINUS(strColum1, strColumn2) with COMPARE(strColumn1,strColumn2)

2021-09-07 Thread GitBox
Jackie-Jiang commented on a change in pull request #7394: URL: https://github.com/apache/pinot/pull/7394#discussion_r703817541 ## File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java ## @@ -389,4 +389,15 @@ public static int hammin

[GitHub] [pinot] richardstartin commented on a change in pull request #7402: Faster association between Java Class and PinotDataType

2021-09-07 Thread GitBox
richardstartin commented on a change in pull request #7402: URL: https://github.com/apache/pinot/pull/7402#discussion_r703821420 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/PinotDataType.java ## @@ -1119,13 +1160,11 @@ public PinotDataType getSingleV

[GitHub] [pinot] richardstartin commented on a change in pull request #7402: memoize association between Java Class and PinotDataType

2021-09-07 Thread GitBox
richardstartin commented on a change in pull request #7402: URL: https://github.com/apache/pinot/pull/7402#discussion_r703820532 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/PinotDataType.java ## @@ -776,30 +777,70 @@ public String toString(Object val

[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7402: memoize association between Java Class and PinotDataType

2021-09-07 Thread GitBox
Jackie-Jiang commented on a change in pull request #7402: URL: https://github.com/apache/pinot/pull/7402#discussion_r703811135 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/PinotDataType.java ## @@ -1119,13 +1160,11 @@ public PinotDataType getSingleVal

[GitHub] [pinot] pedro93 commented on issue #7380: [K8s] Differentiate match labels between headless and non-headless services

2021-09-07 Thread GitBox
pedro93 commented on issue #7380: URL: https://github.com/apache/pinot/issues/7380#issuecomment-914582292 We have defined a ServiceMonitor resource to tell prometheus what to scrape: ``` apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: labels: team:

[GitHub] [pinot] atris opened a new pull request #7405: Introduce Native Text Indices (Core Functionality)

2021-09-07 Thread GitBox
atris opened a new pull request #7405: URL: https://github.com/apache/pinot/pull/7405 https://docs.google.com/document/d/1PMhoRy6WF46C4d4mw0LVe9b8Vjqes6vsXZkmxXzMYzw/edit?usp=sharing This PR implements Phase 1 (core functionality) of the given implementation plan. Part of #73

[GitHub] [pinot] ddcprg commented on issue #7357: Support for Avro logical types in realtime tables

2021-09-07 Thread GitBox
ddcprg commented on issue #7357: URL: https://github.com/apache/pinot/issues/7357#issuecomment-914575470 @mayankshriv my PR should cover both realtime tables and import jobs, please let me know if this is not the case. -- This is an automated message from the Apache Git Service. To resp

[GitHub] [pinot] ddcprg commented on issue #7380: [K8s] Differentiate match labels between headless and non-headless services

2021-09-07 Thread GitBox
ddcprg commented on issue #7380: URL: https://github.com/apache/pinot/issues/7380#issuecomment-914574041 IMHO metrics should be pulled per pod instead of per service. I'm curious to know how you configure Prometheus to use labels instead of annotations and the reason to pull metrics from t

[GitHub] [pinot] ddcprg commented on pull request #7358: WIP [issue-7357] Add support for Avro logical types in realtime ingestion

2021-09-07 Thread GitBox
ddcprg commented on pull request #7358: URL: https://github.com/apache/pinot/pull/7358#issuecomment-914569414 I'm looking at modifying the data used by `RealtimeClusterIntegrationTest` to add fields with logical types. Any guidelines on how this data has been generated? -- This is an au

[GitHub] [pinot] richardstartin commented on a change in pull request #7402: memoize association between Java Class and PinotDataType

2021-09-07 Thread GitBox
richardstartin commented on a change in pull request #7402: URL: https://github.com/apache/pinot/pull/7402#discussion_r703771456 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/PinotDataType.java ## @@ -776,30 +777,70 @@ public String toString(Object val

[GitHub] [pinot] ckharide opened a new issue #7404: startServiceManager() not used in StartServiceManagerCommand

2021-09-07 Thread GitBox
ckharide opened a new issue #7404: URL: https://github.com/apache/pinot/issues/7404 Hello - I just started looking at the code and please ignore if this issue is addressed . It seems the startServiceManager() method in StartServiceManagerCommand is not used also am curious why it

[GitHub] [pinot] klsince commented on a change in pull request #7402: memoize association between Java Class and PinotDataType

2021-09-07 Thread GitBox
klsince commented on a change in pull request #7402: URL: https://github.com/apache/pinot/pull/7402#discussion_r703753717 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/PinotDataType.java ## @@ -776,30 +777,70 @@ public String toString(Object value) {

[GitHub] [pinot] richardstartin opened a new issue #7403: JSONPath cache is inefficient

2021-09-07 Thread GitBox
richardstartin opened a new issue #7403: URL: https://github.com/apache/pinot/issues/7403 In a profile collected from a user, we see over 20% of CPU time in JSONPath's `LRUCache` ![Capture](https://user-images.githubusercontent.com/16439049/132392314-9f279512-2095-40d7-9593-ed6a62ff2dc

[GitHub] [pinot] richardstartin commented on pull request #7402: memoize association between Java Class and PinotDataType

2021-09-07 Thread GitBox
richardstartin commented on pull request #7402: URL: https://github.com/apache/pinot/pull/7402#issuecomment-914518430 Here is the microbenchmark mentioned: ```java @State(Scope.Benchmark) public class PinotDataTypeMappingBenchmark { @Benchmark public void singleValueTy

[GitHub] [pinot] richardstartin commented on a change in pull request #7402: memoize association between Java Class and PinotDataType

2021-09-07 Thread GitBox
richardstartin commented on a change in pull request #7402: URL: https://github.com/apache/pinot/pull/7402#discussion_r703727881 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/PinotDataType.java ## @@ -776,30 +777,70 @@ public String toString(Object val

[GitHub] [pinot] klsince commented on a change in pull request #7402: memoize association between Java Class and PinotDataType

2021-09-07 Thread GitBox
klsince commented on a change in pull request #7402: URL: https://github.com/apache/pinot/pull/7402#discussion_r703717224 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/PinotDataType.java ## @@ -776,30 +777,70 @@ public String toString(Object value) {

[GitHub] [pinot] richardstartin commented on a change in pull request #7402: memoize association between Java Class and PinotDataType

2021-09-07 Thread GitBox
richardstartin commented on a change in pull request #7402: URL: https://github.com/apache/pinot/pull/7402#discussion_r703705968 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/MemoizedClassFunction.java ## @@ -0,0 +1,44 @@ +/** + * Licensed to the Apach

[GitHub] [pinot] klsince commented on a change in pull request #7402: memoize association between Java Class and PinotDataType

2021-09-07 Thread GitBox
klsince commented on a change in pull request #7402: URL: https://github.com/apache/pinot/pull/7402#discussion_r703705469 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/MemoizedClassFunction.java ## @@ -0,0 +1,44 @@ +/** + * Licensed to the Apache Softw

[GitHub] [pinot] richardstartin commented on pull request #7402: memoize association between Java Class and PinotDataType

2021-09-07 Thread GitBox
richardstartin commented on pull request #7402: URL: https://github.com/apache/pinot/pull/7402#issuecomment-914454241 The test failure seems spurious, @Jackie-Jiang do you have any ideas? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [pinot] kishoreg commented on issue #7388: Operational Intelligence: Using Pinot as Backend for Timeseries Data

2021-09-07 Thread GitBox
kishoreg commented on issue #7388: URL: https://github.com/apache/pinot/issues/7388#issuecomment-914453251 @elonazoulay can you please start a design doc on what you plan to add -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [pinot] siddharthteotia commented on issue #7395: Support for Native Text Indexing in Pinot

2021-09-07 Thread GitBox
siddharthteotia commented on issue #7395: URL: https://github.com/apache/pinot/issues/7395#issuecomment-914452426 Thanks @atris . I would like to review. Please give me couple of days to go through the doc. -- This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [pinot] codecov-commenter commented on pull request #7402: memoize association between Java Class and PinotDataType

2021-09-07 Thread GitBox
codecov-commenter commented on pull request #7402: URL: https://github.com/apache/pinot/pull/7402#issuecomment-914319924 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7402?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] richardstartin opened a new pull request #7402: memoize association between Java Class and PinotDataType

2021-09-07 Thread GitBox
richardstartin opened a new pull request #7402: URL: https://github.com/apache/pinot/pull/7402 ## Description This change was motivated by analysing a profile of high CPU usage data ingestion received from a user in the Pinot slack showing 1.5% of the CPU profile in `PinotDataType.getSi