[GitHub] [pinot] richardstartin commented on a change in pull request #7412: Reduce allocation in JSONPath evaluation

richardstartin commented on a change in pull request #7412: URL: https://github.com/apache/pinot/pull/7412#discussion_r705017892 ## File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/JsonFunctions.java ## @@ -97,9 +102,9 @@ public static String jsonF

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7394: Replace MINUS(strColum1, strColumn2) with COMPARE(strColumn1,strColumn2)

codecov-commenter edited a comment on pull request #7394: URL: https://github.com/apache/pinot/pull/7394#issuecomment-912821972 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7394?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7394: Replace MINUS(strColum1, strColumn2) with COMPARE(strColumn1,strColumn2)

codecov-commenter edited a comment on pull request #7394: URL: https://github.com/apache/pinot/pull/7394#issuecomment-912821972 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7394?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7384: Adding presto-pinot-driver module

codecov-commenter edited a comment on pull request #7384: URL: https://github.com/apache/pinot/pull/7384#issuecomment-910162874 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7384?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7384: Adding presto-pinot-driver module

codecov-commenter edited a comment on pull request #7384: URL: https://github.com/apache/pinot/pull/7384#issuecomment-910162874 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7384?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7384: Adding presto-pinot-driver module

codecov-commenter edited a comment on pull request #7384: URL: https://github.com/apache/pinot/pull/7384#issuecomment-910162874 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7384?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] lakshmanan-v commented on a change in pull request #7383: Add HLLSketch and HLLPlusPlus aggregate functions

lakshmanan-v commented on a change in pull request #7383: URL: https://github.com/apache/pinot/pull/7383#discussion_r704995601 ## File path: pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/DistinctCountHLLPlusPlusMVAggregationFunction.java ## @@ -0,0

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7384: Adding presto-pinot-driver module

codecov-commenter edited a comment on pull request #7384: URL: https://github.com/apache/pinot/pull/7384#issuecomment-910162874 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7384?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] lakshmanan-v commented on a change in pull request #7383: Add HLLSketch and HLLPlusPlus aggregate functions

lakshmanan-v commented on a change in pull request #7383: URL: https://github.com/apache/pinot/pull/7383#discussion_r704977748 ## File path: pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/DistinctCountHLLSketchAggregationFunction.java ## @@ -0,0 +1,3

[GitHub] [pinot] amrishlal commented on a change in pull request #7394: Replace MINUS(strColum1, strColumn2) with COMPARE(strColumn1,strColumn2)

amrishlal commented on a change in pull request #7394: URL: https://github.com/apache/pinot/pull/7394#discussion_r704977393 ## File path: pinot-core/src/main/java/org/apache/pinot/core/query/optimizer/statement/StringPredicateFilterOptimizer.java ## @@ -0,0 +1,127 @@ +/** + *

[GitHub] [pinot] lakshmanan-v commented on a change in pull request #7383: Add HLLSketch and HLLPlusPlus aggregate functions

lakshmanan-v commented on a change in pull request #7383: URL: https://github.com/apache/pinot/pull/7383#discussion_r704977422 ## File path: pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/DistinctCountHLLPlusPlusMVAggregationFunction.java ## @@ -0,0

[GitHub] [pinot] amrishlal commented on a change in pull request #7394: Replace MINUS(strColum1, strColumn2) with COMPARE(strColumn1,strColumn2)

amrishlal commented on a change in pull request #7394: URL: https://github.com/apache/pinot/pull/7394#discussion_r704977257 ## File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java ## @@ -389,4 +389,15 @@ public static int hammingDi

[GitHub] [pinot] amrishlal commented on a change in pull request #7394: Replace MINUS(strColum1, strColumn2) with COMPARE(strColumn1,strColumn2)

amrishlal commented on a change in pull request #7394: URL: https://github.com/apache/pinot/pull/7394#discussion_r704976833 ## File path: pinot-core/src/main/java/org/apache/pinot/core/query/optimizer/statement/StringPredicateFilterOptimizer.java ## @@ -0,0 +1,127 @@ +/** + *

[GitHub] [pinot] amrishlal commented on a change in pull request #7394: Replace MINUS(strColum1, strColumn2) with COMPARE(strColumn1,strColumn2)

amrishlal commented on a change in pull request #7394: URL: https://github.com/apache/pinot/pull/7394#discussion_r704976661 ## File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java ## @@ -389,4 +389,15 @@ public static int hammingDi

[GitHub] [pinot] elonazoulay edited a comment on issue #7388: Operational Intelligence: Using Pinot as Backend for Timeseries Data

elonazoulay edited a comment on issue #7388: URL: https://github.com/apache/pinot/issues/7388#issuecomment-915778295 Yes, some example functions: * locf: last observation carried forward * gapfill: similar to the `seq` sql function, fill in a gap with specified granularity * linear

[GitHub] [pinot] elonazoulay commented on issue #7388: Operational Intelligence: Using Pinot as Backend for Timeseries Data

elonazoulay commented on issue #7388: URL: https://github.com/apache/pinot/issues/7388#issuecomment-915778295 Yes, some example functions: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[pinot] branch master updated (58fcd5b -> c27ae81)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git. from 58fcd5b simplify association between Java Class and PinotDataType for faster mapping (#7402) add c27ae81 Cach

[GitHub] [pinot] xiangfu0 merged pull request #7408: Cache ~/.m2/repository to speed up CI speed

xiangfu0 merged pull request #7408: URL: https://github.com/apache/pinot/pull/7408 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[GitHub] [pinot] ckharide commented on issue #7404: startServiceManager() not used in StartServiceManagerCommand

ckharide commented on issue #7404: URL: https://github.com/apache/pinot/issues/7404#issuecomment-915763649 Ok may be am missing something here - in PinotServiceManager constructor _instanceId is already there. _instanceId = String.format("ServiceManager_%s_%d", hostname, port); D

[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7412: Reduce allocation in JSONPath evaluation

Jackie-Jiang commented on a change in pull request #7412: URL: https://github.com/apache/pinot/pull/7412#discussion_r704867325 ## File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/JsonFunctions.java ## @@ -97,9 +102,9 @@ public static String jsonFor

[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7397: [7264] Add Exception to Broker Response When Not All Segments Are Available (Partial Response)

Jackie-Jiang commented on a change in pull request #7397: URL: https://github.com/apache/pinot/pull/7397#discussion_r704865110 ## File path: pinot-common/src/main/java/org/apache/pinot/common/exception/QueryException.java ## @@ -63,19 +63,27 @@ public static void setMaxLinesOf

[GitHub] [pinot] Jackie-Jiang commented on issue #7404: startServiceManager() not used in StartServiceManagerCommand

Jackie-Jiang commented on issue #7404: URL: https://github.com/apache/pinot/issues/7404#issuecomment-915664185 Actually the instanceId is returned after it is started (`StartServiceManagerCommand.startServiceManager()`), and it is logged in `StartServiceManagerCommand.startPinotService()`

[pinot] branch master updated: simplify association between Java Class and PinotDataType for faster mapping (#7402)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 58fcd5b simplify association between Java Class

[GitHub] [pinot] Jackie-Jiang merged pull request #7402: Faster association between Java Class and PinotDataType

Jackie-Jiang merged pull request #7402: URL: https://github.com/apache/pinot/pull/7402 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-uns

[GitHub] [pinot] kkrugler commented on pull request #7400: Fix classpath with plugins java 8

kkrugler commented on pull request #7400: URL: https://github.com/apache/pinot/pull/7400#issuecomment-915660204 @elonazoulay & @xiangfu0 - hoping one of you can take a look, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[GitHub] [pinot] mcvsubbu commented on issue #7413: Race conditions between segment merge/roll-up and purge (or convertToRawIndex) tasks

mcvsubbu commented on issue #7413: URL: https://github.com/apache/pinot/issues/7413#issuecomment-915649144 Does it make sense to have some sort of segment level locks so that the same set of segments are not picked up by multiple tasks? If we can come up with a lock-free algorithms on this

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7407: Obfuscator base implementation

codecov-commenter edited a comment on pull request #7407: URL: https://github.com/apache/pinot/pull/7407#issuecomment-915485767 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7407?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7407: Obfuscator base implementation

codecov-commenter edited a comment on pull request #7407: URL: https://github.com/apache/pinot/pull/7407#issuecomment-915485767 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7407?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7407: Obfuscator base implementation

codecov-commenter edited a comment on pull request #7407: URL: https://github.com/apache/pinot/pull/7407#issuecomment-915485767 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7407?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[pinot] branch config-obfuscator updated (beb7aa0 -> cfb41e9)

This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch config-obfuscator in repository https://gitbox.apache.org/repos/asf/pinot.git. from beb7aa0 dedicated JsonNode path add cfb41e9 handle json string separately No new revisions were ad

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7319: Move offline seg download code to OfflineTableManager

codecov-commenter edited a comment on pull request #7319: URL: https://github.com/apache/pinot/pull/7319#issuecomment-901313573 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7319?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] jtao15 opened a new issue #7413: Race conditions between segment merge/roll-up and purge (or convertToRawIndex) tasks

jtao15 opened a new issue #7413: URL: https://github.com/apache/pinot/issues/7413 There's a potential race condition issue when we run merge/roll-up task & segment upload tasks (e.g. purge, convertToRawIndex, etc) on the same table (purge task for example): 1. Segment A is picked both b

[GitHub] [pinot] klsince commented on a change in pull request #7319: Move offline seg download code to OfflineTableManager

klsince commented on a change in pull request #7319: URL: https://github.com/apache/pinot/pull/7319#discussion_r704818424 ## File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java ## @@ -248,19 +249,27 @@ public boolean is

[GitHub] [pinot] klsince commented on a change in pull request #7319: Move offline seg download code to OfflineTableManager

klsince commented on a change in pull request #7319: URL: https://github.com/apache/pinot/pull/7319#discussion_r704813575 ## File path: pinot-server/src/main/java/org/apache/pinot/server/starter/helix/HelixInstanceDataManager.java ## @@ -311,6 +310,47 @@ private void reloadSeg

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7402: Faster association between Java Class and PinotDataType

codecov-commenter edited a comment on pull request #7402: URL: https://github.com/apache/pinot/pull/7402#issuecomment-914319924 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7402?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] richardstartin commented on pull request #7402: Faster association between Java Class and PinotDataType

richardstartin commented on pull request #7402: URL: https://github.com/apache/pinot/pull/7402#issuecomment-915576580 OK I reordered them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [pinot] mcvsubbu commented on a change in pull request #6778: upload missing LLC segment to segment store by controller periodic task

mcvsubbu commented on a change in pull request #6778: URL: https://github.com/apache/pinot/pull/6778#discussion_r704761568 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/ControllerConf.java ## @@ -173,6 +173,12 @@ public static final String SEGME

[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7402: Faster association between Java Class and PinotDataType

Jackie-Jiang commented on a change in pull request #7402: URL: https://github.com/apache/pinot/pull/7402#discussion_r704755085 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/PinotDataType.java ## @@ -1119,13 +1090,68 @@ public PinotDataType getSingleVal

[GitHub] [pinot] mcvsubbu commented on a change in pull request #7319: Move offline seg download code to OfflineTableManager

mcvsubbu commented on a change in pull request #7319: URL: https://github.com/apache/pinot/pull/7319#discussion_r704738027 ## File path: pinot-core/src/test/java/org/apache/pinot/core/data/manager/BaseTableDataManagerAcquireSegmentTest.java ## @@ -240,7 +239,7 @@ public void t

[GitHub] [pinot] richardstartin commented on pull request #7402: Faster association between Java Class and PinotDataType

richardstartin commented on pull request #7402: URL: https://github.com/apache/pinot/pull/7402#issuecomment-915535374 @Jackie-Jiang do you have any concerns regarding what's above? This is much faster than the `HashMap` in the worst case as well as the best case. -- This is an automated

[GitHub] [pinot] klsince commented on a change in pull request #7319: Move offline seg download code to OfflineTableManager

klsince commented on a change in pull request #7319: URL: https://github.com/apache/pinot/pull/7319#discussion_r704717050 ## File path: pinot-core/src/test/java/org/apache/pinot/core/data/manager/BaseTableDataManagerAcquireSegmentTest.java ## @@ -240,7 +239,7 @@ public void te

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7407: Obfuscator base implementation

codecov-commenter edited a comment on pull request #7407: URL: https://github.com/apache/pinot/pull/7407#issuecomment-915485767 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7407?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] apucher commented on pull request #7407: Obfuscator base implementation

apucher commented on pull request #7407: URL: https://github.com/apache/pinot/pull/7407#issuecomment-915499072 > > The obfuscator is fully configurable. If patterns need to be added or tweaked this can be done at any time > > EDIT: to clarify - the patterns match on key name, not the val

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7407: Obfuscator base implementation

codecov-commenter edited a comment on pull request #7407: URL: https://github.com/apache/pinot/pull/7407#issuecomment-915485767 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7407?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] mcvsubbu commented on pull request #7407: Obfuscator base implementation

mcvsubbu commented on pull request #7407: URL: https://github.com/apache/pinot/pull/7407#issuecomment-915493156 > The obfuscator is fully configurable. If patterns need to be added or tweaked this can be done at any time > > EDIT: to clarify - the patterns match on key name, not the

[GitHub] [pinot] codecov-commenter commented on pull request #7407: Obfuscator base implementation

codecov-commenter commented on pull request #7407: URL: https://github.com/apache/pinot/pull/7407#issuecomment-915485767 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7407?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] apucher edited a comment on pull request #7407: Obfuscator base implementation

apucher edited a comment on pull request #7407: URL: https://github.com/apache/pinot/pull/7407#issuecomment-915476383 The obfuscator is fully configurable. If patterns need to be added or tweaked this can be done at any time EDIT: to clarify - the patterns match on key name, not the

[GitHub] [pinot] apucher edited a comment on pull request #7407: Obfuscator base implementation

apucher edited a comment on pull request #7407: URL: https://github.com/apache/pinot/pull/7407#issuecomment-915476383 The obfuscator is fully configurable. If patterns need to added or tweaked this can be done at any time EDIT: to clarify - the pattern match on key name, not the valu

[GitHub] [pinot] apucher commented on pull request #7407: Obfuscator base implementation

apucher commented on pull request #7407: URL: https://github.com/apache/pinot/pull/7407#issuecomment-915476383 The obfuscator is fully configurable. If patterns need to added or tweaked this can be done at any time -- This is an automated message from the Apache Git Service. To respond t

[pinot] branch config-obfuscator updated (1f996ba -> beb7aa0)

This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch config-obfuscator in repository https://gitbox.apache.org/repos/asf/pinot.git. from 1f996ba license headers add beb7aa0 dedicated JsonNode path No new revisions were added by this up

[GitHub] [pinot] apucher commented on a change in pull request #7407: Obfuscator base implementation

apucher commented on a change in pull request #7407: URL: https://github.com/apache/pinot/pull/7407#discussion_r704659809 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/utils/Obfuscator.java ## @@ -0,0 +1,114 @@ +/** + * Licensed to the Apache Software Foundation (A

[GitHub] [pinot] mcvsubbu commented on a change in pull request #7319: Move offline seg download code to OfflineTableManager

mcvsubbu commented on a change in pull request #7319: URL: https://github.com/apache/pinot/pull/7319#discussion_r704643135 ## File path: pinot-core/src/test/java/org/apache/pinot/core/data/manager/BaseTableDataManagerAcquireSegmentTest.java ## @@ -240,7 +239,7 @@ public void t

[GitHub] [pinot] mcvsubbu commented on a change in pull request #7319: Move offline seg download code to OfflineTableManager

mcvsubbu commented on a change in pull request #7319: URL: https://github.com/apache/pinot/pull/7319#discussion_r704639430 ## File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java ## @@ -256,4 +267,147 @@ public void addSegmentError(S

[GitHub] [pinot] mcvsubbu commented on issue #7229: Pinot Long Term Data Store

mcvsubbu commented on issue #7229: URL: https://github.com/apache/pinot/issues/7229#issuecomment-915422279 Please also add a link o https://docs.google.com/document/d/1Z4FLg3ezHpqvc6zhy0jR6Wi2OL8wLO_lRC6aLkskFgs/edit#heading=h.sbtdm9byxqnc in the design document, and comment on how your so

[GitHub] [pinot] mcvsubbu commented on issue #7378: Support to add multiple directory location for this property in server (pinot.server.instance.dataDir)

mcvsubbu commented on issue #7378: URL: https://github.com/apache/pinot/issues/7378#issuecomment-915421214 So, what you really need is not just multiple directory support, but (also) support for some policy using which some directory may be chosen to keep segments in the server. Perhaps y

[GitHub] [pinot] xiangfu0 edited a comment on pull request #7384: Adding presto-pinot-driver module

xiangfu0 edited a comment on pull request #7384: URL: https://github.com/apache/pinot/pull/7384#issuecomment-915414963 > k. but then why there are other changes than the pom changes? After chatted with Prestodb community, I think it's better to move presto-pinot-driver (https://githu

[GitHub] [pinot] xiangfu0 edited a comment on pull request #7384: Adding presto-pinot-driver module

xiangfu0 edited a comment on pull request #7384: URL: https://github.com/apache/pinot/pull/7384#issuecomment-915414963 > k. but then why there are other changes than the pom changes? After chatted with Prestodb community, I think it's better to move presto-pinot-driver (https://githu

[GitHub] [pinot] xiangfu0 commented on pull request #7384: Adding presto-pinot-driver module

xiangfu0 commented on pull request #7384: URL: https://github.com/apache/pinot/pull/7384#issuecomment-915414963 > k. but then why there are other changes than the pom changes? After chatted with Prestodb community, I think it's better to move presto-pinot-driver (https://github.com/p

[GitHub] [pinot] yupeng9 commented on pull request #7384: Adding presto-pinot-driver module

yupeng9 commented on pull request #7384: URL: https://github.com/apache/pinot/pull/7384#issuecomment-915411000 k. but then why there are other changes than the pom changes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [pinot] klsince commented on a change in pull request #7319: Move offline seg download code to OfflineTableManager

klsince commented on a change in pull request #7319: URL: https://github.com/apache/pinot/pull/7319#discussion_r704605056 ## File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java ## @@ -248,19 +249,27 @@ public boolean is

[GitHub] [pinot] richardstartin commented on a change in pull request #7319: Move offline seg download code to OfflineTableManager

richardstartin commented on a change in pull request #7319: URL: https://github.com/apache/pinot/pull/7319#discussion_r704600925 ## File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java ## @@ -256,4 +267,147 @@ public void addSegmentE

[GitHub] [pinot] klsince commented on a change in pull request #7319: Move offline seg download code to OfflineTableManager

klsince commented on a change in pull request #7319: URL: https://github.com/apache/pinot/pull/7319#discussion_r704597692 ## File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/InstanceDataManager.java ## @@ -86,11 +86,28 @@ void reloadSegment(String tableNa

[GitHub] [pinot] richardstartin commented on a change in pull request #7409: Fix JSONPath cache inefficient issue

richardstartin commented on a change in pull request #7409: URL: https://github.com/apache/pinot/pull/7409#discussion_r704596924 ## File path: pinot-common/src/test/java/org/apache/pinot/common/function/JsonPathMapCacheTest.java ## @@ -0,0 +1,65 @@ +/** + * Licensed to the Apa

[GitHub] [pinot] klsince commented on a change in pull request #7319: Move offline seg download code to OfflineTableManager

klsince commented on a change in pull request #7319: URL: https://github.com/apache/pinot/pull/7319#discussion_r704593911 ## File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java ## @@ -256,4 +267,147 @@ public void addSegmentError(St

[GitHub] [pinot] klsince commented on a change in pull request #7319: Move offline seg download code to OfflineTableManager

klsince commented on a change in pull request #7319: URL: https://github.com/apache/pinot/pull/7319#discussion_r704593059 ## File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java ## @@ -256,4 +267,147 @@ public void addSegmentError(St

[GitHub] [pinot] klsince commented on a change in pull request #7319: Move offline seg download code to OfflineTableManager

klsince commented on a change in pull request #7319: URL: https://github.com/apache/pinot/pull/7319#discussion_r704593059 ## File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java ## @@ -256,4 +267,147 @@ public void addSegmentError(St

[GitHub] [pinot] richardstartin commented on a change in pull request #7409: Fix JSONPath cache inefficient issue

richardstartin commented on a change in pull request #7409: URL: https://github.com/apache/pinot/pull/7409#discussion_r704592708 ## File path: pinot-common/src/test/java/org/apache/pinot/common/function/JsonPathMapCacheTest.java ## @@ -0,0 +1,65 @@ +/** + * Licensed to the Apa

[GitHub] [pinot] klsince commented on a change in pull request #7319: Move offline seg download code to OfflineTableManager

klsince commented on a change in pull request #7319: URL: https://github.com/apache/pinot/pull/7319#discussion_r704588292 ## File path: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/data/manager/TableDataManager.java ## @@ -146,4 +156,12 @@ void addSegment(S

[GitHub] [pinot] xiangfu0 commented on pull request #7384: Adding presto-pinot-driver module

xiangfu0 commented on pull request #7384: URL: https://github.com/apache/pinot/pull/7384#issuecomment-915389720 > Can you explain the purpose of this change? > Can you explain the purpose of this change? presto is on jdk8 and pinot is on jdk 11, so we need to recompile

[GitHub] [pinot] yupeng9 commented on pull request #7384: Adding presto-pinot-driver module

yupeng9 commented on pull request #7384: URL: https://github.com/apache/pinot/pull/7384#issuecomment-915377453 Can you explain the purpose of this change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [pinot] xiangfu0 commented on pull request #7384: Adding presto-pinot-driver module

xiangfu0 commented on pull request #7384: URL: https://github.com/apache/pinot/pull/7384#issuecomment-915346648 > Can you please elaborate more on why we need to replicate some of the classes in this new module? And how is the module going to be released? removed duplicated classes.

[GitHub] [pinot] xiangfu0 commented on pull request #7408: Cache ~/.m2/repository to speed up CI speed

xiangfu0 commented on pull request #7408: URL: https://github.com/apache/pinot/pull/7408#issuecomment-915340296 Seems no significant changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [pinot] richardstartin commented on a change in pull request #7412: Reduce allocation in JSONPath evaluation

richardstartin commented on a change in pull request #7412: URL: https://github.com/apache/pinot/pull/7412#discussion_r704476933 ## File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/JsonFunctions.java ## @@ -97,9 +102,9 @@ public static String jsonF

[GitHub] [pinot] kishoreg commented on a change in pull request #7412: Reduce allocation in JSONPath evaluation

kishoreg commented on a change in pull request #7412: URL: https://github.com/apache/pinot/pull/7412#discussion_r704472738 ## File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/JsonFunctions.java ## @@ -97,9 +102,9 @@ public static String jsonFormat(

[GitHub] [pinot] richardstartin closed issue #7411: User provided JsonPaths should be validated/rejected at startup

richardstartin closed issue #7411: URL: https://github.com/apache/pinot/issues/7411 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [pinot] richardstartin commented on issue #7411: User provided JsonPaths should be validated/rejected at startup

richardstartin commented on issue #7411: URL: https://github.com/apache/pinot/issues/7411#issuecomment-915269328 I managed to reproduce this with a valid JSON path and it's more of a library issue, closing. -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [pinot] richardstartin opened a new pull request #7412: Reduce allocation in JSONPath evaluation

richardstartin opened a new pull request #7412: URL: https://github.com/apache/pinot/pull/7412 ## Description The JSONPath library allocates a stateless object and reinitialises configuration once per JSON path invocation, which contributes to high allocation rate per JSON path eval

[GitHub] [pinot] syed72 commented on issue #7378: Support to add multiple directory location for this property in server (pinot.server.instance.dataDir)

syed72 commented on issue #7378: URL: https://github.com/apache/pinot/issues/7378#issuecomment-915221704 1. Assume, we have 2 disk partitions(/disk1, /disk2)... we can choose any one partition based on the free space available ... 2. I have another case, where we can have 2 disks(one wit

[GitHub] [pinot] suddendust commented on a change in pull request #7397: [7264] Add Exception to Broker Response When Not All Segments Are Available (Partial Response)

suddendust commented on a change in pull request #7397: URL: https://github.com/apache/pinot/pull/7397#discussion_r704331530 ## File path: pinot-common/src/main/java/org/apache/pinot/common/exception/QueryException.java ## @@ -63,19 +63,27 @@ public static void setMaxLinesOfSt

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7409: Fix JSONPath cache inefficient issue

codecov-commenter edited a comment on pull request #7409: URL: https://github.com/apache/pinot/pull/7409#issuecomment-915143514 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7409?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter commented on pull request #7409: Fix JSONPath cache inefficient issue

codecov-commenter commented on pull request #7409: URL: https://github.com/apache/pinot/pull/7409#issuecomment-915143514 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7409?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] richardstartin opened a new issue #7411: User provided JsonPaths should be validated/rejected at startup

richardstartin opened a new issue #7411: URL: https://github.com/apache/pinot/issues/7411 In a profile from a system where invalid JSON paths were configured, a lot of time (> 20%) was spent during ingestion throwing at least one JSON path validation exception per row. The operator of a pr

[GitHub] [pinot] richardstartin opened a new issue #7410: User provided JsonPaths should be trimmed as early as possible

richardstartin opened a new issue #7410: URL: https://github.com/apache/pinot/issues/7410 I noticed defensive calls to `String.trim` actually trimming the JSON path in a profile from a user, where these paths come from ingestion transformation configuration. https://user-images.git

[GitHub] [pinot] Ferrari6 opened a new pull request #7409: Fix JSONPath cache inefficient issue

Ferrari6 opened a new pull request #7409: URL: https://github.com/apache/pinot/pull/7409 ## Description this commit fixes #7403 ### Background > When we used jsonpath transformation functions, we found that there was a delay in consumption, and the CPU usage was very high. Ana

[GitHub] [pinot] codecov-commenter commented on pull request #7408: Cache ~/.m2/repository to speed up CI speed

codecov-commenter commented on pull request #7408: URL: https://github.com/apache/pinot/pull/7408#issuecomment-915089336 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7408?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] xiangfu0 opened a new pull request #7408: Cache ~/.m2/repository to speed up CI speed

xiangfu0 opened a new pull request #7408: URL: https://github.com/apache/pinot/pull/7408 ## Description ## Upgrade Notes Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion) * [ ] Yes (Please label as **backward-incompat**, a

[GitHub] [pinot] richardstartin edited a comment on pull request #7402: Faster association between Java Class and PinotDataType

richardstartin edited a comment on pull request #7402: URL: https://github.com/apache/pinot/pull/7402#issuecomment-914695077 The ordering makes some difference but the point was we're not entirely in control, so shouldn't over-optimise. I created a worst case scenario, where there's one st