Re: [PR] Support server level consumption throttle [pinot]

2024-01-23 Thread via GitHub
xiangfu0 merged PR #12292: URL: https://github.com/apache/pinot/pull/12292 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

(pinot) branch master updated: Support server level consumption throttle (#12292)

2024-01-23 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new f43664dd13 Support server level consumption thro

(pinot) branch master updated: Reduce Disk Footprint for Segment Processor Framework to Avoid Out of Disk Issues (#12220)

2024-01-23 Thread snlee
This is an automated email from the ASF dual-hosted git repository. snlee pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 91ffcc759f Reduce Disk Footprint for Segment Proce

Re: [PR] Reduce Disk Footprint for Segment Processor Framework to Avoid Out of Disk Issues [pinot]

2024-01-23 Thread via GitHub
snleee merged PR #12220: URL: https://github.com/apache/pinot/pull/12220 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

Re: [PR] Reduce Disk Footprint for Segment Processor Framework to Avoid Out of Disk Issues [pinot]

2024-01-23 Thread via GitHub
snleee commented on PR #12220: URL: https://github.com/apache/pinot/pull/12220#issuecomment-1907418232 LGTM! Thank you for working on this 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Reduce Disk Footprint for Segment Processor Framework to Avoid Out of Disk Issues [pinot]

2024-01-23 Thread via GitHub
aishikbh commented on PR #12220: URL: https://github.com/apache/pinot/pull/12220#issuecomment-1907402552 > LGTM! Thank you for working on this feature! Can we make sure that all tests pass? The test was failing because of another change. I have fixed them now. -- This is an automat

(pinot) branch master updated: Support table suffix in ZkBasicAuthAccessControlFactory (#12310)

2024-01-23 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 63e91ef95f Support table suffix in ZkBasicAuthAc

Re: [PR] Support table suffix in ZkBasicAuthAccessControlFactory [pinot]

2024-01-23 Thread via GitHub
xiangfu0 merged PR #12310: URL: https://github.com/apache/pinot/pull/12310 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] Support table suffix in ZkBasicAuthAccessControlFactory [pinot]

2024-01-23 Thread via GitHub
xiangfu0 commented on PR #12310: URL: https://github.com/apache/pinot/pull/12310#issuecomment-1907075282 > The previous assumption during the RBAC (Role-Based Access Control) development was that regular users (referring to users without permissions for all tables) should be agnostic to the

Re: [PR] Support server level consumption throttle [pinot]

2024-01-23 Thread via GitHub
xiangfu0 commented on PR #12292: URL: https://github.com/apache/pinot/pull/12292#issuecomment-1907054267 > LGTM. A couple of comments though: > > 1. Can you please update the main class documentation to reflect the new addition? > 2. The additional logic is straight-forward. Do we

Re: [PR] Support server level consumption throttle [pinot]

2024-01-23 Thread via GitHub
xiangfu0 commented on PR #12292: URL: https://github.com/apache/pinot/pull/12292#issuecomment-1907052596 > Thank you for adding this! > > LGTM! > > QQ: This will require the server restart to change the value. Is my understanding correct? True, we can support the runtime

[I] Support mass data export [pinot]

2024-01-23 Thread via GitHub
egalpin opened a new issue, #12315: URL: https://github.com/apache/pinot/issues/12315 I know mass export isn’t a primary focus of Pinot (understandably, with the focus on real-time ingestion and low latency querying instead), but there are use cases where mass export would be very useful.

Re: [I] Parsing http-headers in RequestContext - broker query event listener [pinot]

2024-01-23 Thread via GitHub
tibrewalpratik17 commented on issue #12273: URL: https://github.com/apache/pinot/issues/12273#issuecomment-1906844579 > that's fine. looks like cluster config is saved in zk Ah then i think we can do this. In ZK, under `propertyStore/configs/cluster` I can create one more property

Re: [PR] Adding support for configuration through environment variables [pinot]

2024-01-23 Thread via GitHub
xiangfu0 commented on code in PR #12307: URL: https://github.com/apache/pinot/pull/12307#discussion_r1463922140 ## pinot-spi/src/main/java/org/apache/pinot/spi/env/PinotConfiguration.java: ## @@ -201,16 +200,16 @@ private static Configuration loadProperties(String configPath) {

Re: [PR] Backwards compatible theta sketch aggregation (#12288) [pinot]

2024-01-23 Thread via GitHub
SabrinaZhaozyf closed pull request #12314: Backwards compatible theta sketch aggregation (#12288) URL: https://github.com/apache/pinot/pull/12314 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Backwards compatible theta sketch aggregation (#12288) [pinot]

2024-01-23 Thread via GitHub
SabrinaZhaozyf closed pull request #12314: Backwards compatible theta sketch aggregation (#12288) URL: https://github.com/apache/pinot/pull/12314 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

(pinot) branch hotfix-theta-sketch updated: Backwards compatible theta sketch aggregation (#12288)

2024-01-23 Thread sajjad
This is an automated email from the ASF dual-hosted git repository. sajjad pushed a commit to branch hotfix-theta-sketch in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/hotfix-theta-sketch by this push: new e54888f597 Backwards c

(pinot) branch hotfix-theta-sketch updated (35faeb6712 -> 298e8d9191)

2024-01-23 Thread sajjad
This is an automated email from the ASF dual-hosted git repository. sajjad pushed a change to branch hotfix-theta-sketch in repository https://gitbox.apache.org/repos/asf/pinot.git omit 35faeb6712 Revert allowing tag override with upserts (#12311) omit eeaf1f0811 [feature] allow dim tabl

(pinot) branch hotfix-theta-sketch created (now 35faeb6712)

2024-01-23 Thread sajjad
This is an automated email from the ASF dual-hosted git repository. sajjad pushed a change to branch hotfix-theta-sketch in repository https://gitbox.apache.org/repos/asf/pinot.git at 35faeb6712 Revert allowing tag override with upserts (#12311) No new revisions were added by this update.

[PR] Backwards compatible theta sketch aggregation (#12288) [pinot]

2024-01-23 Thread via GitHub
SabrinaZhaozyf opened a new pull request, #12314: URL: https://github.com/apache/pinot/pull/12314 * Backwards compatible theta sketch aggregation Servers running on versions before upgrading Pinot to the ThetaSketchAccumulator would return Sketches directly to the merge function. Th

Re: [PR] Add HttpHeaders in broker event listener requestContext [pinot]

2024-01-23 Thread via GitHub
walterddr commented on code in PR #12258: URL: https://github.com/apache/pinot/pull/12258#discussion_r1463832651 ## pinot-spi/src/main/java/org/apache/pinot/spi/eventlistener/query/PinotBrokerQueryEventListenerFactory.java: ## @@ -44,6 +50,8 @@ private PinotBrokerQueryEventListe

Re: [I] Parsing http-headers in RequestContext - broker query event listener [pinot]

2024-01-23 Thread via GitHub
walterddr commented on issue #12273: URL: https://github.com/apache/pinot/issues/12273#issuecomment-1906705914 that's fine. looks like cluster config is saved in zk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [I] Parsing http-headers in RequestContext - broker query event listener [pinot]

2024-01-23 Thread via GitHub
tibrewalpratik17 commented on issue #12273: URL: https://github.com/apache/pinot/issues/12273#issuecomment-1906703892 for step 2 `requires restart`, we might have to store this info in ZK then, is that okay? -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] Reduce Disk Footprint for Segment Processor Framework to Avoid Out of Disk Issues [pinot]

2024-01-23 Thread via GitHub
aishikbh commented on code in PR #12220: URL: https://github.com/apache/pinot/pull/12220#discussion_r1463722709 ## pinot-core/src/test/java/org/apache/pinot/core/segment/processing/framework/SegmentProcessorFrameworkTest.java: ## @@ -55,10 +55,7 @@ import org.testng.annotations

Re: [PR] Revert allowing tag override with upserts [pinot]

2024-01-23 Thread via GitHub
Jackie-Jiang commented on PR #12311: URL: https://github.com/apache/pinot/pull/12311#issuecomment-1906580933 For tracking purpose, this is not reverting the whole PR, but addressing this [comment](https://github.com/apache/pinot/pull/12233#pullrequestreview-1831642018) -- This is an auto

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on PR #12223: URL: https://github.com/apache/pinot/pull/12223#issuecomment-1906539054 @somandal - can you also help take a look if you get a chance ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] Reduce Disk Footprint for Segment Processor Framework to Avoid Out of Disk Issues [pinot]

2024-01-23 Thread via GitHub
snleee commented on code in PR #12220: URL: https://github.com/apache/pinot/pull/12220#discussion_r1463645048 ## pinot-core/src/test/java/org/apache/pinot/core/segment/processing/framework/SegmentProcessorFrameworkTest.java: ## @@ -55,10 +55,7 @@ import org.testng.annotations.B

Re: [PR] [multistage][feature] FunctionRegistry unification [pinot]

2024-01-23 Thread via GitHub
walterddr commented on code in PR #12302: URL: https://github.com/apache/pinot/pull/12302#discussion_r1463618310 ## pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java: ## @@ -94,83 +154,195 @@ private FunctionRegistry() { public static void init

Re: [PR] [multistage][feature] FunctionRegistry unification [pinot]

2024-01-23 Thread via GitHub
walterddr commented on code in PR #12302: URL: https://github.com/apache/pinot/pull/12302#discussion_r1463618310 ## pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java: ## @@ -94,83 +154,195 @@ private FunctionRegistry() { public static void init

Re: [PR] [multistage][feature] FunctionRegistry unification [pinot]

2024-01-23 Thread via GitHub
walterddr commented on code in PR #12302: URL: https://github.com/apache/pinot/pull/12302#discussion_r1463616700 ## pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java: ## @@ -94,83 +154,195 @@ private FunctionRegistry() { public static void init

Re: [PR] [multistage][feature] FunctionRegistry unification [pinot]

2024-01-23 Thread via GitHub
walterddr commented on code in PR #12302: URL: https://github.com/apache/pinot/pull/12302#discussion_r1463614486 ## pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java: ## @@ -94,83 +154,195 @@ private FunctionRegistry() { public static void init

Re: [I] Parsing http-headers in RequestContext - broker query event listener [pinot]

2024-01-23 Thread via GitHub
walterddr commented on issue #12273: URL: https://github.com/apache/pinot/issues/12273#issuecomment-1906488940 so there's 3 - requires no restart --> pick up immediately on next query - requires restart - require changing the image (or modify the files shipped together with the de

Re: [PR] make thread local variable static to avoid the memory leaking issue [pinot]

2024-01-23 Thread via GitHub
wirybeaver commented on code in PR #12242: URL: https://github.com/apache/pinot/pull/12242#discussion_r1452592342 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/forward/VarByteChunkSVForwardIndexReader.java: ## @@ -85,6 +85,10 @@ privat

Re: [PR] Release all segments of a table in releaseAndRemoveAllSegments method [pinot]

2024-01-23 Thread via GitHub
somandal commented on code in PR #12297: URL: https://github.com/apache/pinot/pull/12297#discussion_r1463557078 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java: ## @@ -223,8 +224,12 @@ protected void releaseAndRemoveAllSegments() {

Re: [PR] Release all segments of a table in releaseAndRemoveAllSegments method [pinot]

2024-01-23 Thread via GitHub
somandal commented on code in PR #12297: URL: https://github.com/apache/pinot/pull/12297#discussion_r1463553215 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java: ## @@ -223,8 +224,21 @@ protected void releaseAndRemoveAllSegments() {

Re: [PR] [DNM] Increasing test coverage pinot-common [pinot]

2024-01-23 Thread via GitHub
codecov-commenter commented on PR #12294: URL: https://github.com/apache/pinot/pull/12294#issuecomment-1906346146 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12294?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] Adding support for configuration through environment variables [pinot]

2024-01-23 Thread via GitHub
t0mpere commented on code in PR #12307: URL: https://github.com/apache/pinot/pull/12307#discussion_r1463345813 ## pinot-spi/src/main/java/org/apache/pinot/spi/env/PinotConfiguration.java: ## @@ -201,16 +200,16 @@ private static Configuration loadProperties(String configPath) {

Re: [PR] Adding support for configuration through environment variables [pinot]

2024-01-23 Thread via GitHub
t0mpere commented on code in PR #12307: URL: https://github.com/apache/pinot/pull/12307#discussion_r1463286640 ## pinot-spi/src/main/java/org/apache/pinot/spi/env/PinotConfiguration.java: ## @@ -201,16 +200,16 @@ private static Configuration loadProperties(String configPath) {

Re: [PR] [multistage][feature] FunctionRegistry unification [pinot]

2024-01-23 Thread via GitHub
gortiz commented on code in PR #12302: URL: https://github.com/apache/pinot/pull/12302#discussion_r1463232171 ## pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java: ## @@ -94,83 +154,195 @@ private FunctionRegistry() { public static void init()

Re: [PR] [multistage][feature] FunctionRegistry unification [pinot]

2024-01-23 Thread via GitHub
gortiz commented on code in PR #12302: URL: https://github.com/apache/pinot/pull/12302#discussion_r1463218781 ## pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java: ## @@ -94,83 +154,195 @@ private FunctionRegistry() { public static void init()

Re: [PR] [multistage][feature] FunctionRegistry unification [pinot]

2024-01-23 Thread via GitHub
gortiz commented on code in PR #12302: URL: https://github.com/apache/pinot/pull/12302#discussion_r1463217848 ## pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java: ## @@ -94,83 +154,195 @@ private FunctionRegistry() { public static void init()

Re: [PR] [multistage][feature] FunctionRegistry unification [pinot]

2024-01-23 Thread via GitHub
gortiz commented on code in PR #12302: URL: https://github.com/apache/pinot/pull/12302#discussion_r1463164751 ## pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java: ## @@ -68,22 +83,67 @@ private FunctionRegistry() { } ScalarFunction

Re: [PR] [multistage][feature] FunctionRegistry unification [pinot]

2024-01-23 Thread via GitHub
gortiz commented on code in PR #12302: URL: https://github.com/apache/pinot/pull/12302#discussion_r1463164751 ## pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java: ## @@ -68,22 +83,67 @@ private FunctionRegistry() { } ScalarFunction

Re: [I] Parsing http-headers in RequestContext - broker query event listener [pinot]

2024-01-23 Thread via GitHub
tibrewalpratik17 commented on issue #12273: URL: https://github.com/apache/pinot/issues/12273#issuecomment-1905862904 > i think an allowed list would be desirable. but it is a bit hard to make it configurable/extensible without overriding it. do we have any idea on how one can extend this t

Re: [I] Data getting Truncated when using BigDecimal datatype [pinot]

2024-01-23 Thread via GitHub
skrishnamurthi-zuora commented on issue #12312: URL: https://github.com/apache/pinot/issues/12312#issuecomment-1905853014 Adding more on to this.. for a decimal numbers, it supports only 16 digits excluding decimal point and truncates the digits more than that https://github.com/apache/pi

Re: [PR] Add HttpHeaders in broker event listener requestContext [pinot]

2024-01-23 Thread via GitHub
tibrewalpratik17 commented on code in PR #12258: URL: https://github.com/apache/pinot/pull/12258#discussion_r1463119339 ## pinot-spi/src/main/java/org/apache/pinot/spi/eventlistener/query/PinotBrokerQueryEventListenerFactory.java: ## @@ -44,6 +50,8 @@ private PinotBrokerQueryEve

Re: [I] Error while installing using mvn clean install -DskipTests -Pbin-dist [pinot]

2024-01-23 Thread via GitHub
nakul777 commented on issue #11836: URL: https://github.com/apache/pinot/issues/11836#issuecomment-1905748147 > Hi @nakul777 have you made it work on windows? The previous logs seem to be correct, just need more time to compile it Upon executing this command `mvn clean install

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
gortiz commented on code in PR #12223: URL: https://github.com/apache/pinot/pull/12223#discussion_r1463005976 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/FALFInterner.java: ## @@ -0,0 +1,148 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Add the possibility of configuring ForwardIndexes with compressionCodec [pinot]

2024-01-23 Thread via GitHub
gortiz commented on code in PR #12218: URL: https://github.com/apache/pinot/pull/12218#discussion_r1462968861 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/ForwardIndexConfig.java: ## @@ -99,6 +171,8 @@ public int hashCode() { } public static clas

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on PR #12223: URL: https://github.com/apache/pinot/pull/12223#issuecomment-1905611589 How does the concurrency aspect come into picture during query processing / segment reloads ? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on code in PR #12223: URL: https://github.com/apache/pinot/pull/12223#discussion_r1462867575 ## pinot-common/src/test/java/org/apache/pinot/common/utils/FALFInternerTest.java: ## @@ -0,0 +1,168 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[I] Pinot docker uploading csv data issue. [pinot]

2024-01-23 Thread via GitHub
nowrin-pentabd opened a new issue, #12313: URL: https://github.com/apache/pinot/issues/12313 I have deployed pinot using docker in my local machine. I am using the transcript table and data you have provided in your doc. [](https://docs.pinot.apache.org/basics/data-import/batch-ingestion) .

Re: [PR] Support table suffix in ZkBasicAuthAccessControlFactory [pinot]

2024-01-23 Thread via GitHub
INNOCENT-BOY commented on PR #12310: URL: https://github.com/apache/pinot/pull/12310#issuecomment-1905591585 The previous assumption during the RBAC (Role-Based Access Control) development was that regular users (referring to users without permissions for all tables) should be agnostic to t

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on code in PR #12223: URL: https://github.com/apache/pinot/pull/12223#discussion_r1462929728 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/FALFInterner.java: ## @@ -0,0 +1,148 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on code in PR #12223: URL: https://github.com/apache/pinot/pull/12223#discussion_r1462920239 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/FALFInterner.java: ## @@ -0,0 +1,148 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on code in PR #12223: URL: https://github.com/apache/pinot/pull/12223#discussion_r1462921741 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/FALFInterner.java: ## @@ -0,0 +1,148 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on code in PR #12223: URL: https://github.com/apache/pinot/pull/12223#discussion_r1462920239 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/FALFInterner.java: ## @@ -0,0 +1,148 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on PR #12223: URL: https://github.com/apache/pinot/pull/12223#issuecomment-1905549151 (nit) Please pretty print the sample config used in PR description for readability -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on PR #12223: URL: https://github.com/apache/pinot/pull/12223#issuecomment-1905535717 Haven't gone through all the tests yet so asking if we have covered the following - Correctness test for writing and reading from a Dictionary with this feature enabled

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on code in PR #12223: URL: https://github.com/apache/pinot/pull/12223#discussion_r1462876804 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/DictionaryIndexConfig.java: ## @@ -21,29 +21,46 @@ import com.fasterxml.jackson.annotat

Re: [PR] Support table suffix in ZkBasicAuthAccessControlFactory [pinot]

2024-01-23 Thread via GitHub
codecov-commenter commented on PR #12310: URL: https://github.com/apache/pinot/pull/12310#issuecomment-1905505133 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12310?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on code in PR #12223: URL: https://github.com/apache/pinot/pull/12223#discussion_r1462871078 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/dictionary/DictionaryIndexType.java: ## @@ -308,7 +321,20 @@ public static Dic

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on code in PR #12223: URL: https://github.com/apache/pinot/pull/12223#discussion_r1462869573 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/dictionary/DictionaryIndexType.java: ## @@ -308,7 +321,20 @@ public static Dic

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on code in PR #12223: URL: https://github.com/apache/pinot/pull/12223#discussion_r1462867575 ## pinot-common/src/test/java/org/apache/pinot/common/utils/FALFInternerTest.java: ## @@ -0,0 +1,168 @@ +/** + * Licensed to the Apache Software Foundation (ASF

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on code in PR #12223: URL: https://github.com/apache/pinot/pull/12223#discussion_r1462862679 ## pinot-common/src/test/java/org/apache/pinot/common/utils/FALFInternerTest.java: ## @@ -0,0 +1,168 @@ +/** + * Licensed to the Apache Software Foundation (ASF

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on code in PR #12223: URL: https://github.com/apache/pinot/pull/12223#discussion_r1462861826 ## pinot-common/src/test/java/org/apache/pinot/common/utils/FALFInternerTest.java: ## @@ -0,0 +1,168 @@ +/** + * Licensed to the Apache Software Foundation (ASF

Re: [PR] Reduce Heap Usage of OnHeapStringDictionary [pinot]

2024-01-23 Thread via GitHub
siddharthteotia commented on code in PR #12223: URL: https://github.com/apache/pinot/pull/12223#discussion_r1462857587 ## pinot-common/src/test/java/org/apache/pinot/common/utils/FALFInternerTest.java: ## @@ -0,0 +1,168 @@ +/** + * Licensed to the Apache Software Foundation (ASF