[pinot] branch master updated: Index spi: all types (#10193)

2023-03-27 Thread kharekartik
This is an automated email from the ASF dual-hosted git repository. kharekartik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 1d02d0eb50 Index spi: all types (#10193) 1d0

[GitHub] [pinot] KKcorps merged pull request #10193: Index spi: all types

2023-03-27 Thread via GitHub
KKcorps merged PR #10193: URL: https://github.com/apache/pinot/pull/10193 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

[pinot] branch master updated: Add JOIN support to PinotQuery (#10421)

2023-03-27 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 38211a6c79 Add JOIN support to PinotQuery (#10421

[GitHub] [pinot] Jackie-Jiang merged pull request #10421: Add JOIN support to PinotQuery

2023-03-27 Thread via GitHub
Jackie-Jiang merged PR #10421: URL: https://github.com/apache/pinot/pull/10421 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] 61yao commented on a diff in pull request #10488: [multistage] Adding Stage node support for CorrelateNode and UncollectNode

2023-03-27 Thread via GitHub
61yao commented on code in PR #10488: URL: https://github.com/apache/pinot/pull/10488#discussion_r1150069445 ## pinot-query-planner/src/main/java/org/apache/pinot/query/planner/stage/CorrelateNode.java: ## @@ -0,0 +1,66 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] Vaas-Arora commented on issue #10458: Add API to update ZK metadata of a segment

2023-03-27 Thread via GitHub
Vaas-Arora commented on issue #10458: URL: https://github.com/apache/pinot/issues/10458#issuecomment-1486253151 Hi, can I pick this up? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [pinot] codecov-commenter commented on pull request #10490: Fix shading in spark2 connector pom file

2023-03-27 Thread via GitHub
codecov-commenter commented on PR #10490: URL: https://github.com/apache/pinot/pull/10490#issuecomment-1486236718 ## [Codecov](https://codecov.io/gh/apache/pinot/pull/10490?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Sof

[GitHub] [pinot] cbalci opened a new pull request, #10490: Fix shading in spark2 connector pom file

2023-03-27 Thread via GitHub
cbalci opened a new pull request, #10490: URL: https://github.com/apache/pinot/pull/10490 Adding shading back to the Spark2 Connector pom file for google `common` and `protobuf` libraries, which used to exist but was deleted by mistake in a previous refactoring. These libraries are commo

[GitHub] [pinot] jadami10 commented on a diff in pull request #10444: optimize queries where lhs and rhs of predicate are equal

2023-03-27 Thread via GitHub
jadami10 commented on code in PR #10444: URL: https://github.com/apache/pinot/pull/10444#discussion_r1149974687 ## pinot-core/src/main/java/org/apache/pinot/core/query/optimizer/filter/BaseAndOrBooleanFilterOptimizer.java: ## @@ -0,0 +1,141 @@ +/** + * Licensed to the Apache Sof

[pinot] branch master updated (f136a08831 -> a7b2b67361)

2023-03-27 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from f136a08831 Remove converted H2 type NUMERIC(200, 100) from BIG_DECIMAL (#10483) add a7b2b67361 Fix jackson depen

[GitHub] [pinot] xiangfu0 merged pull request #10477: [cleanup] Remove old `org.codehaus.jackson:*` jackson dependencies

2023-03-27 Thread via GitHub
xiangfu0 merged PR #10477: URL: https://github.com/apache/pinot/pull/10477 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[GitHub] [pinot] codecov-commenter commented on pull request #10489: Clean up temp consuming segment files during server start

2023-03-27 Thread via GitHub
codecov-commenter commented on PR #10489: URL: https://github.com/apache/pinot/pull/10489#issuecomment-1486076618 ## [Codecov](https://codecov.io/gh/apache/pinot/pull/10489?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Sof

[GitHub] [pinot] swaminathanmanish commented on issue #10470: Let's make a generic "jobs" UI in the table page

2023-03-27 Thread via GitHub
swaminathanmanish commented on issue #10470: URL: https://github.com/apache/pinot/issues/10470#issuecomment-1486049923 Yes @jadami10. We need to revisit the UI for this. @Jackie-Jiang and I were talking about this as as well. We may need a drop-down for the list of jobTypes that the user ca

[GitHub] [pinot] Jackie-Jiang opened a new pull request, #10489: Clean up temp consuming segment files during server start

2023-03-27 Thread via GitHub
Jackie-Jiang opened a new pull request, #10489: URL: https://github.com/apache/pinot/pull/10489 When consuming segment is not properly committed, the temporary segment files will be left on the server disk. This PR cleans them up during server start, and also clean up the empty table dat

[GitHub] [pinot] codecov-commenter commented on pull request #10487: Fix Bottleneck for Server Bootstrap by Making maxConnsPerRoute Configurable

2023-03-27 Thread via GitHub
codecov-commenter commented on PR #10487: URL: https://github.com/apache/pinot/pull/10487#issuecomment-1486045217 ## [Codecov](https://codecov.io/gh/apache/pinot/pull/10487?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Sof

[GitHub] [pinot] kirkrodrigues commented on issue #10478: Boolean expression on top of a MV column filter fails

2023-03-27 Thread via GitHub
kirkrodrigues commented on issue #10478: URL: https://github.com/apache/pinot/issues/10478#issuecomment-1486035883 Hey Jackie, yeah, the query is contrived to expose the bug. I actually encountered it while writing a query rewriter: 1. `SELECT * FROM table WHERE clpMatch("message", '*

[GitHub] [pinot] Jackie-Jiang commented on issue #10478: Boolean expression on top of a MV column filter fails

2023-03-27 Thread via GitHub
Jackie-Jiang commented on issue #10478: URL: https://github.com/apache/pinot/issues/10478#issuecomment-1486027001 This query fails because `mvCol = 'test'` is treated as a transform function (instead of filter predicate), and currently `BinaryOperatorTransformFunction` doesn't support MV as

[GitHub] [pinot] soumitra-st commented on a diff in pull request #10465: Provide all stats metadata to queries run by pinot-java-client

2023-03-27 Thread via GitHub
soumitra-st commented on code in PR #10465: URL: https://github.com/apache/pinot/pull/10465#discussion_r1149897959 ## pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/ExecutionStats.java: ## @@ -115,20 +113,6 @@ public long getTimeUsedMs() { @Override

[GitHub] [pinot] ankitsultana opened a new pull request, #10487: Fix Bottleneck for Server Bootstrap by Making maxConnsPerRoute Configurable

2023-03-27 Thread via GitHub
ankitsultana opened a new pull request, #10487: URL: https://github.com/apache/pinot/pull/10487 At present we don't set any configs when we create the HttpClient in `FileUploadDownloadClient`. By default, Apache Commons HttpClient uses maxTotalConns=20 and maxConnsPerRoute=2 [ref](https://

[GitHub] [pinot] abhioncbr commented on a diff in pull request #10446: 10218: Added log statement for jvm options.

2023-03-27 Thread via GitHub
abhioncbr commented on code in PR #10446: URL: https://github.com/apache/pinot/pull/10446#discussion_r1149890188 ## pinot-broker/src/main/java/org/apache/pinot/broker/broker/helix/BaseBrokerStarter.java: ## @@ -223,6 +224,7 @@ public PinotConfiguration getConfig() { public vo

[GitHub] [pinot] mcvsubbu commented on pull request #10475: Fix the double destroy of segment data manager during server shutdown

2023-03-27 Thread via GitHub
mcvsubbu commented on PR #10475: URL: https://github.com/apache/pinot/pull/10475#issuecomment-1486006528 > I found the issue by this CI test failure: https://github.com/apache/pinot/actions/runs/4514162615/jobs/7949914590 > > ``` > Error: org.apache.pinot.integration.tests.Contro

[GitHub] [pinot] Jackie-Jiang commented on pull request #10475: Fix the double destroy of segment data manager during server shutdown

2023-03-27 Thread via GitHub
Jackie-Jiang commented on PR #10475: URL: https://github.com/apache/pinot/pull/10475#issuecomment-1486001968 I found the issue by this CI test failure: https://github.com/apache/pinot/actions/runs/4514162615/jobs/7949914590 ``` Error: org.apache.pinot.integration.tests.Controller

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10465: Provide all stats metadata to queries run by pinot-java-client

2023-03-27 Thread via GitHub
Jackie-Jiang commented on code in PR #10465: URL: https://github.com/apache/pinot/pull/10465#discussion_r1149883338 ## pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/ExecutionStats.java: ## @@ -115,20 +113,6 @@ public long getTimeUsedMs() { @Override

[GitHub] [pinot] Jackie-Jiang commented on issue #10470: Let's make a generic "jobs" UI in the table page

2023-03-27 Thread via GitHub
Jackie-Jiang commented on issue #10470: URL: https://github.com/apache/pinot/issues/10470#issuecomment-1485993954 cc @swaminathanmanish -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10444: optimize queries where lhs and rhs of predicate are equal

2023-03-27 Thread via GitHub
Jackie-Jiang commented on code in PR #10444: URL: https://github.com/apache/pinot/pull/10444#discussion_r1149869941 ## pinot-core/src/main/java/org/apache/pinot/core/query/optimizer/filter/BaseAndOrBooleanFilterOptimizer.java: ## @@ -0,0 +1,141 @@ +/** + * Licensed to the Apache

[GitHub] [pinot] Jackie-Jiang commented on issue #10486: Segments not properly deleted when dropping then recreating a LLC table

2023-03-27 Thread via GitHub
Jackie-Jiang commented on issue #10486: URL: https://github.com/apache/pinot/issues/10486#issuecomment-1485961817 cc @mcvsubbu @sajjad-moradi @navina Any idea what could potentially cause this, or have you encountered similar issues? I'm trying to find what could potentially hold the segmen

[GitHub] [pinot] Jackie-Jiang opened a new issue, #10486: Segments not properly deleted when dropping then recreating a LLC table

2023-03-27 Thread via GitHub
Jackie-Jiang opened a new issue, #10486: URL: https://github.com/apache/pinot/issues/10486 When dropping then recreating a relatively large LLC table, we have observed several issues: - `HelixInstanceDataManager.deleteTable()` will time out because external view is recreated, thus will n

[GitHub] [pinot] jugomezv commented on a diff in pull request #10418: Fix ramping delay caused by long lasting sequence of unfiltered messa…

2023-03-27 Thread via GitHub
jugomezv commented on code in PR #10418: URL: https://github.com/apache/pinot/pull/10418#discussion_r1149851292 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/LLRealtimeSegmentDataManager.java: ## @@ -618,14 +618,18 @@ private boolean processStreamEvents

[GitHub] [pinot] ankitsultana commented on pull request #10463: Segment compaction for upsert real-time tables

2023-03-27 Thread via GitHub
ankitsultana commented on PR #10463: URL: https://github.com/apache/pinot/pull/10463#issuecomment-1485942210 Is there a short design doc for this? It'd be great if we can call out some of the design decisions like when we consider a segment for compaction, how we ensure there's no downtime,

[GitHub] [pinot] cbalci commented on issue #6420: [discuss]integrating Apache DataSketches library

2023-03-27 Thread via GitHub
cbalci commented on issue #6420: URL: https://github.com/apache/pinot/issues/6420#issuecomment-1485930462 Created a proposal document to add some of these sketches: https://docs.google.com/document/d/1ctmKVRi67lpO6x1RYKDvDYf05EZx2Vbs2OnUudYP-bU/edit -- This is an automated message from th

[GitHub] [pinot] dependabot[bot] opened a new pull request, #10485: Bump redis from 3.4.1 to 4.5.3 in /docker/images/pinot-superset

2023-03-27 Thread via GitHub
dependabot[bot] opened a new pull request, #10485: URL: https://github.com/apache/pinot/pull/10485 Bumps [redis](https://github.com/redis/redis-py) from 3.4.1 to 4.5.3. Release notes Sourced from https://github.com/redis/redis-py/releases";>redis's releases. 4.5.3 Changes

[pinot] branch dependabot/pip/docker/images/pinot-superset/redis-4.5.3 created (now 60f4dcf4da)

2023-03-27 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/pip/docker/images/pinot-superset/redis-4.5.3 in repository https://gitbox.apache.org/repos/asf/pinot.git at 60f4dcf4da Bump redis from 3.4.1 to 4.5.3 in /docker/images/pino

[GitHub] [pinot] jugomezv commented on a diff in pull request #10418: Fix ramping delay caused by long lasting sequence of unfiltered messa…

2023-03-27 Thread via GitHub
jugomezv commented on code in PR #10418: URL: https://github.com/apache/pinot/pull/10418#discussion_r1149826701 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/IngestionDelayTracker.java: ## @@ -214,12 +214,20 @@ public void updateIngestionDelay(long inge

[GitHub] [pinot] claudeyj commented on pull request #10445: Replace HashMap with LinkedHashMap to avoid flaky tests

2023-03-27 Thread via GitHub
claudeyj commented on PR #10445: URL: https://github.com/apache/pinot/pull/10445#issuecomment-1485908674 @Jackie-Jiang created a new PR (https://github.com/apache/pinot/pull/10468) regarding this issue https://github.com/apache/pinot/issues/10442, he said the performance won't be affect muc

[GitHub] [pinot] claudeyj commented on pull request #10445: Replace HashMap with LinkedHashMap to avoid flaky tests

2023-03-27 Thread via GitHub
claudeyj commented on PR #10445: URL: https://github.com/apache/pinot/pull/10445#issuecomment-1485907687 @Jackie-Jiang created a new PR (https://github.com/apache/pinot/pull/10468) regarding this issue https://github.com/apache/pinot/issues/10442, he said the performance won't be affect muc

[GitHub] [pinot] jugomezv commented on a diff in pull request #10418: Fix ramping delay caused by long lasting sequence of unfiltered messa…

2023-03-27 Thread via GitHub
jugomezv commented on code in PR #10418: URL: https://github.com/apache/pinot/pull/10418#discussion_r1149823454 ## pinot-spi/src/main/java/org/apache/pinot/spi/stream/MessageBatch.java: ## @@ -116,4 +116,14 @@ default StreamPartitionMsgOffset getOffsetOfNextBatch() { default

[GitHub] [pinot] jugomezv commented on a diff in pull request #10418: Fix ramping delay caused by long lasting sequence of unfiltered messa…

2023-03-27 Thread via GitHub
jugomezv commented on code in PR #10418: URL: https://github.com/apache/pinot/pull/10418#discussion_r1149820316 ## pinot-spi/src/main/java/org/apache/pinot/spi/stream/MessageBatch.java: ## @@ -116,4 +116,14 @@ default StreamPartitionMsgOffset getOffsetOfNextBatch() { default

[pinot] branch master updated: Remove converted H2 type NUMERIC(200, 100) from BIG_DECIMAL (#10483)

2023-03-27 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new f136a08831 Remove converted H2 type NUMERIC(200,

[GitHub] [pinot] xiangfu0 merged pull request #10483: [Test] Revert H2 data type mapping from Pinot BIG_DECIMAL to NUMERIC(1200, 600)

2023-03-27 Thread via GitHub
xiangfu0 merged PR #10483: URL: https://github.com/apache/pinot/pull/10483 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[GitHub] [pinot] jadami10 commented on pull request #10479: wait for full segment commit protocol on force commit

2023-03-27 Thread via GitHub
jadami10 commented on PR #10479: URL: https://github.com/apache/pinot/pull/10479#issuecomment-1485886850 @Jackie-Jiang , do you mind rerunning the failed workflows? looks like a flake -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [pinot] Jackie-Jiang closed issue #9783: API for checking status of forceCommit for realtime tables

2023-03-27 Thread via GitHub
Jackie-Jiang closed issue #9783: API for checking status of forceCommit for realtime tables URL: https://github.com/apache/pinot/issues/9783 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10446: 10218: Added log statement for jvm options.

2023-03-27 Thread via GitHub
Jackie-Jiang commented on code in PR #10446: URL: https://github.com/apache/pinot/pull/10446#discussion_r1149798237 ## pinot-tools/src/main/java/org/apache/pinot/tools/service/PinotServiceManager.java: ## @@ -107,6 +108,7 @@ public String startRole(ServiceRole role, Map properti

[pinot] branch master updated: Adding fields to enable/disable dictionary optimization. (#10484)

2023-03-27 Thread mayanks
This is an automated email from the ASF dual-hosted git repository. mayanks pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new fd08b906c0 Adding fields to enable/disable dicti

[GitHub] [pinot] Jackie-Jiang commented on issue #10457: DATETIMECONVERT function not support pass time offset as parameter, e.g. tz(-09:00)

2023-03-27 Thread via GitHub
Jackie-Jiang commented on issue #10457: URL: https://github.com/apache/pinot/issues/10457#issuecomment-1485863687 We are using the joda library to handle the time conversion, and per the joda [doc](https://www.joda.org/joda-time/timezones.html), `America/Adak` is equivalent to `-10:00`.

[GitHub] [pinot] mayankshriv merged pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
mayankshriv merged PR #10484: URL: https://github.com/apache/pinot/pull/10484 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.

[GitHub] [pinot] swaminathanmanish commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
swaminathanmanish commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149744978 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/builder/TableConfigBuilder.java: ## @@ -256,6 +258,16 @@ public TableConfigBuilder setInvertedIndexColu

[GitHub] [pinot] swaminathanmanish commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
swaminathanmanish commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149744978 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/builder/TableConfigBuilder.java: ## @@ -256,6 +258,16 @@ public TableConfigBuilder setInvertedIndexColu

[GitHub] [pinot] swaminathanmanish commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
swaminathanmanish commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149744978 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/builder/TableConfigBuilder.java: ## @@ -256,6 +258,16 @@ public TableConfigBuilder setInvertedIndexColu

[GitHub] [pinot] navina commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
navina commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149714646 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/builder/TableConfigBuilder.java: ## @@ -256,6 +258,16 @@ public TableConfigBuilder setInvertedIndexColumns(List in

[GitHub] [pinot] swaminathanmanish commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
swaminathanmanish commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149705462 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/builder/TableConfigBuilder.java: ## @@ -103,6 +103,8 @@ public class TableConfigBuilder { private Lis

[GitHub] [pinot] swaminathanmanish commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
swaminathanmanish commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149704765 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/builder/TableConfigBuilder.java: ## @@ -256,6 +258,16 @@ public TableConfigBuilder setInvertedIndexColu

[GitHub] [pinot] codecov-commenter commented on pull request #10483: [Test] Revert H2 data type mapping from Pinot BIG_DECIMAL to NUMERIC(1200, 600)

2023-03-27 Thread via GitHub
codecov-commenter commented on PR #10483: URL: https://github.com/apache/pinot/pull/10483#issuecomment-1485744302 ## [Codecov](https://codecov.io/gh/apache/pinot/pull/10483?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Sof

[GitHub] [pinot] swaminathanmanish commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
swaminathanmanish commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149700724 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/builder/TableConfigBuilder.java: ## @@ -256,6 +258,16 @@ public TableConfigBuilder setInvertedIndexColu

[GitHub] [pinot] navina commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
navina commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149684929 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/builder/TableConfigBuilder.java: ## @@ -256,6 +258,16 @@ public TableConfigBuilder setInvertedIndexColumns(List in

[GitHub] [pinot] swaminathanmanish commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
swaminathanmanish commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149684704 ## pinot-common/src/test/java/org/apache/pinot/common/utils/config/TableConfigSerDeTest.java: ## @@ -64,7 +64,13 @@ public class TableConfigSerDeTest { -

[GitHub] [pinot] swaminathanmanish commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
swaminathanmanish commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149684522 ## pinot-common/src/test/java/org/apache/pinot/common/utils/config/TableConfigSerDeTest.java: ## @@ -64,7 +64,13 @@ public class TableConfigSerDeTest { -

[GitHub] [pinot] snleee commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
snleee commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149670583 ## pinot-common/src/test/java/org/apache/pinot/common/utils/config/TableConfigSerDeTest.java: ## @@ -64,7 +64,13 @@ public class TableConfigSerDeTest { - + @Test +

[GitHub] [pinot] snleee commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
snleee commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149670583 ## pinot-common/src/test/java/org/apache/pinot/common/utils/config/TableConfigSerDeTest.java: ## @@ -64,7 +64,13 @@ public class TableConfigSerDeTest { - + @Test +

[GitHub] [pinot] snleee commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
snleee commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149670583 ## pinot-common/src/test/java/org/apache/pinot/common/utils/config/TableConfigSerDeTest.java: ## @@ -64,7 +64,13 @@ public class TableConfigSerDeTest { - + @Test +

[GitHub] [pinot] snleee commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
snleee commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149668917 ## pinot-common/src/test/java/org/apache/pinot/common/utils/config/TableConfigSerDeTest.java: ## @@ -64,7 +64,13 @@ public class TableConfigSerDeTest { - + @Test +

[GitHub] [pinot] swaminathanmanish commented on pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
swaminathanmanish commented on PR #10484: URL: https://github.com/apache/pinot/pull/10484#issuecomment-1485696092 > TableConfigSerDeTest Added a test. Thanks ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [pinot] snleee commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
snleee commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149664272 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/builder/TableConfigBuilder.java: ## @@ -256,6 +258,15 @@ public TableConfigBuilder setInvertedIndexColumns(List in

[GitHub] [pinot] snleee commented on a diff in pull request #10484: Adding fields in TableConfigBuilder to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
snleee commented on code in PR #10484: URL: https://github.com/apache/pinot/pull/10484#discussion_r1149664272 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/builder/TableConfigBuilder.java: ## @@ -256,6 +258,15 @@ public TableConfigBuilder setInvertedIndexColumns(List in

[GitHub] [pinot] codecov-commenter commented on pull request #10481: [refactor] add physical planner

2023-03-27 Thread via GitHub
codecov-commenter commented on PR #10481: URL: https://github.com/apache/pinot/pull/10481#issuecomment-1485672309 ## [Codecov](https://codecov.io/gh/apache/pinot/pull/10481?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Sof

[GitHub] [pinot] swaminathanmanish commented on pull request #10484: Adding fields to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
swaminathanmanish commented on PR #10484: URL: https://github.com/apache/pinot/pull/10484#issuecomment-1485667064 @snleee - Please take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [pinot] swaminathanmanish opened a new pull request, #10484: Adding fields to enable/disable dictionary optimization for metric columns

2023-03-27 Thread via GitHub
swaminathanmanish opened a new pull request, #10484: URL: https://github.com/apache/pinot/pull/10484 This [PR](https://github.com/apache/pinot/pull/8398) introduced index config options to enable/disable dictionary optimization for metrics based on a custom threshold (to enable/disable dict

[GitHub] [pinot] xiangfu0 opened a new pull request, #10483: Remove converted H2 type NUMERIC(200, 100) from BIG_DECIMAL

2023-03-27 Thread via GitHub
xiangfu0 opened a new pull request, #10483: URL: https://github.com/apache/pinot/pull/10483 fix the tests. https://github.com/apache/pinot/issues/10482 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [pinot] walterddr opened a new issue, #10482: [test regression]

2023-03-27 Thread via GitHub
walterddr opened a new issue, #10482: URL: https://github.com/apache/pinot/issues/10482 related to #10456 , test speed is 1000 slower after H2 upgrade See: https://github.com/apache/pinot/actions/runs/4485055239/jobs/7886201102 https://github.com/apache/pinot/actions/runs/4492805

[GitHub] [pinot] walterddr opened a new pull request, #10481: [refactor] add physical planner

2023-03-27 Thread via GitHub
walterddr opened a new pull request, #10481: URL: https://github.com/apache/pinot/pull/10481 refactor post logical plan to a single global physical planner Details === - PhysicalPlanContext to include QueryPlan and other peripherals needed to add parallelism & worker assignment

[GitHub] [pinot] Jackie-Jiang merged pull request #10453: Fix null handling in streaming selection operator

2023-03-27 Thread via GitHub
Jackie-Jiang merged PR #10453: URL: https://github.com/apache/pinot/pull/10453 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[pinot] branch master updated (a44d5354aa -> 549d565ddd)

2023-03-27 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from a44d5354aa [feature] [backward-incompat] [null support # 2] Preserve null literal information in literal context and l

[GitHub] [pinot] klsince commented on a diff in pull request #10184: Complete index spi

2023-03-27 Thread via GitHub
klsince commented on code in PR #10184: URL: https://github.com/apache/pinot/pull/10184#discussion_r1149557871 ## pinot-compatibility-verifier/pom.xml: ## @@ -92,6 +92,11 @@ ${project.version} test-jar + + org.testng + testng Review Comment:

[GitHub] [pinot] walterddr merged pull request #10380: [feature] [backward-incompat] [null support # 2] Preserve null literal information in literal context and literal transform

2023-03-27 Thread via GitHub
walterddr merged PR #10380: URL: https://github.com/apache/pinot/pull/10380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

[pinot] branch master updated (0e6c386094 -> a44d5354aa)

2023-03-27 Thread rongr
This is an automated email from the ASF dual-hosted git repository. rongr pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 0e6c386094 Upgraded joda-time library version from 2.10.5 to 2.11.2 (#10474) add a44d5354aa [feature] [backward-in

[GitHub] [pinot] gortiz commented on a diff in pull request #10184: Complete index spi

2023-03-27 Thread via GitHub
gortiz commented on code in PR #10184: URL: https://github.com/apache/pinot/pull/10184#discussion_r1149369006 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/text/NativeTextIndexCreator.java: ## @@ -73,7 +75,8 @@ public class NativeTextIn

[GitHub] [pinot] gortiz commented on a diff in pull request #10184: Complete index spi

2023-03-27 Thread via GitHub
gortiz commented on code in PR #10184: URL: https://github.com/apache/pinot/pull/10184#discussion_r1149366291 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/inv/geospatial/BaseH3IndexCreator.java: ## @@ -101,6 +102,11 @@ public abstract

[GitHub] [pinot] gortiz commented on a diff in pull request #10184: Complete index spi

2023-03-27 Thread via GitHub
gortiz commented on code in PR #10184: URL: https://github.com/apache/pinot/pull/10184#discussion_r1149359436 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/fwd/SameValueForwardIndexCreator.java: ## @@ -0,0 +1,137 @@ +/** + * Licensed to

[GitHub] [pinot] gortiz commented on a diff in pull request #10184: Complete index spi

2023-03-27 Thread via GitHub
gortiz commented on code in PR #10184: URL: https://github.com/apache/pinot/pull/10184#discussion_r1149356183 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/fwd/AbstractForwardIndexCreator.java: ## @@ -16,10 +16,16 @@ * specific langua

[GitHub] [pinot] gortiz commented on a diff in pull request #10184: Complete index spi

2023-03-27 Thread via GitHub
gortiz commented on code in PR #10184: URL: https://github.com/apache/pinot/pull/10184#discussion_r1149064068 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java: ## @@ -379,340 +330,94 @@ private boolean sho

[GitHub] [pinot] gortiz commented on a diff in pull request #10184: Complete index spi

2023-03-27 Thread via GitHub
gortiz commented on code in PR #10184: URL: https://github.com/apache/pinot/pull/10184#discussion_r1149056600 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java: ## @@ -133,183 +122,144 @@ public void init(Se

[GitHub] [pinot] gortiz commented on a diff in pull request #10184: Complete index spi

2023-03-27 Thread via GitHub
gortiz commented on code in PR #10184: URL: https://github.com/apache/pinot/pull/10184#discussion_r1149052802 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java: ## @@ -133,183 +122,144 @@ public void init(Se

[GitHub] [pinot] gortiz commented on a diff in pull request #10184: Complete index spi

2023-03-27 Thread via GitHub
gortiz commented on code in PR #10184: URL: https://github.com/apache/pinot/pull/10184#discussion_r1149038820 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentDictionaryCreator.java: ## @@ -62,9 +64,20 @@ public class SegmentDiction

[GitHub] [pinot] gortiz commented on a diff in pull request #10184: Complete index spi

2023-03-27 Thread via GitHub
gortiz commented on code in PR #10184: URL: https://github.com/apache/pinot/pull/10184#discussion_r1148962745 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/JsonIndexConfig.java: ## @@ -38,14 +39,25 @@ * to be excluded. * - excludeFields: Exclud

[GitHub] [pinot] gortiz commented on a diff in pull request #10184: Complete index spi

2023-03-27 Thread via GitHub
gortiz commented on code in PR #10184: URL: https://github.com/apache/pinot/pull/10184#discussion_r1148961507 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/creator/H3IndexConfig.java: ## @@ -18,20 +18,38 @@ */ package org.apache.pinot.segment.spi.index

[GitHub] [pinot] gortiz commented on a diff in pull request #10184: Complete index spi

2023-03-27 Thread via GitHub
gortiz commented on code in PR #10184: URL: https://github.com/apache/pinot/pull/10184#discussion_r1148959765 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java: ## @@ -133,183 +122,144 @@ public void init(Se

[GitHub] [pinot] saurabhd336 opened a new pull request, #10480: Fix FS props handling when using /ingestFromUri

2023-03-27 Thread via GitHub
saurabhd336 opened a new pull request, #10480: URL: https://github.com/apache/pinot/pull/10480 Doubling stripping of "input.fs.prop" prefix leads to FS props not being parsed correctly when using /ingestFromUri -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [pinot] gortiz commented on a diff in pull request #10184: Complete index spi

2023-03-27 Thread via GitHub
gortiz commented on code in PR #10184: URL: https://github.com/apache/pinot/pull/10184#discussion_r1148868990 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/converter/RealtimeSegmentConverter.java: ## @@ -75,9 +81,8 @@ public void build(@Nullable Se

[GitHub] [pinot] gortiz commented on a diff in pull request #10184: Complete index spi

2023-03-27 Thread via GitHub
gortiz commented on code in PR #10184: URL: https://github.com/apache/pinot/pull/10184#discussion_r1148867232 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/immutable/ImmutableSegmentLoader.java: ## @@ -166,6 +165,8 @@ public static ImmutableSeg