[GitHub] [pinot] Jackie-Jiang commented on issue #11018: LOOKUP (dimension table) does not work with GROUP BY

2023-06-30 Thread via GitHub
Jackie-Jiang commented on issue #11018: URL: https://github.com/apache/pinot/issues/11018#issuecomment-1615567230 Can you also share the query? In order to do post-aggregation, the function must be registered as scalar function. There is no scalar function version of `look_up`, thus the e

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11019: update freemaker to fix Server-side Template Injection

2023-06-30 Thread via GitHub
Jackie-Jiang commented on code in PR #11019: URL: https://github.com/apache/pinot/pull/11019#discussion_r1248581637 ## contrib/pinot-fmpp-maven-plugin/pom.xml: ## @@ -66,6 +66,12 @@ net.sourceforge.fmpp fmpp ${fmpp.version} + Review Comment: Do we

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11015: Apply some allocation optimizations on GrpcSendingMailbox

2023-06-30 Thread via GitHub
Jackie-Jiang commented on code in PR #11015: URL: https://github.com/apache/pinot/pull/11015#discussion_r1248573641 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/BaseDataBlock.java: ## @@ -387,7 +387,10 @@ public RoaringBitmap getNullRowIds(int colId) { */

[GitHub] [pinot] codecov-commenter commented on pull request #11022: Fix a flaky test OfflineGRPCServerIntegrationTest.testQueryingGrpcServer.

2023-06-30 Thread via GitHub
codecov-commenter commented on PR #11022: URL: https://github.com/apache/pinot/pull/11022#issuecomment-1615452555 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11022?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] shenyu0127 opened a new pull request, #11022: Fix a flaky test OfflineGRPCServerIntegrationTest.testQueryingGrpcServer.

2023-06-30 Thread via GitHub
shenyu0127 opened a new pull request, #11022: URL: https://github.com/apache/pinot/pull/11022 The test is flaky because the query uses the default `LIMIT` of 10 and there is a race condition in [`ConcurrentIndexedTable::upsertWithoutOrderBy`](https://github.com/apache/pinot/blob/f3068bc9316

[GitHub] [pinot] chenboat commented on a diff in pull request #10993: Update getTenantInstances call for controller and separate POST operations on it

2023-06-30 Thread via GitHub
chenboat commented on code in PR #10993: URL: https://github.com/apache/pinot/pull/10993#discussion_r1248392840 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTenantRestletResource.java: ## @@ -260,7 +276,7 @@ private String getTablesServedFromT

[GitHub] [pinot] chenboat commented on a diff in pull request #10993: Update getTenantInstances call for controller and separate POST operations on it

2023-06-30 Thread via GitHub
chenboat commented on code in PR #10993: URL: https://github.com/apache/pinot/pull/10993#discussion_r1248389459 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTenantRestletResource.java: ## @@ -276,27 +292,17 @@ private String toggleTenantState(

[GitHub] [pinot] chenboat merged pull request #11006: Add CLPDecodeRewriter to make it easier to call clpDecode with a column-group name rather than the individual columns.

2023-06-30 Thread via GitHub
chenboat merged PR #11006: URL: https://github.com/apache/pinot/pull/11006 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[pinot] branch master updated (9821cee011 -> b9cec629b7)

2023-06-30 Thread tingchen
This is an automated email from the ASF dual-hosted git repository. tingchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 9821cee011 Updated the code for showing the sql parsing exception. (#11009) add b9cec629b7 Add CLPDecodeRewrite

[GitHub] [pinot] abhioncbr commented on issue #10921: Improve Syntax Error Messages

2023-06-30 Thread via GitHub
abhioncbr commented on issue #10921: URL: https://github.com/apache/pinot/issues/10921#issuecomment-1615321856 It seems good enough to me. https://github.com/apache/pinot/assets/1076944/d432909f-d12b-4c5b-974e-4093963c9ce1";> -- This is an automated message from the Apache Git S

[GitHub] [pinot] snleee commented on a diff in pull request #10463: Segment compaction for upsert real-time tables

2023-06-30 Thread via GitHub
snleee commented on code in PR #10463: URL: https://github.com/apache/pinot/pull/10463#discussion_r1248357624 ## pinot-core/src/main/java/org/apache/pinot/core/common/MinionConstants.java: ## @@ -136,4 +136,18 @@ public static class SegmentGenerationAndPushTask { public sta

[GitHub] [pinot] abhioncbr commented on pull request #10790: [multistage]: Changes for supporting Json scalar functions.

2023-06-30 Thread via GitHub
abhioncbr commented on PR #10790: URL: https://github.com/apache/pinot/pull/10790#issuecomment-1615311406 I'll continue on this once these two PR's(https://github.com/apache/pinot/pull/10845, https://github.com/apache/pinot/pull/10846) get merged. Following the same approach will be helpfu

[GitHub] [pinot] abhioncbr commented on issue #10921: Improve Syntax Error Messages

2023-06-30 Thread via GitHub
abhioncbr commented on issue #10921: URL: https://github.com/apache/pinot/issues/10921#issuecomment-1615310104 Got it; that's a very going point. I'll confirm it here. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [pinot] snleee commented on a diff in pull request #10758: Add a new controller endpoint for segment deletion with a time window

2023-06-30 Thread via GitHub
snleee commented on code in PR #10758: URL: https://github.com/apache/pinot/pull/10758#discussion_r1248355285 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java: ## @@ -884,6 +865,52 @@ public SuccessResponse deleteSegmen

[GitHub] [pinot] snleee commented on a diff in pull request #10758: Add a new controller endpoint for segment deletion with a time window

2023-06-30 Thread via GitHub
snleee commented on code in PR #10758: URL: https://github.com/apache/pinot/pull/10758#discussion_r1248355285 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java: ## @@ -884,6 +865,52 @@ public SuccessResponse deleteSegmen

[GitHub] [pinot] snleee commented on a diff in pull request #10758: Add a new controller endpoint for segment deletion with a time window

2023-06-30 Thread via GitHub
snleee commented on code in PR #10758: URL: https://github.com/apache/pinot/pull/10758#discussion_r1248355285 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java: ## @@ -884,6 +865,52 @@ public SuccessResponse deleteSegmen

[GitHub] [pinot] walterddr commented on issue #10921: Improve Syntax Error Messages

2023-06-30 Thread via GitHub
walterddr commented on issue #10921: URL: https://github.com/apache/pinot/issues/10921#issuecomment-1615305989 ah. actually i have a follow up @abhioncbr in your PR's description. you can see that the query is single-lined --> if the query is multiple line. can you confirm that if th

[GitHub] [pinot] abhioncbr commented on issue #10921: Improve Syntax Error Messages

2023-06-30 Thread via GitHub
abhioncbr commented on issue #10921: URL: https://github.com/apache/pinot/issues/10921#issuecomment-1615303999 @ege-st, can we close this issue or any further work you have in mind? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [pinot] abhioncbr commented on issue #11003: pinot-client unable to dynamically choose broker for multi-table query

2023-06-30 Thread via GitHub
abhioncbr commented on issue #11003: URL: https://github.com/apache/pinot/issues/11003#issuecomment-1615304459 I want to give it a try. Please assign it to me. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [pinot] deemoliu commented on pull request #10915: [Upsert TTL] Add Watermark for each partitions for Primary key cleanup

2023-06-30 Thread via GitHub
deemoliu commented on PR #10915: URL: https://github.com/apache/pinot/pull/10915#issuecomment-1615301143 Addressed @Jackie-Jiang 's comment 1. used a double comparison column value in upsertConfig to replace upsertTTLConfig so it doesn't need to be timeInMills. 2. Persist largest

[GitHub] [pinot] abhioncbr commented on pull request #10891: Ensure min/max value generation in the segment metadata.

2023-06-30 Thread via GitHub
abhioncbr commented on PR #10891: URL: https://github.com/apache/pinot/pull/10891#issuecomment-1615296482 @Jackie-Jiang, please have a look. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10991: [enhance]: When enable case-insensitive, don't allow to add newly col…

2023-06-30 Thread via GitHub
Jackie-Jiang commented on code in PR #10991: URL: https://github.com/apache/pinot/pull/10991#discussion_r1248349644 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSchemaRestletResource.java: ## @@ -359,7 +364,10 @@ private void validateSchemaInt

[GitHub] [pinot] kirkrodrigues commented on a diff in pull request #11006: Add CLPDecodeRewriter to make it easier to call clpDecode with a column-group name rather than the individual columns.

2023-06-30 Thread via GitHub
kirkrodrigues commented on code in PR #11006: URL: https://github.com/apache/pinot/pull/11006#discussion_r1248344244 ## pinot-common/src/main/java/org/apache/pinot/sql/parsers/rewriter/CLPDecodeRewriter.java: ## @@ -0,0 +1,176 @@ +/** + * Licensed to the Apache Software Foundati

[GitHub] [pinot] chenboat commented on a diff in pull request #11006: Add CLPDecodeRewriter to make it easier to call clpDecode with a column-group name rather than the individual columns.

2023-06-30 Thread via GitHub
chenboat commented on code in PR #11006: URL: https://github.com/apache/pinot/pull/11006#discussion_r1248338626 ## pinot-common/src/main/java/org/apache/pinot/sql/parsers/rewriter/CLPDecodeRewriter.java: ## @@ -0,0 +1,176 @@ +/** + * Licensed to the Apache Software Foundation (A

[GitHub] [pinot] chenboat commented on a diff in pull request #11006: Add CLPDecodeRewriter to make it easier to call clpDecode with a column-group name rather than the individual columns.

2023-06-30 Thread via GitHub
chenboat commented on code in PR #11006: URL: https://github.com/apache/pinot/pull/11006#discussion_r1248338357 ## pinot-common/src/main/java/org/apache/pinot/sql/parsers/rewriter/CLPDecodeRewriter.java: ## @@ -0,0 +1,176 @@ +/** + * Licensed to the Apache Software Foundation (A

[GitHub] [pinot] walterddr merged pull request #11009: [multistage]: Better SQL Syntax Error Messages

2023-06-30 Thread via GitHub
walterddr merged PR #11009: URL: https://github.com/apache/pinot/pull/11009 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

[pinot] branch master updated: Updated the code for showing the sql parsing exception. (#11009)

2023-06-30 Thread rongr
This is an automated email from the ASF dual-hosted git repository. rongr pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 9821cee011 Updated the code for showing the sql pa

[GitHub] [pinot] xiangfu0 commented on issue #11002: Incorrect Results when using OR Operator in WHERE Clause with Pinot Connector on Trino

2023-06-30 Thread via GitHub
xiangfu0 commented on issue #11002: URL: https://github.com/apache/pinot/issues/11002#issuecomment-1615274498 I think the valid query should be ```WITH fd AS ( SELECT user_id, affiliate FROM ( SELECT user_id, affiliate, row_number() OVER (PARTITION BY user_id ORDER BY co

[GitHub] [pinot] codecov-commenter commented on pull request #11021: update jersey to fix Denial of Service (DoS)

2023-06-30 Thread via GitHub
codecov-commenter commented on PR #11021: URL: https://github.com/apache/pinot/pull/11021#issuecomment-1615272014 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11021?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] tibrewalpratik17 commented on a diff in pull request #10993: Update getTenantInstances call for controller and separate POST operations on it

2023-06-30 Thread via GitHub
tibrewalpratik17 commented on code in PR #10993: URL: https://github.com/apache/pinot/pull/10993#discussion_r1248335682 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTenantRestletResource.java: ## @@ -203,19 +204,36 @@ public TenantsList getAll

[GitHub] [pinot] tibrewalpratik17 commented on a diff in pull request #10993: Update getTenantInstances call for controller and separate POST operations on it

2023-06-30 Thread via GitHub
tibrewalpratik17 commented on code in PR #10993: URL: https://github.com/apache/pinot/pull/10993#discussion_r1248335615 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTenantRestletResource.java: ## @@ -203,19 +204,36 @@ public TenantsList getAll

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-30 Thread via GitHub
Jackie-Jiang commented on code in PR #10982: URL: https://github.com/apache/pinot/pull/10982#discussion_r1248332662 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/TableConfigUtils.java: ## @@ -42,6 +42,7 @@ import org.apache.pinot.common.request.conte

[GitHub] [pinot] Jackie-Jiang closed issue #10975: Text Index does not match against mutable segments for MV Columns

2023-06-30 Thread via GitHub
Jackie-Jiang closed issue #10975: Text Index does not match against mutable segments for MV Columns URL: https://github.com/apache/pinot/issues/10975 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[pinot] branch master updated (25173a3f28 -> e1c4e2fdd1)

2023-06-30 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 25173a3f28 Add fromController method that accepts a PinotClientTransport (#11013) add e1c4e2fdd1 add support for

[GitHub] [pinot] Jackie-Jiang merged pull request #11000: add support for mv cols with realtime lucene index

2023-06-30 Thread via GitHub
Jackie-Jiang merged PR #11000: URL: https://github.com/apache/pinot/pull/11000 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[pinot] branch master updated (fac02cf4df -> 25173a3f28)

2023-06-30 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from fac02cf4df When extracting root-level field name for complex type handling, use the whole delimiter (#11005) add

[GitHub] [pinot] Jackie-Jiang closed issue #10924: Implementing PinotClientTransport using Vert.x

2023-06-30 Thread via GitHub
Jackie-Jiang closed issue #10924: Implementing PinotClientTransport using Vert.x URL: https://github.com/apache/pinot/issues/10924 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [pinot] Jackie-Jiang merged pull request #11013: Add fromController method that accepts a PinotClientTransport

2023-06-30 Thread via GitHub
Jackie-Jiang merged PR #11013: URL: https://github.com/apache/pinot/pull/11013 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10905: Startree index build enhancement

2023-06-30 Thread via GitHub
Jackie-Jiang commented on code in PR #10905: URL: https://github.com/apache/pinot/pull/10905#discussion_r1248326912 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilder.java: ## @@ -104,7 +107,7 @@ public MultipleTreesBuild

[GitHub] [pinot] codecov-commenter commented on pull request #11020: allow to preload segments with upsert snapshots to speedup table loading

2023-06-30 Thread via GitHub
codecov-commenter commented on PR #11020: URL: https://github.com/apache/pinot/pull/11020#issuecomment-1615258595 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11020?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] zhtaoxiang opened a new pull request, #11021: update jersey to fix Denial of Service (DoS)

2023-06-30 Thread via GitHub
zhtaoxiang opened a new pull request, #11021: URL: https://github.com/apache/pinot/pull/11021 the problem is in jersey's dependency: [com.fasterxml.jackson.core:jackson-databind](https://security.snyk.io/package/maven/com.fasterxml.jackson.core%3Ajackson-databind) -- This is an automated

[pinot] branch master updated: When extracting root-level field name for complex type handling, use the whole delimiter (#11005)

2023-06-30 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new fac02cf4df When extracting root-level field name

[GitHub] [pinot] Jackie-Jiang merged pull request #11005: `bugfix` Fix issue with extracting root-level field names for complex handling

2023-06-30 Thread via GitHub
Jackie-Jiang merged PR #11005: URL: https://github.com/apache/pinot/pull/11005 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] Jackie-Jiang commented on pull request #11016: Rbac impl

2023-06-30 Thread via GitHub
Jackie-Jiang commented on PR #11016: URL: https://github.com/apache/pinot/pull/11016#issuecomment-1615251276 Do you have a design doc that can be linked to the PR? Or add more descriptions -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11016: Rbac impl

2023-06-30 Thread via GitHub
Jackie-Jiang commented on code in PR #11016: URL: https://github.com/apache/pinot/pull/11016#discussion_r1248317872 ## pinot-broker/src/main/java/org/apache/pinot/broker/api/AccessControl.java: ## @@ -58,4 +58,19 @@ default boolean hasAccess(RequesterIdentity requesterIdentity)

[GitHub] [pinot] codecov-commenter commented on pull request #11017: Remove support for High level consumers in Apache Pinot

2023-06-30 Thread via GitHub
codecov-commenter commented on PR #11017: URL: https://github.com/apache/pinot/pull/11017#issuecomment-1615248955 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11017?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] egalpin commented on issue #11018: LOOKUP (dimension table) does not work with GROUP BY

2023-06-30 Thread via GitHub
egalpin commented on issue #11018: URL: https://github.com/apache/pinot/issues/11018#issuecomment-1615242306 It turns out that the root cause of the issue is that the fields being fetched via `LOOKUP` function were not included in the `GROUP BY` clause. The above error did not mention this

[GitHub] [pinot] abhioncbr closed pull request #10891: Ensure min/max value generation in the segment metadata.

2023-06-30 Thread via GitHub
abhioncbr closed pull request #10891: Ensure min/max value generation in the segment metadata. URL: https://github.com/apache/pinot/pull/10891 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[pinot] branch master updated (2e346d4a6a -> 49efcfa587)

2023-06-30 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 2e346d4a6a Upgrade commons-codec:commons-codec@1.11 to commons-codec:commons-codec@1.15 in pinot-pulsar (#11011)

[GitHub] [pinot] xiangfu0 merged pull request #11012: Upgrade circe-parser from 0.13.0 to 0.14.2

2023-06-30 Thread via GitHub
xiangfu0 merged PR #11012: URL: https://github.com/apache/pinot/pull/11012 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[pinot] branch master updated (bde2d01153 -> 2e346d4a6a)

2023-06-30 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from bde2d01153 Update dockerfile with recent jdk distro changes (#10963) add 2e346d4a6a Upgrade commons-codec:commons

[GitHub] [pinot] xiangfu0 merged pull request #11011: Upgrade commons-codec:commons-codec from 1.11 to 1.15 in pinot-pulsar

2023-06-30 Thread via GitHub
xiangfu0 merged PR #11011: URL: https://github.com/apache/pinot/pull/11011 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[GitHub] [pinot] klsince opened a new pull request, #11020: allow to preload segments with upsert snapshots to speedup table loading

2023-06-30 Thread via GitHub
klsince opened a new pull request, #11020: URL: https://github.com/apache/pinot/pull/11020 `feature` This PR adds a feature to preload segments from table that uses the upsert snapshot feature. The segments with validDocIds snapshots can be preloaded in a more efficient manner to spe

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10993: Update getTenantInstances call for controller and separate POST operations on it

2023-06-30 Thread via GitHub
Jackie-Jiang commented on code in PR #10993: URL: https://github.com/apache/pinot/pull/10993#discussion_r1248312080 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/builder/ControllerRequestURLBuilder.java: ## @@ -163,6 +163,12 @@ public String forBrokerTablesGet(String sta

[GitHub] [pinot] Jackie-Jiang commented on issue #11002: Incorrect Results when using OR Operator in WHERE Clause with Pinot Connector on Trino

2023-06-30 Thread via GitHub
Jackie-Jiang commented on issue #11002: URL: https://github.com/apache/pinot/issues/11002#issuecomment-1615225342 cc @elonazoulay @xiangfu0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [pinot] zhtaoxiang opened a new pull request, #11019: update freemaker to fix Server-side Template Injection

2023-06-30 Thread via GitHub
zhtaoxiang opened a new pull request, #11019: URL: https://github.com/apache/pinot/pull/11019 follow the suggestion on https://security.snyk.io/vuln/SNYK-JAVA-ORGFREEMARKER-1076795 to fix Server-side Template Injection -- This is an automated message from the Apache Git Service. To respo

[GitHub] [pinot] egalpin opened a new issue, #11018: LOOKUP (dimension table) does not work with GROUP BY

2023-06-30 Thread via GitHub
egalpin opened a new issue, #11018: URL: https://github.com/apache/pinot/issues/11018 From the broker logs: ```log 2023/06/30 21:28:35.954 ERROR [PinotClientRequest] [jersey-server-managed-async-executor-1] Caught exception while processing POST request java.lang.IllegalArgumen

[GitHub] [pinot] jtao15 commented on a diff in pull request #10964: Allow custom segment grouping in MergeRollupTask

2023-06-30 Thread via GitHub
jtao15 commented on code in PR #10964: URL: https://github.com/apache/pinot/pull/10964#discussion_r1248298273 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotTaskManager.java: ## @@ -475,6 +476,14 @@ public void registerTaskGenerator(PinotTas

[GitHub] [pinot] navina opened a new pull request, #11017: Remove support for High level consumers in Apache Pinot

2023-06-30 Thread via GitHub
navina opened a new pull request, #11017: URL: https://github.com/apache/pinot/pull/11017 * Controller will throw runtime exception during startup if there are hlc tables in the cluster * Add table api will disallow table creation with consumer.type = highlevel * Marked StreamLevelCons

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #10963: Update dockerfile with recent jdk distro changes

2023-06-30 Thread via GitHub
xiangfu0 commented on code in PR #10963: URL: https://github.com/apache/pinot/pull/10963#discussion_r1248293147 ## docker/images/pinot-base/pinot-base-build/openjdk.dockerfile: ## @@ -16,33 +16,37 @@ # specific language governing permissions and limitations # under the License

[pinot] branch master updated: Update dockerfile with recent jdk distro changes (#10963)

2023-06-30 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new bde2d01153 Update dockerfile with recent jdk dis

[GitHub] [pinot] xiangfu0 merged pull request #10963: Update dockerfile with recent jdk distro changes

2023-06-30 Thread via GitHub
xiangfu0 merged PR #10963: URL: https://github.com/apache/pinot/pull/10963 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[GitHub] [pinot] somandal commented on a diff in pull request #10846: [multistage] Initial planner changes to support the v1 aggregation functions in multistage

2023-06-30 Thread via GitHub
somandal commented on code in PR #10846: URL: https://github.com/apache/pinot/pull/10846#discussion_r1248289893 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/BaseSingleInputAggregationFunction.java: ## @@ -40,7 +40,7 @@ public BaseSingleInputAggreg

[GitHub] [pinot] somandal commented on a diff in pull request #10846: [multistage] Initial planner changes to support the v1 aggregation functions in multistage

2023-06-30 Thread via GitHub
somandal commented on code in PR #10846: URL: https://github.com/apache/pinot/pull/10846#discussion_r1248289636 ## pinot-core/src/main/java/org/apache/pinot/core/query/reduce/function/InternalReduceFunctions.java: ## @@ -35,17 +33,10 @@ private InternalReduceFunctions() { }

[GitHub] [pinot] walterddr commented on a diff in pull request #10845: [Multistage] Runtime changes for leveraging V1 Aggregation Functions

2023-06-30 Thread via GitHub
walterddr commented on code in PR #10845: URL: https://github.com/apache/pinot/pull/10845#discussion_r1248265090 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MultistageAggregationExecutor.java: ## @@ -0,0 +1,192 @@ +/** + * Licensed to the Apache

[GitHub] [pinot] vvivekiyer commented on a diff in pull request #10845: [Multistage] Runtime changes for leveraging V1 Aggregation Functions

2023-06-30 Thread via GitHub
vvivekiyer commented on code in PR #10845: URL: https://github.com/apache/pinot/pull/10845#discussion_r1248214238 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/BaseBooleanAggregationFunction.java: ## @@ -233,7 +233,8 @@ public Integer merge(Integer

[GitHub] [pinot] walterddr commented on pull request #11009: [multistage]: Better SQL Syntax Error Messages

2023-06-30 Thread via GitHub
walterddr commented on PR #11009: URL: https://github.com/apache/pinot/pull/11009#issuecomment-1615098549 > @abhioncbr this is great, thanks! > > @walterddr do we also want to fix error messages in the V1 query engine? I believe this change is on the common code path -- This

[GitHub] [pinot] somandal commented on a diff in pull request #10846: [multistage] Initial planner changes to support the v1 aggregation functions in multistage

2023-06-30 Thread via GitHub
somandal commented on code in PR #10846: URL: https://github.com/apache/pinot/pull/10846#discussion_r1248183763 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotAggregateReduceFunctionsRule.java: ## @@ -0,0 +1,427 @@ +/** + * Licensed to the Apache Software

[GitHub] [pinot] walterddr commented on a diff in pull request #10846: [multistage] Initial planner changes to support the v1 aggregation functions in multistage

2023-06-30 Thread via GitHub
walterddr commented on code in PR #10846: URL: https://github.com/apache/pinot/pull/10846#discussion_r1248167425 ## pinot-core/src/main/java/org/apache/pinot/core/query/reduce/function/InternalReduceFunctions.java: ## @@ -35,17 +33,10 @@ private InternalReduceFunctions() { }

[GitHub] [pinot] navina commented on issue #10924: Implementing PinotClientTransport using Vert.x

2023-06-30 Thread via GitHub
navina commented on issue #10924: URL: https://github.com/apache/pinot/issues/10924#issuecomment-1615025457 @lucifer4j : Thanks for sharing the doc and looking forward to the demo app. Follow-up questions: 1. Is the code for the client going to be in open-source (I am assuming so)?

[GitHub] [pinot] vvivekiyer commented on a diff in pull request #10845: [Multistage] Runtime changes for leveraging V1 Aggregation Functions

2023-06-30 Thread via GitHub
vvivekiyer commented on code in PR #10845: URL: https://github.com/apache/pinot/pull/10845#discussion_r1248152940 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MultistageAggregationExecutor.java: ## @@ -0,0 +1,192 @@ +/** + * Licensed to the Apache

[GitHub] [pinot] walterddr commented on a diff in pull request #10845: [Multistage] Runtime changes for leveraging V1 Aggregation Functions

2023-06-30 Thread via GitHub
walterddr commented on code in PR #10845: URL: https://github.com/apache/pinot/pull/10845#discussion_r1248121775 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/FourthMomentAggregationFunction.java: ## @@ -30,7 +30,7 @@ import org.apache.pinot.segme

[GitHub] [pinot] walterddr commented on a diff in pull request #10845: [Multistage] Runtime changes for leveraging V1 Aggregation Functions

2023-06-30 Thread via GitHub
walterddr commented on code in PR #10845: URL: https://github.com/apache/pinot/pull/10845#discussion_r1248121185 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/BaseBooleanAggregationFunction.java: ## @@ -233,7 +233,8 @@ public Integer merge(Integer

[GitHub] [pinot] somandal commented on a diff in pull request #10846: [multistage] Initial planner changes to support the v1 aggregation functions in multistage

2023-06-30 Thread via GitHub
somandal commented on code in PR #10846: URL: https://github.com/apache/pinot/pull/10846#discussion_r1248091869 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/BaseSingleInputAggregationFunction.java: ## @@ -40,7 +40,7 @@ public BaseSingleInputAggreg

[GitHub] [pinot] codecov-commenter commented on pull request #11016: Rbac impl

2023-06-30 Thread via GitHub
codecov-commenter commented on PR #11016: URL: https://github.com/apache/pinot/pull/11016#issuecomment-1614927288 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11016?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] soumitra-st opened a new pull request, #11016: Rbac impl

2023-06-30 Thread via GitHub
soumitra-st opened a new pull request, #11016: URL: https://github.com/apache/pinot/pull/11016 Adding support Role Based Access Control in Pinot. This feature is a work in progress. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [pinot] walterddr commented on a diff in pull request #10846: [multistage] Initial planner changes to support the v1 aggregation functions in multistage

2023-06-30 Thread via GitHub
walterddr commented on code in PR #10846: URL: https://github.com/apache/pinot/pull/10846#discussion_r1248049043 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/BaseSingleInputAggregationFunction.java: ## @@ -40,7 +40,7 @@ public BaseSingleInputAggre

[GitHub] [pinot] atris commented on issue #10934: TEXT_CONTAINS fails with NullPointerException

2023-06-30 Thread via GitHub
atris commented on issue #10934: URL: https://github.com/apache/pinot/issues/10934#issuecomment-1614877364 I am looking into this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [pinot] egalpin closed issue #10998: Block-level, intra-segment bloom filters

2023-06-30 Thread via GitHub
egalpin closed issue #10998: Block-level, intra-segment bloom filters URL: https://github.com/apache/pinot/issues/10998 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

[GitHub] [pinot] egalpin commented on issue #10998: Block-level, intra-segment bloom filters

2023-06-30 Thread via GitHub
egalpin commented on issue #10998: URL: https://github.com/apache/pinot/issues/10998#issuecomment-1614872611 Thanks all 👍 I'll close this issue out so it doesn't appear as outstanding work etc. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [pinot] walterddr commented on a diff in pull request #10845: [Multistage] Runtime changes for leveraging V1 Aggregation Functions

2023-06-30 Thread via GitHub
walterddr commented on code in PR #10845: URL: https://github.com/apache/pinot/pull/10845#discussion_r1248016907 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/DataBlockUtils.java: ## @@ -268,4 +269,381 @@ private static Object[] extractRowFromDataBlock(DataBloc

[GitHub] [pinot] walterddr commented on a diff in pull request #10845: [Multistage] Runtime changes for leveraging V1 Aggregation Functions

2023-06-30 Thread via GitHub
walterddr commented on code in PR #10845: URL: https://github.com/apache/pinot/pull/10845#discussion_r1248017898 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/DataBlockUtils.java: ## @@ -268,4 +269,381 @@ private static Object[] extractRowFromDataBlock(DataBloc

[GitHub] [pinot] codecov-commenter commented on pull request #11015: Apply some allocation optimizations on GrpcSendingMailbox

2023-06-30 Thread via GitHub
codecov-commenter commented on PR #11015: URL: https://github.com/apache/pinot/pull/11015#issuecomment-1614841523 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11015?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] ege-st commented on pull request #11009: [multistage]: Better SQL Syntax Error Messages

2023-06-30 Thread via GitHub
ege-st commented on PR #11009: URL: https://github.com/apache/pinot/pull/11009#issuecomment-1614822293 @abhioncbr this is great, thanks! @walterddr do we want to fix error messages in the V1 query engine? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [pinot] gortiz opened a new pull request, #11015: Apply some allocation optimizations on GrpcSendingMailbox

2023-06-30 Thread via GitHub
gortiz opened a new pull request, #11015: URL: https://github.com/apache/pinot/pull/11015 This is a simple PR that tries to reduce the amount of byte[] that are created by `GrpcSendingMailbox`. There are still some places that can be improved, but these changes are easy to apply and very ef

[GitHub] [pinot] klsince commented on issue #10998: Block-level, intra-segment bloom filters

2023-06-30 Thread via GitHub
klsince commented on issue #10998: URL: https://github.com/apache/pinot/issues/10998#issuecomment-1614669230 Yes. This idea has been implemented as sparse index, an extension available in StarTree Pinot: https://dev.startree.ai/docs/startree-extensions/sparse-index for more details. It wor

[GitHub] [pinot] andimiller commented on a diff in pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-30 Thread via GitHub
andimiller commented on code in PR #10982: URL: https://github.com/apache/pinot/pull/10982#discussion_r1247760748 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/TableConfigUtils.java: ## @@ -42,6 +42,7 @@ import org.apache.pinot.common.request.context

[GitHub] [pinot] andimiller commented on a diff in pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-30 Thread via GitHub
andimiller commented on code in PR #10982: URL: https://github.com/apache/pinot/pull/10982#discussion_r1247701457 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/TableConfigUtils.java: ## @@ -42,6 +42,7 @@ import org.apache.pinot.common.request.context

[GitHub] [pinot] INNOCENT-BOY commented on a diff in pull request #10991: [enhance]: When enable case-insensitive, don't allow to add newly col…

2023-06-30 Thread via GitHub
INNOCENT-BOY commented on code in PR #10991: URL: https://github.com/apache/pinot/pull/10991#discussion_r1247696908 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSchemaRestletResource.java: ## @@ -359,7 +359,8 @@ private void validateSchemaInte

[GitHub] [pinot] itschrispeck commented on a diff in pull request #11000: add support for mv cols with realtime lucene index

2023-06-30 Thread via GitHub
itschrispeck commented on code in PR #11000: URL: https://github.com/apache/pinot/pull/11000#discussion_r1247626645 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/mutable/MutableTextIndex.java: ## @@ -31,12 +31,18 @@ default void add(@Nonnull Object value,

[GitHub] [pinot] snleee commented on a diff in pull request #10969: Instance drop operation validation check API

2023-06-30 Thread via GitHub
snleee commented on code in PR #10969: URL: https://github.com/apache/pinot/pull/10969#discussion_r1247597496 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotInstanceRestletResource.java: ## @@ -390,4 +391,33 @@ public SuccessResponse updateBroke

[GitHub] [pinot] richardstartin commented on issue #10986: Count discrepancy among `in`/`=` and `<` queries using timestamp index

2023-06-30 Thread via GitHub
richardstartin commented on issue #10986: URL: https://github.com/apache/pinot/issues/10986#issuecomment-1614280375 the best thing to do right now would be to replace the call to `eq(x)` in the range index reader with `between(x, x)` and see if this makes things better, I'll come back at a

[GitHub] [pinot] Jackie-Jiang commented on issue #10986: Count discrepancy among `in`/`=` and `<` queries using timestamp index

2023-06-30 Thread via GitHub
Jackie-Jiang commented on issue #10986: URL: https://github.com/apache/pinot/issues/10986#issuecomment-1614275957 @shwin Can you also share the stats from the query response? In the stats, `numSegmentsProcessed` is the segment count passing the pruner, and `numSegmentsMatched` is the segmen

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11000: wip: add support for mv cols with realtime lucene index

2023-06-30 Thread via GitHub
Jackie-Jiang commented on code in PR #11000: URL: https://github.com/apache/pinot/pull/11000#discussion_r1247542426 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/mutable/MutableTextIndex.java: ## @@ -31,12 +31,18 @@ default void add(@Nonnull Object value,

[GitHub] [pinot] Jackie-Jiang commented on issue #10998: Block-level, intra-segment bloom filters

2023-06-30 Thread via GitHub
Jackie-Jiang commented on issue #10998: URL: https://github.com/apache/pinot/issues/10998#issuecomment-1614255698 cc @klsince -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [pinot] codecov-commenter commented on pull request #11013: Add fromController method that accepts a PinotClientTransport

2023-06-30 Thread via GitHub
codecov-commenter commented on PR #11013: URL: https://github.com/apache/pinot/pull/11013#issuecomment-1614248658 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11013?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-30 Thread via GitHub
Jackie-Jiang commented on code in PR #10982: URL: https://github.com/apache/pinot/pull/10982#discussion_r1247531397 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/TableConfigUtils.java: ## @@ -42,6 +42,7 @@ import org.apache.pinot.common.request.conte

[GitHub] [pinot] codecov-commenter commented on pull request #11014: Fix double-checked locking in ConnectionFactory

2023-06-30 Thread via GitHub
codecov-commenter commented on PR #11014: URL: https://github.com/apache/pinot/pull/11014#issuecomment-1614246056 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11014?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] gortiz commented on issue #10960: Large off-heap memory usage?

2023-06-30 Thread via GitHub
gortiz commented on issue #10960: URL: https://github.com/apache/pinot/issues/10960#issuecomment-1614239155 Newer images will be built using amazon corretto and microsoft base images and the plan is to try to keep them updated. -- This is an automated message from the Apache Git Service.

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10463: Segment compaction for upsert real-time tables

2023-06-30 Thread via GitHub
Jackie-Jiang commented on code in PR #10463: URL: https://github.com/apache/pinot/pull/10463#discussion_r1247523772 ## pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/upsertcompaction/UpsertCompactionTaskGenerator.java:

  1   2   >