Re: [I] COUNT(DISTINCT col1, col2) does not throw a syntax error and returns an incorrect result [pinot]

prajyotkcsu commented on issue #11503: URL: https://github.com/apache/pinot/issues/11503#issuecomment-2071478815 good observation! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [I] [multistage] Subqueries return different results depending on where clause [pinot]

Jackie-Jiang commented on issue #12949: URL: https://github.com/apache/pinot/issues/12949#issuecomment-2071169310 @gortiz Can you help take a look at this query? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Multi stage stats [pinot]

Jackie-Jiang commented on code in PR #12704: URL: https://github.com/apache/pinot/pull/12704#discussion_r1575464681 ## pinot-common/src/main/java/org/apache/pinot/common/response/broker/BrokerResponseNative.java: ## @@ -51,6 +59,7 @@ "segmentStatistics", "traceInfo", "parti

(pinot) branch master updated: handle absent segments so that catchup checker doesn't get stuck on them (#12883)

This is an automated email from the ASF dual-hosted git repository. xbli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 8e10320595 handle absent segments so that catchup c

Re: [PR] handle absent segments so that catchup checker doesn't get stuck on them [pinot]

klsince merged PR #12883: URL: https://github.com/apache/pinot/pull/12883 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

Re: [PR] Upgrade Calcite to 1.36.0 [pinot]

robertzych commented on PR #12754: URL: https://github.com/apache/pinot/pull/12754#issuecomment-2071110780 I just filed the Calcite issue: https://issues.apache.org/jira/browse/CALCITE-6381 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Support NOT in StarTree Index [pinot]

codecov-commenter commented on PR #12988: URL: https://github.com/apache/pinot/pull/12988#issuecomment-2071048585 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12988?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Upgrade lucene to 9.10.0 and compatibility changes to code. [pinot]

Jackie-Jiang commented on PR #12866: URL: https://github.com/apache/pinot/pull/12866#issuecomment-2071043920 I don't fully follow why did it break. Following the same logic in #11857 should work. Basically when downgrading and Pinot doesn't understand the new format, it should try to genera

Re: [PR] Add threadLocal reusable byte array for VarByteChunkReaderV4 [pinot]

itschrispeck commented on PR #12977: URL: https://github.com/apache/pinot/pull/12977#issuecomment-2071036509 Have you found any issues from these allocations? In our experience w/ V4 we have not seen GC issues even when using lower throughput GC algorithms (ZGC/generational ZGC). I

Re: [PR] Add back pinot spi shading [pinot]

Jackie-Jiang commented on PR #12969: URL: https://github.com/apache/pinot/pull/12969#issuecomment-2071014624 Solved with #12979 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] Add back pinot spi shading [pinot]

Jackie-Jiang closed pull request #12969: Add back pinot spi shading URL: https://github.com/apache/pinot/pull/12969 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [I] Upsert table backfill enhancement: support externally partitioned data [pinot]

Jackie-Jiang commented on issue #12987: URL: https://github.com/apache/pinot/issues/12987#issuecomment-2071013541 For real-time ingested data, the partition must match the upstream partition id to ensure the upsert assumption of all data of the same partition served by the same server, and

[PR] Support NOT in StarTree Index [pinot]

Jackie-Jiang opened a new pull request, #12988: URL: https://github.com/apache/pinot/pull/12988 Support NOT in StarTree Index when it is followed by predicate or NOT (not with nested AND/OR) E.g. - ` WHERE NOT d1 > 10` - ` WHERE d1 > 10 AND NOT d2 < 10` - ` WHERE d1__COLUMN_NAME

Re: [PR] [WIP] add locking logic to get consistent table view for upsert tables [pinot]

tibrewalpratik17 commented on code in PR #12976: URL: https://github.com/apache/pinot/pull/12976#discussion_r1575309595 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java: ## @@ -1103,6 +1210,48 @@ private static Mu

[I] Upsert table backfill enhancement: support externally partitioned data [pinot]

rohityadav1993 opened a new issue, #12987: URL: https://github.com/apache/pinot/issues/12987 ## Problem #6567 allows uploading a batch generated segment to Pinot upsert realtime table. Partitioned data is handled by defining the partition column in `segmentPartitionConfig.columnPartition

(pinot) branch master updated (a852c8a424 -> a5c728f549)

This is an automated email from the ASF dual-hosted git repository. mcvsubbu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from a852c8a424 Update ORC and Hive dependency versions in the license binary file (#12986) add a5c728f549 Add back

Re: [PR] Add back shade profile for shade removed in #12712 [pinot]

mcvsubbu merged PR #12979: URL: https://github.com/apache/pinot/pull/12979 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] Issue #12367 testCaseTransformFunctionWithIntResults test fix [pinot]

Jackie-Jiang commented on code in PR #12922: URL: https://github.com/apache/pinot/pull/12922#discussion_r1575173156 ## pinot-core/src/test/java/org/apache/pinot/core/operator/transform/function/CaseTransformFunctionTest.java: ## @@ -116,6 +145,90 @@ public void testCaseTransform

Re: [PR] Issue #12367 testCaseTransformFunctionWithIntResults test fix [pinot]

aditya0811 commented on PR #12922: URL: https://github.com/apache/pinot/pull/12922#issuecomment-2070471323 Hello @Jackie-Jiang, please let me know your thoughts on the newly added negative scenario test. -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [PR] Custom configuration property reader for segment metadata files [pinot]

abhioncbr commented on code in PR #12440: URL: https://github.com/apache/pinot/pull/12440#discussion_r1575127192 ## pinot-spi/src/main/java/org/apache/pinot/spi/env/PropertyIOFactoryKind.java: ## @@ -0,0 +1,25 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under on

(pinot) branch master updated (c9d513aef0 -> a852c8a424)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from c9d513aef0 Bump org.apache.maven.plugins:maven-jar-plugin from 3.4.0 to 3.4.1 (#12983) add a852c8a424 Update ORC

Re: [PR] Update ORC and Hive dependency versions in the license binary file [pinot]

Jackie-Jiang merged PR #12986: URL: https://github.com/apache/pinot/pull/12986 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Bump org.apache.maven.plugins:maven-jar-plugin from 3.4.0 to 3.4.1 [pinot]

Jackie-Jiang merged PR #12983: URL: https://github.com/apache/pinot/pull/12983 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: Bump org.apache.maven.plugins:maven-jar-plugin from 3.4.0 to 3.4.1 (#12983)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new c9d513aef0 Bump org.apache.maven.plugins:maven-ja

(pinot) branch dependabot/maven/org.apache.maven.plugins-maven-jar-plugin-3.4.1 deleted (was a6c9ec1350)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/org.apache.maven.plugins-maven-jar-plugin-3.4.1 in repository https://gitbox.apache.org/repos/asf/pinot.git was a6c9ec1350 Bump org.apache.maven.plugins:maven-jar-plugin fr

Re: [PR] Update ORC and Hive dependency versions in the license binary file [pinot]

codecov-commenter commented on PR #12986: URL: https://github.com/apache/pinot/pull/12986#issuecomment-2070261379 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12986?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Add schema as input to the decoder. [pinot]

Jackie-Jiang commented on code in PR #12981: URL: https://github.com/apache/pinot/pull/12981#discussion_r1575052394 ## pinot-spi/src/main/java/org/apache/pinot/spi/stream/StreamMessageDecoder.java: ## @@ -46,9 +47,25 @@ public interface StreamMessageDecoder { * @param topicN

Re: [PR] Add some multi-stage metrics [pinot]

Jackie-Jiang commented on code in PR #12982: URL: https://github.com/apache/pinot/pull/12982#discussion_r1575044559 ## pinot-common/src/main/java/org/apache/pinot/common/metrics/BrokerMeter.java: ## @@ -102,7 +107,23 @@ public enum BrokerMeter implements AbstractMetrics.Meter {

Re: [PR] Add some multi-stage metrics [pinot]

Jackie-Jiang commented on code in PR #12982: URL: https://github.com/apache/pinot/pull/12982#discussion_r1575043817 ## pinot-common/src/main/java/org/apache/pinot/common/metrics/BrokerMeter.java: ## @@ -102,7 +107,23 @@ public enum BrokerMeter implements AbstractMetrics.Meter {

[PR] Update ORC and Hive dependency versions in the license binary file [pinot]

yashmayya opened a new pull request, #12986: URL: https://github.com/apache/pinot/pull/12986 - The ORC and Hive dependency versions were upgraded in https://github.com/apache/pinot/pull/12956. This patch updates the license binary file with the new versions. -- This is an automated messa

(pinot) branch dependabot/maven/aws.sdk.version-2.25.35 deleted (was 469990d428)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/aws.sdk.version-2.25.35 in repository https://gitbox.apache.org/repos/asf/pinot.git was 469990d428 Bump aws.sdk.version from 2.25.34 to 2.25.35 The revisions that were on t

(pinot) branch master updated (f83e466c42 -> 7b68aa369d)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from f83e466c42 Bump org.roaringbitmap:RoaringBitmap from 1.0.5 to 1.0.6 (#12985) add 7b68aa369d Bump aws.sdk.version

Re: [PR] Bump aws.sdk.version from 2.25.34 to 2.25.35 [pinot]

Jackie-Jiang merged PR #12984: URL: https://github.com/apache/pinot/pull/12984 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated (e1b0e5357e -> f83e466c42)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from e1b0e5357e Refactor PinotTaskManager class (#12964) add f83e466c42 Bump org.roaringbitmap:RoaringBitmap from 1.0.5

(pinot) branch dependabot/maven/org.roaringbitmap-RoaringBitmap-1.0.6 deleted (was ccfb0d3a55)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/org.roaringbitmap-RoaringBitmap-1.0.6 in repository https://gitbox.apache.org/repos/asf/pinot.git was ccfb0d3a55 Bump org.roaringbitmap:RoaringBitmap from 1.0.5 to 1.0.6 T

Re: [PR] Bump org.roaringbitmap:RoaringBitmap from 1.0.5 to 1.0.6 [pinot]

Jackie-Jiang merged PR #12985: URL: https://github.com/apache/pinot/pull/12985 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Metric for count of tables configured with various tier backends [pinot]

shounakmk219 commented on code in PR #12940: URL: https://github.com/apache/pinot/pull/12940#discussion_r1574879993 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/SegmentStatusChecker.java: ## @@ -135,6 +137,13 @@ protected void postprocess(Context context)

(pinot) branch dependabot/maven/org.roaringbitmap-RoaringBitmap-1.0.6 created (now ccfb0d3a55)

This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.roaringbitmap-RoaringBitmap-1.0.6 in repository https://gitbox.apache.org/repos/asf/pinot.git at ccfb0d3a55 Bump org.roaringbitmap:RoaringBitmap from 1.0.5 to 1.0.

[PR] Bump org.roaringbitmap:RoaringBitmap from 1.0.5 to 1.0.6 [pinot]

dependabot[bot] opened a new pull request, #12985: URL: https://github.com/apache/pinot/pull/12985 Bumps [org.roaringbitmap:RoaringBitmap](https://github.com/RoaringBitmap/RoaringBitmap) from 1.0.5 to 1.0.6. Release notes Sourced from https://github.com/RoaringBitmap/RoaringBitmap

(pinot) branch dependabot/maven/aws.sdk.version-2.25.35 created (now 469990d428)

This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/aws.sdk.version-2.25.35 in repository https://gitbox.apache.org/repos/asf/pinot.git at 469990d428 Bump aws.sdk.version from 2.25.34 to 2.25.35 No new revisions were ad

[PR] Bump aws.sdk.version from 2.25.34 to 2.25.35 [pinot]

dependabot[bot] opened a new pull request, #12984: URL: https://github.com/apache/pinot/pull/12984 Bumps `aws.sdk.version` from 2.25.34 to 2.25.35. Updates `software.amazon.awssdk:bom` from 2.25.34 to 2.25.35 Updates `software.amazon.awssdk:apache-client` from 2.25.34 to 2.25.35

[PR] Bump org.apache.maven.plugins:maven-jar-plugin from 3.4.0 to 3.4.1 [pinot]

dependabot[bot] opened a new pull request, #12983: URL: https://github.com/apache/pinot/pull/12983 Bumps [org.apache.maven.plugins:maven-jar-plugin](https://github.com/apache/maven-jar-plugin) from 3.4.0 to 3.4.1. Release notes Sourced from https://github.com/apache/maven-jar-plug

(pinot) branch dependabot/maven/org.apache.maven.plugins-maven-jar-plugin-3.4.1 created (now a6c9ec1350)

This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.apache.maven.plugins-maven-jar-plugin-3.4.1 in repository https://gitbox.apache.org/repos/asf/pinot.git at a6c9ec1350 Bump org.apache.maven.plugins:maven-jar-plugi

Re: [PR] Add some multi-stage metrics [pinot]

gortiz commented on PR #12982: URL: https://github.com/apache/pinot/pull/12982#issuecomment-2069098588 I've simplified the PR and modified the title and description of the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] Add some multi-stage metrics [pinot]

gortiz commented on code in PR #12982: URL: https://github.com/apache/pinot/pull/12982#discussion_r1574559296 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/MultiStageBrokerRequestHandler.java: ## @@ -260,10 +276,18 @@ protected BrokerResponse handleRequest

Re: [PR] Add some multi-stage metrics [pinot]

gortiz commented on code in PR #12982: URL: https://github.com/apache/pinot/pull/12982#discussion_r1574558083 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java: ## @@ -309,7 +309,7 @@ protected BrokerResponse handleRequest(long re

Re: [PR] Add some multi-stage metrics [pinot]

codecov-commenter commented on PR #12982: URL: https://github.com/apache/pinot/pull/12982#issuecomment-2069078164 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12982?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Add some multi-stage metrics [pinot]

gortiz commented on code in PR #12982: URL: https://github.com/apache/pinot/pull/12982#discussion_r1574536612 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java: ## @@ -309,7 +309,7 @@ protected BrokerResponse handleRequest(long re

Re: [PR] Add some multi-stage metrics [pinot]

tibrewalpratik17 commented on code in PR #12982: URL: https://github.com/apache/pinot/pull/12982#discussion_r1574511131 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java: ## @@ -309,7 +309,7 @@ protected BrokerResponse handleReque

[PR] Add some multi-stage metrics [pinot]

gortiz opened a new pull request, #12982: URL: https://github.com/apache/pinot/pull/12982 Including: - Report BROKER_RESPONSES_WITH_NUM_GROUPS_LIMIT_REACHED - Report QUERY_TOTAL_TIME_MS - Report QUERY_QUOTA_EXCEEDED - Report REQUEST_DROPPED_DUE_TO_ACCESS_ERROR - Report QUERIES

Re: [PR] Add back shade profile for shade removed in #12712 [pinot]

xiangfu0 commented on PR #12979: URL: https://github.com/apache/pinot/pull/12979#issuecomment-2068839101 It's disabled by default. Plugins modules are enabled by default. We added this profile to enable shading for modules like pinot-spi, or disable plugin modules like pinot-s3. -- T

Re: [I] Grouping data by time interval [pinot]

synoxbm commented on issue #48: URL: https://github.com/apache/pinot/issues/48#issuecomment-2068670535 [Jackie-Jiang](https://github.com/Jackie-Jiang) thanks ;) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Add back shade profile for shade removed in #12712 [pinot]

gortiz commented on PR #12979: URL: https://github.com/apache/pinot/pull/12979#issuecomment-2068638072 So... IICU: - Shading is enabled by default - Some projects disable it - In some projects we can tune whether it is enabled or not by using a property Is that correct?

Re: [PR] Add back shade profile for shade removed in #12712 [pinot]

gortiz commented on code in PR #12979: URL: https://github.com/apache/pinot/pull/12979#discussion_r1574222460 ## pinot-clients/pinot-jdbc-client/pom.xml: ## @@ -82,4 +81,18 @@ jsr305 + + + build-shaded-jar + + + skipShade +