Re: [PR] feat(cleanup): Implement pauseless table segment cleanup logic [pinot]

2025-02-12 Thread via GitHub
9aman closed pull request #15002: feat(cleanup): Implement pauseless table segment cleanup logic URL: https://github.com/apache/pinot/pull/15002 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Add debug API for pauseless tables. Add pauselessFSM as an option in realtimeQuickStart [pinot]

2025-02-12 Thread via GitHub
9aman commented on PR #14961: URL: https://github.com/apache/pinot/pull/14961#issuecomment-2655777832 We took a different approach for the debug API as this approach required fetching ZK metadata for all the segments. Alternate Approach: https://github.com/apache/pinot/pull/15016 -- T

Re: [PR] Add debug API for pauseless tables. Add pauselessFSM as an option in realtimeQuickStart [pinot]

2025-02-12 Thread via GitHub
9aman closed pull request #14961: Add debug API for pauseless tables. Add pauselessFSM as an option in realtimeQuickStart URL: https://github.com/apache/pinot/pull/14961 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] feat(cleanup): Implement pauseless table segment cleanup logic [pinot]

2025-02-12 Thread via GitHub
9aman commented on PR #15002: URL: https://github.com/apache/pinot/pull/15002#issuecomment-2655774919 This approach is not taken for now as we don't feel a need to expose bulk deletion of segments via is safe -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] feat(cleanup): Implement pauseless table segment cleanup logic [pinot]

2025-02-12 Thread via GitHub
9aman closed pull request #15002: feat(cleanup): Implement pauseless table segment cleanup logic URL: https://github.com/apache/pinot/pull/15002 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] [Build] Add Maven Wrapper + increase Maven version in Docker images [pinot]

2025-02-12 Thread via GitHub
xiangfu0 commented on PR #15035: URL: https://github.com/apache/pinot/pull/15035#issuecomment-2655710725 Overall lgtm. Can you also modify the steps in the Readme -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Fixing aliased columns bug in GAPFILL based queries [pinot]

2025-02-12 Thread via GitHub
codecov-commenter commented on PR #15052: URL: https://github.com/apache/pinot/pull/15052#issuecomment-2655619281 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15052?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Fixing aliased columns bug in GAPFILL based queries [pinot]

2025-02-12 Thread via GitHub
shauryachats opened a new pull request, #15052: URL: https://github.com/apache/pinot/pull/15052 A bug exists in GAPFILL queries. Aliased columns in subqueries within GAPFILL are not correctly referenced, causing a `NullPointerException` to be thrown. This issue happens for almost all types

Re: [I] [Proposal] Query Resource Isolation in Pinot [pinot]

2025-02-12 Thread via GitHub
ankitsultana commented on issue #14551: URL: https://github.com/apache/pinot/issues/14551#issuecomment-2655477021 @vvivekiyer : overall lgtm. Had a question though: how does this sit with the binary scheduler feature you folks shipped last year? I guess the scope of this issue

Re: [PR] Track the actor that triggers the minion task [pinot]

2025-02-12 Thread via GitHub
swaminathanmanish merged PR #14829: URL: https://github.com/apache/pinot/pull/14829 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@

(pinot) branch master updated: Track the actor that triggers the minion task (#14829)

2025-02-12 Thread manishswaminathan
This is an automated email from the ASF dual-hosted git repository. manishswaminathan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new eb7489c453 Track the actor that trigge

Re: [I] Add a null check in thread sampling instrumentation [pinot]

2025-02-12 Thread via GitHub
vrajat commented on issue #15051: URL: https://github.com/apache/pinot/issues/15051#issuecomment-2655438742 cc @vvivekiyer @jasperjiaguo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Fix for NPE in subqueries in SSE when OOM protection is ON [pinot]

2025-02-12 Thread via GitHub
vrajat commented on code in PR #15046: URL: https://github.com/apache/pinot/pull/15046#discussion_r1953766412 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandler.java: ## @@ -299,601 +299,609 @@ protected BrokerResponse handle

Re: [PR] Fix for NPE in subqueries in SSE when OOM protection is ON [pinot]

2025-02-12 Thread via GitHub
vrajat commented on PR #15046: URL: https://github.com/apache/pinot/pull/15046#issuecomment-2655415009 @Jackie-Jiang, +1 to what @yashmayya said. The "refactor" is the fix. It mostly consists of whitespace changes. The body of the function _has to be_ moved out of the try/finally block. Tha

Re: [PR] Fix for NPE in subqueries in SSE when OOM protection is ON [pinot]

2025-02-12 Thread via GitHub
yashmayya commented on PR #15046: URL: https://github.com/apache/pinot/pull/15046#issuecomment-2655350626 > and quite hard to follow the actual fix @Jackie-Jiang you can view the diff in GitHub UI with the `Hide whitespace` option checked, the actual diff here is very small and there'

Re: [PR] Add a dry-run summary mode for TableRebalance which only returns a summary of the dry-run results [pinot]

2025-02-12 Thread via GitHub
codecov-commenter commented on PR #15050: URL: https://github.com/apache/pinot/pull/15050#issuecomment-2655273599 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15050?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Add a dry-run summary mode for TableRebalance which only returns a summary of the dry-run results [pinot]

2025-02-12 Thread via GitHub
somandal opened a new pull request, #15050: URL: https://github.com/apache/pinot/pull/15050 This PR adds a `summary` option to the TableRebalance API which is meant to be used with `dryRun`. If `summary` is set to `true` a summary of the dry-run is returned rather than the full dry-run. Thi

Re: [PR] Adding a testcase for testing a table where kafka partition has been reduced after creating the table [pinot]

2025-02-12 Thread via GitHub
soumitra-st commented on code in PR #15028: URL: https://github.com/apache/pinot/pull/15028#discussion_r1953594944 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/KafkaIncreaseDecreasePartitionsIntegrationTest.java: ## @@ -0,0 +1,158 @@ +/** + * Licen

Re: [PR] Adding a testcase for testing a table where kafka partition has been reduced after creating the table [pinot]

2025-02-12 Thread via GitHub
soumitra-st commented on code in PR #15028: URL: https://github.com/apache/pinot/pull/15028#discussion_r1953593644 ## pinot-spi/src/main/java/org/apache/pinot/spi/stream/StreamDataServerStartable.java: ## @@ -54,6 +54,22 @@ public interface StreamDataServerStartable { */

(pinot) branch master updated (9c32d53874 -> 02b3046b2b)

2025-02-12 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 9c32d53874 Bump flink.version from 1.20.0 to 1.20.1 (#15042) add 02b3046b2b add option to delete table and schema

Re: [PR] add option to delete table and schema immediately [pinot]

2025-02-12 Thread via GitHub
Jackie-Jiang merged PR #14736: URL: https://github.com/apache/pinot/pull/14736 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [I] Error with streaming Kafka Protobuf Messages into Pinot table [pinot]

2025-02-12 Thread via GitHub
Jackie-Jiang commented on issue #15043: URL: https://github.com/apache/pinot/issues/15043#issuecomment-2654969448 Check the server log which should provides more details cc @KKcorps -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [I] Do we have a timeline for when CLP querying will be production ready? [pinot]

2025-02-12 Thread via GitHub
Jackie-Jiang commented on issue #15003: URL: https://github.com/apache/pinot/issues/15003#issuecomment-2654964078 cc @jackluo923 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Improve JSON_MATCH performance. [pinot]

2025-02-12 Thread via GitHub
codecov-commenter commented on PR #15049: URL: https://github.com/apache/pinot/pull/15049#issuecomment-2654956052 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15049?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Improve JSON_MATCH performance. [pinot]

2025-02-12 Thread via GitHub
bziobrowski opened a new pull request, #15049: URL: https://github.com/apache/pinot/pull/15049 PR improves performance and reduces temporary object generation in `JSON_MATCH()` function calls by: - reducing and/or delaying immutable to mutable conversion as much as possible - using i

Re: [PR] Added support to pause and resume ingestion based on resource utilization [pinot]

2025-02-12 Thread via GitHub
rajagopr commented on code in PR #15008: URL: https://github.com/apache/pinot/pull/15008#discussion_r1953226440 ## pinot-controller/src/main/java/org/apache/pinot/controller/validation/DiskUtilizationChecker.java: ## @@ -0,0 +1,134 @@ +/** + * Licensed to the Apache Software Fou

Re: [PR] Add support for performing pre-checks for TableRebalance [pinot]

2025-02-12 Thread via GitHub
somandal commented on code in PR #15029: URL: https://github.com/apache/pinot/pull/15029#discussion_r1953369779 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/DefaultRebalancePreChecker.java: ## @@ -0,0 +1,126 @@ +/** + * Licensed to the Apach

Re: [PR] Add support for performing pre-checks for TableRebalance [pinot]

2025-02-12 Thread via GitHub
somandal commented on code in PR #15029: URL: https://github.com/apache/pinot/pull/15029#discussion_r1953368390 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -186,13 +190,21 @@ private RebalanceResult doRebalance(T

Re: [PR] Add support for performing pre-checks for TableRebalance [pinot]

2025-02-12 Thread via GitHub
somandal commented on code in PR #15029: URL: https://github.com/apache/pinot/pull/15029#discussion_r1953368390 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -186,13 +190,21 @@ private RebalanceResult doRebalance(T

Re: [PR] Add support for performing pre-checks for TableRebalance [pinot]

2025-02-12 Thread via GitHub
somandal commented on code in PR #15029: URL: https://github.com/apache/pinot/pull/15029#discussion_r1953366481 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalancePreChecker.java: ## @@ -0,0 +1,29 @@ +/** + * Licensed to the Apache Softwa

(pinot) branch master updated: Bump flink.version from 1.20.0 to 1.20.1 (#15042)

2025-02-12 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 9c32d53874 Bump flink.version from 1.20.0 to 1.20

Re: [PR] Add support for performing pre-checks for TableRebalance [pinot]

2025-02-12 Thread via GitHub
somandal commented on code in PR #15029: URL: https://github.com/apache/pinot/pull/15029#discussion_r1953298187 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/DefaultRebalancePreChecker.java: ## @@ -0,0 +1,126 @@ +/** + * Licensed to the Apach

(pinot) branch dependabot/maven/net.minidev-json-smart-2.5.2 deleted (was 1dfd4c0c4e)

2025-02-12 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/net.minidev-json-smart-2.5.2 in repository https://gitbox.apache.org/repos/asf/pinot.git was 1dfd4c0c4e Bump net.minidev:json-smart from 2.5.1 to 2.5.2 The revisions that w

(pinot) branch dependabot/maven/flink.version-1.20.1 deleted (was bb03bff090)

2025-02-12 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/flink.version-1.20.1 in repository https://gitbox.apache.org/repos/asf/pinot.git was bb03bff090 Bump flink.version from 1.20.0 to 1.20.1 The revisions that were on this bra

Re: [PR] Bump flink.version from 1.20.0 to 1.20.1 [pinot]

2025-02-12 Thread via GitHub
Jackie-Jiang merged PR #15042: URL: https://github.com/apache/pinot/pull/15042 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: Bump net.minidev:json-smart from 2.5.1 to 2.5.2 (#15041)

2025-02-12 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 2af35d4ade Bump net.minidev:json-smart from 2.5.1

Re: [PR] Bump net.minidev:json-smart from 2.5.1 to 2.5.2 [pinot]

2025-02-12 Thread via GitHub
Jackie-Jiang merged PR #15041: URL: https://github.com/apache/pinot/pull/15041 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch dependabot/maven/com.mchange-c3p0-0.11.0-pre1 deleted (was 3eed52fdd5)

2025-02-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.mchange-c3p0-0.11.0-pre1 in repository https://gitbox.apache.org/repos/asf/pinot.git was 3eed52fdd5 Bump com.mchange:c3p0 from 0.10.2 to 0.11.0-pre1 The revisions

(pinot) branch dependabot/maven/com.mchange-mchange-commons-java-0.3.2 deleted (was c66a52f573)

2025-02-12 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/com.mchange-mchange-commons-java-0.3.2 in repository https://gitbox.apache.org/repos/asf/pinot.git was c66a52f573 Bump com.mchange:mchange-commons-java from 0.3.1 to 0.3.2

Re: [PR] Bump com.mchange:mchange-commons-java from 0.3.1 to 0.3.2 [pinot]

2025-02-12 Thread via GitHub
Jackie-Jiang merged PR #15039: URL: https://github.com/apache/pinot/pull/15039 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: Bump com.mchange:mchange-commons-java from 0.3.1 to 0.3.2 (#15039)

2025-02-12 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new daa8229cae Bump com.mchange:mchange-commons-java

Re: [PR] Bump com.mchange:c3p0 from 0.10.2 to 0.11.0-pre1 [pinot]

2025-02-12 Thread via GitHub
dependabot[bot] commented on PR #15038: URL: https://github.com/apache/pinot/pull/15038#issuecomment-2654695552 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] Bump com.mchange:c3p0 from 0.10.2 to 0.11.0-pre1 [pinot]

2025-02-12 Thread via GitHub
Jackie-Jiang closed pull request #15038: Bump com.mchange:c3p0 from 0.10.2 to 0.11.0-pre1 URL: https://github.com/apache/pinot/pull/15038 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Fix for NPE in subqueries in SSE when OOM protection is ON [pinot]

2025-02-12 Thread via GitHub
Jackie-Jiang commented on code in PR #15046: URL: https://github.com/apache/pinot/pull/15046#discussion_r1953295349 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandler.java: ## @@ -299,601 +299,609 @@ protected BrokerResponse

(pinot) branch master updated: Adding jute.maxBuffer in java opts as an example (#15047)

2025-02-12 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 09a7ab51c9 Adding jute.maxBuffer in java opts as

Re: [PR] Add jute.maxBuffer in java opts as an example [pinot]

2025-02-12 Thread via GitHub
xiangfu0 merged PR #15047: URL: https://github.com/apache/pinot/pull/15047 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] Rely on segment ZK download url for deleting segment in the deletion manager [pinot]

2025-02-12 Thread via GitHub
codecov-commenter commented on PR #15048: URL: https://github.com/apache/pinot/pull/15048#issuecomment-2654637385 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15048?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Add support for performing pre-checks for TableRebalance [pinot]

2025-02-12 Thread via GitHub
klsince commented on code in PR #15029: URL: https://github.com/apache/pinot/pull/15029#discussion_r1953229112 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -186,13 +190,21 @@ private RebalanceResult doRebalance(Ta

Re: [PR] Add support for performing pre-checks for TableRebalance [pinot]

2025-02-12 Thread via GitHub
klsince commented on code in PR #15029: URL: https://github.com/apache/pinot/pull/15029#discussion_r1953233568 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -186,13 +190,21 @@ private RebalanceResult doRebalance(Ta

Re: [PR] Add support for performing pre-checks for TableRebalance [pinot]

2025-02-12 Thread via GitHub
klsince commented on code in PR #15029: URL: https://github.com/apache/pinot/pull/15029#discussion_r1953238433 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalancePreCheckerFactory.java: ## @@ -0,0 +1,42 @@ +/** + * Licensed to the Apache

Re: [PR] Do not convert IN/NOT_IN to OR lists [pinot]

2025-02-12 Thread via GitHub
Jackie-Jiang commented on PR #15027: URL: https://github.com/apache/pinot/pull/15027#issuecomment-2654588365 @gortiz The idea is to reverse the effect of Calcite converting the IN/NOT_IN clause into a `JOIN` back to their original form. I don't see a scenario where they can be converted int

Re: [PR] Add jute.maxBuffer in java opts as an example [pinot]

2025-02-12 Thread via GitHub
codecov-commenter commented on PR #15047: URL: https://github.com/apache/pinot/pull/15047#issuecomment-2654584498 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15047?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

(pinot) branch master updated (42335ae68d -> 7fcb82cc00)

2025-02-12 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 42335ae68d Improve error message when colocated join is requested but workermanager is disabled (#14947) add 7fcb

Re: [PR] Pauseless Consumption #3: Disaster Recovery with Reingestion [pinot]

2025-02-12 Thread via GitHub
Jackie-Jiang merged PR #14920: URL: https://github.com/apache/pinot/pull/14920 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[PR] Add jute.maxBuffer in java opts as an example [pinot]

2025-02-12 Thread via GitHub
xiangfu0 opened a new pull request, #15047: URL: https://github.com/apache/pinot/pull/15047 We recently observed more users facing issues with znode size issue: https://docs.pinot.apache.org/reference/troubleshooting/troubleshoot-zookeeper#adjust-zookeeper-znode-size Hence add this

Re: [PR] Fix for NPE in subqueries in SSE when OOM protection is ON [pinot]

2025-02-12 Thread via GitHub
yashmayya commented on code in PR #15046: URL: https://github.com/apache/pinot/pull/15046#discussion_r1953100410 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OOMProtectionIntegrationTest.java: ## @@ -0,0 +1,110 @@ +/** + * Licensed to the Apache So

Re: [PR] Parallelize segment metadata file generation. [pinot]

2025-02-12 Thread via GitHub
abhishekbafna commented on code in PR #15030: URL: https://github.com/apache/pinot/pull/15030#discussion_r1953091793 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/SegmentPushUtils.java: ## @@ -458,10 +438,71 @@ public static void sendSegmentsUriAndMe

Re: [PR] Parallelize segment metadata file generation. [pinot]

2025-02-12 Thread via GitHub
rajagopr commented on code in PR #15030: URL: https://github.com/apache/pinot/pull/15030#discussion_r1953067309 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/SegmentPushUtils.java: ## @@ -458,10 +438,71 @@ public static void sendSegmentsUriAndMetadat

[PR] Fix for NPE in subqueries in SSE when OOM protection is ON [pinot]

2025-02-12 Thread via GitHub
vrajat opened a new pull request, #15046: URL: https://github.com/apache/pinot/pull/15046 The diff in `BaseSingleStageBrokerRequestHandler`. However the body of the fn. has not changed. The code has been moved to `handleRequestImpl` which other internal functions can use. Since the try/catc

Re: [PR] Fix for NPE in subqueries in SSE when OOM protection is ON [pinot]

2025-02-12 Thread via GitHub
codecov-commenter commented on PR #15046: URL: https://github.com/apache/pinot/pull/15046#issuecomment-2654143725 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15046?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Refactor query error handling to use QueryErrorCode and QueryErrorMessage for improved clarity and consistency [pinot]

2025-02-12 Thread via GitHub
codecov-commenter commented on PR #15037: URL: https://github.com/apache/pinot/pull/15037#issuecomment-2654058735 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15037?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[I] Error with streaming Kafka Protobuf Messages into Pinot table [pinot]

2025-02-12 Thread via GitHub
csdavidc opened a new issue, #15043: URL: https://github.com/apache/pinot/issues/15043 i'm running pinot on a docker container, I created a realtime table with the following configuration: ``` json { "REALTIME": { "tableName": "sso_REALTIME", "tableType": "REAL

Re: [PR] Decouple tracking and storage in MinionEventObserver using MinionTaskProgressManager [pinot]

2025-02-12 Thread via GitHub
codecov-commenter commented on PR #15044: URL: https://github.com/apache/pinot/pull/15044#issuecomment-2653935825 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15044?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[I] Subqueries in Single Stage Engine fail with NPE if OOM protection is on. [pinot]

2025-02-12 Thread via GitHub
vrajat opened a new issue, #15045: URL: https://github.com/apache/pinot/issues/15045 A reproducer is available in https://github.com/apache/pinot/compare/master...vrajat:rv-oom-subquery-npe The issue is that subqueries make recursive calls to `BaseSingleStageBrokerRequestHandler.hand

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.30.18 created (now 956c572e62)

2025-02-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.30.18 in repository https://gitbox.apache.org/repos/asf/pinot.git at 956c572e62 Bump software.amazon.awssdk:bom from 2.30.16 to 2.30.18 N

[PR] Decouple tracking and storage in MinionEventObserver using MinionTaskProgressManager [pinot]

2025-02-12 Thread via GitHub
shounakmk219 opened a new pull request, #15044: URL: https://github.com/apache/pinot/pull/15044 Splitting #14521 into separate PRs # Description This PR tries to decouple the way how minion task progress is being tracked and stored so that we can reuse the implementations of `M

Re: [PR] Bump software.amazon.awssdk:bom from 2.30.16 to 2.30.17 [pinot]

2025-02-12 Thread via GitHub
dependabot[bot] commented on PR #15032: URL: https://github.com/apache/pinot/pull/15032#issuecomment-2653479513 Superseded by #15040. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Do not convert IN/NOT_IN to OR lists [pinot]

2025-02-12 Thread via GitHub
gortiz commented on code in PR #15027: URL: https://github.com/apache/pinot/pull/15027#discussion_r1952517625 ## pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotLeftJoinToNotInClauseRule.java: ## @@ -0,0 +1,133 @@ +/** + * Licensed to the Apache Softwar

Re: [PR] Added support to pause and resume ingestion based on resource utilization [pinot]

2025-02-12 Thread via GitHub
swaminathanmanish commented on code in PR #15008: URL: https://github.com/apache/pinot/pull/15008#discussion_r1952516625 ## pinot-controller/src/main/java/org/apache/pinot/controller/validation/ResourceUtilizationInfo.java: ## @@ -0,0 +1,45 @@ +/** + * Licensed to the Apache Sof

Re: [PR] Spool intermediate stage fix [pinot]

2025-02-12 Thread via GitHub
gortiz commented on code in PR #15024: URL: https://github.com/apache/pinot/pull/15024#discussion_r1952497149 ## pinot-query-planner/src/main/java/org/apache/pinot/query/planner/logical/PlanFragmenter.java: ## @@ -89,12 +89,26 @@ private PlanNode process(PlanNode node, Context c

Re: [PR] Do not convert IN/NOT_IN to OR lists [pinot]

2025-02-12 Thread via GitHub
gortiz commented on code in PR #15027: URL: https://github.com/apache/pinot/pull/15027#discussion_r1952504314 ## pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotRuleUtils.java: ## @@ -46,10 +46,12 @@ private PinotRuleUtils() { public static final Rel

Re: [PR] Improve error message when colocated join is requested but workermanager is disabled [pinot]

2025-02-12 Thread via GitHub
gortiz merged PR #14947: URL: https://github.com/apache/pinot/pull/14947 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

(pinot) branch master updated: Improve error message when colocated join is requested but workermanager is disabled (#14947)

2025-02-12 Thread gortiz
This is an automated email from the ASF dual-hosted git repository. gortiz pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 42335ae68d Improve error message when colocated j

(pinot) branch dependabot/maven/flink.version-1.20.1 created (now bb03bff090)

2025-02-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/flink.version-1.20.1 in repository https://gitbox.apache.org/repos/asf/pinot.git at bb03bff090 Bump flink.version from 1.20.0 to 1.20.1 No new revisions were added by

[PR] Bump flink.version from 1.20.0 to 1.20.1 [pinot]

2025-02-12 Thread via GitHub
dependabot[bot] opened a new pull request, #15042: URL: https://github.com/apache/pinot/pull/15042 Bumps `flink.version` from 1.20.0 to 1.20.1. Updates `org.apache.flink:flink-clients` from 1.20.0 to 1.20.1 Commits See full diff in https://github.com/apache/flink/commits";>co

Re: [PR] Spool intermediate stage fix [pinot]

2025-02-12 Thread via GitHub
gortiz merged PR #15024: URL: https://github.com/apache/pinot/pull/15024 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

(pinot) branch master updated (a209f9b0b1 -> 2bb5cd0441)

2025-02-12 Thread gortiz
This is an automated email from the ASF dual-hosted git repository. gortiz pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from a209f9b0b1 Adding a new list in the propertystore to capture the committing segments. (#15016) add 2bb5cd0441 Spo

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.30.17 deleted (was 1eb42e0bc1)

2025-02-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.30.17 in repository https://gitbox.apache.org/repos/asf/pinot.git was 1eb42e0bc1 Bump software.amazon.awssdk:bom from 2.30.16 to 2.30.17 T

(pinot) branch dependabot/maven/com.mchange-mchange-commons-java-0.3.2 created (now c66a52f573)

2025-02-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.mchange-mchange-commons-java-0.3.2 in repository https://gitbox.apache.org/repos/asf/pinot.git at c66a52f573 Bump com.mchange:mchange-commons-java from 0.3.1 to 0.

[PR] Bump com.mchange:mchange-commons-java from 0.3.1 to 0.3.2 [pinot]

2025-02-12 Thread via GitHub
dependabot[bot] opened a new pull request, #15039: URL: https://github.com/apache/pinot/pull/15039 Bumps [com.mchange:mchange-commons-java](https://github.com/swaldman/mchange-commons-java) from 0.3.1 to 0.3.2. Commits https://github.com/swaldman/mchange-commons-java/commit/e7

(pinot) branch dependabot/maven/net.minidev-json-smart-2.5.2 created (now 1dfd4c0c4e)

2025-02-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/net.minidev-json-smart-2.5.2 in repository https://gitbox.apache.org/repos/asf/pinot.git at 1dfd4c0c4e Bump net.minidev:json-smart from 2.5.1 to 2.5.2 No new revision

[PR] Bump net.minidev:json-smart from 2.5.1 to 2.5.2 [pinot]

2025-02-12 Thread via GitHub
dependabot[bot] opened a new pull request, #15041: URL: https://github.com/apache/pinot/pull/15041 Bumps [net.minidev:json-smart](https://github.com/netplex/json-smart-v2) from 2.5.1 to 2.5.2. Release notes Sourced from https://github.com/netplex/json-smart-v2/releases";>net.minide

[PR] Bump software.amazon.awssdk:bom from 2.30.16 to 2.30.18 [pinot]

2025-02-12 Thread via GitHub
dependabot[bot] opened a new pull request, #15040: URL: https://github.com/apache/pinot/pull/15040 Bumps software.amazon.awssdk:bom from 2.30.16 to 2.30.18. Most Recent Ignore Conditions Applied to This Pull Request | Dependency Name | Ignore Conditions | | --- | --- |

Re: [PR] Bump software.amazon.awssdk:bom from 2.30.16 to 2.30.17 [pinot]

2025-02-12 Thread via GitHub
dependabot[bot] closed pull request #15032: Bump software.amazon.awssdk:bom from 2.30.16 to 2.30.17 URL: https://github.com/apache/pinot/pull/15032 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[PR] Bump com.mchange:c3p0 from 0.10.2 to 0.11.0-pre1 [pinot]

2025-02-12 Thread via GitHub
dependabot[bot] opened a new pull request, #15038: URL: https://github.com/apache/pinot/pull/15038 Bumps [com.mchange:c3p0](https://github.com/swaldman/c3p0) from 0.10.2 to 0.11.0-pre1. Commits https://github.com/swaldman/c3p0/commit/bbebcc22e2f9b6080e8354042a91f6d02e50e392";>b

(pinot) branch dependabot/maven/com.mchange-c3p0-0.11.0-pre1 created (now 3eed52fdd5)

2025-02-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.mchange-c3p0-0.11.0-pre1 in repository https://gitbox.apache.org/repos/asf/pinot.git at 3eed52fdd5 Bump com.mchange:c3p0 from 0.10.2 to 0.11.0-pre1 No new revisio

[PR] Refactor query error handling to use QueryErrorCode and QueryErrorMessage for improved clarity and consistency [pinot]

2025-02-12 Thread via GitHub
gortiz opened a new pull request, #15037: URL: https://github.com/apache/pinot/pull/15037 This PR derives from https://github.com/apache/pinot/pull/14994. Here, I'm applying the bigger refactors required to clean up the error hierarchy. Although the number of lines changed is pretty extensi

Re: [PR] Added support to pause and resume ingestion based on resource utilization [pinot]

2025-02-12 Thread via GitHub
swaminathanmanish commented on code in PR #15008: URL: https://github.com/apache/pinot/pull/15008#discussion_r1952034182 ## pinot-controller/src/main/java/org/apache/pinot/controller/validation/DiskUtilizationChecker.java: ## @@ -0,0 +1,134 @@ +/** + * Licensed to the Apache Sof

Re: [PR] Added support to pause and resume ingestion based on resource utilization [pinot]

2025-02-12 Thread via GitHub
swaminathanmanish commented on code in PR #15008: URL: https://github.com/apache/pinot/pull/15008#discussion_r1952046465 ## pinot-controller/src/main/java/org/apache/pinot/controller/validation/DiskUtilizationChecker.java: ## @@ -0,0 +1,134 @@ +/** + * Licensed to the Apache Sof

Re: [PR] Parallelize segment metadata file generation. [pinot]

2025-02-12 Thread via GitHub
abhishekbafna commented on PR #15030: URL: https://github.com/apache/pinot/pull/15030#issuecomment-2653379168 > > pushParallelism > > @abhishekbafna - Please update description. Done. -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [I] Issue with NULL storage in column with 1.4.0 (must be there earlier versions too) [pinot]

2025-02-12 Thread via GitHub
raghukn commented on issue #14922: URL: https://github.com/apache/pinot/issues/14922#issuecomment-2653339359 Oh! got it 1. Table Config: "nullHandlingEnabled": true 2. Schema Config: "enableColumnBasedNullHandling": true 3. Query: enableNullHandling=true This is what is

Re: [I] [Doc] How long till data is ready to be consumed at speed? [pinot]

2025-02-12 Thread via GitHub
hpvd commented on issue #15036: URL: https://github.com/apache/pinot/issues/15036#issuecomment-2653309463 btw: at operation of Pinot this kind of metric of "e2e freshness" would also have relevance in many use cases and would allow to - to find problems - help to opzimize - proove

Re: [PR] Parallelize segment metadata file generation. [pinot]

2025-02-12 Thread via GitHub
swaminathanmanish commented on PR #15030: URL: https://github.com/apache/pinot/pull/15030#issuecomment-2653235068 > pushParallelism @abhishekbafna - Please update description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

(pinot) branch master updated: Adding a new list in the propertystore to capture the committing segments. (#15016)

2025-02-12 Thread manishswaminathan
This is an automated email from the ASF dual-hosted git repository. manishswaminathan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new a209f9b0b1 Adding a new list in the pr

Re: [PR] Adding a new list in the propertystore to capture the committing segments. [pinot]

2025-02-12 Thread via GitHub
swaminathanmanish merged PR #15016: URL: https://github.com/apache/pinot/pull/15016 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@

Re: [I] Improve/Add completeness and freshness metrics for Pinot [pinot]

2025-02-12 Thread via GitHub
hpvd commented on issue #4007: URL: https://github.com/apache/pinot/issues/4007#issuecomment-2653140252 would be also important to ad something like last ingest /not ingested for xxx ms to the metrics to easily find things like Pausing of consumptions see https://github.com/apache/pinot/i

Re: [I] Pinot real-time data ingestion freshness guarantee [pinot]

2025-02-12 Thread via GitHub
hpvd commented on issue #14815: URL: https://github.com/apache/pinot/issues/14815#issuecomment-2653123132 old but open and strongly related to the field of freshness: Improve/Add completeness and freshness metrics for Pinot https://github.com/apache/pinot/issues/4007 with the proposal

Re: [PR] Parallelize segment metadata file generation. [pinot]

2025-02-12 Thread via GitHub
swaminathanmanish commented on code in PR #15030: URL: https://github.com/apache/pinot/pull/15030#discussion_r1952215580 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/SegmentPushUtils.java: ## @@ -458,10 +438,71 @@ public static void sendSegmentsUriA

Re: [PR] Adding a testcase for testing a table where kafka partition has been reduced after creating the table [pinot]

2025-02-12 Thread via GitHub
noob-se7en commented on code in PR #15028: URL: https://github.com/apache/pinot/pull/15028#discussion_r1952197917 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/KafkaIncreaseDecreasePartitionsIntegrationTest.java: ## @@ -0,0 +1,158 @@ +/** + * Licens

Re: [PR] Adding a testcase for testing a table where kafka partition has been reduced after creating the table [pinot]

2025-02-12 Thread via GitHub
noob-se7en commented on code in PR #15028: URL: https://github.com/apache/pinot/pull/15028#discussion_r1952192959 ## pinot-spi/src/main/java/org/apache/pinot/spi/stream/StreamDataServerStartable.java: ## @@ -54,6 +54,22 @@ public interface StreamDataServerStartable { */ v

  1   2   >