Re: [PR] Clean up query executor configs [pinot]

2025-02-19 Thread via GitHub
bziobrowski commented on code in PR #15095: URL: https://github.com/apache/pinot/pull/15095#discussion_r1963011735 ## pinot-core/src/main/java/org/apache/pinot/core/plan/maker/InstancePlanMakerImplV2.java: ## @@ -65,72 +65,76 @@ * The InstancePlanMakerImplV2 class is the defau

Re: [PR] Add a dry-run summary mode for TableRebalance which only returns a summary of the dry-run results [pinot]

2025-02-19 Thread via GitHub
somandal commented on code in PR #15050: URL: https://github.com/apache/pinot/pull/15050#discussion_r1962234112 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceSummaryResult.java: ## @@ -0,0 +1,325 @@ +/** + * Licensed to the Apache So

(pinot) branch master updated: Make fields protected for DefaultRebalancePreChecker (#15091)

2025-02-19 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new f9750e2da0 Make fields protected for DefaultRebal

Re: [PR] Make fields protected for DefaultRebalancePreChecker [pinot]

2025-02-19 Thread via GitHub
Jackie-Jiang merged PR #15091: URL: https://github.com/apache/pinot/pull/15091 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Fixing ordering issue in subquery for time column in GAPFILL based queries [pinot]

2025-02-19 Thread via GitHub
codecov-commenter commented on PR #15096: URL: https://github.com/apache/pinot/pull/15096#issuecomment-2670293445 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15096?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Fixing ordering issue in subquery for time column in GAPFILL based queries [pinot]

2025-02-19 Thread via GitHub
shauryachats opened a new pull request, #15096: URL: https://github.com/apache/pinot/pull/15096 The current GAPFILL implementation expects the time column series to always be the first column in any subquery executed before the GAPFILL operation, which is not an ideal behaviour since it bre

Re: [PR] Clean up query executor configs [pinot]

2025-02-19 Thread via GitHub
codecov-commenter commented on PR #15095: URL: https://github.com/apache/pinot/pull/15095#issuecomment-2670127838 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15095?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Clean up query executor configs [pinot]

2025-02-19 Thread via GitHub
Jackie-Jiang commented on PR #15095: URL: https://github.com/apache/pinot/pull/15095#issuecomment-2670075974 cc @bziobrowski -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Clean up query executor configs [pinot]

2025-02-19 Thread via GitHub
Jackie-Jiang commented on code in PR #15095: URL: https://github.com/apache/pinot/pull/15095#discussion_r1962570761 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/AggregateOperator.java: ## @@ -108,33 +110,28 @@ public AggregateOperator(OpChainExecu

[PR] Clean up query executor configs [pinot]

2025-02-19 Thread via GitHub
Jackie-Jiang opened a new pull request, #15095: URL: https://github.com/apache/pinot/pull/15095 Main changes: - Move config constants in `InstancePlanMakerImplV2` and `SegmentPrunerConfig` into `CommonConstants` - Remove config wrapper `QueryExecutorConfig` - Rename the following co

Re: [PR] Changes default load mode to MMAP [pinot]

2025-02-19 Thread via GitHub
Jackie-Jiang merged PR #15089: URL: https://github.com/apache/pinot/pull/15089 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: Changes default load mode to MMAP (#15089)

2025-02-19 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new a1e8ab26f0 Changes default load mode to MMAP (#15

(pinot) branch hotfix-empty-schema updated: protect usage MSQE compiler for empty schema polyfill with config param disabled by default (#15078)

2025-02-19 Thread jtao
This is an automated email from the ASF dual-hosted git repository. jtao pushed a commit to branch hotfix-empty-schema in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/hotfix-empty-schema by this push: new 85e4761a2d protect usage

(pinot) branch hotfix-empty-schema created (now c7fcda980b)

2025-02-19 Thread jtao
This is an automated email from the ASF dual-hosted git repository. jtao pushed a change to branch hotfix-empty-schema in repository https://gitbox.apache.org/repos/asf/pinot.git at c7fcda980b [timeseries] Add Support for limit and numGroupsLimit (#14945) No new revisions were added by t

Re: [PR] Delete Old Files containing Google Analytics in new-static-prod branch [pinot-site]

2025-02-19 Thread via GitHub
niallkp commented on PR #140: URL: https://github.com/apache/pinot-site/pull/140#issuecomment-2669867499 Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

(pinot) branch master updated: Add a dry-run summary mode for TableRebalance which only returns a summary of the dry-run results (#15050)

2025-02-19 Thread xbli
This is an automated email from the ASF dual-hosted git repository. xbli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 31e94d3b18 Add a dry-run summary mode for TableReba

Re: [PR] Add `DATE_TRUNC` Optimizer [pinot]

2025-02-19 Thread via GitHub
jadami10 commented on code in PR #14385: URL: https://github.com/apache/pinot/pull/14385#discussion_r1962401280 ## pinot-core/src/test/java/org/apache/pinot/core/query/optimizer/filter/TimePredicateFilterOptimizerTest.java: ## @@ -121,38 +114,79 @@ public void testEpochToEpochDa

Re: [PR] Add a dry-run summary mode for TableRebalance which only returns a summary of the dry-run results [pinot]

2025-02-19 Thread via GitHub
klsince merged PR #15050: URL: https://github.com/apache/pinot/pull/15050 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

Re: [PR] Make KafkaConsumerFactory compatible with multi-topics ingestion [pinot]

2025-02-19 Thread via GitHub
codecov-commenter commented on PR #15094: URL: https://github.com/apache/pinot/pull/15094#issuecomment-2669709709 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15094?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Make KafkaConsumerFactory compatible with multi-topics ingestion [pinot]

2025-02-19 Thread via GitHub
lnbest0707-uber opened a new pull request, #15094: URL: https://github.com/apache/pinot/pull/15094 `bugfix` The previous change on https://github.com/apache/pinot/blob/6c80b932d993027620d69a979769e9fac8e913f9/pinot-spi/src/main/java/org/apache/pinot/spi/stream/StreamConsumerFactory.ja

Re: [PR] Add a dry-run summary mode for TableRebalance which only returns a summary of the dry-run results [pinot]

2025-02-19 Thread via GitHub
somandal commented on code in PR #15050: URL: https://github.com/apache/pinot/pull/15050#discussion_r1962235392 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -559,6 +588,146 @@ private RebalanceResult doRebalance(T

Re: [PR] Add a dry-run summary mode for TableRebalance which only returns a summary of the dry-run results [pinot]

2025-02-19 Thread via GitHub
somandal commented on code in PR #15050: URL: https://github.com/apache/pinot/pull/15050#discussion_r1962234873 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -291,43 +305,58 @@ private RebalanceResult doRebalance(T

Re: [PR] Add a dry-run summary mode for TableRebalance which only returns a summary of the dry-run results [pinot]

2025-02-19 Thread via GitHub
klsince commented on code in PR #15050: URL: https://github.com/apache/pinot/pull/15050#discussion_r1962208544 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java: ## @@ -559,6 +588,146 @@ private RebalanceResult doRebalance(Ta

Re: [I] Pauseless Consumption [pinot]

2025-02-19 Thread via GitHub
Jackie-Jiang closed issue #10147: Pauseless Consumption URL: https://github.com/apache/pinot/issues/10147 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail

Re: [PR] Make fields protected for DefaultRebalancePreChecker [pinot]

2025-02-19 Thread via GitHub
somandal commented on PR #15091: URL: https://github.com/apache/pinot/pull/15091#issuecomment-2669451928 test failure seems transient, passes when i run it locally: ``` Error: OfflineClusterIntegrationTest.testStringComparisonInFilter:3494->BaseClusterIntegrationTest.getType:742

Re: [PR] disable polyfill until we face the performance penalty [pinot]

2025-02-19 Thread via GitHub
albertobastos commented on PR #15075: URL: https://github.com/apache/pinot/pull/15075#issuecomment-2669438432 @gortiz My bad, this PR has been abandoned in favor of https://github.com/apache/pinot/pull/15078 and should have been closed. -- This is an automated message from the Apache Git

Re: [PR] disable polyfill until we face the performance penalty [pinot]

2025-02-19 Thread via GitHub
albertobastos closed pull request #15075: disable polyfill until we face the performance penalty URL: https://github.com/apache/pinot/pull/15075 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Make fields protected for DefaultRebalancePreChecker [pinot]

2025-02-19 Thread via GitHub
codecov-commenter commented on PR #15091: URL: https://github.com/apache/pinot/pull/15091#issuecomment-2669413703 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15091?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [I] Support Pinot as a DataSource in Grafana plugin [pinot]

2025-02-19 Thread via GitHub
PeterCorless commented on issue #6639: URL: https://github.com/apache/pinot/issues/6639#issuecomment-2669368975 Note that there is a corresponding Grafana issue: [Grafana #20141 ](https://github.com/grafana/grafana/issues/20141) -- This is an automated message from the Apache Git Service.

[I] Make MMAP_SIZE_THRESHOLD in OffHeapSingleTreeBuilder builder is configurable [pinot]

2025-02-19 Thread via GitHub
chrajeshbabu opened a new issue, #15093: URL: https://github.com/apache/pinot/issues/15093 Currently when the buffer size required for sort and aggregate the records require for star tree building above 500mb using MMAP based files used which is making star tree building slower compared to

Re: [I] Make MAX_NUM_THREADS_PER_QUERY configurable [pinot]

2025-02-19 Thread via GitHub
chrajeshbabu commented on issue #15092: URL: https://github.com/apache/pinot/issues/15092#issuecomment-2669328989 Validated performance of a query to be processed 100 segments per server with 10, 20 threads latency of the query halved which is 100% performance benefit. So would be better to

[I] Make MAX_NUM_THREADS_PER_QUERY configurable [pinot]

2025-02-19 Thread via GitHub
chrajeshbabu opened a new issue, #15092: URL: https://github.com/apache/pinot/issues/15092 Currently max number of threads to be used for a query having combine operator is 10 which can be made configurable when we have more number of segments to be processed and have high number of cores i

[PR] Make fields protected for DefaultRebalancePreChecker [pinot]

2025-02-19 Thread via GitHub
somandal opened a new pull request, #15091: URL: https://github.com/apache/pinot/pull/15091 This PR makes the `executorService` and `pinotHelixResourceManager` fields of `DefaultRebalancePreChecker` class as protected -- This is an automated message from the Apache Git Service. To respond

Re: [PR] protect usage MSQE compiler for empty schema polyfill with config param [pinot]

2025-02-19 Thread via GitHub
gortiz merged PR #15078: URL: https://github.com/apache/pinot/pull/15078 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

(pinot) branch master updated: protect usage MSQE compiler for empty schema polyfill with config param disabled by default (#15078)

2025-02-19 Thread gortiz
This is an automated email from the ASF dual-hosted git repository. gortiz pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 6c80b932d9 protect usage MSQE compiler for empty

Re: [PR] disable polyfill until we face the performance penalty [pinot]

2025-02-19 Thread via GitHub
gortiz commented on PR #15075: URL: https://github.com/apache/pinot/pull/15075#issuecomment-2668866321 integration tests are not very happy with these changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Changes default load mode to MMAP [pinot]

2025-02-19 Thread via GitHub
gortiz commented on PR #15089: URL: https://github.com/apache/pinot/pull/15089#issuecomment-2668846515 I'm not sure if we can change this default. It would be a breaking change. In case we do, at least we would need to have a specific section in the release notes -- This is an automated

Re: [PR] Changes default load mode to MMAP [pinot]

2025-02-19 Thread via GitHub
codecov-commenter commented on PR #15089: URL: https://github.com/apache/pinot/pull/15089#issuecomment-2668563020 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15089?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.30.22 deleted (was 24cdee91b6)

2025-02-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.30.22 in repository https://gitbox.apache.org/repos/asf/pinot.git was 24cdee91b6 Bump software.amazon.awssdk:bom from 2.30.16 to 2.30.22 T

[PR] Changes default load mode to MMAP [pinot]

2025-02-19 Thread via GitHub
noob-se7en opened a new pull request, #15089: URL: https://github.com/apache/pinot/pull/15089 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

[PR] Bump software.amazon.awssdk:bom from 2.30.16 to 2.30.23 [pinot]

2025-02-19 Thread via GitHub
dependabot[bot] opened a new pull request, #15090: URL: https://github.com/apache/pinot/pull/15090 Bumps software.amazon.awssdk:bom from 2.30.16 to 2.30.23. Most Recent Ignore Conditions Applied to This Pull Request | Dependency Name | Ignore Conditions | | --- | --- |

Re: [PR] Bump software.amazon.awssdk:bom from 2.30.16 to 2.30.22 [pinot]

2025-02-19 Thread via GitHub
dependabot[bot] commented on PR #15083: URL: https://github.com/apache/pinot/pull/15083#issuecomment-2668536696 Superseded by #15090. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Bump software.amazon.awssdk:bom from 2.30.16 to 2.30.22 [pinot]

2025-02-19 Thread via GitHub
dependabot[bot] closed pull request #15083: Bump software.amazon.awssdk:bom from 2.30.16 to 2.30.22 URL: https://github.com/apache/pinot/pull/15083 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.30.23 created (now ec811db870)

2025-02-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.30.23 in repository https://gitbox.apache.org/repos/asf/pinot.git at ec811db870 Bump software.amazon.awssdk:bom from 2.30.16 to 2.30.23 N

Re: [PR] Add BrokerGrpcServer for a streaming response fetching endpoint [pinot]

2025-02-19 Thread via GitHub
codecov-commenter commented on PR #15088: URL: https://github.com/apache/pinot/pull/15088#issuecomment-2668381050 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15088?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Add BrokerGrpcServer for a streaming response fetching endpoint [pinot]

2025-02-19 Thread via GitHub
xiangfu0 opened a new pull request, #15088: URL: https://github.com/apache/pinot/pull/15088 - Add BrokerGrpcServer which is a grpc based broker query endpoint - Add Grpc client support in pinot-java-client with only broker list as the constructor - In integration tests, randomly pick g

(pinot) branch master updated (4e61bb3d6b -> 24e5bb3bc7)

2025-02-19 Thread manishswaminathan
This is an automated email from the ASF dual-hosted git repository. manishswaminathan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 4e61bb3d6b [Bugfix] Adds Check to Ignore Committing Segments as Completed (#15065) add 24e5bb3bc7 Fixi

Re: [PR] Fixing flaky integration test for pauseless failure scenario [pinot]

2025-02-19 Thread via GitHub
swaminathanmanish merged PR #15087: URL: https://github.com/apache/pinot/pull/15087 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@

Re: [PR] Fixing flaky integration test for pauseless failure scenario [pinot]

2025-02-19 Thread via GitHub
codecov-commenter commented on PR #15087: URL: https://github.com/apache/pinot/pull/15087#issuecomment-2668078246 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15087?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Update FailureDetector recovery logic to not break if only one query engine is being used [pinot]

2025-02-19 Thread via GitHub
codecov-commenter commented on PR #15086: URL: https://github.com/apache/pinot/pull/15086#issuecomment-2667970699 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15086?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [I] [Flaky-test] PauselessRealtimeIngestionNewSegmentMetadataCreationFailureTest.testSegmentAssignment [pinot]

2025-02-19 Thread via GitHub
9aman commented on issue #15059: URL: https://github.com/apache/pinot/issues/15059#issuecomment-2667969351 Have raised a PR with the fix: https://github.com/apache/pinot/pull/15087 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] Fixing flaky integration test for pauseless failure scenario [pinot]

2025-02-19 Thread via GitHub
9aman opened a new pull request, #15087: URL: https://github.com/apache/pinot/pull/15087 Issue: https://github.com/apache/pinot/issues/15059 - The setup() function only waits till all the documents have been loaded for the pauseless table - This has lead to race condition between t

(pinot) branch master updated (802226ec8c -> 4e61bb3d6b)

2025-02-19 Thread manishswaminathan
This is an automated email from the ASF dual-hosted git repository. manishswaminathan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 802226ec8c Emits metric before Consumer gets stuck in postStopConsumedMsg Method (#15070) add 4e61bb3d

Re: [PR] [Bugfix] Adds Check to Ignore Committing Segments as Completed [pinot]

2025-02-19 Thread via GitHub
swaminathanmanish merged PR #15065: URL: https://github.com/apache/pinot/pull/15065 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@

[PR] Update FailureDetector recovery logic to not break if only one query engine is being used [pinot]

2025-02-19 Thread via GitHub
yashmayya opened a new pull request, #15086: URL: https://github.com/apache/pinot/pull/15086 - After https://github.com/apache/pinot/pull/15005, the failure detector recovery mechanism relied on each handler reporting the server as healthy to be safe. - However, in use cases where only o