Re: [I] Support for recursive queries (for graph / tree use-cases) [pinot]

2024-12-02 Thread via GitHub
hpvd commented on issue #14568: URL: https://github.com/apache/pinot/issues/14568#issuecomment-2513782692 from strategic USP POV: being able to efficiently use Pinot for production graph use-cases, may give a new unique solution hardly to match in this combination by "real" graph DBs: -

Re: [PR] Fix crc mismatch during deepstore upload retry task [pinot]

2024-12-02 Thread via GitHub
tibrewalpratik17 merged PR #14506: URL: https://github.com/apache/pinot/pull/14506 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@p

(pinot) branch master updated (d0d4419488 -> e5d5bad371)

2024-12-02 Thread pratik
This is an automated email from the ASF dual-hosted git repository. pratik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from d0d4419488 [SparkConnector] Add proper time column support for Spark connector segment writer (#14556) add e5d5ba

Re: [PR] Make RecordExtractor preserve empty array/map and map entries with empty value [pinot]

2024-12-02 Thread via GitHub
KKcorps commented on code in PR #14547: URL: https://github.com/apache/pinot/pull/14547#discussion_r1867101992 ## pinot-plugins/pinot-input-format/pinot-parquet/src/main/java/org/apache/pinot/plugin/inputformat/parquet/ParquetNativeRecordExtractor.java: ## @@ -209,79 +207,26 @@

Re: [PR] Make RecordExtractor preserve empty array/map and map entries with empty value [pinot]

2024-12-02 Thread via GitHub
KKcorps commented on code in PR #14547: URL: https://github.com/apache/pinot/pull/14547#discussion_r1867101375 ## pinot-plugins/pinot-input-format/pinot-parquet/src/main/java/org/apache/pinot/plugin/inputformat/parquet/ParquetNativeRecordExtractor.java: ## @@ -209,79 +207,26 @@

Re: [PR] [timeseries] Part-1: Misc. Refactor to Enable Broker Reduce [pinot]

2024-12-02 Thread via GitHub
codecov-commenter commented on PR #14582: URL: https://github.com/apache/pinot/pull/14582#issuecomment-2513602843 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14582?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] [timeseries] Part-1: Misc. Refactor to Enable Broker Reduce [pinot]

2024-12-02 Thread via GitHub
ankitsultana opened a new pull request, #14582: URL: https://github.com/apache/pinot/pull/14582 ### Summary First of the 3-part PRs to enable Broker Reduce for the Time Series Engine. This PR does the following: * Renames `child` references to `inputs` to be consistent wi

Re: [PR] [timeseries] Part-1: Misc. Refactor to Enable Broker Reduce [pinot]

2024-12-02 Thread via GitHub
ankitsultana commented on code in PR #14582: URL: https://github.com/apache/pinot/pull/14582#discussion_r1867054388 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/GrpcSendingMailbox.java: ## @@ -135,17 +133,7 @@ private MailboxContent toMailboxContent(Transf

(pinot) branch dependabot/maven/com.puppycrawl.tools-checkstyle-10.20.2 deleted (was 4ce1cfc62b)

2024-12-02 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a change to branch dependabot/maven/com.puppycrawl.tools-checkstyle-10.20.2 in repository https://gitbox.apache.org/repos/asf/pinot.git was 4ce1cfc62b Bump com.puppycrawl.tools:checkstyle from 10.20.1 to 1

Re: [PR] [SparkConnector] Add proper time column support for Spark connector segment writer [pinot]

2024-12-02 Thread via GitHub
ankitsultana merged PR #14556: URL: https://github.com/apache/pinot/pull/14556 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: [SparkConnector] Add proper time column support for Spark connector segment writer (#14556)

2024-12-02 Thread ankitsultana
This is an automated email from the ASF dual-hosted git repository. ankitsultana pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new d0d4419488 [SparkConnector] Add proper time

Re: [PR] [Flaky-test] Fix ForwardIndexHandlerTest.testAddOtherIndexForForwardIndexDisabledColumn() [pinot]

2024-12-02 Thread via GitHub
Jackie-Jiang merged PR #14580: URL: https://github.com/apache/pinot/pull/14580 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: [Flaky-test] Fix ForwardIndexHandlerTest.testAddOtherIndexForForwardIndexDisabledColumn() (#14580)

2024-12-02 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new af6fd6c28a [Flaky-test] Fix ForwardIndexHandlerT

Re: [PR] Raise alert for replicas in ERROR state when Replica Group is configured in table Config [pinot]

2024-12-02 Thread via GitHub
codecov-commenter commented on PR #14581: URL: https://github.com/apache/pinot/pull/14581#issuecomment-2513220360 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14581?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

(pinot) branch master updated (8f262e074d -> ca83cf1aa4)

2024-12-02 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 8f262e074d Bump software.amazon.awssdk:bom from 2.29.23 to 2.29.24 (#14577) add ca83cf1aa4 Bump nl.jqno.equals

Re: [PR] [Flaky-test] Fix ForwardIndexHandlerTest.testAddOtherIndexForForwardIndexDisabledColumn() [pinot]

2024-12-02 Thread via GitHub
codecov-commenter commented on PR #14580: URL: https://github.com/apache/pinot/pull/14580#issuecomment-2513183651 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14580?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Raise alert for replicas in ERROR state when Replica Group is configured in table Config [pinot]

2024-12-02 Thread via GitHub
deepthi912 opened a new pull request, #14581: URL: https://github.com/apache/pinot/pull/14581 **Context:** We have an alert to show the percent of replicas that are healthy but for Replica Group Setting when strictReplicaGroup config is enabled for upsert/dedup tables it is hard to detec

Re: [PR] Bump org.asynchttpclient:async-http-client from 3.0.0 to 3.0.1 [pinot]

2024-12-02 Thread via GitHub
Jackie-Jiang merged PR #14579: URL: https://github.com/apache/pinot/pull/14579 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[PR] [Flaky-test] Fix ForwardIndexHandlerTest.testAddOtherIndexForForwardIndexDisabledColumn() [pinot]

2024-12-02 Thread via GitHub
Jackie-Jiang opened a new pull request, #14580: URL: https://github.com/apache/pinot/pull/14580 The test will fail when max MV entries is 1 (random between 1 - 500). Revise the testing logic ``` Error: org.apache.pinot.segment.local.segment.index.loader.ForwardIndexHandlerTest.te

Re: [PR] Bump software.amazon.awssdk:bom from 2.29.23 to 2.29.24 [pinot]

2024-12-02 Thread via GitHub
yashmayya merged PR #14577: URL: https://github.com/apache/pinot/pull/14577 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

Re: [PR] [WIP] Adds Support of maxNumRowsPerTask in RealtimeToOfflineSegmentsTasksGe… [pinot]

2024-12-02 Thread via GitHub
codecov-commenter commented on PR #14578: URL: https://github.com/apache/pinot/pull/14578#issuecomment-2511811726 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14578?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

(pinot) branch master updated: Exclude secondary workload queries from some critical metrics (#14553)

2024-12-02 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 0ccc6471e0 Exclude secondary workload queries from

Re: [PR] Add some critical monitoring metrics for secondary workload queries [pinot]

2024-12-02 Thread via GitHub
jackjlli merged PR #14553: URL: https://github.com/apache/pinot/pull/14553 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] Cleanup Redundant JMX Exporter Rules [pinot]

2024-12-02 Thread via GitHub
soumitra-st commented on PR #14348: URL: https://github.com/apache/pinot/pull/14348#issuecomment-2512255004 Few questions: 1. Did you validate this change (number and name of metrics) using the unit tests and running on an actual cluster? 2. Were you able to run a performance test to s

(pinot) branch master updated: Bump software.amazon.awssdk:bom from 2.29.23 to 2.29.24 (#14577)

2024-12-02 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 8f262e074d Bump software.amazon.awssdk:bom fro

Re: [PR] Bump nl.jqno.equalsverifier:equalsverifier from 3.17.3 to 3.17.4 [pinot]

2024-12-02 Thread via GitHub
yashmayya merged PR #14576: URL: https://github.com/apache/pinot/pull/14576 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

(pinot) branch dependabot/maven/org.asynchttpclient-async-http-client-3.0.1 created (now fab6101902)

2024-12-02 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.asynchttpclient-async-http-client-3.0.1 in repository https://gitbox.apache.org/repos/asf/pinot.git at fab6101902 Bump org.asynchttpclient:async-http-client from 3

[PR] Bump org.asynchttpclient:async-http-client from 3.0.0 to 3.0.1 [pinot]

2024-12-02 Thread via GitHub
dependabot[bot] opened a new pull request, #14579: URL: https://github.com/apache/pinot/pull/14579 Bumps [org.asynchttpclient:async-http-client](https://github.com/AsyncHttpClient/async-http-client) from 3.0.0 to 3.0.1. Release notes Sourced from https://github.com/AsyncHttpClient

(pinot) branch master updated (ca83cf1aa4 -> 2910caac80)

2024-12-02 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from ca83cf1aa4 Bump nl.jqno.equalsverifier:equalsverifier from 3.17.3 to 3.17.4 (#14576) add 2910caac80 Bump com.p

[PR] Adds Support of maxNumRowsPerTask in RealtimeToOfflineSegmentsTasksGe… [pinot]

2024-12-02 Thread via GitHub
Harnoor7 opened a new pull request, #14578: URL: https://github.com/apache/pinot/pull/14578 [https://github.com/apache/pinot/issues/12857](https://github.com/apache/pinot/issues/12857 ) Instructions: 1. The PR has to be tagged with at least one of the following labels (*): 1.

Re: [PR] Cleanup Redundant JMX Exporter Rules [pinot]

2024-12-02 Thread via GitHub
soumitra-st commented on code in PR #14348: URL: https://github.com/apache/pinot/pull/14348#discussion_r1866307806 ## docker/images/pinot/etc/jmx_prometheus_javaagent/configs/controller.yml: ## @@ -2,151 +2,20 @@ rules: - pattern: "\"org\\.apache\\.pinot\\.common\\.metrics\"<>

Re: [PR] [SparkConnector] Add proper time column support for Spark connector segment writer [pinot]

2024-12-02 Thread via GitHub
ankitsultana commented on code in PR #14556: URL: https://github.com/apache/pinot/pull/14556#discussion_r1866418294 ## pinot-connectors/pinot-spark-3-connector/src/main/scala/org/apache/pinot/connector/spark/v3/datasource/PinotDataWriter.scala: ## @@ -94,6 +115,8 @@ class PinotD

Re: [PR] Bump com.puppycrawl.tools:checkstyle from 10.20.1 to 10.20.2 [pinot]

2024-12-02 Thread via GitHub
yashmayya merged PR #14575: URL: https://github.com/apache/pinot/pull/14575 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

(pinot) branch dependabot/maven/nl.jqno.equalsverifier-equalsverifier-3.17.4 deleted (was 1da6e45695)

2024-12-02 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a change to branch dependabot/maven/nl.jqno.equalsverifier-equalsverifier-3.17.4 in repository https://gitbox.apache.org/repos/asf/pinot.git was 1da6e45695 Bump nl.jqno.equalsverifier:equalsverifier from 3

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.29.24 deleted (was 3e1548e16b)

2024-12-02 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.29.24 in repository https://gitbox.apache.org/repos/asf/pinot.git was 3e1548e16b Bump software.amazon.awssdk:bom from 2.29.23 to 2.29.24 Th

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.29.24 created (now 3e1548e16b)

2024-12-02 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.29.24 in repository https://gitbox.apache.org/repos/asf/pinot.git at 3e1548e16b Bump software.amazon.awssdk:bom from 2.29.23 to 2.29.24 N

[PR] Bump software.amazon.awssdk:bom from 2.29.23 to 2.29.24 [pinot]

2024-12-02 Thread via GitHub
dependabot[bot] opened a new pull request, #14577: URL: https://github.com/apache/pinot/pull/14577 Bumps software.amazon.awssdk:bom from 2.29.23 to 2.29.24. Most Recent Ignore Conditions Applied to This Pull Request | Dependency Name | Ignore Conditions | | --- | --- |

(pinot) branch dependabot/maven/nl.jqno.equalsverifier-equalsverifier-3.17.4 created (now 1da6e45695)

2024-12-02 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/nl.jqno.equalsverifier-equalsverifier-3.17.4 in repository https://gitbox.apache.org/repos/asf/pinot.git at 1da6e45695 Bump nl.jqno.equalsverifier:equalsverifier from

[PR] Bump nl.jqno.equalsverifier:equalsverifier from 3.17.3 to 3.17.4 [pinot]

2024-12-02 Thread via GitHub
dependabot[bot] opened a new pull request, #14576: URL: https://github.com/apache/pinot/pull/14576 Bumps [nl.jqno.equalsverifier:equalsverifier](https://github.com/jqno/equalsverifier) from 3.17.3 to 3.17.4. Release notes Sourced from https://github.com/jqno/equalsverifier/release

(pinot) branch dependabot/maven/com.puppycrawl.tools-checkstyle-10.20.2 created (now 4ce1cfc62b)

2024-12-02 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.puppycrawl.tools-checkstyle-10.20.2 in repository https://gitbox.apache.org/repos/asf/pinot.git at 4ce1cfc62b Bump com.puppycrawl.tools:checkstyle from 10.20.1 to

[PR] Bump com.puppycrawl.tools:checkstyle from 10.20.1 to 10.20.2 [pinot]

2024-12-02 Thread via GitHub
dependabot[bot] opened a new pull request, #14575: URL: https://github.com/apache/pinot/pull/14575 Bumps [com.puppycrawl.tools:checkstyle](https://github.com/checkstyle/checkstyle) from 10.20.1 to 10.20.2. Release notes Sourced from https://github.com/checkstyle/checkstyle/release

(pinot) branch master updated: Specify start index of file name in GenerateDataCommand (#14573)

2024-12-02 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 76e1506336 Specify start index of file name in

Re: [PR] Specify start index of file name in GenerateDataCommand [pinot]

2024-12-02 Thread via GitHub
yashmayya merged PR #14573: URL: https://github.com/apache/pinot/pull/14573 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

Re: [PR] Specify start index of file name in GenerateDataCommand [pinot]

2024-12-02 Thread via GitHub
codecov-commenter commented on PR #14573: URL: https://github.com/apache/pinot/pull/14573#issuecomment-2511199782 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14573?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Add cluster configuration to allow limiting the number of multi-stage queries running concurrently [pinot]

2024-12-02 Thread via GitHub
codecov-commenter commented on PR #14574: URL: https://github.com/apache/pinot/pull/14574#issuecomment-2511160903 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14574?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Add a Server API to list segments that need to be refreshed for a table [pinot]

2024-12-02 Thread via GitHub
vrajat commented on code in PR #14451: URL: https://github.com/apache/pinot/pull/14451#discussion_r1865604773 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java: ## @@ -1046,6 +1064,276 @@ public boolean needReloadSegments() return need

Re: [PR] Specify start index of file name in GenerateDataCommand [pinot]

2024-12-02 Thread via GitHub
vrajat commented on PR #14573: URL: https://github.com/apache/pinot/pull/14573#issuecomment-2511134031 > The changes look fine to me but what's the use case for this exactly? I've updated the description. Tell me if it is more informative. -- This is an automated message from the A

Re: [PR] Specify start index of file name in GenerateDataCommand [pinot]

2024-12-02 Thread via GitHub
vrajat commented on code in PR #14573: URL: https://github.com/apache/pinot/pull/14573#discussion_r1865580434 ## pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/GenerateDataCommand.java: ## @@ -120,9 +123,9 @@ public boolean execute() gen.init(spec); i

Re: [PR] Specify start index of file name in GenerateDataCommand [pinot]

2024-12-02 Thread via GitHub
yashmayya commented on code in PR #14573: URL: https://github.com/apache/pinot/pull/14573#discussion_r1865573770 ## pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/GenerateDataCommand.java: ## @@ -120,9 +123,9 @@ public boolean execute() gen.init(spec);

[PR] Add cluster configuration to allow limiting the number of multi-stage queries running concurrently [pinot]

2024-12-02 Thread via GitHub
yashmayya opened a new pull request, #14574: URL: https://github.com/apache/pinot/pull/14574 - Currently, there is no limit on the number of multi-stage queries that are executed concurrently. Every request that comes in to a broker is immediately compiled into a query plan and dispatched t

[PR] Specify start index of file name in GenerateDataCommand [pinot]

2024-12-02 Thread via GitHub
vrajat opened a new pull request, #14573: URL: https://github.com/apache/pinot/pull/14573 `GenerateDataCommand` can generate data in multiple files. The number of files is specified using `-numFiles`. File name are generated with an int as the suffix starting from 0. This PR adds the ab

(pinot) branch master updated: Add immtuable CLPForwardIndex creator and related classes (#14288)

2024-12-02 Thread lqc
This is an automated email from the ASF dual-hosted git repository. lqc pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 585e8e Add immtuable CLPForwardIndex creator and

Re: [PR] Add immtuable CLPForwardIndex creator and related classes [pinot]

2024-12-02 Thread via GitHub
deemoliu merged PR #14288: URL: https://github.com/apache/pinot/pull/14288 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa