Re: [PR] add clientQueryId and its cancel operation [pinot]

2025-01-15 Thread via GitHub
codecov-commenter commented on PR #14823: URL: https://github.com/apache/pinot/pull/14823#issuecomment-2594748226 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14823?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Adding error message in the SchemUtils.java [pinot]

2025-01-15 Thread via GitHub
codecov-commenter commented on PR #14804: URL: https://github.com/apache/pinot/pull/14804#issuecomment-2594743423 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14804?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [I] [proposal] Reuse common expressions in a query [pinot]

2025-01-15 Thread via GitHub
siddharthteotia commented on issue #14196: URL: https://github.com/apache/pinot/issues/14196#issuecomment-2594732326 How far are we with this feature? I am interested in helping with reviews (while not slowing it down) -- This is an automated message from the Apache Git Service. To respon

Re: [PR] #bugfix add numServersQueried and numServersResponded to MSQE response [pinot]

2025-01-15 Thread via GitHub
siddharthteotia commented on code in PR #14806: URL: https://github.com/apache/pinot/pull/14806#discussion_r1917872631 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/MultiStageBrokerRequestHandler.java: ## @@ -277,8 +286,12 @@ protected BrokerResponse handl

Re: [PR] #bugfix add numServersQueried and numServersResponded to MSQE response [pinot]

2025-01-15 Thread via GitHub
siddharthteotia commented on PR #14806: URL: https://github.com/apache/pinot/pull/14806#issuecomment-2594712114 I think a better approach would be to rethink how we are propagating the execution state during the course of query execution. This would be beneficial for debugging and in fact

Re: [PR] Adds metrics for task states [pinot]

2025-01-15 Thread via GitHub
siddharthteotia commented on code in PR #14785: URL: https://github.com/apache/pinot/pull/14785#discussion_r1917855814 ## pinot-common/src/main/java/org/apache/pinot/common/metrics/ControllerGauge.java: ## @@ -64,11 +64,15 @@ public enum ControllerGauge implements AbstractMetri

Re: [PR] support configuring initial broker tags even when single-tenant mode is enabled [pinot]

2025-01-15 Thread via GitHub
codecov-commenter commented on PR #14824: URL: https://github.com/apache/pinot/pull/14824#issuecomment-2594676518 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14824?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Add serverStats to BrokerQueryEventListener [pinot]

2025-01-15 Thread via GitHub
siddharthteotia commented on PR #14807: URL: https://github.com/apache/pinot/pull/14807#issuecomment-2594683308 We use this within LinkedIn. Essentially our internal piece of code (which links with OSS libraries) publishes metrics to a kafka topic (at a query level with sampling). W

Re: [PR] Support exchange type hint to allow broadcast join [pinot]

2025-01-15 Thread via GitHub
Jackie-Jiang commented on PR #14797: URL: https://github.com/apache/pinot/pull/14797#issuecomment-2594665174 Updated the PR to decouple exchange type from join strategy. Allow customizing exchange type for both left and right side. -- This is an automated message from the Apache Git Servi

Re: [PR] add clientQueryId and its cancel operation [pinot]

2025-01-15 Thread via GitHub
siddharthteotia commented on PR #14823: URL: https://github.com/apache/pinot/pull/14823#issuecomment-2594664930 Is ClientQueryID a new concept? Is it same as requestID ? How does the support added here improve the existing Query Cancellation (which is also exposed to user IIRC) ? --

(pinot) branch hotfix-npe created (now 43254690fc)

2025-01-15 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch hotfix-npe in repository https://gitbox.apache.org/repos/asf/pinot.git at 43254690fc Fix NPE caused by MultiStageQueryThrottler (#14805) This branch includes the following new commits:

(pinot) 01/01: Fix NPE caused by MultiStageQueryThrottler (#14805)

2025-01-15 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch hotfix-npe in repository https://gitbox.apache.org/repos/asf/pinot.git commit 43254690fc1ed8f699196e23ca579f87b010ac15 Author: Jialiang Li AuthorDate: Mon Jan 13 18:00:29 2025 -0800 Fix NPE ca

Re: [PR] add clientQueryId and its cancel operation [pinot]

2025-01-15 Thread via GitHub
siddharthteotia commented on code in PR #14823: URL: https://github.com/apache/pinot/pull/14823#discussion_r1917835462 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandler.java: ## @@ -865,6 +896,48 @@ protected BrokerResponse

Re: [PR] Resolve failures pauseless ingestion [pinot]

2025-01-15 Thread via GitHub
9aman commented on code in PR #14798: URL: https://github.com/apache/pinot/pull/14798#discussion_r1917781890 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentZKMetadataUtils.java: ## @@ -0,0 +1,80 @@ +/** + * Licensed to the Apache Software Foundatio

Re: [PR] Resolve failures pauseless ingestion [pinot]

2025-01-15 Thread via GitHub
9aman commented on code in PR #14798: URL: https://github.com/apache/pinot/pull/14798#discussion_r1917770366 ## pinot-controller/src/main/java/org/apache/pinot/controller/validation/RealtimeSegmentValidationManager.java: ## @@ -171,6 +172,9 @@ private void runSegmentLevelValidat

Re: [PR] support configuring initial broker tags even when single-tenant mode is enabled [pinot]

2025-01-15 Thread via GitHub
Jackie-Jiang merged PR #14824: URL: https://github.com/apache/pinot/pull/14824 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Add support for KLL sketch aggregation in minion jobs [pinot]

2025-01-15 Thread via GitHub
yashmayya commented on code in PR #14702: URL: https://github.com/apache/pinot/pull/14702#discussion_r1917563511 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/Constants.java: ## @@ -33,4 +33,5 @@ private Constants() { public static final String THETA_TUPLE_S

(pinot) branch master updated (a0433aa9c6 -> f37bc3d4cb)

2025-01-15 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from a0433aa9c6 support configuring broker tags even when cluster is configured for single tenant (#14824) add f37b

Re: [PR] Add support for KLL sketch aggregation in minion jobs [pinot]

2025-01-15 Thread via GitHub
yashmayya merged PR #14702: URL: https://github.com/apache/pinot/pull/14702 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

Re: [I] Pinot real-time data ingestion freshness guarantee [pinot]

2025-01-15 Thread via GitHub
wirybeaver commented on issue #14815: URL: https://github.com/apache/pinot/issues/14815#issuecomment-2594312265 backfill through minion jobs makes more sense, creating a temporary topic would cause more overhead to address the failure scenario, for example: the number of topics would blows

(pinot) branch master updated: support configuring broker tags even when cluster is configured for single tenant (#14824)

2025-01-15 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new a0433aa9c6 support configuring broker tags even w

Re: [PR] Supports Force Committing Segments in Batches [pinot]

2025-01-15 Thread via GitHub
Jackie-Jiang commented on code in PR #14811: URL: https://github.com/apache/pinot/pull/14811#discussion_r1917549332 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java: ## @@ -1728,15 +1736,114 @@ private boolean

[PR] support configuring initial broker tags even when single-tenant mode is enabled [pinot]

2025-01-15 Thread via GitHub
dang-stripe opened a new pull request, #14824: URL: https://github.com/apache/pinot/pull/14824 This allows configuring initial broker tags, regardless of cluster isolation mode. Cluster isolation mode is a rather obscure setting that defaults to true. It controls whether brokers/serv

Re: [I] Add support for lazy reload during schema evolution [pinot]

2025-01-15 Thread via GitHub
deemoliu commented on issue #14499: URL: https://github.com/apache/pinot/issues/14499#issuecomment-2594144592 @rohityadav1993 do you want to work on this feature? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Supports Force Committing Segments in Batches [pinot]

2025-01-15 Thread via GitHub
noob-se7en commented on code in PR #14811: URL: https://github.com/apache/pinot/pull/14811#discussion_r1917468414 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java: ## @@ -1728,15 +1736,111 @@ private boolean is

Re: [PR] Supports Force Committing Segments in Batches [pinot]

2025-01-15 Thread via GitHub
noob-se7en commented on code in PR #14811: URL: https://github.com/apache/pinot/pull/14811#discussion_r1917460537 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java: ## @@ -1728,15 +1736,111 @@ private boolean is

Re: [PR] Fixes int overflow for num values [pinot]

2025-01-15 Thread via GitHub
noob-se7en commented on code in PR #14423: URL: https://github.com/apache/pinot/pull/14423#discussion_r1917467762 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/writer/impl/FixedBitMVForwardIndexWriter.java: ## @@ -74,7 +74,7 @@ public class FixedBitMVFor

Re: [PR] Fixes int overflow for num values [pinot]

2025-01-15 Thread via GitHub
noob-se7en commented on code in PR #14423: URL: https://github.com/apache/pinot/pull/14423#discussion_r1917467762 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/writer/impl/FixedBitMVForwardIndexWriter.java: ## @@ -74,7 +74,7 @@ public class FixedBitMVFor

Re: [PR] Supports Force Committing Segments in Batches [pinot]

2025-01-15 Thread via GitHub
noob-se7en commented on code in PR #14811: URL: https://github.com/apache/pinot/pull/14811#discussion_r1917460537 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java: ## @@ -1728,15 +1736,111 @@ private boolean is

Re: [PR] Supports Force Committing Segments in Batches [pinot]

2025-01-15 Thread via GitHub
noob-se7en commented on code in PR #14811: URL: https://github.com/apache/pinot/pull/14811#discussion_r1917460537 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java: ## @@ -1728,15 +1736,111 @@ private boolean is

Re: [PR] Supports Force Committing Segments in Batches [pinot]

2025-01-15 Thread via GitHub
noob-se7en commented on code in PR #14811: URL: https://github.com/apache/pinot/pull/14811#discussion_r1917453197 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java: ## @@ -1728,15 +1736,105 @@ private boolean is

[PR] add clientQueryId and its cancel operation [pinot]

2025-01-15 Thread via GitHub
albertobastos opened a new pull request, #14823: URL: https://github.com/apache/pinot/pull/14823 Adds the option to provide a "clientQueryId" as a QueryOption and eventually use it to cancel the running query. -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] Upsert Deadlock Condition Present in Pinot 1.1 [pinot]

2025-01-15 Thread via GitHub
Jackie-Jiang commented on issue #14822: URL: https://github.com/apache/pinot/issues/14822#issuecomment-2594113681 #12886 Refactored the message handling logic, and consolidated this lock into the regular segment lock. With #12886 and #13632 this problem should be solved. -- This is an au

[I] Critical Upsert Deadlock Condition Present in Pinot 1.1 [pinot]

2025-01-15 Thread via GitHub
ankitsultana opened a new issue, #14822: URL: https://github.com/apache/pinot/issues/14822 We had been seeing frequent deadlocked threads on our Upsert enabled servers, and we were able to trace the root-cause to this commit: #12241. At a high-level, the issue is that: 1. The c

Re: [PR] Add serverStats to BrokerQueryEventListener [pinot]

2025-01-15 Thread via GitHub
gortiz commented on code in PR #14807: URL: https://github.com/apache/pinot/pull/14807#discussion_r1917400525 ## pinot-spi/src/main/java/org/apache/pinot/spi/trace/ServerStatsInfo.java: ## @@ -0,0 +1,56 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * o

Re: [PR] Supports Force Committing Segments in Batches [pinot]

2025-01-15 Thread via GitHub
noob-se7en commented on PR #14811: URL: https://github.com/apache/pinot/pull/14811#issuecomment-2594036760 @Jackie-Jiang I understand the edge case. Fixing the implementation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Add serverStats to BrokerQueryEventListener [pinot]

2025-01-15 Thread via GitHub
gortiz commented on PR #14807: URL: https://github.com/apache/pinot/pull/14807#issuecomment-2594035816 I don't have enough context. What is BrokerQueryEventListener used for? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Support BROADCAST join strategy [pinot]

2025-01-15 Thread via GitHub
gortiz commented on PR #14797: URL: https://github.com/apache/pinot/pull/14797#issuecomment-2594005921 You are right @siddharthteotia. The distribution is a property of the exchange. I was discussing that with @bziobrowski yesterday, and he rightfully mentioned that exchange types also affe

Re: [PR] Support BROADCAST join strategy [pinot]

2025-01-15 Thread via GitHub
siddharthteotia commented on PR #14797: URL: https://github.com/apache/pinot/pull/14797#issuecomment-2594015264 Sounds good. > I'm thinking changing the join_strategy hint to exchange_type hint (still under joinOptions) I am ok with this for now. It's fine to provide excha

Re: [PR] #bugfix add numServersQueried and numServersResponded to MSQE response [pinot]

2025-01-15 Thread via GitHub
gortiz commented on code in PR #14806: URL: https://github.com/apache/pinot/pull/14806#discussion_r1917371935 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/MultiStageBrokerRequestHandler.java: ## @@ -199,8 +201,15 @@ protected BrokerResponse handleRequest(

Re: [PR] Allows multiple requests per server per request ID [pinot]

2025-01-15 Thread via GitHub
egalpin commented on code in PR #13742: URL: https://github.com/apache/pinot/pull/13742#discussion_r1917366074 ## pinot-core/src/main/java/org/apache/pinot/core/transport/AsyncQueryResponse.java: ## @@ -39,10 +44,13 @@ @ThreadSafe public class AsyncQueryResponse implements Que

Re: [PR] #bugfix add numServersQueried and numServersResponded to MSQE response [pinot]

2025-01-15 Thread via GitHub
gortiz commented on code in PR #14806: URL: https://github.com/apache/pinot/pull/14806#discussion_r1917355976 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/MultiStageBrokerRequestHandler.java: ## @@ -277,8 +286,12 @@ protected BrokerResponse handleRequest(

Re: [PR] Allows multiple requests per server per request ID [pinot]

2025-01-15 Thread via GitHub
egalpin commented on code in PR #13742: URL: https://github.com/apache/pinot/pull/13742#discussion_r1917355766 ## pinot-core/src/main/java/org/apache/pinot/core/transport/AsyncQueryResponse.java: ## @@ -39,10 +44,13 @@ @ThreadSafe public class AsyncQueryResponse implements Que

Re: [PR] Supports Force Committing Segments in Batches [pinot]

2025-01-15 Thread via GitHub
Jackie-Jiang commented on PR #14811: URL: https://github.com/apache/pinot/pull/14811#issuecomment-2593814818 Taking a concrete example: when `batchSize` is set to 1, and there are 2 consuming segment `seg0` and `seg1`, 2 servers (replicas) per segment on `server0` and `server1`. After throt

Re: [PR] Adds metrics for task states [pinot]

2025-01-15 Thread via GitHub
noob-se7en commented on PR #14785: URL: https://github.com/apache/pinot/pull/14785#issuecomment-2593743403 > Thanks for adding these metrics. These are table level right ? Yes @siddharthteotia there will be table level metrics as well like: `pinot.controller.numMinionSubtasksDro

Re: [PR] Resolve failures pauseless ingestion [pinot]

2025-01-15 Thread via GitHub
9aman commented on code in PR #14798: URL: https://github.com/apache/pinot/pull/14798#discussion_r1917191824 ## pinot-common/src/main/java/org/apache/pinot/common/utils/FileUploadDownloadClient.java: ## @@ -968,26 +970,25 @@ public String uploadToSegmentStore(String uri) * U

Re: [PR] Resolve failures pauseless ingestion [pinot]

2025-01-15 Thread via GitHub
9aman commented on code in PR #14798: URL: https://github.com/apache/pinot/pull/14798#discussion_r1917189209 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java: ## @@ -543,13 +555,22 @@ private void commitSegment

Re: [PR] Resolve failures pauseless ingestion [pinot]

2025-01-15 Thread via GitHub
9aman commented on code in PR #14798: URL: https://github.com/apache/pinot/pull/14798#discussion_r1917190185 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentZKMetadataUtils.java: ## @@ -0,0 +1,80 @@ +/** + * Licensed to the Apache Software Foundatio

Re: [PR] Resolve failures pauseless ingestion [pinot]

2025-01-15 Thread via GitHub
9aman commented on code in PR #14798: URL: https://github.com/apache/pinot/pull/14798#discussion_r1917189510 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java: ## @@ -1764,6 +1794,86 @@ public void uploadToDeepS

Re: [PR] Resolve failures pauseless ingestion [pinot]

2025-01-15 Thread via GitHub
9aman commented on code in PR #14798: URL: https://github.com/apache/pinot/pull/14798#discussion_r1917188787 ## pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java: ## @@ -884,7 +892,7 @@ public String uploadLLCSegment( @ApiResponse(code =

Re: [PR] Supports Force Committing Segments in Batches [pinot]

2025-01-15 Thread via GitHub
noob-se7en commented on PR #14811: URL: https://github.com/apache/pinot/pull/14811#issuecomment-2593706606 > Is it not possible to solve the problem on controller / coordinate from the controller ? Pushing this down to the individual server will likely lead to error-prone situations Logi

Re: [PR] Supports Force Committing Segments in Batches [pinot]

2025-01-15 Thread via GitHub
noob-se7en commented on PR #14811: URL: https://github.com/apache/pinot/pull/14811#issuecomment-2593699308 @Jackie-Jiang I don't quite get what is meant by > and others get throttled? Regarding Deadlock or any edge case - Server will use the same logic which is used `/tables/force

Re: [PR] Bump software.amazon.awssdk:bom from 2.29.51 to 2.29.52 [pinot]

2025-01-15 Thread via GitHub
Jackie-Jiang merged PR #14820: URL: https://github.com/apache/pinot/pull/14820 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Bump org.apache.datasketches:datasketches-java from 6.1.1 to 7.0.0 [pinot]

2025-01-15 Thread via GitHub
dependabot[bot] commented on PR #14821: URL: https://github.com/apache/pinot/pull/14821#issuecomment-2593648639 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

(pinot) branch master updated (bd57573ca1 -> fe7e9e2793)

2025-01-15 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from bd57573ca1 Bump com.diffplug.spotless:spotless-maven-plugin from 2.44.1 to 2.44.2 (#14819) add fe7e9e2793 Bump so

(pinot) branch dependabot/maven/org.apache.datasketches-datasketches-java-7.0.0 deleted (was 8036f903f4)

2025-01-15 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.apache.datasketches-datasketches-java-7.0.0 in repository https://gitbox.apache.org/repos/asf/pinot.git was 8036f903f4 Bump org.apache.datasketches:datasketches-jav

Re: [PR] Bump com.diffplug.spotless:spotless-maven-plugin from 2.44.1 to 2.44.2 [pinot]

2025-01-15 Thread via GitHub
Jackie-Jiang merged PR #14819: URL: https://github.com/apache/pinot/pull/14819 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Bump org.apache.datasketches:datasketches-java from 6.1.1 to 7.0.0 [pinot]

2025-01-15 Thread via GitHub
Jackie-Jiang closed pull request #14821: Bump org.apache.datasketches:datasketches-java from 6.1.1 to 7.0.0 URL: https://github.com/apache/pinot/pull/14821 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

(pinot) branch dependabot/maven/dropwizard-metrics.version-4.2.30 deleted (was 6585faab5a)

2025-01-15 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/dropwizard-metrics.version-4.2.30 in repository https://gitbox.apache.org/repos/asf/pinot.git was 6585faab5a Bump dropwizard-metrics.version from 4.2.29 to 4.2.30 The revi

(pinot) branch master updated: Bump com.diffplug.spotless:spotless-maven-plugin from 2.44.1 to 2.44.2 (#14819)

2025-01-15 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new bd57573ca1 Bump com.diffplug.spotless:spotless-ma

(pinot) branch dependabot/maven/com.diffplug.spotless-spotless-maven-plugin-2.44.2 deleted (was d5da133a29)

2025-01-15 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/com.diffplug.spotless-spotless-maven-plugin-2.44.2 in repository https://gitbox.apache.org/repos/asf/pinot.git was d5da133a29 Bump com.diffplug.spotless:spotless-maven-plug

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.29.52 deleted (was 69f6a85bcb)

2025-01-15 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.29.52 in repository https://gitbox.apache.org/repos/asf/pinot.git was 69f6a85bcb Bump software.amazon.awssdk:bom from 2.29.51 to 2.29.52 The r

(pinot) branch master updated: Bump dropwizard-metrics.version from 4.2.29 to 4.2.30 (#14818)

2025-01-15 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 759369f636 Bump dropwizard-metrics.version from 4

Re: [PR] Bump dropwizard-metrics.version from 4.2.29 to 4.2.30 [pinot]

2025-01-15 Thread via GitHub
Jackie-Jiang merged PR #14818: URL: https://github.com/apache/pinot/pull/14818 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Short Circuit And Filter Operator During Index Evaluation [pinot]

2025-01-15 Thread via GitHub
ankitsultana commented on PR #14700: URL: https://github.com/apache/pinot/pull/14700#issuecomment-2593414307 That's great. I am hoping to wrap this up over the coming long weekend, so do share any other ideas if you can. I have also added a new benchmark to make it easy to benchmark filter

Re: [PR] Support BROADCAST join strategy [pinot]

2025-01-15 Thread via GitHub
ankitsultana commented on PR #14797: URL: https://github.com/apache/pinot/pull/14797#issuecomment-2593395149 @siddharthteotia I am right now working on refactoring the optimizer where many of the optimizations like coloration, skipping of partial aggregates, etc. will become automatic. Will

Re: [PR] Add support for KLL sketch aggregation in minion jobs [pinot]

2025-01-15 Thread via GitHub
raghavagrawal commented on code in PR #14702: URL: https://github.com/apache/pinot/pull/14702#discussion_r1916674053 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/Constants.java: ## @@ -33,4 +33,5 @@ private Constants() { public static final String THETA_TUP

Re: [PR] [flaky-test] BrokerManagedAsyncExecutorProviderTest#testRejectHandler [pinot]

2025-01-15 Thread via GitHub
codecov-commenter commented on PR #14814: URL: https://github.com/apache/pinot/pull/14814#issuecomment-2592680278 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14814?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

(pinot) branch dependabot/maven/org.apache.datasketches-datasketches-java-7.0.0 created (now 8036f903f4)

2025-01-15 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.apache.datasketches-datasketches-java-7.0.0 in repository https://gitbox.apache.org/repos/asf/pinot.git at 8036f903f4 Bump org.apache.datasketches:datasketches-jav

[PR] Bump org.apache.datasketches:datasketches-java from 6.1.1 to 7.0.0 [pinot]

2025-01-15 Thread via GitHub
dependabot[bot] opened a new pull request, #14821: URL: https://github.com/apache/pinot/pull/14821 Bumps org.apache.datasketches:datasketches-java from 6.1.1 to 7.0.0. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-

[PR] Bump software.amazon.awssdk:bom from 2.29.51 to 2.29.52 [pinot]

2025-01-15 Thread via GitHub
dependabot[bot] opened a new pull request, #14820: URL: https://github.com/apache/pinot/pull/14820 Bumps software.amazon.awssdk:bom from 2.29.51 to 2.29.52. Most Recent Ignore Conditions Applied to This Pull Request | Dependency Name | Ignore Conditions | | --- | --- |

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.29.52 created (now 69f6a85bcb)

2025-01-15 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.29.52 in repository https://gitbox.apache.org/repos/asf/pinot.git at 69f6a85bcb Bump software.amazon.awssdk:bom from 2.29.51 to 2.29.52 N

(pinot) branch dependabot/maven/com.diffplug.spotless-spotless-maven-plugin-2.44.2 created (now d5da133a29)

2025-01-15 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.diffplug.spotless-spotless-maven-plugin-2.44.2 in repository https://gitbox.apache.org/repos/asf/pinot.git at d5da133a29 Bump com.diffplug.spotless:spotless-maven-

[PR] Bump com.diffplug.spotless:spotless-maven-plugin from 2.44.1 to 2.44.2 [pinot]

2025-01-15 Thread via GitHub
dependabot[bot] opened a new pull request, #14819: URL: https://github.com/apache/pinot/pull/14819 Bumps [com.diffplug.spotless:spotless-maven-plugin](https://github.com/diffplug/spotless) from 2.44.1 to 2.44.2. Release notes Sourced from https://github.com/diffplug/spotless/relea

(pinot) branch dependabot/maven/dropwizard-metrics.version-4.2.30 created (now 6585faab5a)

2025-01-15 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/dropwizard-metrics.version-4.2.30 in repository https://gitbox.apache.org/repos/asf/pinot.git at 6585faab5a Bump dropwizard-metrics.version from 4.2.29 to 4.2.30 No n

[PR] Bump dropwizard-metrics.version from 4.2.29 to 4.2.30 [pinot]

2025-01-15 Thread via GitHub
dependabot[bot] opened a new pull request, #14818: URL: https://github.com/apache/pinot/pull/14818 Bumps `dropwizard-metrics.version` from 4.2.29 to 4.2.30. Updates `io.dropwizard.metrics:metrics-core` from 4.2.29 to 4.2.30 Release notes Sourced from https://github.com/dropwizard

Re: [I] support simple file ingestion from public remote file [pinot]

2025-01-15 Thread via GitHub
hpvd commented on issue #14812: URL: https://github.com/apache/pinot/issues/14812#issuecomment-2592169450 +1, would also simplify testing with standardized/public data -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [I] HW SKU cost optimization [pinot]

2025-01-15 Thread via GitHub
hpvd commented on issue #14817: URL: https://github.com/apache/pinot/issues/14817#issuecomment-2592152261 yeah sounds pretty interesting! Looking forward to your presentation and hope for sharing with community! -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] Resolve failures pauseless ingestion [pinot]

2025-01-15 Thread via GitHub
codecov-commenter commented on PR #14798: URL: https://github.com/apache/pinot/pull/14798#issuecomment-2592121597 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14798?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] [flaky-test] BrokerManagedAsyncExecutorProviderTest#testRejectHandler [pinot]

2025-01-15 Thread via GitHub
DaniilRoman commented on code in PR #14814: URL: https://github.com/apache/pinot/pull/14814#discussion_r1916231941 ## pinot-broker/src/test/java/org/apache/pinot/broker/broker/BrokerManagedAsyncExecutorProviderTest.java: ## @@ -49,9 +50,12 @@ public class BrokerManagedAsyncExecu

[I] HW SKU cost optimization [pinot]

2025-01-15 Thread via GitHub
siddharthteotia opened a new issue, #14817: URL: https://github.com/apache/pinot/issues/14817 Our work on a Multi SKU recommender / cost-to-serve optimizer recently got accepted in USENIX and we will be presenting the work in March '25. We have built the entire machinery in-house and have a

Re: [PR] Resolve failures pauseless ingestion [pinot]

2025-01-15 Thread via GitHub
9aman commented on code in PR #14798: URL: https://github.com/apache/pinot/pull/14798#discussion_r1916189375 ## pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java: ## @@ -884,7 +892,7 @@ public String uploadLLCSegment( @ApiResponse(code =

Re: [PR] Resolve failures pauseless ingestion [pinot]

2025-01-15 Thread via GitHub
9aman commented on code in PR #14798: URL: https://github.com/apache/pinot/pull/14798#discussion_r1916183101 ## pinot-controller/src/main/java/org/apache/pinot/controller/validation/RealtimeSegmentValidationManager.java: ## @@ -171,6 +172,9 @@ private void runSegmentLevelValidat

Re: [PR] Resolve failures pauseless ingestion [pinot]

2025-01-15 Thread via GitHub
9aman commented on code in PR #14798: URL: https://github.com/apache/pinot/pull/14798#discussion_r1916173997 ## pinot-common/src/main/java/org/apache/pinot/common/utils/FileUploadDownloadClient.java: ## @@ -1248,6 +1249,63 @@ public File downloadUntarFileStreamed(URI uri, File

Re: [PR] Short Circuit And Filter Operator During Index Evaluation [pinot]

2025-01-15 Thread via GitHub
siddharthteotia commented on PR #14700: URL: https://github.com/apache/pinot/pull/14700#issuecomment-2591981125 Interesting. I think one of me or @jasperjiaguo or @vvivekiyer had an internal JIRA about AND filter optimizations. I don't recall if this is similar. Jia / Vivek - if you remem

Re: [PR] Re-implement DistinctCombineOperator to utilize more threads to merge [pinot]

2025-01-15 Thread via GitHub
siddharthteotia commented on code in PR #14714: URL: https://github.com/apache/pinot/pull/14714#discussion_r1916143288 ## pinot-core/src/main/java/org/apache/pinot/core/operator/combine/DistinctCombineOperator.java: ## @@ -19,26 +19,130 @@ package org.apache.pinot.core.operator

Re: [PR] Support BROADCAST join strategy [pinot]

2025-01-15 Thread via GitHub
Jackie-Jiang commented on PR #14797: URL: https://github.com/apache/pinot/pull/14797#issuecomment-2591909132 I'm thinking changing the `join_strategy` hint to `exchange_type` hint (still under `joinOptions`). For this particular PR's purpose, I can add `local_broadcast` to represent left lo