Re: [PR] Add multistage thread limiting configs at the broker and server level [pinot]

2025-06-12 Thread via GitHub
gortiz commented on PR #16080: URL: https://github.com/apache/pinot/pull/16080#issuecomment-2969264781 I'm fine with this PR but I would suggest to include the beta prefix, given this is something we may want to change and deprecate in the future. We are discussing how to apply this limit l

Re: [PR] Add Filter for segment state in the Table UI [pinot]

2025-06-12 Thread via GitHub
codecov-commenter commented on PR #16085: URL: https://github.com/apache/pinot/pull/16085#issuecomment-2969234176 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/16085?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Add reset segment operation to UI [pinot]

2025-06-12 Thread via GitHub
codecov-commenter commented on PR #16078: URL: https://github.com/apache/pinot/pull/16078#issuecomment-2969235447 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/16078?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Fix null join key matching in multi-stage hash joins [pinot]

2025-06-12 Thread via GitHub
gortiz commented on code in PR #16038: URL: https://github.com/apache/pinot/pull/16038#discussion_r2144303921 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/HashJoinOperator.java: ## @@ -132,13 +171,28 @@ protected List buildJoinedRows(MseBlock.Data

Re: [PR] Add exception for resource limit exceeded in MultiStageOperator [pinot]

2025-06-12 Thread via GitHub
gortiz merged PR #16077: URL: https://github.com/apache/pinot/pull/16077 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

Re: [PR] Add exception for resource limit exceeded in MultiStageOperator [pinot]

2025-06-12 Thread via GitHub
gortiz commented on PR #16077: URL: https://github.com/apache/pinot/pull/16077#issuecomment-2969203642 We are going to receive a ServerResourceLimitExceededError, which is the same we return when we reach max number of rows in a join, for example. This may not be perfect, but customers will

Re: [I] Throw a separate exception when a query is terminated because of OOM protection [pinot]

2025-06-12 Thread via GitHub
gortiz commented on issue #14999: URL: https://github.com/apache/pinot/issues/14999#issuecomment-2969211085 Now that #16077 has been merged, we will receive a ServerResourceLimitExceededError (error code 245) when the OOM killer terminates a query. This is the same error code we send when a

(pinot) branch master updated: Add exception for resource limit exceeded in MultiStageOperator (#16077)

2025-06-12 Thread gortiz
This is an automated email from the ASF dual-hosted git repository. gortiz pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 6e8e3d2984 Add exception for resource limit excee

Re: [PR] try fixing CI error by running unit tests single-threadedly [pinot]

2025-06-12 Thread via GitHub
songwdfu commented on PR #16098: URL: https://github.com/apache/pinot/pull/16098#issuecomment-2969122921 Running all unit tests single-threadedly seems to solve the wierd Guice problem. Will try running which modules single-threaded are sufficient. -- This is an automated message from th

(pinot) branch master updated: try fixing CI error by running unit tests single-threadedly (#16098)

2025-06-12 Thread kharekartik
This is an automated email from the ASF dual-hosted git repository. kharekartik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 217b731085 try fixing CI error by running un

Re: [PR] try fixing CI error by running unit tests single-threadedly [pinot]

2025-06-12 Thread via GitHub
KKcorps merged PR #16098: URL: https://github.com/apache/pinot/pull/16098 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

Re: [PR] try fixing CI error by running unit tests single-threadedly [pinot]

2025-06-12 Thread via GitHub
KKcorps commented on PR #16098: URL: https://github.com/apache/pinot/pull/16098#issuecomment-2969141392 Merging this to unblock other PRs. We can revert it later on once we find actual fix -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[PR] Unit test ci fix by changing failing modules to single threaded [pinot]

2025-06-12 Thread via GitHub
songwdfu opened a new pull request, #16099: URL: https://github.com/apache/pinot/pull/16099 Unit test ci fix by changing failing modules to single threaded -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] TableDoesNotExistError for MSE queries on hybrid tables [pinot]

2025-06-12 Thread via GitHub
vrajat commented on issue #16032: URL: https://github.com/apache/pinot/issues/16032#issuecomment-2969106046 I am able to reproduce the issue. It is because of #15773 . I'll open a PR with a fix & test soon. -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] try fixing CI error by running unit tests single-threadedly [pinot]

2025-06-12 Thread via GitHub
codecov-commenter commented on PR #16098: URL: https://github.com/apache/pinot/pull/16098#issuecomment-2969068600 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/16098?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] try fixing unit test CI failure by downgrading custom user data maven extension [pinot]

2025-06-12 Thread via GitHub
songwdfu closed pull request #16095: try fixing unit test CI failure by downgrading custom user data maven extension URL: https://github.com/apache/pinot/pull/16095 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Disable develocity until build issue is fixed [pinot]

2025-06-12 Thread via GitHub
yashmayya closed pull request #16097: Disable develocity until build issue is fixed URL: https://github.com/apache/pinot/pull/16097 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[PR] try fixing CI error by running unit tests single-threadedly [pinot]

2025-06-12 Thread via GitHub
songwdfu opened a new pull request, #16098: URL: https://github.com/apache/pinot/pull/16098 try fixing CI error by running unit tests single-threadedly -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[PR] Disable develocity until build issue is fixed [pinot]

2025-06-12 Thread via GitHub
yashmayya opened a new pull request, #16097: URL: https://github.com/apache/pinot/pull/16097 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

[PR] [WIP] Force Commit Consuming Segments Before Moved during Table Rebalance [pinot]

2025-06-12 Thread via GitHub
J-HowHuang opened a new pull request, #16096: URL: https://github.com/apache/pinot/pull/16096 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

[PR] add -Ddevelocity.extension.skip=true into CI job as temporarily fix [pinot]

2025-06-12 Thread via GitHub
songwdfu opened a new pull request, #16095: URL: https://github.com/apache/pinot/pull/16095 add -Ddevelocity.extension.skip=true into CI job as temporarily fix to Develocity failure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Support for Virtual DataSource [pinot]

2025-06-12 Thread via GitHub
praveenc7 commented on code in PR #15350: URL: https://github.com/apache/pinot/pull/15350#discussion_r2143945284 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/immutable/ImmutableSegmentImpl.java: ## @@ -239,12 +240,14 @@ public SegmentMetadataI

Re: [PR] Support for Virtual DataSource [pinot]

2025-06-12 Thread via GitHub
praveenc7 commented on code in PR #15350: URL: https://github.com/apache/pinot/pull/15350#discussion_r2143874936 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/immutable/ImmutableSegmentImpl.java: ## @@ -239,12 +240,14 @@ public SegmentMetadataI

Re: [PR] Support for Virtual DataSource [pinot]

2025-06-12 Thread via GitHub
Jackie-Jiang commented on code in PR #15350: URL: https://github.com/apache/pinot/pull/15350#discussion_r2143908804 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/immutable/ImmutableSegmentImpl.java: ## @@ -239,12 +240,14 @@ public SegmentMetada

Re: [PR] Support for Virtual DataSource [pinot]

2025-06-12 Thread via GitHub
Jackie-Jiang commented on PR #15350: URL: https://github.com/apache/pinot/pull/15350#issuecomment-2968471995 Applied a commit for some misc fixes and cleanups. Changes worth noting: - Do not add virtual data source into immutable segment to avoid thread safety issue - Do not check inva

Re: [PR] Support for Virtual DataSource [pinot]

2025-06-12 Thread via GitHub
Jackie-Jiang commented on code in PR #15350: URL: https://github.com/apache/pinot/pull/15350#discussion_r2143824933 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/immutable/ImmutableSegmentImpl.java: ## @@ -234,6 +236,17 @@ public SegmentMetadat

Re: [PR] Adds remove ingestion-metrics API [pinot]

2025-06-12 Thread via GitHub
Jackie-Jiang commented on code in PR #16045: URL: https://github.com/apache/pinot/pull/16045#discussion_r2143735230 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java: ## @@ -1257,6 +1261,59 @@ public SuccessResponse dele

[PR] Add bad data handling for some IndexCreator::add() functions to skip record or add dummy record [pinot]

2025-06-12 Thread via GitHub
somandal opened a new pull request, #16094: URL: https://github.com/apache/pinot/pull/16094 This PR adds handling for potential bad data for some of the `IndexCreator::add()` functions: - LuceneFSTIndexCreator - adds handling in the constructor and add() to catch exceptions and skip

Re: [PR] Adds remove ingestion-metrics API [pinot]

2025-06-12 Thread via GitHub
noob-se7en commented on code in PR #16045: URL: https://github.com/apache/pinot/pull/16045#discussion_r2143619970 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java: ## @@ -1257,6 +1261,59 @@ public SuccessResponse delete

Re: [PR] Adds remove ingestion-metrics API [pinot]

2025-06-12 Thread via GitHub
Jackie-Jiang commented on code in PR #16045: URL: https://github.com/apache/pinot/pull/16045#discussion_r2143581069 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java: ## @@ -1257,6 +1261,59 @@ public SuccessResponse dele

Re: [PR] Adds BEST_EFFORT disaster recovery mode in dedup [pinot]

2025-06-12 Thread via GitHub
noob-se7en commented on code in PR #16071: URL: https://github.com/apache/pinot/pull/16071#discussion_r2143581178 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java: ## @@ -1472,9 +1472,10 @@ public void goOnlineFromConsuming(

Re: [PR] Adds BEST_EFFORT disaster recovery mode in dedup [pinot]

2025-06-12 Thread via GitHub
noob-se7en commented on code in PR #16071: URL: https://github.com/apache/pinot/pull/16071#discussion_r2143581178 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java: ## @@ -1472,9 +1472,10 @@ public void goOnlineFromConsuming(

Re: [PR] Adds BEST_EFFORT disaster recovery mode in dedup [pinot]

2025-06-12 Thread via GitHub
noob-se7en commented on code in PR #16071: URL: https://github.com/apache/pinot/pull/16071#discussion_r2143580884 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java: ## @@ -1472,9 +1472,10 @@ public void goOnlineFromConsuming(

Re: [PR] [Do not merge] [Bugfix] [UpsertCompactMerge] Resolve data inconsistency in merge compaction caused by replica create time mismatch [pinot]

2025-06-12 Thread via GitHub
tarun11Mavani commented on PR #15863: URL: https://github.com/apache/pinot/pull/15863#issuecomment-2967735952 We added support to use creation time from ZK metadata while resolving the upsert tie breaker in #16034. This allows us to use consistent time across replica and make sure all repli

Re: [PR] [Do not merge] [Bugfix] [UpsertCompactMerge] Resolve data inconsistency in merge compaction caused by replica create time mismatch [pinot]

2025-06-12 Thread via GitHub
tarun11Mavani closed pull request #15863: [Do not merge] [Bugfix] [UpsertCompactMerge] Resolve data inconsistency in merge compaction caused by replica create time mismatch URL: https://github.com/apache/pinot/pull/15863 -- This is an automated message from the Apache Git Service. To respond

Re: [PR] [timeseries] Adding requester level access control check for timeseries queries [pinot]

2025-06-12 Thread via GitHub
shauryachats commented on code in PR #16066: URL: https://github.com/apache/pinot/pull/16066#discussion_r2143317652 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/TimeSeriesAuthIntegrationTest.java: ## @@ -0,0 +1,177 @@ +/** + * Licensed to the Apach

[I] Support custom headers for uploadSegments in ClusterTest [pinot]

2025-06-12 Thread via GitHub
shauryachats opened a new issue, #16093: URL: https://github.com/apache/pinot/issues/16093 [CursorWithAuthIntegrationTest.java](https://github.com/apache/pinot/blob/master/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/CursorWithAuthIntegrationTest.java#L135) has i

Re: [PR] Fix null join key matching in multi-stage hash joins [pinot]

2025-06-12 Thread via GitHub
soupam05 commented on code in PR #16038: URL: https://github.com/apache/pinot/pull/16038#discussion_r2143313385 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/HashJoinOperator.java: ## @@ -59,16 +60,21 @@ public class HashJoinOperator extends BaseJo

Re: [PR] Fix null join key matching in multi-stage hash joins [pinot]

2025-06-12 Thread via GitHub
soupam05 commented on code in PR #16038: URL: https://github.com/apache/pinot/pull/16038#discussion_r2143312070 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/HashJoinOperator.java: ## @@ -98,8 +104,36 @@ public String toExplainString() { protect

Re: [PR] Fix null join key matching in multi-stage hash joins [pinot]

2025-06-12 Thread via GitHub
soupam05 commented on code in PR #16038: URL: https://github.com/apache/pinot/pull/16038#discussion_r2143308293 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/HashJoinOperator.java: ## @@ -59,16 +60,21 @@ public class HashJoinOperator extends BaseJo

(pinot) branch master updated: Continue Segment Reset post instance errors (#15903)

2025-06-12 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 44dde02db1 Continue Segment Reset post instance e

Re: [PR] Continue Segment Reset post instance errors [pinot]

2025-06-12 Thread via GitHub
Jackie-Jiang merged PR #15903: URL: https://github.com/apache/pinot/pull/15903 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Add new ImplicitRealtimeTablePartitionSelector strategy for instance assignment [pinot]

2025-06-12 Thread via GitHub
Jackie-Jiang commented on PR #15930: URL: https://github.com/apache/pinot/pull/15930#issuecomment-2967638536 Great work! Please also help update pinot documentation for this new strategy -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] Fix null join key matching in multi-stage hash joins [pinot]

2025-06-12 Thread via GitHub
soupam05 commented on code in PR #16038: URL: https://github.com/apache/pinot/pull/16038#discussion_r2143256969 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/HashJoinOperator.java: ## @@ -234,6 +289,11 @@ private List buildJoinedDataBlockAnti(MseB

(pinot) branch master updated: Fix up the schema validation to perform most validations for virtual columns as well (#16091)

2025-06-12 Thread somandal
This is an automated email from the ASF dual-hosted git repository. somandal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new ed62f82a88 Fix up the schema validation to perf

Re: [PR] Fix up the schema validation to perform most validations for virtual columns as well [pinot]

2025-06-12 Thread via GitHub
somandal merged PR #16091: URL: https://github.com/apache/pinot/pull/16091 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

(pinot) branch master updated: [MSE] For constant expression query, solve it with a single random server (#16083)

2025-06-12 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 99cde4d164 [MSE] For constant expression query, s

Re: [PR] [MSE] For constant expression query, solve it with a single random server [pinot]

2025-06-12 Thread via GitHub
Jackie-Jiang merged PR #16083: URL: https://github.com/apache/pinot/pull/16083 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [I] Pinot fanout constant queries to all servers which could cause unexpected DNS query spikes [pinot]

2025-06-12 Thread via GitHub
Jackie-Jiang closed issue #16024: Pinot fanout constant queries to all servers which could cause unexpected DNS query spikes URL: https://github.com/apache/pinot/issues/16024 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] [MSE] For constant expression query, solve it with a single random server [pinot]

2025-06-12 Thread via GitHub
Jackie-Jiang commented on code in PR #16083: URL: https://github.com/apache/pinot/pull/16083#discussion_r2143245694 ## pinot-query-planner/src/main/java/org/apache/pinot/query/routing/WorkerManager.java: ## @@ -319,48 +340,44 @@ private boolean isPrePartitionAssignment(List chi

Re: [PR] [timeseries] Adding requester level access control check for timeseries queries [pinot]

2025-06-12 Thread via GitHub
ankitsultana commented on code in PR #16066: URL: https://github.com/apache/pinot/pull/16066#discussion_r2143199973 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/TimeSeriesAuthIntegrationTest.java: ## @@ -0,0 +1,177 @@ +/** + * Licensed to the Apach

Re: [PR] [timeseries] Adding requester level access control check for timeseries queries [pinot]

2025-06-12 Thread via GitHub
shauryachats commented on code in PR #16066: URL: https://github.com/apache/pinot/pull/16066#discussion_r2143189579 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/TimeSeriesAuthIntegrationTest.java: ## @@ -0,0 +1,177 @@ +/** + * Licensed to the Apach

[I] [multistage] Scenarios with Silent Query Failures [pinot]

2025-06-12 Thread via GitHub
ankitsultana opened a new issue, #16092: URL: https://github.com/apache/pinot/issues/16092 In #15741 we had reported one scenario where queries fail silently without any error messages. I have found more such scenarios in the past few days so creating this broader ticket for tracking

Re: [I] [multistage] Mailbox Errs Without Logs on Inbound Message Limit Breach [pinot]

2025-06-12 Thread via GitHub
ankitsultana commented on issue #15741: URL: https://github.com/apache/pinot/issues/15741#issuecomment-2967476638 @shauryachats : can you confirm if your patch has fixed this issue? As in, before your patch, queries with this error would fail silently and block till timeout. Af

Re: [PR] Add multistage thread limiting configs at the broker and server level [pinot]

2025-06-12 Thread via GitHub
gortiz commented on code in PR #16080: URL: https://github.com/apache/pinot/pull/16080#discussion_r2143153222 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java: ## @@ -382,6 +382,9 @@ public static class Broker { public static final String CONFIG_OF

Re: [PR] [Query Resource Isolation] Workload Configs [pinot]

2025-06-12 Thread via GitHub
praveenc7 commented on code in PR #15109: URL: https://github.com/apache/pinot/pull/15109#discussion_r2112671214 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java: ## @@ -4764,6 +4777,48 @@ public Map minimumInstancesRequire

Re: [PR] [timeseries] Adding requester level access control check for timeseries queries [pinot]

2025-06-12 Thread via GitHub
ankitsultana commented on code in PR #16066: URL: https://github.com/apache/pinot/pull/16066#discussion_r2143133449 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/TimeSeriesAuthIntegrationTest.java: ## @@ -0,0 +1,177 @@ +/** + * Licensed to the Apach

Re: [PR] [timeseries] Adding requester level access control check for timeseries queries [pinot]

2025-06-12 Thread via GitHub
ankitsultana commented on code in PR #16066: URL: https://github.com/apache/pinot/pull/16066#discussion_r2143134249 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/TimeSeriesAuthIntegrationTest.java: ## @@ -0,0 +1,177 @@ +/** + * Licensed to the Apach

Re: [PR] [timeseries] Adding requester level access control check for timeseries queries [pinot]

2025-06-12 Thread via GitHub
ankitsultana commented on code in PR #16066: URL: https://github.com/apache/pinot/pull/16066#discussion_r2143133449 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/TimeSeriesAuthIntegrationTest.java: ## @@ -0,0 +1,177 @@ +/** + * Licensed to the Apach

Re: [PR] Add validations for MV columns: reject MV primary-keys for upsert/dedup, BIG_DECIMAL, and JSON [pinot]

2025-06-12 Thread via GitHub
somandal commented on code in PR #16079: URL: https://github.com/apache/pinot/pull/16079#discussion_r2143133273 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/SchemaUtils.java: ## @@ -147,6 +147,9 @@ public static void validate(Schema schema, boolean

Re: [PR] [timeseries] Adding requester level access control check for timeseries queries [pinot]

2025-06-12 Thread via GitHub
ankitsultana commented on code in PR #16066: URL: https://github.com/apache/pinot/pull/16066#discussion_r2143131169 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/TimeSeriesAuthIntegrationTest.java: ## @@ -0,0 +1,177 @@ +/** + * Licensed to the Apach

Re: [I] TableDoesNotExistError for MSE queries on hybrid tables [pinot]

2025-06-12 Thread via GitHub
jadami10 commented on issue #16032: URL: https://github.com/apache/pinot/issues/16032#issuecomment-2967385541 I reproduced with `HybridTableQuickstart`. `MultiStageEngineQuickStart` seems to only have offline tables -- This is an automated message from the Apache Git Service. To respond t

[PR] Fix up the schema validation to perform most validations for virtual columns as well [pinot]

2025-06-12 Thread via GitHub
somandal opened a new pull request, #16091: URL: https://github.com/apache/pinot/pull/16091 Follow-up for https://github.com/apache/pinot/pull/16079 where the ask was to also do most of the schema validations for virtual columns as well. Left out the part where the primary key candidates ar

Re: [PR] Bump com.gradle:common-custom-user-data-maven-extension from 2.0.2 to 2.0.3 [pinot]

2025-06-12 Thread via GitHub
noob-se7en commented on PR #16049: URL: https://github.com/apache/pinot/pull/16049#issuecomment-2967216928 Hmm revert PR also failed. Trying to see what change caused the issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Revert "Bump com.gradle:common-custom-user-data-maven-extension from 2.0.2 to 2.0.3" [pinot]

2025-06-12 Thread via GitHub
noob-se7en closed pull request #16090: Revert "Bump com.gradle:common-custom-user-data-maven-extension from 2.0.2 to 2.0.3" URL: https://github.com/apache/pinot/pull/16090 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[PR] Revert "Bump com.gradle:common-custom-user-data-maven-extension from 2.0.2 to 2.0.3" [pinot]

2025-06-12 Thread via GitHub
noob-se7en opened a new pull request, #16090: URL: https://github.com/apache/pinot/pull/16090 Reverts apache/pinot#16049 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Reduce fanout of intermediate stage [pinot]

2025-06-12 Thread via GitHub
ankitsultana commented on PR #16084: URL: https://github.com/apache/pinot/pull/16084#issuecomment-2967172806 > // For MSE queries, when this option is set to true, only use servers for leaf stages as the workers for the > // intermediate stages. This is useful to control the fanout of the

Re: [PR] Bump com.gradle:common-custom-user-data-maven-extension from 2.0.2 to 2.0.3 [pinot]

2025-06-12 Thread via GitHub
noob-se7en commented on PR #16049: URL: https://github.com/apache/pinot/pull/16049#issuecomment-2967169291 Can we try reverting this change?? Seeing these errors in CIs: Error: Failed to execute goal org.jacoco:jacoco-maven-plugin:0.8.13:prepare-agent (prepare-agent) on projec

Re: [PR] Bump com.gradle:develocity-maven-extension from 2.0 to 2.0.1 [pinot]

2025-06-12 Thread via GitHub
noob-se7en commented on PR #15937: URL: https://github.com/apache/pinot/pull/15937#issuecomment-2967156663 Can we try reverting this change?? @xiangfu0 @Jackie-Jiang Seeing these errors in CIs: ``` Error: Failed to execute goal org.jacoco:jacoco-maven-plugin:0.8.13:prepare

Re: [PR] Adding changes for supporting RLS [pinot]

2025-06-12 Thread via GitHub
9aman commented on code in PR #16043: URL: https://github.com/apache/pinot/pull/16043#discussion_r2142889370 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/plan/server/ServerPlanRequestUtils.java: ## @@ -76,19 +78,26 @@ private ServerPlanRequestUtils() {

[PR] [UpsertCompactMerge] Adding UTs for UpsertCompactMergeTask [pinot]

2025-06-12 Thread via GitHub
tarun11Mavani opened a new pull request, #16089: URL: https://github.com/apache/pinot/pull/16089 Adding UTs for UpsertCompactMergeTask -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Adding changes for supporting RLS [pinot]

2025-06-12 Thread via GitHub
9aman commented on code in PR #16043: URL: https://github.com/apache/pinot/pull/16043#discussion_r2142858076 ## pinot-core/src/main/java/org/apache/pinot/core/auth/BasicAuthUtils.java: ## @@ -101,9 +122,10 @@ public static List extractBasicAuthPrincipals(List Collections.emptyL

Re: [PR] Adding changes for supporting RLS [pinot]

2025-06-12 Thread via GitHub
9aman commented on code in PR #16043: URL: https://github.com/apache/pinot/pull/16043#discussion_r2142848705 ## pinot-core/src/main/java/org/apache/pinot/core/auth/BasicAuthPrincipal.java: ## @@ -31,14 +34,17 @@ public class BasicAuthPrincipal { private final Set _tables;

Re: [PR] [UpsertCompactMerge] Use consistent creation time for merged segments to prevent replica inconsistencies [pinot]

2025-06-12 Thread via GitHub
tarun11Mavani commented on PR #16086: URL: https://github.com/apache/pinot/pull/16086#issuecomment-2966890209 cc @tibrewalpratik17 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [I] TableDoesNotExistError for MSE queries on hybrid tables [pinot]

2025-06-12 Thread via GitHub
vrajat commented on issue #16032: URL: https://github.com/apache/pinot/issues/16032#issuecomment-2966817753 @jadami10 I am trying to reproduce the issue. I started `MultiStageEngineQuickStart` and ran the following curl command as well as ran the query through the UI. ``` curl --

Re: [PR] Adding changes for supporting RLS [pinot]

2025-06-12 Thread via GitHub
9aman commented on code in PR #16043: URL: https://github.com/apache/pinot/pull/16043#discussion_r2142735397 ## pinot-broker/src/main/java/org/apache/pinot/broker/broker/BasicAuthAccessControlFactory.java: ## @@ -129,12 +132,43 @@ public TableAuthorizationResult authorize(Reque

Re: [PR] Adding changes for supporting RLS [pinot]

2025-06-12 Thread via GitHub
9aman commented on code in PR #16043: URL: https://github.com/apache/pinot/pull/16043#discussion_r2142707078 ## pinot-core/src/main/java/org/apache/pinot/core/auth/BasicAuthPrincipal.java: ## @@ -65,13 +72,27 @@ public boolean hasPermission(String permission) { return _perm

(pinot) branch dependabot/maven/org.webjars-swagger-ui-5.24.1-1 created (now f345e8d459)

2025-06-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.webjars-swagger-ui-5.24.1-1 in repository https://gitbox.apache.org/repos/asf/pinot.git at f345e8d459 Bump org.webjars:swagger-ui from 5.24.1 to 5.24.1-1 No new r

Re: [PR] Add exception for resource limit exceeded in MultiStageOperator [pinot]

2025-06-12 Thread via GitHub
vrajat commented on code in PR #16077: URL: https://github.com/apache/pinot/pull/16077#discussion_r2142392974 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MultiStageOperator.java: ## @@ -78,6 +79,8 @@ protected void sampleAndCheckInterruption() {

[PR] Bump org.webjars:swagger-ui from 5.24.1 to 5.24.1-1 [pinot]

2025-06-12 Thread via GitHub
dependabot[bot] opened a new pull request, #16088: URL: https://github.com/apache/pinot/pull/16088 Bumps [org.webjars:swagger-ui](https://github.com/swagger-api/swagger-ui) from 5.24.1 to 5.24.1-1. Commits See full diff in https://github.com/swagger-api/swagger-ui/commits";>com

Re: [PR] Add exception for resource limit exceeded in MultiStageOperator [pinot]

2025-06-12 Thread via GitHub
vrajat commented on code in PR #16077: URL: https://github.com/apache/pinot/pull/16077#discussion_r2142387657 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MultiStageOperator.java: ## @@ -78,6 +79,8 @@ protected void sampleAndCheckInterruption() {

Re: [I] [UpsertMergeCompactTask] Data inconsistency issue across segment replica due to different segment creation time after enabling UpsertMergeCompactTask [pinot]

2025-06-12 Thread via GitHub
tarun11Mavani closed issue #15846: [UpsertMergeCompactTask] Data inconsistency issue across segment replica due to different segment creation time after enabling UpsertMergeCompactTask URL: https://github.com/apache/pinot/issues/15846 -- This is an automated message from the Apache Git Servi

Re: [PR] Adding changes for supporting RLS [pinot]

2025-06-12 Thread via GitHub
vrajat commented on code in PR #16043: URL: https://github.com/apache/pinot/pull/16043#discussion_r2141774043 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandler.java: ## @@ -919,6 +922,20 @@ private CompileResult compileReque

[PR] Add Filter for segment state in the Table UI [pinot]

2025-06-12 Thread via GitHub
KKcorps opened a new pull request, #16085: URL: https://github.com/apache/pinot/pull/16085 This pull request enhances the user interface of several components in the Pinot Controller application by introducing a new `StatusFilter` component, improving layout flexibility with additional cont

Re: [PR] Adding changes for supporting RLS [pinot]

2025-06-12 Thread via GitHub
9aman commented on code in PR #16043: URL: https://github.com/apache/pinot/pull/16043#discussion_r2141950515 ## pinot-broker/src/main/java/org/apache/pinot/broker/api/AccessControl.java: ## @@ -120,4 +128,39 @@ default TableAuthorizationResult authorize(RequesterIdentity reques

Re: [PR] Add exception for resource limit exceeded in MultiStageOperator [pinot]

2025-06-12 Thread via GitHub
vrajat commented on code in PR #16077: URL: https://github.com/apache/pinot/pull/16077#discussion_r2142392974 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MultiStageOperator.java: ## @@ -78,6 +79,8 @@ protected void sampleAndCheckInterruption() {

[PR] Bump software.amazon.awssdk:bom from 2.31.61 to 2.31.62 [pinot]

2025-06-12 Thread via GitHub
dependabot[bot] opened a new pull request, #16087: URL: https://github.com/apache/pinot/pull/16087 Bumps software.amazon.awssdk:bom from 2.31.61 to 2.31.62. Most Recent Ignore Conditions Applied to This Pull Request | Dependency Name | Ignore Conditions | | --- | --- |

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.31.62 created (now da4b0eb70f)

2025-06-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.31.62 in repository https://gitbox.apache.org/repos/asf/pinot.git at da4b0eb70f Bump software.amazon.awssdk:bom from 2.31.61 to 2.31.62 N

[PR] Reduce fanout of intermediate stage [pinot]

2025-06-12 Thread via GitHub
Jackie-Jiang opened a new pull request, #16084: URL: https://github.com/apache/pinot/pull/16084 Fix #15932 Reduce the fanout of intermediate stage workers: - Do not include tables used as lookup table when calculating the candidate servers for intermediate stage - Add a config/

Re: [PR] Adding changes for supporting RLS [pinot]

2025-06-12 Thread via GitHub
9aman commented on code in PR #16043: URL: https://github.com/apache/pinot/pull/16043#discussion_r2141920553 ## pinot-common/src/main/java/org/apache/pinot/sql/parsers/rewriter/RlsFiltersRewriter.java: ## @@ -0,0 +1,39 @@ +package org.apache.pinot.sql.parsers.rewriter; + +import

Re: [PR] Add multistage thread limiting configs at the broker and server level [pinot]

2025-06-12 Thread via GitHub
satwik-pachigolla commented on code in PR #16080: URL: https://github.com/apache/pinot/pull/16080#discussion_r2141752700 ## pinot-spi/src/main/java/org/apache/pinot/spi/executor/HardLimitExecutor.java: ## @@ -46,21 +50,31 @@ public HardLimitExecutor(int max, ExecutorService exe

Re: [PR] [Priority Replica Group Routing] Support orderedPreferredReplicas query option for customizable routing strategy [pinot]

2025-06-12 Thread via GitHub
yashmayya commented on PR #15203: URL: https://github.com/apache/pinot/pull/15203#issuecomment-2964791938 @wirybeaver the multiple tenants / pools thing is a secondary concern. The primary concern is calling instance pools replica groups (query option, query logs, metrics), when we already

Re: [PR] [multistage] Enable Tests for New Optimizer + Bug Fixes [pinot]

2025-06-12 Thread via GitHub
wirybeaver commented on code in PR #15958: URL: https://github.com/apache/pinot/pull/15958#discussion_r2141672531 ## pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotRuleUtils.java: ## @@ -122,4 +135,129 @@ public static String extractFunctionName(RexCal

[PR] Pass creation time as task input [pinot]

2025-06-12 Thread via GitHub
tarun11Mavani opened a new pull request, #16086: URL: https://github.com/apache/pinot/pull/16086 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] Add Filter for segment state in the Table UI [pinot]

2025-06-12 Thread via GitHub
Copilot commented on code in PR #16085: URL: https://github.com/apache/pinot/pull/16085#discussion_r2142037357 ## pinot-controller/src/main/resources/app/pages/TenantDetails.tsx: ## @@ -878,6 +904,19 @@ const TenantPageDetails = ({ match }: RouteComponentProps) => {

Re: [I] [UpsertMergeCompactTask] Data inconsistency issue across segment replica due to different segment creation time after enabling UpsertMergeCompactTask [pinot]

2025-06-12 Thread via GitHub
tarun11Mavani commented on issue #15846: URL: https://github.com/apache/pinot/issues/15846#issuecomment-2965770505 This has been fixed in #16034 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Adding changes for supporting RLS [pinot]

2025-06-12 Thread via GitHub
9aman commented on code in PR #16043: URL: https://github.com/apache/pinot/pull/16043#discussion_r2141911530 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandler.java: ## @@ -919,6 +922,32 @@ private CompileResult compileReques

Re: [PR] Adding changes for supporting RLS [pinot]

2025-06-12 Thread via GitHub
9aman commented on code in PR #16043: URL: https://github.com/apache/pinot/pull/16043#discussion_r2141952074 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandler.java: ## @@ -919,6 +922,20 @@ private CompileResult compileReques

Re: [PR] Adding changes for supporting RLS [pinot]

2025-06-12 Thread via GitHub
9aman commented on code in PR #16043: URL: https://github.com/apache/pinot/pull/16043#discussion_r2141951090 ## pinot-broker/src/main/java/org/apache/pinot/broker/api/AccessControl.java: ## @@ -120,4 +128,39 @@ default TableAuthorizationResult authorize(RequesterIdentity reques

Re: [PR] Adding changes for supporting RLS [pinot]

2025-06-12 Thread via GitHub
9aman commented on code in PR #16043: URL: https://github.com/apache/pinot/pull/16043#discussion_r2141912186 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandler.java: ## @@ -919,6 +922,32 @@ private CompileResult compileReques

  1   2   >