Re: [PR] Revert "Consolidate Auth and Validation in SSE Request Handler" [pinot]

2025-03-21 Thread via GitHub
vrajat commented on PR #15346: URL: https://github.com/apache/pinot/pull/15346#issuecomment-2745029954 > Is there a new function that super-sedes this deprecated function and does have the BrokerRequest in the method signature ? > > I understand something has been deprecated but ideal

Re: [PR] Model Hybrid Table behaviour in SSE query planning using HybridTable class [pinot]

2025-03-21 Thread via GitHub
vrajat commented on PR #15297: URL: https://github.com/apache/pinot/pull/15297#issuecomment-2745034726 @siddharthteotia @vvivekiyer @Yuyaoo Another refactor of `BaseSingleStageBrokerRequestHandler` that we are working on. -- This is an automated message from the Apache Git Service. To re

Re: [PR] Revert "Consolidate Auth and Validation in SSE Request Handler" [pinot]

2025-03-21 Thread via GitHub
vrajat commented on PR #15346: URL: https://github.com/apache/pinot/pull/15346#issuecomment-2745032294 I re-read the code to see if the call to broker request can be added on instead of reverting the commit. The only other option I see is : * Add the function to authorize using broker req

Re: [PR] Close `ChunkReaderContext` to improve projection/scan based filter memory for raw columns [pinot]

2025-03-21 Thread via GitHub
Jackie-Jiang commented on code in PR #15139: URL: https://github.com/apache/pinot/pull/15139#discussion_r2008612350 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/forward/ChunkReaderContext.java: ## @@ -52,11 +51,8 @@ public ChunkReader

Re: [PR] Support size based threshold for pauseless consumption [pinot]

2025-03-21 Thread via GitHub
codecov-commenter commented on PR #15347: URL: https://github.com/apache/pinot/pull/15347#issuecomment-2744904326 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15347?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Support size based threshold for pauseless consumption [pinot]

2025-03-21 Thread via GitHub
Jackie-Jiang commented on PR #15347: URL: https://github.com/apache/pinot/pull/15347#issuecomment-2744879836 cc @KKcorps @9aman -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Revert "Consolidate Auth and Validation in SSE Request Handler" [pinot]

2025-03-21 Thread via GitHub
siddharthteotia commented on PR #15346: URL: https://github.com/apache/pinot/pull/15346#issuecomment-2744736110 @vrajat @gortiz - Appreciate your help here !! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Revert "Consolidate Auth and Validation in SSE Request Handler" [pinot]

2025-03-21 Thread via GitHub
siddharthteotia commented on PR #15346: URL: https://github.com/apache/pinot/pull/15346#issuecomment-2744735939 Can we merge this please ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Fix Direct Memory OOM on Server [pinot]

2025-03-21 Thread via GitHub
xiangfu0 commented on PR #15335: URL: https://github.com/apache/pinot/pull/15335#issuecomment-2744622459 Is there a way to refactor this and #11496 to simplify the DirectOOMHandler implementation? -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [I] Upgrade Pinot service source version to Java 21 [pinot]

2025-03-21 Thread via GitHub
hpvd commented on issue #15343: URL: https://github.com/apache/pinot/issues/15343#issuecomment-2744585813 related to: https://github.com/apache/pinot/issues/14325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Bump com.mycila:license-maven-plugin from 4.6 to 5.0.0 [pinot]

2025-03-21 Thread via GitHub
tibrewalpratik17 merged PR #15329: URL: https://github.com/apache/pinot/pull/15329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@p

Re: [PR] Add metrics to track external view's node size [pinot]

2025-03-21 Thread via GitHub
codecov-commenter commented on PR #15338: URL: https://github.com/apache/pinot/pull/15338#issuecomment-2744502802 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15338?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Consumes segments in strict order of sequence number [pinot]

2025-03-21 Thread via GitHub
noob-se7en commented on code in PR #15261: URL: https://github.com/apache/pinot/pull/15261#discussion_r2008211190 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/ingestion/StreamIngestionConfig.java: ## @@ -43,6 +43,12 @@ public class StreamIngestionConfig extends B

Re: [PR] Revert "Consolidate Auth and Validation in SSE Request Handler" [pinot]

2025-03-21 Thread via GitHub
Jackie-Jiang commented on PR #15346: URL: https://github.com/apache/pinot/pull/15346#issuecomment-2744437818 cc @vrajat @gortiz -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Close `ChunkReaderContext` to improve projection/scan based filter memory for raw columns [pinot]

2025-03-21 Thread via GitHub
Jackie-Jiang commented on code in PR #15139: URL: https://github.com/apache/pinot/pull/15139#discussion_r2008311704 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/forward/ChunkReaderContext.java: ## @@ -52,11 +51,8 @@ public ChunkReader

Re: [PR] Revert "Consolidate Auth and Validation in SSE Request Handler" [pinot]

2025-03-21 Thread via GitHub
siddharthteotia commented on PR #15346: URL: https://github.com/apache/pinot/pull/15346#issuecomment-2744408922 Is there a new function in that super-sedes this deprecated function and does have the BrokerRequest in the method signature ? I understand something has been deprecated but

[PR] Revert "Consolidate Auth and Validation in SSE Request Handler" [pinot]

2025-03-21 Thread via GitHub
Yuyaoo opened a new pull request, #15346: URL: https://github.com/apache/pinot/pull/15346 PR apache/pinot#15240 introduced a backwards-incompatible change that has broken our build. Specifically, it removed the use of `accessControl.authorize` with the `brokerRequest` ([previous usage](htt

Re: [PR] Close `ChunkReaderContext` to improve projection/scan based filter memory for raw columns [pinot]

2025-03-21 Thread via GitHub
itschrispeck commented on code in PR #15139: URL: https://github.com/apache/pinot/pull/15139#discussion_r2008163230 ## pinot-core/src/main/java/org/apache/pinot/core/operator/dociditerators/MVScanDocIdIterator.java: ## @@ -251,4 +251,11 @@ public boolean doesValueMatch(int docId

Re: [PR] Allows consumption during build for Dedup [pinot]

2025-03-21 Thread via GitHub
noob-se7en commented on code in PR #15296: URL: https://github.com/apache/pinot/pull/15296#discussion_r2008245657 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java: ## @@ -1066,7 +1067,7 @@ protected SegmentBuildDescriptor b

Re: [PR] Consumes segments in strict order of sequence number [pinot]

2025-03-21 Thread via GitHub
noob-se7en commented on code in PR #15261: URL: https://github.com/apache/pinot/pull/15261#discussion_r2008207970 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/ingestion/StreamIngestionConfig.java: ## @@ -43,6 +43,12 @@ public class StreamIngestionConfig extends B

Re: [PR] Consumes segments in strict order of sequence number [pinot]

2025-03-21 Thread via GitHub
noob-se7en commented on code in PR #15261: URL: https://github.com/apache/pinot/pull/15261#discussion_r2008204346 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/SemaphoreAccessCoordinator.java: ## @@ -0,0 +1,251 @@ +/** + * Licensed to the Apache Softwar

Re: [PR] Consumes segments in strict order of sequence number [pinot]

2025-03-21 Thread via GitHub
Jackie-Jiang commented on code in PR #15261: URL: https://github.com/apache/pinot/pull/15261#discussion_r2008073447 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/SemaphoreAccessCoordinator.java: ## @@ -0,0 +1,251 @@ +/** + * Licensed to the Apache Softw

Re: [PR] Fix real-time table segment download [pinot]

2025-03-21 Thread via GitHub
Jackie-Jiang commented on code in PR #15316: URL: https://github.com/apache/pinot/pull/15316#discussion_r2008058288 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -564,77 +562,59 @@ private void doAddConsumingSegment(

[PR] Bump software.amazon.awssdk:bom from 2.31.4 to 2.31.5 [pinot]

2025-03-21 Thread via GitHub
dependabot[bot] opened a new pull request, #15344: URL: https://github.com/apache/pinot/pull/15344 Bumps software.amazon.awssdk:bom from 2.31.4 to 2.31.5. Most Recent Ignore Conditions Applied to This Pull Request | Dependency Name | Ignore Conditions | | --- | --- | |

Re: [PR] Fix up parsing of controller timers tableRebalanceExecutionTimeMs and cronSchedulerJobExecutionTimeMs [pinot]

2025-03-21 Thread via GitHub
somandal commented on code in PR #15330: URL: https://github.com/apache/pinot/pull/15330#discussion_r2007871341 ## docker/images/pinot/etc/jmx_prometheus_javaagent/configs/controller.yml: ## @@ -48,6 +48,21 @@ rules: table: "$2$4" tableType: "$5" taskType: "$6" +

Re: [PR] Fix up parsing of controller timers tableRebalanceExecutionTimeMs and cronSchedulerJobExecutionTimeMs [pinot]

2025-03-21 Thread via GitHub
somandal commented on PR #15330: URL: https://github.com/apache/pinot/pull/15330#issuecomment-2743955014 test failure seems to be due to a flaky test (passes locally for me, unrelated to this change): ``` Error: Failures: Error: DateTimeFieldSpecHybridClusterIntegrationTes

Re: [PR] Fix real-time table segment download [pinot]

2025-03-21 Thread via GitHub
9aman commented on code in PR #15316: URL: https://github.com/apache/pinot/pull/15316#discussion_r2007006292 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -564,77 +562,59 @@ private void doAddConsumingSegment(String

Re: [PR] Fix up parsing of controller timers tableRebalanceExecutionTimeMs and cronSchedulerJobExecutionTimeMs [pinot]

2025-03-21 Thread via GitHub
somandal commented on code in PR #15330: URL: https://github.com/apache/pinot/pull/15330#discussion_r2007876312 ## docker/images/pinot/etc/jmx_prometheus_javaagent/configs/controller.yml: ## @@ -48,6 +48,21 @@ rules: table: "$2$4" tableType: "$5" taskType: "$6" +

Re: [I] Changing Field Type between Dimension / Metric for Auto Generated Column Leads to Segment Loss [pinot]

2025-03-21 Thread via GitHub
ankitsultana commented on issue #15340: URL: https://github.com/apache/pinot/issues/15340#issuecomment-2743894378 @Jackie-Jiang : I am probably missing something, but since the deepstore copy is uploaded from the servers wouldn't that also have the auto-generated column? -- This is an au

Re: [PR] Bump org.apache.kafka:kafka-clients from 2.8.2 to 3.7.1 [pinot]

2025-03-21 Thread via GitHub
Jackie-Jiang closed pull request #15337: Bump org.apache.kafka:kafka-clients from 2.8.2 to 3.7.1 URL: https://github.com/apache/pinot/pull/15337 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Fix up parsing of controller timers tableRebalanceExecutionTimeMs and cronSchedulerJobExecutionTimeMs [pinot]

2025-03-21 Thread via GitHub
somandal commented on code in PR #15330: URL: https://github.com/apache/pinot/pull/15330#discussion_r2007876312 ## docker/images/pinot/etc/jmx_prometheus_javaagent/configs/controller.yml: ## @@ -48,6 +48,21 @@ rules: table: "$2$4" tableType: "$5" taskType: "$6" +

Re: [PR] Fix up parsing of controller timers tableRebalanceExecutionTimeMs and cronSchedulerJobExecutionTimeMs [pinot]

2025-03-21 Thread via GitHub
somandal commented on code in PR #15330: URL: https://github.com/apache/pinot/pull/15330#discussion_r2007876312 ## docker/images/pinot/etc/jmx_prometheus_javaagent/configs/controller.yml: ## @@ -48,6 +48,21 @@ rules: table: "$2$4" tableType: "$5" taskType: "$6" +

Re: [PR] Fix up parsing of controller timers tableRebalanceExecutionTimeMs and cronSchedulerJobExecutionTimeMs [pinot]

2025-03-21 Thread via GitHub
somandal commented on code in PR #15330: URL: https://github.com/apache/pinot/pull/15330#discussion_r2007871341 ## docker/images/pinot/etc/jmx_prometheus_javaagent/configs/controller.yml: ## @@ -48,6 +48,21 @@ rules: table: "$2$4" tableType: "$5" taskType: "$6" +

Re: [PR] Enforce schema for all tables [pinot]

2025-03-21 Thread via GitHub
jadami10 commented on PR #15333: URL: https://github.com/apache/pinot/pull/15333#issuecomment-2743548976 > Enforce schema for all tables Meaning enforce schema name == table name? We've been enforcing this already as well, so no concerns. -- This is an automated message from the Ap

(pinot) branch dependabot/maven/com.mycila-license-maven-plugin-5.0.0 deleted (was c61d9e4467)

2025-03-21 Thread pratik
This is an automated email from the ASF dual-hosted git repository. pratik pushed a change to branch dependabot/maven/com.mycila-license-maven-plugin-5.0.0 in repository https://gitbox.apache.org/repos/asf/pinot.git was c61d9e4467 Bump com.mycila:license-maven-plugin from 4.6 to 5.0.0 The

(pinot) branch master updated: Bump software.amazon.awssdk:bom from 2.31.4 to 2.31.5 (#15344)

2025-03-21 Thread pratik
This is an automated email from the ASF dual-hosted git repository. pratik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 8783eb1017 Bump software.amazon.awssdk:bom from 2

Re: [PR] Consumes segments in strict order of sequence number [pinot]

2025-03-21 Thread via GitHub
noob-se7en commented on code in PR #15261: URL: https://github.com/apache/pinot/pull/15261#discussion_r2007453854 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/SemaphoreAccessCoordinator.java: ## @@ -0,0 +1,251 @@ +/** + * Licensed to the Apache Softwar

Re: [PR] switch log.debug to log.info on query request [pinot]

2025-03-21 Thread via GitHub
gortiz commented on code in PR #15264: URL: https://github.com/apache/pinot/pull/15264#discussion_r2007193302 ## pinot-broker/src/main/java/org/apache/pinot/broker/querylog/QueryLogger.java: ## @@ -88,9 +87,12 @@ public void log(QueryLogParams params) { queryLogBuilder.ap

Re: [PR] Fix real-time table segment download [pinot]

2025-03-21 Thread via GitHub
9aman commented on code in PR #15316: URL: https://github.com/apache/pinot/pull/15316#discussion_r2007039924 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -564,77 +562,59 @@ private void doAddConsumingSegment(String

Re: [PR] Remove larray [pinot]

2025-03-21 Thread via GitHub
gortiz commented on PR #13545: URL: https://github.com/apache/pinot/pull/13545#issuecomment-2742621217 I'm finally removing LArray in https://github.com/apache/pinot/pull/15342 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[PR] Remove LArray usage [pinot]

2025-03-21 Thread via GitHub
gortiz opened a new pull request, #15342: URL: https://github.com/apache/pinot/pull/15342 This PR removes the dependency and all usages of LArray, as the last final PR to implement https://github.com/apache/pinot/issues/12810. As mentioned there, LArray is the library we used to use i

Re: [PR] Fix real-time table segment download [pinot]

2025-03-21 Thread via GitHub
9aman commented on code in PR #15316: URL: https://github.com/apache/pinot/pull/15316#discussion_r2007035192 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -564,77 +562,59 @@ private void doAddConsumingSegment(String

Re: [PR] Fix real-time table segment download [pinot]

2025-03-21 Thread via GitHub
9aman commented on code in PR #15316: URL: https://github.com/apache/pinot/pull/15316#discussion_r2007034972 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -564,77 +562,59 @@ private void doAddConsumingSegment(String