Re: [PR] Support apply logical type recursively to decode Avro message [pinot]

2024-07-24 Thread via GitHub
xiangfu0 merged PR #13669: URL: https://github.com/apache/pinot/pull/13669 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

(pinot) branch master updated: Support apply logical type recursively to decode Avro message (#13669)

2024-07-24 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 66c8dfef78 Support apply logical type recursivel

Re: [I] AvroSchemaUtils doesn't apply logical type in the nested structure [pinot]

2024-07-24 Thread via GitHub
xiangfu0 closed issue #13670: AvroSchemaUtils doesn't apply logical type in the nested structure URL: https://github.com/apache/pinot/issues/13670 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[PR] Update to latest version of clp-ffi that has support for Linux and macOS ARM platforms. [pinot]

2024-07-24 Thread via GitHub
kirkrodrigues opened a new pull request, #13687: URL: https://github.com/apache/pinot/pull/13687 This updates to the latest release of clp-ffi that has support for Linux and macOS ARM platforms. This partly addresses #10945. # Validation performed On an Apple Silicon Mac

(pinot) branch SegmentProcessorFrameworkImprovement created (now 5ef41d81bd)

2024-07-24 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch SegmentProcessorFrameworkImprovement in repository https://gitbox.apache.org/repos/asf/pinot.git at 5ef41d81bd Remove flaky test: testUpdateAndRemoveGauge (#13684) No new revisions were a

Re: [PR] Database query quota [pinot]

2024-07-24 Thread via GitHub
shounakmk219 commented on PR #13544: URL: https://github.com/apache/pinot/pull/13544#issuecomment-2249434537 > Are there tests for setting/updating qps due to helix messages for table qps ? Do you plan to tests those out as well ? @vrajat I am not sure how to specifically test that, w

Re: [PR] Database query quota [pinot]

2024-07-24 Thread via GitHub
shounakmk219 commented on code in PR #13544: URL: https://github.com/apache/pinot/pull/13544#discussion_r1690805600 ## pinot-broker/src/test/java/org/apache/pinot/broker/queryquota/HelixExternalViewBasedQueryQuotaManagerTest.java: ## @@ -82,21 +93,13 @@ private HelixManager init

Re: [PR] Database query quota [pinot]

2024-07-24 Thread via GitHub
vrajat commented on code in PR #13544: URL: https://github.com/apache/pinot/pull/13544#discussion_r1690780744 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/QuotaConfig.java: ## @@ -35,6 +35,7 @@ public class QuotaConfig extends BaseJsonConfig { private static f

Re: [PR] Remove ingestion metrics when consuming segment relocates [pinot]

2024-07-24 Thread via GitHub
KKcorps commented on code in PR #13668: URL: https://github.com/apache/pinot/pull/13668#discussion_r1690763064 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java: ## @@ -1029,6 +1039,47 @@ void updateInstanceSta

Re: [PR] Reduce Kinesis default rate limit to 1 to account for replication [pinot]

2024-07-24 Thread via GitHub
KKcorps commented on code in PR #13649: URL: https://github.com/apache/pinot/pull/13649#discussion_r1690763727 ## pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisConfig.java: ## @@ -70,7 +70,12 @@ public class Kinesi

Re: [PR] Fix bug when evaluating resource status during service startup check [pinot]

2024-07-24 Thread via GitHub
itschrispeck commented on PR #13541: URL: https://github.com/apache/pinot/pull/13541#issuecomment-2249074665 > My bad, now I see the problem. Should we consider first looping over the ideal state and create a map from serving partition to state, then only loop over this map in the second lo

Re: [PR] Add broker routing config: newSegmentExpirationTimeInSeconds [pinot]

2024-07-24 Thread via GitHub
codecov-commenter commented on PR #13686: URL: https://github.com/apache/pinot/pull/13686#issuecomment-2248945603 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13686?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

(pinot) branch master updated: Remove flaky test: testUpdateAndRemoveGauge (#13684)

2024-07-24 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 5ef41d81bd Remove flaky test: testUpdateAndRemoveGa

Re: [PR] Remove flaky test: testUpdateAndRemoveGauge [pinot]

2024-07-24 Thread via GitHub
codecov-commenter commented on PR #13684: URL: https://github.com/apache/pinot/pull/13684#issuecomment-2248901059 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13684?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Remove flaky test: testUpdateAndRemoveGauge [pinot]

2024-07-24 Thread via GitHub
jackjlli merged PR #13684: URL: https://github.com/apache/pinot/pull/13684 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[PR] Add broker routing config: newSegmentExpirationTimeInSeconds [pinot]

2024-07-24 Thread via GitHub
tibrewalpratik17 opened a new pull request, #13686: URL: https://github.com/apache/pinot/pull/13686 Fixes #13683 Adds a new broker / table level config: newSegmentExpirationTimeInSeconds which will help in configuring the maximum amount of time broker will wait for new consuming seg

[I] Add more UTs for AbstractMetrics class [pinot]

2024-07-24 Thread via GitHub
tibrewalpratik17 opened a new issue, #13685: URL: https://github.com/apache/pinot/issues/13685 The [AbstractMetrics](https://github.com/apache/pinot/blob/master/pinot-common/src/main/java/org/apache/pinot/common/metrics/AbstractMetrics.java) class has very less related UTs - https://github

[PR] Remove flaky test: testUpdateAndRemoveGauge [pinot]

2024-07-24 Thread via GitHub
tibrewalpratik17 opened a new pull request, #13684: URL: https://github.com/apache/pinot/pull/13684 Remove flaky test introduced in #13679. This test was not testing as well of what we want to so removing for now. We need more UTs for `AbstractMetrics` class and created an issue for

Re: [I] Improve InstanceSelector routing logic for new segments [pinot]

2024-07-24 Thread via GitHub
tibrewalpratik17 commented on issue #13683: URL: https://github.com/apache/pinot/issues/13683#issuecomment-2248804579 Hey @rohityadav1993 yes looks like #13256 might be the reason. I see the new consuming thread blocked for 5-6 minutes when the old consuming segment was getting replaced by

Re: [I] Improve InstanceSelector routing logic for new segments [pinot]

2024-07-24 Thread via GitHub
rohityadav1993 commented on issue #13683: URL: https://github.com/apache/pinot/issues/13683#issuecomment-2248641953 Thanks @tibrewalpratik17 for the analysis. This could be aggravated due to https://github.com/apache/pinot/pull/13256. Configuring NEW_SEGMENT_EXPIRATION_MILLIS can be a

Re: [PR] Improve realtime Lucene text index freshness/cpu/disk io usage [pinot]

2024-07-24 Thread via GitHub
itschrispeck commented on code in PR #13503: URL: https://github.com/apache/pinot/pull/13503#discussion_r1690224690 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneIndexRefreshManager.java: ## @@ -0,0 +1,268 @@ +/** +

[I] Improve InstanceSelector routing logic for new segments [pinot]

2024-07-24 Thread via GitHub
tibrewalpratik17 opened a new issue, #13683: URL: https://github.com/apache/pinot/issues/13683 Recently, we experienced an incident in our production environment where numerous query failures occurred due to the absence of a replica group to route the queries to. We are using the `strictRep

Re: [PR] Improve realtime Lucene text index freshness/cpu/disk io usage [pinot]

2024-07-24 Thread via GitHub
chenboat commented on code in PR #13503: URL: https://github.com/apache/pinot/pull/13503#discussion_r1690197552 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneIndexRefreshManager.java: ## @@ -0,0 +1,268 @@ +/** + * L

(pinot) branch master updated: Refactor PinotSchemaRestletResource for repetitive code pattern (#13680)

2024-07-24 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 00871f242f Refactor PinotSchemaRestletResource for

Re: [PR] Refactor PinotSchemaRestletResource for repetitive code pattern [pinot]

2024-07-24 Thread via GitHub
jackjlli merged PR #13680: URL: https://github.com/apache/pinot/pull/13680 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

(pinot) branch master updated (870cf57635 -> c603cd47f7)

2024-07-24 Thread mcvsubbu
This is an automated email from the ASF dual-hosted git repository. mcvsubbu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 870cf57635 Fix flaky gauge value issue (#13679) add c603cd47f7 Fix query response time metric (#13661) No new r

Re: [PR] Fix query response time metric [pinot]

2024-07-24 Thread via GitHub
mcvsubbu merged PR #13661: URL: https://github.com/apache/pinot/pull/13661 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] TLS Configuration Support for QueryServer and Dispatch Client [pinot]

2024-07-24 Thread via GitHub
codecov-commenter commented on PR #13645: URL: https://github.com/apache/pinot/pull/13645#issuecomment-2247874775 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13645?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Use debug logs in case we run into rate limit exceeded exception [pinot]

2024-07-24 Thread via GitHub
KKcorps commented on code in PR #13649: URL: https://github.com/apache/pinot/pull/13649#discussion_r1689646609 ## pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisConfig.java: ## @@ -70,7 +70,12 @@ public class Kinesi

Re: [PR] Use debug logs in case we run into rate limit exceeded exception [pinot]

2024-07-24 Thread via GitHub
KKcorps commented on code in PR #13649: URL: https://github.com/apache/pinot/pull/13649#discussion_r1689643390 ## pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisConsumer.java: ## @@ -122,7 +131,7 @@ private void rat

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.26.22 deleted (was 71126e369d)

2024-07-24 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.26.22 in repository https://gitbox.apache.org/repos/asf/pinot.git was 71126e369d Bump software.amazon.awssdk:bom from 2.26.21 to 2.26.22 T

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.26.23 created (now 909a217047)

2024-07-24 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.26.23 in repository https://gitbox.apache.org/repos/asf/pinot.git at 909a217047 Bump software.amazon.awssdk:bom from 2.26.21 to 2.26.23 N

[PR] Bump software.amazon.awssdk:bom from 2.26.21 to 2.26.23 [pinot]

2024-07-24 Thread via GitHub
dependabot[bot] opened a new pull request, #13682: URL: https://github.com/apache/pinot/pull/13682 Bumps software.amazon.awssdk:bom from 2.26.21 to 2.26.23. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=softwa

Re: [PR] Bump software.amazon.awssdk:bom from 2.26.21 to 2.26.22 [pinot]

2024-07-24 Thread via GitHub
dependabot[bot] closed pull request #13681: Bump software.amazon.awssdk:bom from 2.26.21 to 2.26.22 URL: https://github.com/apache/pinot/pull/13681 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Bump software.amazon.awssdk:bom from 2.26.21 to 2.26.22 [pinot]

2024-07-24 Thread via GitHub
dependabot[bot] commented on PR #13681: URL: https://github.com/apache/pinot/pull/13681#issuecomment-2247716625 Superseded by #13682. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] TLS Configuration Support for QueryServer and Dispatch Client [pinot]

2024-07-24 Thread via GitHub
anandheritage commented on PR #13645: URL: https://github.com/apache/pinot/pull/13645#issuecomment-2247089004 @Jackie-Jiang @xiangfu0 Can you enable the workflow here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u