Re: [PR] Pauseless Consumption #3: Disaster Recovery with Reingestion [pinot]

2025-02-06 Thread via GitHub
KKcorps commented on code in PR #14920: URL: https://github.com/apache/pinot/pull/14920#discussion_r1946106811 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/writer/StatelessRealtimeSegmentWriter.java: ## @@ -0,0 +1,582 @@ +/** + * Licensed to the A

Re: [PR] Pauseless Consumption #3: Disaster Recovery with Reingestion [pinot]

2025-02-06 Thread via GitHub
9aman commented on code in PR #14920: URL: https://github.com/apache/pinot/pull/14920#discussion_r1946074533 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java: ## @@ -2096,6 +2104,137 @@ URI createSegmentPath(St

Re: [PR] Pauseless Consumption #3: Disaster Recovery with Reingestion [pinot]

2025-02-06 Thread via GitHub
9aman commented on code in PR #14920: URL: https://github.com/apache/pinot/pull/14920#discussion_r1946087815 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java: ## @@ -389,15 +390,21 @@ protected void replaceSegmentIfCrcMismatch(SegmentData

Re: [PR] Pauseless Consumption #3: Disaster Recovery with Reingestion [pinot]

2025-02-06 Thread via GitHub
9aman commented on code in PR #14920: URL: https://github.com/apache/pinot/pull/14920#discussion_r1946082135 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java: ## @@ -2096,6 +2104,131 @@ URI createSegmentPath(St

Re: [PR] Add configs to customize http client/request timeouts and improve peer download reliability [pinot]

2025-02-06 Thread via GitHub
codecov-commenter commented on PR #15010: URL: https://github.com/apache/pinot/pull/15010#issuecomment-2642059184 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15010?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [I] Segments are lost [pinot]

2025-02-06 Thread via GitHub
raghukn commented on issue #14993: URL: https://github.com/apache/pinot/issues/14993#issuecomment-2642012129 Oh that must be the reason then ! Interesting, my understanding was 7 day period starts from the date the segment is created !! Since a segment can contain data from mul

[PR] Add configs to customize http client/request timeouts and improve peer download reliability [pinot]

2025-02-06 Thread via GitHub
itschrispeck opened a new pull request, #15010: URL: https://github.com/apache/pinot/pull/15010 Makes http request/client timeouts configurable, primarily for peer download. We have found peer downloads often fail while waiting for a connection from the pool. Retry means the peer download l

(pinot) branch master updated: [timeseries] Add Support for Passing Raw Time Values to Leaf Stage (#15000)

2025-02-06 Thread ankitsultana
This is an automated email from the ASF dual-hosted git repository. ankitsultana pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 159fc6716e [timeseries] Add Support for Pas

Re: [PR] [timeseries] Add Support for Passing Raw Time Values to Leaf Stage [pinot]

2025-02-06 Thread via GitHub
ankitsultana merged PR #15000: URL: https://github.com/apache/pinot/pull/15000 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] refactor get table from tenant, so that we also get tables that have tag overwrite [pinot]

2025-02-06 Thread via GitHub
codecov-commenter commented on PR #15009: URL: https://github.com/apache/pinot/pull/15009#issuecomment-2641862105 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15009?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Added support to pause and resume ingestion based on resource utilization [pinot]

2025-02-06 Thread via GitHub
codecov-commenter commented on PR #15008: URL: https://github.com/apache/pinot/pull/15008#issuecomment-2641837909 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15008?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] refactor get table from tenant, so that we also get tables that have tag overwrite [pinot]

2025-02-06 Thread via GitHub
jasperjiaguo opened a new pull request, #15009: URL: https://github.com/apache/pinot/pull/15009 refactor get table from tenant, so that we also get tables that have tag overwrite -- This is an automated message from the Apache Git Service. To respond to the message, please log

[PR] Added support to pause and resume ingestion based on resource utilization [pinot]

2025-02-06 Thread via GitHub
rajagopr opened a new pull request, #15008: URL: https://github.com/apache/pinot/pull/15008 ## Description Added support to pause and resume ingestion for OFFLINE and REALTIME tables when disk utilization exceeds a configured threshold. ## Key Components Introduced `Resourc

Re: [I] [Time series Gap-fill V2] Server side gap-filling of time series data [pinot]

2025-02-06 Thread via GitHub
chenboat commented on issue #9492: URL: https://github.com/apache/pinot/issues/9492#issuecomment-2641219382 @lakshmanan-v Any update on this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] support to enable Map encoded custom forward index [pinot]

2025-02-06 Thread via GitHub
codecov-commenter commented on PR #15006: URL: https://github.com/apache/pinot/pull/15006#issuecomment-2641189491 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15006?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[I] helm chart metrics with multiple replicas [pinot]

2025-02-06 Thread via GitHub
guzzijones opened a new issue, #15007: URL: https://github.com/apache/pinot/issues/15007 When setting multiple replicas for the controller, broker, and server the Prometheus metrics are in conflict. The metric names do not have a label for the pod they came from or other type of unique ide

[PR] support to enable Map encoded custom forward index [pinot]

2025-02-06 Thread via GitHub
raghavyadav01 opened a new pull request, #15006: URL: https://github.com/apache/pinot/pull/15006 PR add support for enabling Custom MAP encoded forwarded index creator and reader. Summary : - immutable Segment Index Build Path > Collect MapColumnPreIndexStatsCollector during g

Re: [I] [Proposal] Query Resource Isolation in Pinot [pinot]

2025-02-06 Thread via GitHub
vvivekiyer commented on issue #14551: URL: https://github.com/apache/pinot/issues/14551#issuecomment-2640589903 We are starting with the execution of this effort. Gentle ping here to check if there are any suggestions/feedback you'd like us to incorporate. cc: @Jackie-Jiang @ankitsultana @m

Re: [PR] [WIP] add schema validation mechanism for table config [pinot]

2025-02-06 Thread via GitHub
galalen closed pull request #6485: [WIP] add schema validation mechanism for table config URL: https://github.com/apache/pinot/pull/6485 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

(pinot) branch master updated: Introducing MSE result holder config to minimize rehashing for high cardinality group by (#14981)

2025-02-06 Thread ankitsultana
This is an automated email from the ASF dual-hosted git repository. ankitsultana pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 465c811675 Introducing MSE result holder co

Re: [PR] Rewrite FailureDetector interface and implementations to also work with the multi-stage engine [pinot]

2025-02-06 Thread via GitHub
yashmayya commented on code in PR #15005: URL: https://github.com/apache/pinot/pull/15005#discussion_r1945118622 ## pinot-common/src/main/java/org/apache/pinot/common/failuredetector/ConnectionFailureDetector.java: ## @@ -16,29 +16,18 @@ * specific language governing permissio

Re: [PR] Introducing MSE result holder config to minimize rehashing for high cardinality group by [pinot]

2025-02-06 Thread via GitHub
ankitsultana merged PR #14981: URL: https://github.com/apache/pinot/pull/14981 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Rewrite FailureDetector interface and implementations to also work with the multi-stage engine [pinot]

2025-02-06 Thread via GitHub
codecov-commenter commented on PR #15005: URL: https://github.com/apache/pinot/pull/15005#issuecomment-2640093683 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15005?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Improve exceptions broker [pinot]

2025-02-06 Thread via GitHub
gortiz commented on PR #14994: URL: https://github.com/apache/pinot/pull/14994#issuecomment-2640098151 I'm convering the PR into a draft given I've found some stack traces leaking in the tests. My plan is to apply some extra changes. If it ends up being too difficult to review I'll try to s

Re: [PR] Improve exceptions broker [pinot]

2025-02-06 Thread via GitHub
gortiz commented on code in PR #14994: URL: https://github.com/apache/pinot/pull/14994#discussion_r1944727930 ## pinot-spi/src/main/java/org/apache/pinot/spi/exception/QException.java: ## @@ -0,0 +1,108 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * o

Re: [PR] feat(cleanup): Implement pauseless table segment cleanup logic [pinot]

2025-02-06 Thread via GitHub
codecov-commenter commented on PR #15002: URL: https://github.com/apache/pinot/pull/15002#issuecomment-2639761940 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15002?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Pauseless Consumption #3: Disaster Recovery with Reingestion [pinot]

2025-02-06 Thread via GitHub
KKcorps commented on code in PR #14920: URL: https://github.com/apache/pinot/pull/14920#discussion_r1944609599 ## pinot-common/src/main/java/org/apache/pinot/common/metrics/ControllerGauge.java: ## @@ -180,7 +180,10 @@ public enum ControllerGauge implements AbstractMetrics.Gaug

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.30.14 created (now a57d169bdd)

2025-02-06 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.30.14 in repository https://gitbox.apache.org/repos/asf/pinot.git at a57d169bdd Bump software.amazon.awssdk:bom from 2.30.13 to 2.30.14 N

[PR] Bump software.amazon.awssdk:bom from 2.30.13 to 2.30.14 [pinot]

2025-02-06 Thread via GitHub
dependabot[bot] opened a new pull request, #15004: URL: https://github.com/apache/pinot/pull/15004 Bumps software.amazon.awssdk:bom from 2.30.13 to 2.30.14. Most Recent Ignore Conditions Applied to This Pull Request | Dependency Name | Ignore Conditions | | --- | --- |

[I] Do we have a timeline for when CLP querying will be production ready? [pinot]

2025-02-06 Thread via GitHub
leongshengmin opened a new issue, #15003: URL: https://github.com/apache/pinot/issues/15003 We're interested in using [CLP](https://docs.pinot.apache.org/basics/data-import/clp#searching-and-decoding-clp-encoded-fields) for storing unstructured logs but note that this is still an experiment