[PR] Catch-all Regex for JXM -> Prom Exporter [pinot]

2023-11-29 Thread via GitHub
suddendust opened a new pull request, #12073: URL: https://github.com/apache/pinot/pull/12073 Recently, we detected that the catch-all label was incorrect - It failed to capture a metric for which a specific regex had not been added. It also does not capture metrics that contain the kafka t

Re: [PR] Add support for retention on deleted keys of upsert tables [pinot]

2023-11-29 Thread via GitHub
Jackie-Jiang commented on code in PR #12037: URL: https://github.com/apache/pinot/pull/12037#discussion_r1410118166 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java: ## @@ -323,11 +347,13 @@ static class

Re: [PR] Add support for retention on deleted keys of upsert tables [pinot]

2023-11-29 Thread via GitHub
Jackie-Jiang commented on code in PR #12037: URL: https://github.com/apache/pinot/pull/12037#discussion_r1410119462 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java: ## @@ -323,11 +347,13 @@ static class

Re: [PR] Add support for retention on deleted keys of upsert tables [pinot]

2023-11-29 Thread via GitHub
Jackie-Jiang commented on code in PR #12037: URL: https://github.com/apache/pinot/pull/12037#discussion_r1410118166 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java: ## @@ -323,11 +347,13 @@ static class

Re: [PR] Add support for murmur3 as a partition function [pinot]

2023-11-29 Thread via GitHub
walterddr commented on code in PR #12049: URL: https://github.com/apache/pinot/pull/12049#discussion_r1410057271 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/partition/Murmur3PartitionFunction.java: ## @@ -0,0 +1,69 @@ +/** + * Licensed to the Apache Software

Re: [PR] Add support for murmur3 as a partition function [pinot]

2023-11-29 Thread via GitHub
walterddr commented on code in PR #12049: URL: https://github.com/apache/pinot/pull/12049#discussion_r1410057271 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/partition/Murmur3PartitionFunction.java: ## @@ -0,0 +1,69 @@ +/** + * Licensed to the Apache Software

Re: [PR] Add support for murmur3 as a partition function [pinot]

2023-11-29 Thread via GitHub
walterddr commented on code in PR #12049: URL: https://github.com/apache/pinot/pull/12049#discussion_r1410055786 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/partition/Murmur3PartitionFunction.java: ## @@ -0,0 +1,69 @@ +/** + * Licensed to the Apache Software

Re: [PR] Add support for murmur3 as a partition function [pinot]

2023-11-29 Thread via GitHub
walterddr commented on code in PR #12049: URL: https://github.com/apache/pinot/pull/12049#discussion_r1410054731 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/partition/Murmur3PartitionFunction.java: ## @@ -0,0 +1,69 @@ +/** + * Licensed to the Apache Software

Re: [PR] Add support for murmur3 as a partition function [pinot]

2023-11-29 Thread via GitHub
walterddr commented on code in PR #12049: URL: https://github.com/apache/pinot/pull/12049#discussion_r1410054731 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/partition/Murmur3PartitionFunction.java: ## @@ -0,0 +1,69 @@ +/** + * Licensed to the Apache Software

Re: [PR] [multistage] don't prune project after agg during relBuilder [pinot]

2023-11-29 Thread via GitHub
walterddr commented on code in PR #12058: URL: https://github.com/apache/pinot/pull/12058#discussion_r1410047421 ## pinot-query-planner/src/main/java/org/apache/pinot/query/QueryEnvironment.java: ## @@ -122,7 +122,8 @@ public QueryEnvironment(TypeFactory typeFactory, CalciteSch

Re: [PR] Maintain pool selection for the minimizeDataMovement instance partition assignment strategy [pinot]

2023-11-29 Thread via GitHub
Jackie-Jiang commented on code in PR #11953: URL: https://github.com/apache/pinot/pull/11953#discussion_r1410042951 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/instance/InstanceTagPoolSelector.java: ## @@ -109,11 +121,46 @@ public Map> se

Re: [PR] Introduce low disk mode to table rebalance [pinot]

2023-11-29 Thread via GitHub
codecov-commenter commented on PR #12072: URL: https://github.com/apache/pinot/pull/12072#issuecomment-1832925957 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12072?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[PR] Introduce low disk mode to table rebalance [pinot]

2023-11-29 Thread via GitHub
Jackie-Jiang opened a new pull request, #12072: URL: https://github.com/apache/pinot/pull/12072 Address #11895 Introduce a new boolean config `lowDiskMode` (by default false) to rebalance config. When enabled, it will offload segments first, then add segments after offload is done.

Re: [PR] Extend strictReplicaGroup policy to pick a replica group even if there are unavaliable segments for more availablity [pinot]

2023-11-29 Thread via GitHub
klsince commented on code in PR #11847: URL: https://github.com/apache/pinot/pull/11847#discussion_r1367558654 ## pinot-broker/src/main/java/org/apache/pinot/broker/routing/instanceselector/FallbackReplicaGroupInstanceSelector.java: ## @@ -0,0 +1,428 @@ +/** + * Licensed to the

Re: [PR] Consolidate two for loops when poll data Block from multi stream consumers [pinot]

2023-11-29 Thread via GitHub
codecov-commenter commented on PR #12071: URL: https://github.com/apache/pinot/pull/12071#issuecomment-1832833341 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12071?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[PR] Consolidate two for loops when poll data Block from multi stream consumers [pinot]

2023-11-29 Thread via GitHub
wirybeaver opened a new pull request, #12071: URL: https://github.com/apache/pinot/pull/12071 Leverage the modular arithmetic to avoid code duplication. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

(pinot) branch master updated: [multistage][refactor] clean up planner (#12070)

2023-11-29 Thread rongr
This is an automated email from the ASF dual-hosted git repository. rongr pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new b9ed378355 [multistage][refactor] clean up planner

Re: [PR] [multistage][refactor] clean up planner [pinot]

2023-11-29 Thread via GitHub
walterddr merged PR #12070: URL: https://github.com/apache/pinot/pull/12070 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

Re: [PR] [multistage][refactor] clean up planner [pinot]

2023-11-29 Thread via GitHub
codecov-commenter commented on PR #12070: URL: https://github.com/apache/pinot/pull/12070#issuecomment-1832613717 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12070?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[PR] [multistage][refactor] clean up planner [pinot]

2023-11-29 Thread via GitHub
walterddr opened a new pull request, #12070: URL: https://github.com/apache/pinot/pull/12070 - colocated visitor is no longer needed, we derived using partition now - plan fragment metadata is not needed, we use dispatchablePlanFragment & dispatchablePlanMetadata - move several physic

Re: [PR] Allow tenant level config override [pinot]

2023-11-29 Thread via GitHub
klsince commented on code in PR #12039: URL: https://github.com/apache/pinot/pull/12039#discussion_r1409728866 ## pinot-common/src/main/java/org/apache/pinot/common/utils/ServiceStartableUtils.java: ## @@ -36,56 +40,91 @@ private ServiceStartableUtils() { private static fin

Re: [I] [feature] Add support for oauth2 to Pulsar ingest plugin [pinot]

2023-11-29 Thread via GitHub
JeffBolle commented on issue #12067: URL: https://github.com/apache/pinot/issues/12067#issuecomment-1832385241 Documentation PR here: https://github.com/pinot-contrib/pinot-docs/pull/263 -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] Correct path of default log4j configures [pinot]

2023-11-29 Thread via GitHub
goodseeyou commented on PR #12069: URL: https://github.com/apache/pinot/pull/12069#issuecomment-1832180673 Any instruction to add labels? If it's necessary. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[PR] Correct path of default log4j configures [pinot]

2023-11-29 Thread via GitHub
goodseeyou opened a new pull request, #12069: URL: https://github.com/apache/pinot/pull/12069 This PR is to correct default configure path for log4j in helm chart. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] [feature] add support for StreamNative OAuth2 authentication for pulsar. [pinot]

2023-11-29 Thread via GitHub
codecov-commenter commented on PR #12068: URL: https://github.com/apache/pinot/pull/12068#issuecomment-1832145059 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12068?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

(pinot) branch master updated: [multistage] don't prune project after agg during relBuilder (#12058)

2023-11-29 Thread rongr
This is an automated email from the ASF dual-hosted git repository. rongr pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new c57117a750 [multistage] don't prune project after

Re: [I] [multistage][bug] multi semi-join hint not applied in weird conditions [pinot]

2023-11-29 Thread via GitHub
walterddr closed issue #12013: [multistage][bug] multi semi-join hint not applied in weird conditions URL: https://github.com/apache/pinot/issues/12013 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] [multistage] don't prune project after agg during relBuilder [pinot]

2023-11-29 Thread via GitHub
walterddr merged PR #12058: URL: https://github.com/apache/pinot/pull/12058 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

Re: [PR] [multistage][cleanup] remove some unused rules [pinot]

2023-11-29 Thread via GitHub
walterddr merged PR #12052: URL: https://github.com/apache/pinot/pull/12052 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

(pinot) branch master updated: [multistage][cleanup] remove some unused rules (#12052)

2023-11-29 Thread rongr
This is an automated email from the ASF dual-hosted git repository. rongr pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 37270f749c [multistage][cleanup] remove some unuse

Re: [I] [feature] Add support for oauth2 to Pulsar ingest plugin [pinot]

2023-11-29 Thread via GitHub
JeffBolle commented on issue #12067: URL: https://github.com/apache/pinot/issues/12067#issuecomment-1832027306 PR here: https://github.com/apache/pinot/pull/12068 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[PR] [feature] add support for StreamNative OAuth2 authentication for pulsar. [pinot]

2023-11-29 Thread via GitHub
JeffBolle opened a new pull request, #12068: URL: https://github.com/apache/pinot/pull/12068 Adds support for StreamNative OAuth2 authentication, allowing Pinot to connect to StreamNative hosted Pulsar cluster. fixes #12067 -- This is an automated message from the Apache Git Servic

Re: [PR] Bump net.alchim31.maven:scala-maven-plugin from 3.2.2 to 4.8.1 [pinot]

2023-11-29 Thread via GitHub
codecov-commenter commented on PR #12066: URL: https://github.com/apache/pinot/pull/12066#issuecomment-1831817325 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12066?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] Bump com.adobe.testing:s3mock-testcontainers from 2.12.2 to 3.2.0 [pinot]

2023-11-29 Thread via GitHub
codecov-commenter commented on PR #12065: URL: https://github.com/apache/pinot/pull/12065#issuecomment-1831813130 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12065?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

(pinot) branch dependabot/maven/com.adobe.testing-s3mock-testcontainers-3.2.0 created (now 76f7e19f0b)

2023-11-29 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.adobe.testing-s3mock-testcontainers-3.2.0 in repository https://gitbox.apache.org/repos/asf/pinot.git at 76f7e19f0b Bump com.adobe.testing:s3mock-testcontainers fr

[PR] Bump net.alchim31.maven:scala-maven-plugin from 3.2.2 to 4.8.1 [pinot]

2023-11-29 Thread via GitHub
dependabot[bot] opened a new pull request, #12066: URL: https://github.com/apache/pinot/pull/12066 Bumps net.alchim31.maven:scala-maven-plugin from 3.2.2 to 4.8.1. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name

(pinot) branch dependabot/maven/net.alchim31.maven-scala-maven-plugin-4.8.1 created (now 750b2658f2)

2023-11-29 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/net.alchim31.maven-scala-maven-plugin-4.8.1 in repository https://gitbox.apache.org/repos/asf/pinot.git at 750b2658f2 Bump net.alchim31.maven:scala-maven-plugin from 3

[PR] Bump com.adobe.testing:s3mock-testcontainers from 2.12.2 to 3.2.0 [pinot]

2023-11-29 Thread via GitHub
dependabot[bot] opened a new pull request, #12065: URL: https://github.com/apache/pinot/pull/12065 Bumps com.adobe.testing:s3mock-testcontainers from 2.12.2 to 3.2.0. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-n

[I] Lack of JSON_EXISTS_PATH [pinot]

2023-11-29 Thread via GitHub
INNOCENT-BOY opened a new issue, #12064: URL: https://github.com/apache/pinot/issues/12064 > Json Index have a limitation: > The key (left-hand side) of the filter expression must be the leaf level of the JSON object, for example, "$.addresses[*]"='main st' won't work. In some sce

Re: [PR] Theta Sketch Aggregation Enhancements [pinot]

2023-11-29 Thread via GitHub
davecromberge commented on code in PR #12042: URL: https://github.com/apache/pinot/pull/12042#discussion_r1408415739 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/CustomSerDeUtils.java: ## @@ -243,9 +243,14 @@ public TDigest deserialize(ByteBuffer byt

Re: [PR] Theta Sketch Aggregation Enhancements [pinot]

2023-11-29 Thread via GitHub
davecromberge commented on code in PR #12042: URL: https://github.com/apache/pinot/pull/12042#discussion_r1408414659 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/customobject/ThetaSketchAccumulator.java: ## @@ -0,0 +1,133 @@ +/** + * Licensed to the Apache