Re: [PR] Add some additional metrics for Minion tasks [pinot]

2024-03-26 Thread via GitHub
vvivekiyer commented on code in PR #12710: URL: https://github.com/apache/pinot/pull/12710#discussion_r1540548233 ## pinot-common/src/main/java/org/apache/pinot/common/metrics/MinionMeter.java: ## @@ -31,7 +31,13 @@ public enum MinionMeter implements AbstractMetrics.Meter { N

Re: [PR] Add minionInstanceTag config to minion-tasks for resource isolation [pinot]

2024-03-26 Thread via GitHub
vvivekiyer commented on code in PR #12459: URL: https://github.com/apache/pinot/pull/12459#discussion_r1540440738 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTaskRestletResource.java: ## @@ -615,12 +615,15 @@ public Map scheduleTasks(@ApiPara

Re: [PR] Add minionInstanceTag config to minion-tasks for resource isolation [pinot]

2024-03-26 Thread via GitHub
vvivekiyer commented on code in PR #12459: URL: https://github.com/apache/pinot/pull/12459#discussion_r1540440738 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTaskRestletResource.java: ## @@ -615,12 +615,15 @@ public Map scheduleTasks(@ApiPara

[I] Broker cannot build resource map if there are any server with invalid configs [pinot]

2024-03-26 Thread via GitHub
lnbest0707-uber opened a new issue, #12732: URL: https://github.com/apache/pinot/issues/12732 After servers with invalid configuration joined the cluster, e.g. like below `{ "id": "", "simpleFields": { "HELIX_HOST": "", "HELIX_PORT": "", }, "mapFields": {}

Re: [PR] do not allow pruning exception to fail queries [pinot]

2024-03-26 Thread via GitHub
codecov-commenter commented on PR #12731: URL: https://github.com/apache/pinot/pull/12731#issuecomment-2021605320 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12731?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [I] TimeSegmentPruner exception based on query and schema datetime column format [pinot]

2024-03-26 Thread via GitHub
jadami10 commented on issue #12721: URL: https://github.com/apache/pinot/issues/12721#issuecomment-2021559505 also to provide more concrete feedback for the schema docs, https://docs.pinot.apache.org/configuration-reference/schema, there are ~1700 words on that page and all of 1 schema with

[PR] do not allow pruning exception to fail queries [pinot]

2024-03-26 Thread via GitHub
jadami10 opened a new pull request, #12731: URL: https://github.com/apache/pinot/pull/12731 This is a `bugfix` addressing a portion of #12721. Failing to Prune should not fail the query itself. Instead we log the table, the pruner, and the stack trace leading to the error. -- This is an

Re: [I] TimeSegmentPruner exception based on query and schema datetime column format [pinot]

2024-03-26 Thread via GitHub
jadami10 commented on issue #12721: URL: https://github.com/apache/pinot/issues/12721#issuecomment-2021537161 between the new/old formats, https://docs.pinot.apache.org/configuration-reference/schema#new-datetime-formats, `1:MILLISECONDS:TIMESTAMP` isn't mentioned anywhere. Though if I read

(pinot) branch master updated: Remove commons-logging exclusion (#12730)

2024-03-26 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 3697552290 Remove commons-logging exclusion (#127

Re: [PR] Remove commons-logging exclusion [pinot]

2024-03-26 Thread via GitHub
Jackie-Jiang merged PR #12730: URL: https://github.com/apache/pinot/pull/12730 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Add some additional metrics for Minion tasks [pinot]

2024-03-26 Thread via GitHub
sajjad-moradi commented on code in PR #12710: URL: https://github.com/apache/pinot/pull/12710#discussion_r1539860670 ## pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/BaseTaskExecutor.java: ## @@ -68,4 +71,10 @@ pro

Re: [PR] Remove commons-logging exclusion [pinot]

2024-03-26 Thread via GitHub
codecov-commenter commented on PR #12730: URL: https://github.com/apache/pinot/pull/12730#issuecomment-2021347783 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12730?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [I] Minion task for hybrid table erroring about a retry attempt exhaustion [pinot]

2024-03-26 Thread via GitHub
estebanz01 commented on issue #12547: URL: https://github.com/apache/pinot/issues/12547#issuecomment-2021333062 OK, after lots of trial and error, this is what I did to have a working hybrid table with minion tasks and S3 deep storage: Controller helm config ```yaml cont

Re: [I] Minion task for hybrid table erroring about a retry attempt exhaustion [pinot]

2024-03-26 Thread via GitHub
estebanz01 closed issue #12547: Minion task for hybrid table erroring about a retry attempt exhaustion URL: https://github.com/apache/pinot/issues/12547 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] Remove commons-logging exclusion [pinot]

2024-03-26 Thread via GitHub
xiangfu0 opened a new pull request, #12730: URL: https://github.com/apache/pinot/pull/12730 1. All the `commons-logging` are excluded from the root pom dependencies. 2. Add `enforce-banned-dependencies` execution in `maven-enforcer-plugin` to ensure no such library is included in the proj

Re: [I] Ingest from Pulsar: relying on/ compatible to EOL version of Pulsar [pinot]

2024-03-26 Thread via GitHub
hpvd commented on issue #12713: URL: https://github.com/apache/pinot/issues/12713#issuecomment-2021226848 hmm when looking into their release policy it looks like this: ![2024-03-26_19h46_02](https://github.com/apache/pinot/assets/5681880/064daf94-bd19-4865-84e0-038af9361fa2) https://p

Re: [I] CalciteSQLParser error even when query executes succesfully [pinot]

2024-03-26 Thread via GitHub
SnehaHegde24 commented on issue #11823: URL: https://github.com/apache/pinot/issues/11823#issuecomment-2021191022 Hi @Jackie-Jiang , Any update here ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-26 Thread via GitHub
Jackie-Jiang commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1539859364 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/forward/BaseChunkForwardIndexReader.java: ## @@ -446,9 +446,10 @@ public

Re: [PR] Consistency in API response for live broker [pinot]

2024-03-26 Thread via GitHub
Jackie-Jiang merged PR #12201: URL: https://github.com/apache/pinot/pull/12201 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated (5201d2f8ee -> 2b47523faf)

2024-03-26 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 5201d2f8ee upgrade redis version in superset pinot docker image (#12729) add 2b47523faf Consistency in API respon

Re: [I] Response parity between Table/getLiveBrokers and Table/getLiveBrokersForTable [pinot]

2024-03-26 Thread via GitHub
Jackie-Jiang closed issue #11841: Response parity between Table/getLiveBrokers and Table/getLiveBrokersForTable URL: https://github.com/apache/pinot/issues/11841 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] upgrade redis version in superset pinot docker image [pinot]

2024-03-26 Thread via GitHub
xiangfu0 merged PR #12729: URL: https://github.com/apache/pinot/pull/12729 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

(pinot) branch master updated: upgrade redis version in superset pinot docker image (#12729)

2024-03-26 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 5201d2f8ee upgrade redis version in superset pin

[PR] upgrade redis version in superset pinot docker image [pinot]

2024-03-26 Thread via GitHub
xiangfu0 opened a new pull request, #12729: URL: https://github.com/apache/pinot/pull/12729 New superset requires redis version >4.6.0,<5.0 ``` Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/pkg_resources/__init__.py", line 615, in _build_master

Re: [I] Ingest from Pulsar: relying on/ compatible to EOL version of Pulsar [pinot]

2024-03-26 Thread via GitHub
Jackie-Jiang commented on issue #12713: URL: https://github.com/apache/pinot/issues/12713#issuecomment-2020954157 From what I can tell, Pulsar is actively maintaining 2 major versions, and `3.2.1` and `2.11.4` are released on the same day (03/08/2024). We can evaluate `3.2.*`, but I guess i

(pinot) branch master updated: Bump aws.sdk.version from 2.25.3 to 2.25.17 (#12725)

2024-03-26 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new eceffb9bca Bump aws.sdk.version from 2.25.3 to 2.

(pinot) branch dependabot/maven/aws.sdk.version-2.25.17 deleted (was 9a59c46a15)

2024-03-26 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/aws.sdk.version-2.25.17 in repository https://gitbox.apache.org/repos/asf/pinot.git was 9a59c46a15 Bump aws.sdk.version from 2.25.3 to 2.25.17 The revisions that were on th

Re: [PR] Bump aws.sdk.version from 2.25.3 to 2.25.17 [pinot]

2024-03-26 Thread via GitHub
Jackie-Jiang merged PR #12725: URL: https://github.com/apache/pinot/pull/12725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-26 Thread via GitHub
marregui commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1539288679 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/compression/GzipCompressor.java: ## @@ -0,0 +1,62 @@ +/** + * Licensed to the Apache Software Fo

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-26 Thread via GitHub
marregui commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1539258153 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/compression/GzipCompressor.java: ## @@ -0,0 +1,62 @@ +/** + * Licensed to the Apache Software Fo

Re: [I] Proposal to Harmonize Pinot Dependency Management [pinot]

2024-03-26 Thread via GitHub
gortiz commented on issue #12676: URL: https://github.com/apache/pinot/issues/12676#issuecomment-2020439630 I'm fine with this approach. Maybe I would suggest to: 1. Do not use properties if we expect to only use it once. Otherwise we need to add one extra line per dependency family and i

Re: [I] Ingest from Pulsar: relying on/ compatible to EOL version of Pulsar [pinot]

2024-03-26 Thread via GitHub
hpvd commented on issue #12713: URL: https://github.com/apache/pinot/issues/12713#issuecomment-2020378224 latest version is `3.2.1.` `depend bot` seems to have been canceled for this https://github.com/apache/pinot/pull/12123 -- This is an automated message from the Apache Git Service.

Re: [I] Ingest from Pulsar: relying on/ compatible to EOL version of Pulsar [pinot]

2024-03-26 Thread via GitHub
hpvd commented on issue #12713: URL: https://github.com/apache/pinot/issues/12713#issuecomment-2020345338 Just for the overview and to put the work in the right order, here are some open pull request in the field of pulsar.. https://github.com/apache/pinot/pulls?q=is%3Apr+is%3Aopen+pulsar

Re: [PR] Bugfix. Avoid passing null table name input to translation util [pinot]

2024-03-26 Thread via GitHub
saurabhd336 merged PR #12726: URL: https://github.com/apache/pinot/pull/12726 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.

(pinot) branch master updated: Bugfix. Avoid passing null table name input to translation util (#12726)

2024-03-26 Thread saurabhd336
This is an automated email from the ASF dual-hosted git repository. saurabhd336 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 0c61bb67af Bugfix. Avoid passing null table

(pinot) branch dependabot/maven/hk2.version-3.1.0 created (now 23bd28f3e4)

2024-03-26 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/hk2.version-3.1.0 in repository https://gitbox.apache.org/repos/asf/pinot.git at 23bd28f3e4 Bump hk2.version from 2.6.1 to 3.1.0 No new revisions were added by this up

(pinot) branch dependabot/maven/org.apache.maven.plugins-maven-gpg-plugin-3.2.2 created (now bf65017993)

2024-03-26 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.apache.maven.plugins-maven-gpg-plugin-3.2.2 in repository https://gitbox.apache.org/repos/asf/pinot.git at bf65017993 Bump org.apache.maven.plugins:maven-gpg-plugi

[PR] Bump org.apache.maven.plugins:maven-gpg-plugin from 3.2.0 to 3.2.2 [pinot]

2024-03-26 Thread via GitHub
dependabot[bot] opened a new pull request, #12728: URL: https://github.com/apache/pinot/pull/12728 Bumps [org.apache.maven.plugins:maven-gpg-plugin](https://github.com/apache/maven-gpg-plugin) from 3.2.0 to 3.2.2. Release notes Sourced from https://github.com/apache/maven-gpg-plug

[PR] Bump hk2.version from 2.6.1 to 3.1.0 [pinot]

2024-03-26 Thread via GitHub
dependabot[bot] opened a new pull request, #12727: URL: https://github.com/apache/pinot/pull/12727 Bumps `hk2.version` from 2.6.1 to 3.1.0. Updates `org.glassfish.hk2:hk2-locator` from 2.6.1 to 3.1.0 Release notes Sourced from https://github.com/eclipse-ee4j/glassfish-hk2/release

Re: [PR] Bugfix. Avoid passing null table name input to translation util [pinot]

2024-03-26 Thread via GitHub
codecov-commenter commented on PR #12726: URL: https://github.com/apache/pinot/pull/12726#issuecomment-2020112929 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12726?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Bugfix. Avoid passing null table name input to translation util [pinot]

2024-03-26 Thread via GitHub
saurabhd336 commented on code in PR #12726: URL: https://github.com/apache/pinot/pull/12726#discussion_r1538945992 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTaskRestletResource.java: ## @@ -556,7 +556,7 @@ public List getCronSchedulerJobKey

Re: [PR] Bugfix. Avoid passing null table name input to translation util [pinot]

2024-03-26 Thread via GitHub
saurabhd336 commented on code in PR #12726: URL: https://github.com/apache/pinot/pull/12726#discussion_r1538945992 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTaskRestletResource.java: ## @@ -556,7 +556,7 @@ public List getCronSchedulerJobKey

[PR] Bugfix. Avoid passing null table name input to translation util [pinot]

2024-03-26 Thread via GitHub
shounakmk219 opened a new pull request, #12726: URL: https://github.com/apache/pinot/pull/12726 This PR fixes the bug introduced in #12417 which blocked `/tasks/schedule` API with no table name input -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] Ingest from Pulsar: relying on/ compatible to EOL version of Pulsar [pinot]

2024-03-26 Thread via GitHub
hpvd commented on issue #12713: URL: https://github.com/apache/pinot/issues/12713#issuecomment-2019535411 yep, good to see there was an upgrade with the shiny new 1.10! Looking into Pulsars Release Policy, even for 2.11 active and also security support were already terminated on 11