Re: [PR] Detect expired messages in Kafka. Log and set a guage. [pinot]

2023-12-21 Thread via GitHub
vrajat commented on PR #12157: URL: https://github.com/apache/pinot/pull/12157#issuecomment-1867294975 A simple script to show message expiration in kafka Assume all the components are installed - Kafka - Kafka CLIs - Faker ``` # Create a topic with retention of 1m

Re: [I] Http Response Headers for Error Codes [pinot]

2023-12-21 Thread via GitHub
ankitsultana commented on issue #12190: URL: https://github.com/apache/pinot/issues/12190#issuecomment-1867294997 I think we can set it to -1, but open to suggestions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [I] SegmentTarPushJobRunner NullPointerException import csv [pinot]

2023-12-21 Thread via GitHub
Jackie-Jiang commented on issue #11940: URL: https://github.com/apache/pinot/issues/11940#issuecomment-1867292405 @xiangfu0 Can you help take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] Http Response Headers for Error Codes [pinot]

2023-12-21 Thread via GitHub
abhioncbr commented on issue #12190: URL: https://github.com/apache/pinot/issues/12190#issuecomment-1867161617 Thanks for explanation. Question, in-case of successful response, header value would be 200(I am not sure about Pinot code incase of successful result)? -- This is an automat

Re: [PR] fix: Pulsar OAuth2 Authentication Factory [pinot]

2023-12-21 Thread via GitHub
KKcorps commented on code in PR #12195: URL: https://github.com/apache/pinot/pull/12195#discussion_r1434655645 ## pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarPartitionLevelConnectionHandler.java: ## @@ -73,10 +73,10

Re: [PR] fix: Pulsar OAuth2 Authentication Factory [pinot]

2023-12-21 Thread via GitHub
KKcorps commented on code in PR #12195: URL: https://github.com/apache/pinot/pull/12195#discussion_r1434655332 ## pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/test/java/org/apache/pinot/plugin/stream/pulsar/PulsarConfigTest.java: ## @@ -147,7 +152,6 @@ public void testP

Re: [PR] fix: Pulsar OAuth2 Authentication Factory [pinot]

2023-12-21 Thread via GitHub
KKcorps commented on code in PR #12195: URL: https://github.com/apache/pinot/pull/12195#discussion_r1434655165 ## pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/test/java/org/apache/pinot/plugin/stream/pulsar/PulsarConfigTest.java: ## @@ -31,6 +31,8 @@ import org.testng.

(pinot) branch master updated: Proper computation of realtime "segment.flush.threshold.size" in case of force-commit (#12188)

2023-12-21 Thread sajjad
This is an automated email from the ASF dual-hosted git repository. sajjad pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 9cfce82385 Proper computation of realtime "segme

Re: [PR] Proper computation of realtime "segment.flush.threshold.size" in case of force-commit [pinot]

2023-12-21 Thread via GitHub
sajjad-moradi merged PR #12188: URL: https://github.com/apache/pinot/pull/12188 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pino

Re: [PR] Shared aggregations in StarTree [pinot]

2023-12-21 Thread via GitHub
Jackie-Jiang commented on code in PR #12164: URL: https://github.com/apache/pinot/pull/12164#discussion_r1434553727 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/store/StarTreeIndexReader.java: ## @@ -121,7 +124,9 @@ private void mapBufferEntries(in

(pinot) branch master updated (9b6d0b7cb8 -> 50153796c6)

2023-12-21 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 9b6d0b7cb8 Bump org.freemarker:freemarker from 2.3.30 to 2.3.32 (#12192) add 50153796c6 Bump com.azure:azure-core

(pinot) branch dependabot/maven/com.azure-azure-core-1.45.1 deleted (was 8aa7bd5f6a)

2023-12-21 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/com.azure-azure-core-1.45.1 in repository https://gitbox.apache.org/repos/asf/pinot.git was 8aa7bd5f6a Bump com.azure:azure-core from 1.37.0 to 1.45.1 The revisions that we

Re: [PR] Bump com.azure:azure-core from 1.37.0 to 1.45.1 [pinot]

2023-12-21 Thread via GitHub
Jackie-Jiang merged PR #12193: URL: https://github.com/apache/pinot/pull/12193 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch dependabot/maven/org.apache.spark-spark-core_2.11-2.4.8 deleted (was bd65ad6dd9)

2023-12-21 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.apache.spark-spark-core_2.11-2.4.8 in repository https://gitbox.apache.org/repos/asf/pinot.git was bd65ad6dd9 Bump org.apache.spark:spark-core_2.11 from 2.4.6 to 2.

Re: [PR] Bump org.apache.spark:spark-core_2.11 from 2.4.6 to 2.4.8 [pinot]

2023-12-21 Thread via GitHub
dependabot[bot] commented on PR #12191: URL: https://github.com/apache/pinot/pull/12191#issuecomment-1866988386 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] Bump org.apache.spark:spark-core_2.11 from 2.4.6 to 2.4.8 [pinot]

2023-12-21 Thread via GitHub
Jackie-Jiang closed pull request #12191: Bump org.apache.spark:spark-core_2.11 from 2.4.6 to 2.4.8 URL: https://github.com/apache/pinot/pull/12191 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

(pinot) branch dependabot/maven/org.freemarker-freemarker-2.3.32 deleted (was 3a52fbd3c6)

2023-12-21 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/org.freemarker-freemarker-2.3.32 in repository https://gitbox.apache.org/repos/asf/pinot.git was 3a52fbd3c6 Bump org.freemarker:freemarker from 2.3.30 to 2.3.32 The revisi

(pinot) branch master updated: Bump org.freemarker:freemarker from 2.3.30 to 2.3.32 (#12192)

2023-12-21 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 9b6d0b7cb8 Bump org.freemarker:freemarker from 2.

Re: [PR] Bump org.freemarker:freemarker from 2.3.30 to 2.3.32 [pinot]

2023-12-21 Thread via GitHub
Jackie-Jiang merged PR #12192: URL: https://github.com/apache/pinot/pull/12192 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [I] Http Response Headers for Error Codes [pinot]

2023-12-21 Thread via GitHub
ankitsultana commented on issue #12190: URL: https://github.com/apache/pinot/issues/12190#issuecomment-1866987690 I think I can explain better with an example. The existing API returns responses like this: ``` < HTTP/1.1 200 OK < Content-Type: application/json < Content

Re: [PR] fix: Pulsar OAuth2 Authentication Factory [pinot]

2023-12-21 Thread via GitHub
codecov-commenter commented on PR #12195: URL: https://github.com/apache/pinot/pull/12195#issuecomment-1866934613 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12195?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[I] Pulsar OAuth configuration is not used [pinot]

2023-12-21 Thread via GitHub
JeffBolle opened a new issue, #12196: URL: https://github.com/apache/pinot/issues/12196 When providing the required configuration for authenticating with OAuth to a pulsar cluster, the necessary authentication factory is not created. This is a result of a typo in the recently merged PR: h

Re: [I] Http Response Headers for Error Codes [pinot]

2023-12-21 Thread via GitHub
abhioncbr commented on issue #12190: URL: https://github.com/apache/pinot/issues/12190#issuecomment-1866899055 I think error code 422 make sense here. Thoughts? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[PR] fix: Pulsar OAuth2 Authentication Factory [pinot]

2023-12-21 Thread via GitHub
JeffBolle opened a new pull request, #12195: URL: https://github.com/apache/pinot/pull/12195 Fixes the creation of the configured pulsar authentication factory when oauth configuration is provided. -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [I] SegmentTarPushJobRunner NullPointerException import csv [pinot]

2023-12-21 Thread via GitHub
pbombo commented on issue #11940: URL: https://github.com/apache/pinot/issues/11940#issuecomment-1866830773 Also > @hugosjoberg Your data dir is named `data`, but your job spec is looking for `rawdata`: `Caused by: java.nio.file.NoSuchFileException: /tmp/pinot-quick-start/rawdata`

(pinot) branch master updated (24f8433d5f -> 9d939a007e)

2023-12-21 Thread sajjad
This is an automated email from the ASF dual-hosted git repository. sajjad pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 24f8433d5f Bump com.google.auto.service:auto-service from 1.0.1 to 1.1.1 (#12183) add 9d939a007e Add partition le

Re: [PR] Add partition level Force Commit [pinot]

2023-12-21 Thread via GitHub
sajjad-moradi merged PR #12088: URL: https://github.com/apache/pinot/pull/12088 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pino

[PR] Enhance SSL Context Handling in Client Transports for Dynamic Certificate Rotation [pinot]

2023-12-21 Thread via GitHub
kay-owolabi opened a new pull request, #12194: URL: https://github.com/apache/pinot/pull/12194 ## Summary This pull request introduces improvements to the SSL context handling in `JsonAsyncHttpPinotClientTransport` and `PinotControllerTransport` within Apache Pinot. The primary goal is t

Re: [PR] Proper computation of realtime "segment.flush.threshold.size" in case of force-commit [pinot]

2023-12-21 Thread via GitHub
sajjad-moradi commented on code in PR #12188: URL: https://github.com/apache/pinot/pull/12188#discussion_r1434369527 ## pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/realtime/segment/SegmentFlushThresholdComputerTest.java: ## @@ -105,6 +106,28 @@ public v

Re: [PR] Shared aggregations in StarTree [pinot]

2023-12-21 Thread via GitHub
davecromberge commented on code in PR #12164: URL: https://github.com/apache/pinot/pull/12164#discussion_r1434289093 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/store/StarTreeIndexReader.java: ## @@ -121,7 +124,9 @@ private void mapBufferEntries(i

Re: [PR] Shared aggregations in StarTree [pinot]

2023-12-21 Thread via GitHub
davecromberge commented on code in PR #12164: URL: https://github.com/apache/pinot/pull/12164#discussion_r1434289568 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/BaseSingleTreeBuilder.java: ## @@ -133,7 +133,9 @@ static class Record {

Re: [PR] Shared aggregations in StarTree [pinot]

2023-12-21 Thread via GitHub
davecromberge commented on code in PR #12164: URL: https://github.com/apache/pinot/pull/12164#discussion_r1434289093 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/store/StarTreeIndexReader.java: ## @@ -121,7 +124,9 @@ private void mapBufferEntries(i

Re: [PR] Bump org.apache.spark:spark-core_2.11 from 2.4.6 to 2.4.8 [pinot]

2023-12-21 Thread via GitHub
codecov-commenter commented on PR #12191: URL: https://github.com/apache/pinot/pull/12191#issuecomment-1866165840 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12191?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] Bump com.azure:azure-core from 1.37.0 to 1.45.1 [pinot]

2023-12-21 Thread via GitHub
codecov-commenter commented on PR #12193: URL: https://github.com/apache/pinot/pull/12193#issuecomment-1866165887 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12193?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

(pinot) branch dependabot/maven/com.azure-azure-core-1.45.1 created (now 8aa7bd5f6a)

2023-12-21 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.azure-azure-core-1.45.1 in repository https://gitbox.apache.org/repos/asf/pinot.git at 8aa7bd5f6a Bump com.azure:azure-core from 1.37.0 to 1.45.1 No new revisions

[PR] Bump com.azure:azure-core from 1.37.0 to 1.45.1 [pinot]

2023-12-21 Thread via GitHub
dependabot[bot] opened a new pull request, #12193: URL: https://github.com/apache/pinot/pull/12193 Bumps [com.azure:azure-core](https://github.com/Azure/azure-sdk-for-java) from 1.37.0 to 1.45.1. Commits https://github.com/Azure/azure-sdk-for-java/commit/e8138571cef1fe885ed67d8

(pinot) branch dependabot/maven/org.freemarker-freemarker-2.3.32 created (now 3a52fbd3c6)

2023-12-21 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.freemarker-freemarker-2.3.32 in repository https://gitbox.apache.org/repos/asf/pinot.git at 3a52fbd3c6 Bump org.freemarker:freemarker from 2.3.30 to 2.3.32 No new

[PR] Bump org.apache.spark:spark-core_2.11 from 2.4.6 to 2.4.8 [pinot]

2023-12-21 Thread via GitHub
dependabot[bot] opened a new pull request, #12191: URL: https://github.com/apache/pinot/pull/12191 Bumps org.apache.spark:spark-core_2.11 from 2.4.6 to 2.4.8. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=org.

(pinot) branch dependabot/maven/org.apache.spark-spark-core_2.11-2.4.8 created (now bd65ad6dd9)

2023-12-21 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.apache.spark-spark-core_2.11-2.4.8 in repository https://gitbox.apache.org/repos/asf/pinot.git at bd65ad6dd9 Bump org.apache.spark:spark-core_2.11 from 2.4.6 to 2.

[PR] Bump org.freemarker:freemarker from 2.3.30 to 2.3.32 [pinot]

2023-12-21 Thread via GitHub
dependabot[bot] opened a new pull request, #12192: URL: https://github.com/apache/pinot/pull/12192 Bumps org.freemarker:freemarker from 2.3.30 to 2.3.32. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=org.freem

Re: [PR] Shared aggregations in StarTree [pinot]

2023-12-21 Thread via GitHub
davecromberge commented on code in PR #12164: URL: https://github.com/apache/pinot/pull/12164#discussion_r1433848301 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/aggregator/DistinctCountHLLPlusValueAggregator.java: ## @@ -31,6 +31,7 @@ public class Dist