Re: [PR] Pauseless ingestion without failure scenarios [pinot]

2025-01-06 Thread via GitHub
9aman commented on PR #14741: URL: https://github.com/apache/pinot/pull/14741#issuecomment-2574612264 > Good job! > > Currently pauseless is determined by table config on both controller and server. Will it have problem if user changes table config from blocking to pauseless in the m

Re: [PR] Pauseless ingestion without failure scenarios [pinot]

2025-01-06 Thread via GitHub
9aman commented on code in PR #14741: URL: https://github.com/apache/pinot/pull/14741#discussion_r1905041131 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentZKMetadata.java: ## @@ -44,6 +44,8 @@ public class SegmentZKMetadata implements ZKMetadata {

Re: [PR] Timestamp in MSE [pinot]

2025-01-06 Thread via GitHub
gortiz commented on code in PR #14690: URL: https://github.com/apache/pinot/pull/14690#discussion_r1905018209 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/ExplainIntegrationTestTrait.java: ## @@ -0,0 +1,124 @@ +/** + * Licensed to the Apache Softwa

Re: [PR] Short Circuit And Filter Operator During Index Evaluation [pinot]

2025-01-06 Thread via GitHub
Jackie-Jiang commented on code in PR #14700: URL: https://github.com/apache/pinot/pull/14700#discussion_r1904952965 ## pinot-core/src/main/java/org/apache/pinot/core/common/BlockDocIdSet.java: ## @@ -48,12 +49,28 @@ public interface BlockDocIdSet { */ long getNumEntriesSc

Re: [PR] Calculate size based flush threshold per topic [pinot]

2025-01-06 Thread via GitHub
itschrispeck commented on PR #14765: URL: https://github.com/apache/pinot/pull/14765#issuecomment-2574381377 > Can you elaborate more on why do we want to separate size info for each topic? If the data is ingested into the same table, they should have the same schema, and most of the case s

Re: [PR] Calculate size based flush threshold per topic [pinot]

2025-01-06 Thread via GitHub
codecov-commenter commented on PR #14765: URL: https://github.com/apache/pinot/pull/14765#issuecomment-2574361554 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14765?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Upgrading Pinot version(1.4.0) for next release. [pinot]

2025-01-06 Thread via GitHub
Jackie-Jiang commented on PR #14761: URL: https://github.com/apache/pinot/pull/14761#issuecomment-2574353977 Are you working on preparing the 1.3.0 release? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Add a new rule PinotSeminJoinDistinctProjectRule to apply a distinct to a semi join right side project [pinot]

2025-01-06 Thread via GitHub
Jackie-Jiang commented on PR #14758: URL: https://github.com/apache/pinot/pull/14758#issuecomment-2574352116 Will this rule prevent us from doing: ``` SELECT ... FROM t1 WHERE t1.col IN (SELECT t2.col FROM t2) ``` We don't always want to pay overhead of distinct when we already kn

[PR] Calculate size based flush threshold per topic [pinot]

2025-01-06 Thread via GitHub
itschrispeck opened a new pull request, #14765: URL: https://github.com/apache/pinot/pull/14765 With multi-stream ingestion added, it makes sense to calculate segment flush thresholds based on individual topics, not at a table level. This patch has been running internally, behavior was vali

(pinot) branch master updated: Updated more packages in License-binary (#14757)

2025-01-06 Thread abhioncbr
This is an automated email from the ASF dual-hosted git repository. abhioncbr pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 3ee3dd4fba Updated more packages in License-bi

Re: [PR] Updated more packages in the License-binary file. [pinot]

2025-01-06 Thread via GitHub
abhioncbr merged PR #14757: URL: https://github.com/apache/pinot/pull/14757 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

Re: [PR] Config for max output segment size in UpsertCompactMerge task [pinot]

2025-01-06 Thread via GitHub
tibrewalpratik17 commented on code in PR #14742: URL: https://github.com/apache/pinot/pull/14742#discussion_r1904767265 ## pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/upsertcompactmerge/UpsertCompactMergeTaskGenerator.ja

Re: [PR] [perf] Run Inverted Index Before Other Operators [pinot]

2025-01-06 Thread via GitHub
codecov-commenter commented on PR #14764: URL: https://github.com/apache/pinot/pull/14764#issuecomment-2574130994 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14764?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] [perf] Run Inverted Index Before Other Operators [pinot]

2025-01-06 Thread via GitHub
ankitsultana opened a new pull request, #14764: URL: https://github.com/apache/pinot/pull/14764 Resolves #14744 This is still not the "ideal approach". Currently we *always* reorder AND predicates based on these priorities, but sometimes users may have selective filters which are low

Re: [PR] Fix typo in RefreshSegmentTaskExecutor logger [pinot]

2025-01-06 Thread via GitHub
codecov-commenter commented on PR #14763: URL: https://github.com/apache/pinot/pull/14763#issuecomment-2573969022 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14763?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

(pinot) branch master updated: Fix logger class (#14763)

2025-01-06 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 88805f2605 Fix logger class (#14763) 88805f26

Re: [PR] Fix typo in RefreshSegmentTaskExecutor logger [pinot]

2025-01-06 Thread via GitHub
vvivekiyer merged PR #14763: URL: https://github.com/apache/pinot/pull/14763 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

[PR] Fix typo in RefreshSegmentTaskExecutor logger [pinot]

2025-01-06 Thread via GitHub
vvivekiyer opened a new pull request, #14763: URL: https://github.com/apache/pinot/pull/14763 Fixes typo in RefreshSegmentTaskExecutor. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] Segments count and indexes mismatch [pinot]

2025-01-06 Thread via GitHub
s4iko closed issue #14692: Segments count and indexes mismatch URL: https://github.com/apache/pinot/issues/14692 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [I] Segments count and indexes mismatch [pinot]

2025-01-06 Thread via GitHub
s4iko commented on issue #14692: URL: https://github.com/apache/pinot/issues/14692#issuecomment-2573597733 Hi, thanks for the answer. I guess the missing part was running `RealtimeSegmentValidationManager`. I will try this if the problem occurs again and then open a new issue if I did

Re: [PR] Config for max output segment size in UpsertCompactMerge task [pinot]

2025-01-06 Thread via GitHub
tibrewalpratik17 commented on code in PR #14742: URL: https://github.com/apache/pinot/pull/14742#discussion_r1904416335 ## pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/upsertcompactmerge/UpsertCompactMergeTaskGenerator.ja

Re: [I] [flaky-test] SSB Integration Test Failed Due to Failure to Download Segment [pinot]

2025-01-06 Thread via GitHub
cutiepie-10 commented on issue #14655: URL: https://github.com/apache/pinot/issues/14655#issuecomment-2573523255 Hi @ankitsultana, How to produce this error? I have tried many times but the test succeeded. -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] Improving logs and metric update logic [pinot]

2025-01-06 Thread via GitHub
codecov-commenter commented on PR #14762: URL: https://github.com/apache/pinot/pull/14762#issuecomment-2573504852 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14762?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Upgrading Pinot version(1.4.0) for next release. [pinot]

2025-01-06 Thread via GitHub
codecov-commenter commented on PR #14761: URL: https://github.com/apache/pinot/pull/14761#issuecomment-2573403056 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14761?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Improving logs and metric update logic [pinot]

2025-01-06 Thread via GitHub
9aman opened a new pull request, #14762: URL: https://github.com/apache/pinot/pull/14762 - The fetchLatestStreamOffset doesn't throw an exception when it's not able to fetch the latest offset. - The logs are not printed in case the consumer is not able to fetch the latest offset. -

Re: [PR] Config for max output segment size in UpsertCompactMerge task [pinot]

2025-01-06 Thread via GitHub
swaminathanmanish commented on code in PR #14742: URL: https://github.com/apache/pinot/pull/14742#discussion_r1904286958 ## pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/upsertcompactmerge/UpsertCompactMergeTaskGenerator.j

Re: [PR] Config for max output segment size in UpsertCompactMerge task [pinot]

2025-01-06 Thread via GitHub
swaminathanmanish commented on code in PR #14742: URL: https://github.com/apache/pinot/pull/14742#discussion_r1904286958 ## pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/upsertcompactmerge/UpsertCompactMergeTaskGenerator.j

[PR] Upgrading Pinot version(1.4.0) for next release. [pinot]

2025-01-06 Thread via GitHub
abhioncbr opened a new pull request, #14761: URL: https://github.com/apache/pinot/pull/14761 This PR increments the Pinot version for the next release. This PR will be merged once we cut the Pinot 1.3.0 release from the master branch. -- This is an automated message from the Apache Git S

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.29.45 deleted (was 34d487fa5b)

2025-01-06 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.29.45 in repository https://gitbox.apache.org/repos/asf/pinot.git was 34d487fa5b Bump software.amazon.awssdk:bom from 2.29.44 to 2.29.45 Th

Re: [PR] Bump software.amazon.awssdk:bom from 2.29.44 to 2.29.45 [pinot]

2025-01-06 Thread via GitHub
yashmayya merged PR #14760: URL: https://github.com/apache/pinot/pull/14760 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

(pinot) branch master updated: Bump com.nimbusds:nimbus-jose-jwt from 10.0 to 10.0.1 (#14759)

2025-01-06 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new c7c4cb1b46 Bump com.nimbusds:nimbus-jose-jwt f

(pinot) branch master updated (c7c4cb1b46 -> cad5cf8a62)

2025-01-06 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from c7c4cb1b46 Bump com.nimbusds:nimbus-jose-jwt from 10.0 to 10.0.1 (#14759) add cad5cf8a62 Bump software.amazon.

(pinot) branch dependabot/maven/com.nimbusds-nimbus-jose-jwt-10.0.1 deleted (was fbbe8cc524)

2025-01-06 Thread yashmayya
This is an automated email from the ASF dual-hosted git repository. yashmayya pushed a change to branch dependabot/maven/com.nimbusds-nimbus-jose-jwt-10.0.1 in repository https://gitbox.apache.org/repos/asf/pinot.git was fbbe8cc524 Bump com.nimbusds:nimbus-jose-jwt from 10.0 to 10.0.1 The

Re: [PR] Bump com.nimbusds:nimbus-jose-jwt from 10.0 to 10.0.1 [pinot]

2025-01-06 Thread via GitHub
yashmayya merged PR #14759: URL: https://github.com/apache/pinot/pull/14759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

[PR] Bump software.amazon.awssdk:bom from 2.29.44 to 2.29.45 [pinot]

2025-01-06 Thread via GitHub
dependabot[bot] opened a new pull request, #14760: URL: https://github.com/apache/pinot/pull/14760 Bumps software.amazon.awssdk:bom from 2.29.44 to 2.29.45. Most Recent Ignore Conditions Applied to This Pull Request | Dependency Name | Ignore Conditions | | --- | --- |

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.29.45 created (now 34d487fa5b)

2025-01-06 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.29.45 in repository https://gitbox.apache.org/repos/asf/pinot.git at 34d487fa5b Bump software.amazon.awssdk:bom from 2.29.44 to 2.29.45 N

(pinot) branch dependabot/maven/com.nimbusds-nimbus-jose-jwt-10.0.1 created (now fbbe8cc524)

2025-01-06 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.nimbusds-nimbus-jose-jwt-10.0.1 in repository https://gitbox.apache.org/repos/asf/pinot.git at fbbe8cc524 Bump com.nimbusds:nimbus-jose-jwt from 10.0 to 10.0.1 No

[PR] Bump com.nimbusds:nimbus-jose-jwt from 10.0 to 10.0.1 [pinot]

2025-01-06 Thread via GitHub
dependabot[bot] opened a new pull request, #14759: URL: https://github.com/apache/pinot/pull/14759 Bumps [com.nimbusds:nimbus-jose-jwt](https://bitbucket.org/connect2id/nimbus-jose-jwt) from 10.0 to 10.0.1. Changelog Sourced from https://bitbucket.org/connect2id/nimbus-jose-jwt/sr