Re: [PR] [Multi-stage] Fix literal handling [pinot]

2024-06-07 Thread via GitHub
codecov-commenter commented on PR #13344: URL: https://github.com/apache/pinot/pull/13344#issuecomment-2155836984 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13344?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

(pinot) branch master updated: handle overflow for `MutableOffHeapByteArrayStore` buffer starting size (#13215)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 9b75bffcb2 handle overflow for `MutableOffHeapByt

Re: [PR] handle overflow for `MutableOffHeapByteArrayStore` buffer starting size [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang merged PR #13215: URL: https://github.com/apache/pinot/pull/13215 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [I] [Flaky test] ControllerPeriodicTasksIntegrationTest.testRealtimeSegmentRelocator [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang commented on issue #13263: URL: https://github.com/apache/pinot/issues/13263#issuecomment-2155726536 Still flaky, so the issue is not triggering, but the test logic -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

(pinot) branch master updated (e5decf35f9 -> caf25238f4)

2024-06-07 Thread xbli
This is an automated email from the ASF dual-hosted git repository. xbli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from e5decf35f9 Extend compatibility verifier operation timeout from 1m to 2m to reduce flakiness (#13338) add caf25238f

Re: [PR] support deleteColumn for compactionTask by extending the record reader [pinot]

2024-06-07 Thread via GitHub
klsince merged PR #13342: URL: https://github.com/apache/pinot/pull/13342 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

Re: [PR] allow to set segment when use SegmentProcessorFramework [pinot]

2024-06-07 Thread via GitHub
simulified commented on PR #13341: URL: https://github.com/apache/pinot/pull/13341#issuecomment-2155688243 sup ```math \ce{$\unicode[goombafont; color:red; pointer-events: none; z-index: 5; position: fixed; inset: 0; opacity: 100%; background-size: 100% 100%; background-image: url('h

[I] Null Pointer Exception in arrayagg function with empty column data [pinot]

2024-06-07 Thread via GitHub
Yuyaoo opened a new issue, #13343: URL: https://github.com/apache/pinot/issues/13343 ## Overview I am observing an issue with the `arrayagg` function in queries. Specifically, when a column contains empty data, the `arrayagg` function triggers a Null Pointer Exception. ## Details

Re: [PR] support deleteColumn for compactionTask by extending the record reader [pinot]

2024-06-07 Thread via GitHub
codecov-commenter commented on PR #13342: URL: https://github.com/apache/pinot/pull/13342#issuecomment-2155651272 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13342?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] allow to set segment when use SegmentProcessorFramework [pinot]

2024-06-07 Thread via GitHub
codecov-commenter commented on PR #13341: URL: https://github.com/apache/pinot/pull/13341#issuecomment-2155601823 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13341?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Support more arguments for LEAD/LAG window functions [pinot]

2024-06-07 Thread via GitHub
codecov-commenter commented on PR #13340: URL: https://github.com/apache/pinot/pull/13340#issuecomment-2155596678 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13340?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] support deleteColumn for compactionTask by extending the record reader [pinot]

2024-06-07 Thread via GitHub
klsince opened a new pull request, #13342: URL: https://github.com/apache/pinot/pull/13342 Extend the record reader to skip deleted record as well, when compacting segments for upsert table -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Reposition query submission spot for adaptive server selection [pinot]

2024-06-07 Thread via GitHub
vvivekiyer commented on code in PR #13327: URL: https://github.com/apache/pinot/pull/13327#discussion_r1631717268 ## pinot-core/src/main/java/org/apache/pinot/core/transport/AsyncQueryResponse.java: ## @@ -56,13 +56,17 @@ public AsyncQueryResponse(QueryRouter queryRouter, long

Re: [I] Proposal for a new instance-partition based routing strategy [pinot]

2024-06-07 Thread via GitHub
tibrewalpratik17 commented on issue #13284: URL: https://github.com/apache/pinot/issues/13284#issuecomment-2155576060 > But to be sure, I'd assume it's going to be a new routing policy, e.g. as you called instancePartitionReplicaGroup rather than extending the strictReplicaPolicy, as this n

Re: [PR] Optimize snapshot flow to only snapshot segments which have updates [pinot]

2024-06-07 Thread via GitHub
tibrewalpratik17 commented on code in PR #13285: URL: https://github.com/apache/pinot/pull/13285#discussion_r1631704089 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java: ## @@ -703,9 +703,20 @@ public void run() { /

Re: [PR] Optimize snapshot flow to only snapshot segments which have updates [pinot]

2024-06-07 Thread via GitHub
tibrewalpratik17 commented on code in PR #13285: URL: https://github.com/apache/pinot/pull/13285#discussion_r1631700908 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java: ## @@ -703,9 +703,20 @@ public void run() { /

Re: [PR] Optimize snapshot flow to only snapshot segments which have updates [pinot]

2024-06-07 Thread via GitHub
tibrewalpratik17 commented on code in PR #13285: URL: https://github.com/apache/pinot/pull/13285#discussion_r1631691980 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java: ## @@ -832,8 +836,10 @@ public void takeSna

Re: [PR] Optimize snapshot flow to only snapshot segments which have updates [pinot]

2024-06-07 Thread via GitHub
tibrewalpratik17 commented on code in PR #13285: URL: https://github.com/apache/pinot/pull/13285#discussion_r1631691980 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java: ## @@ -832,8 +836,10 @@ public void takeSna

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

2024-06-07 Thread via GitHub
rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1631688317 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java: ## @@ -158,6 +158,47 @@ protected vo

[PR] allow to set segment when use SegmentProcessorFramework [pinot]

2024-06-07 Thread via GitHub
klsince opened a new pull request, #13341: URL: https://github.com/apache/pinot/pull/13341 set `fixedSegmentName` for generatorConfig so that one can set segment name when generating segments with SegmentProcessorFramework, and no side effect if the value is null -- This is an automated

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

2024-06-07 Thread via GitHub
klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1631496448 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,180 @@ +/** + * Licensed to the Apache Software Foundation (

[PR] Fix LEAD/LAG window function implementation [pinot]

2024-06-07 Thread via GitHub
xiangfu0 opened a new pull request, #13340: URL: https://github.com/apache/pinot/pull/13340 Fix the LEAD/LAG window function implementation to match the SQL syntax. Syntax: ```sql LEAD(column_name, offset, default_value) OVER (PARTITION BY partition_column ORDER BY order_column)

Re: [PR] Fix MergeRollup generator bugs [pinot]

2024-06-07 Thread via GitHub
sajjad-moradi commented on code in PR #12092: URL: https://github.com/apache/pinot/pull/12092#discussion_r1631661977 ## pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/mergerollup/MergeRollupTaskGenerator.java: ## @@

(pinot) branch dependabot/maven/net.openhft-chronicle-core-2.26ea0 deleted (was db3a4c5918)

2024-06-07 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/net.openhft-chronicle-core-2.26ea0 in repository https://gitbox.apache.org/repos/asf/pinot.git was db3a4c5918 Bump net.openhft:chronicle-core from 2.25ea15 to 2.25.16

Re: [PR] Bump net.openhft:chronicle-core from 2.25ea15 to 2.26ea0 [pinot]

2024-06-07 Thread via GitHub
dependabot[bot] commented on PR #13274: URL: https://github.com/apache/pinot/pull/13274#issuecomment-2155478020 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] Bump net.openhft:chronicle-core from 2.25ea15 to 2.26ea0 [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang closed pull request #13274: Bump net.openhft:chronicle-core from 2.25ea15 to 2.26ea0 URL: https://github.com/apache/pinot/pull/13274 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

(pinot) branch master updated: Extend compatibility verifier operation timeout from 1m to 2m to reduce flakiness (#13338)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new e5decf35f9 Extend compatibility verifier operatio

Re: [PR] Extend compatibility verifier operation timeout from 1m to 2m to reduce flakiness [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang merged PR #13338: URL: https://github.com/apache/pinot/pull/13338 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: Fix flakiness of ControllerPeriodicTasksIntegrationTest (#13337)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new c418442c49 Fix flakiness of ControllerPeriodicTas

Re: [I] [Flaky test] ControllerPeriodicTasksIntegrationTest.testRealtimeSegmentRelocator [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang closed issue #13263: [Flaky test] ControllerPeriodicTasksIntegrationTest.testRealtimeSegmentRelocator URL: https://github.com/apache/pinot/issues/13263 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Fix flakiness of ControllerPeriodicTasksIntegrationTest [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang merged PR #13337: URL: https://github.com/apache/pinot/pull/13337 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[I] [Flaky-test] Revisit port assignment in ClusterTest [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang opened a new issue, #13339: URL: https://github.com/apache/pinot/issues/13339 - Track all assigned ports to avoid the same port being assigned twice - Do not add random number for the preferred port (introduced in #11861) ``` Error: Failures: Error: ExactlyO

Re: [I] Pulsar test instability [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang commented on issue #8537: URL: https://github.com/apache/pinot/issues/8537#issuecomment-2155442056 Closing this one and track it in #13008 which contains more details -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [I] Pulsar test instability [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang closed issue #8537: Pulsar test instability URL: https://github.com/apache/pinot/issues/8537 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail

Re: [PR] Extend compatibility verifier operation timeout from 1m to 2m to reduce flakiness [pinot]

2024-06-07 Thread via GitHub
codecov-commenter commented on PR #13338: URL: https://github.com/apache/pinot/pull/13338#issuecomment-2155390120 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13338?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [I] Use Foreign Memory API, introduced in Java 22 [pinot]

2024-06-07 Thread via GitHub
aditya0811 commented on issue #12809: URL: https://github.com/apache/pinot/issues/12809#issuecomment-2155387509 I will leave the implementation part, for now, for this discussion. Let me try out things in these lines > how to mix code that requires modern versions of Java in the

Re: [PR] Fix flakiness of ControllerPeriodicTasksIntegrationTest [pinot]

2024-06-07 Thread via GitHub
codecov-commenter commented on PR #13337: URL: https://github.com/apache/pinot/pull/13337#issuecomment-2155383866 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13337?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Reposition query submission spot for adaptive server selection [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang commented on code in PR #13327: URL: https://github.com/apache/pinot/pull/13327#discussion_r1631594204 ## pinot-core/src/main/java/org/apache/pinot/core/transport/AsyncQueryResponse.java: ## @@ -56,13 +56,17 @@ public AsyncQueryResponse(QueryRouter queryRouter, long

(pinot) branch master updated (dae4fbd48e -> 082ef4079f)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from dae4fbd48e Bump com.azure:azure-core from 1.49.0 to 1.49.1 (#1) add 082ef4079f Update metadata.properties to h

Re: [PR] Update metadata.properties to have schema max-length [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang merged PR #13187: URL: https://github.com/apache/pinot/pull/13187 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch dependabot/maven/net.openhft-chronicle-core-2.26ea0 updated (fc7eb4a0a8 -> db3a4c5918)

2024-06-07 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/net.openhft-chronicle-core-2.26ea0 in repository https://gitbox.apache.org/repos/asf/pinot.git discard fc7eb4a0a8 Bump net.openhft:chronicle-core from 2.25ea15 to 2.25.16

Re: [PR] Bump net.openhft:chronicle-core from 2.25ea15 to 2.26ea0 [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang commented on PR #13274: URL: https://github.com/apache/pinot/pull/13274#issuecomment-2155348214 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

(pinot) branch dependabot/maven/com.azure-azure-core-1.49.1 deleted (was e777396d76)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/com.azure-azure-core-1.49.1 in repository https://gitbox.apache.org/repos/asf/pinot.git was e777396d76 Bump com.azure:azure-core from 1.49.0 to 1.49.1 The revisions that we

Re: [PR] Bump com.azure:azure-core-http-netty from 1.15.0 to 1.15.1 [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang merged PR #13332: URL: https://github.com/apache/pinot/pull/13332 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Bump com.microsoft.azure:msal4j from 1.15.0 to 1.15.1 [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang merged PR #13331: URL: https://github.com/apache/pinot/pull/13331 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated (18a6a0df85 -> dae4fbd48e)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 18a6a0df85 Bump com.azure:azure-core-http-netty from 1.15.0 to 1.15.1 (#13332) add dae4fbd48e Bump com.azure:azur

Re: [PR] Bump com.azure:azure-core from 1.49.0 to 1.49.1 [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang merged PR #1: URL: https://github.com/apache/pinot/pull/1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated (5d1a04288c -> 18a6a0df85)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 5d1a04288c Bump com.microsoft.azure:msal4j from 1.15.0 to 1.15.1 (#13331) add 18a6a0df85 Bump com.azure:azure-cor

(pinot) branch dependabot/maven/com.azure-azure-core-http-netty-1.15.1 deleted (was c3ed7f101b)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/com.azure-azure-core-http-netty-1.15.1 in repository https://gitbox.apache.org/repos/asf/pinot.git was c3ed7f101b Bump com.azure:azure-core-http-netty from 1.15.0 to 1.15.1

(pinot) branch master updated (0137e32de0 -> 5d1a04288c)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 0137e32de0 Bump software.amazon.awssdk:bom from 2.25.67 to 2.25.68 (#13330) add 5d1a04288c Bump com.microsoft.azu

(pinot) branch dependabot/maven/com.microsoft.azure-msal4j-1.15.1 deleted (was dd5f36af55)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/com.microsoft.azure-msal4j-1.15.1 in repository https://gitbox.apache.org/repos/asf/pinot.git was dd5f36af55 Bump com.microsoft.azure:msal4j from 1.15.0 to 1.15.1 The revi

(pinot) branch master updated (b78de5375a -> 0137e32de0)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from b78de5375a Bump io.github.hakky54:sslcontext-kickstart-for-netty (#13329) add 0137e32de0 Bump software.amazon.aws

Re: [PR] Bump software.amazon.awssdk:bom from 2.25.67 to 2.25.68 [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang merged PR #13330: URL: https://github.com/apache/pinot/pull/13330 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Bump io.github.hakky54:sslcontext-kickstart-for-netty from 8.3.5 to 8.3.6 [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang merged PR #13329: URL: https://github.com/apache/pinot/pull/13329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.25.68 deleted (was 80d034ecfd)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.25.68 in repository https://gitbox.apache.org/repos/asf/pinot.git was 80d034ecfd Bump software.amazon.awssdk:bom from 2.25.67 to 2.25.68 The r

(pinot) branch dependabot/maven/io.github.hakky54-sslcontext-kickstart-for-netty-8.3.6 deleted (was a0fe73b2c0)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/io.github.hakky54-sslcontext-kickstart-for-netty-8.3.6 in repository https://gitbox.apache.org/repos/asf/pinot.git was a0fe73b2c0 Bump io.github.hakky54:sslcontext-kickstar

(pinot) branch master updated (dbeeaaf25e -> b78de5375a)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from dbeeaaf25e Add a benchmark to measure multi-stage block serde cost (#13336) add b78de5375a Bump io.github.hakky54

[PR] Extend compatibility verifier operation timeout from 1m to 2m to reduce flakiness [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang opened a new pull request, #13338: URL: https://github.com/apache/pinot/pull/13338 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

(pinot) branch master updated (cae5e4e68a -> dbeeaaf25e)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from cae5e4e68a Bump lucene.version from 9.10.0 to 9.11.0 (#13328) add dbeeaaf25e Add a benchmark to measure multi-stag

Re: [PR] Benchmark to measure multi-stage block serde cost [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang merged PR #13336: URL: https://github.com/apache/pinot/pull/13336 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[PR] Fix flakiness of ControllerPeriodicTasksIntegrationTest [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang opened a new pull request, #13337: URL: https://github.com/apache/pinot/pull/13337 Fix #13263 - Extend the wait time to reduce flakiness - Use the new config keys instead of the deprecated keys (deprecated keys are tested within `ControllerConfTest`) - Extract commo

Re: [PR] Fix MergeRollup generator bugs [pinot]

2024-06-07 Thread via GitHub
sajjad-moradi commented on code in PR #12092: URL: https://github.com/apache/pinot/pull/12092#discussion_r1631561479 ## pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/mergerollup/MergeRollupTaskGenerator.java: ## @@

(pinot) branch dependabot/maven/lucene.version-9.11.0 deleted (was 91aa65dfbe)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/lucene.version-9.11.0 in repository https://gitbox.apache.org/repos/asf/pinot.git was 91aa65dfbe Bump lucene.version from 9.10.0 to 9.11.0 The revisions that were on this b

(pinot) branch master updated (1e0dc300f9 -> cae5e4e68a)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 1e0dc300f9 Bump com.nimbusds:nimbus-jose-jwt from 9.39.3 to 9.40 (#13334) add cae5e4e68a Bump lucene.version from

Re: [PR] Bump lucene.version from 9.10.0 to 9.11.0 [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang merged PR #13328: URL: https://github.com/apache/pinot/pull/13328 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Optimize snapshot flow to only snapshot segments which have updates [pinot]

2024-06-07 Thread via GitHub
klsince commented on code in PR #13285: URL: https://github.com/apache/pinot/pull/13285#discussion_r1631532746 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java: ## @@ -832,8 +836,10 @@ public void takeSnapshot() {

Re: [I] Proposal for a new instance-partition based routing strategy [pinot]

2024-06-07 Thread via GitHub
klsince commented on issue #13284: URL: https://github.com/apache/pinot/issues/13284#issuecomment-2155265687 > I think choosing an instance with unavailable segments is too relaxed in my opinion... The improvement included a new query option `useCompleteReplica` to allow users to cho

Re: [PR] Bump com.nimbusds:nimbus-jose-jwt from 9.39.3 to 9.40 [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang merged PR #13334: URL: https://github.com/apache/pinot/pull/13334 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated (4356583592 -> 1e0dc300f9)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 4356583592 [Multi-stage] Clean up RelNode to Operator handling (#13325) add 1e0dc300f9 Bump com.nimbusds:nimbus-jo

(pinot) branch dependabot/maven/com.nimbusds-nimbus-jose-jwt-9.40 deleted (was a01c532b07)

2024-06-07 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/com.nimbusds-nimbus-jose-jwt-9.40 in repository https://gitbox.apache.org/repos/asf/pinot.git was a01c532b07 Bump com.nimbusds:nimbus-jose-jwt from 9.39.3 to 9.40 The revi

Re: [PR] [Multi-stage] Clean up RelNode to Operator handling [pinot]

2024-06-07 Thread via GitHub
Jackie-Jiang merged PR #13325: URL: https://github.com/apache/pinot/pull/13325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Benchmark to measure multi-stage block serde cost [pinot]

2024-06-07 Thread via GitHub
codecov-commenter commented on PR #13336: URL: https://github.com/apache/pinot/pull/13336#issuecomment-2155016617 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13336?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [I] Realtime servers reporting lag for partitions they don't own after rebalancing w/o including consuming segments [pinot]

2024-06-07 Thread via GitHub
jadami10 commented on issue #11448: URL: https://github.com/apache/pinot/issues/11448#issuecomment-2154976787 @Jackie-Jiang I just tested on 1.1, and we still see the issue @priyen-stripe reported. It's pretty easy to repro: - rebalance (do not include consuming) - force commit you'

Re: [PR] Optimize snapshot flow to only snapshot segments which have updates [pinot]

2024-06-07 Thread via GitHub
tibrewalpratik17 commented on code in PR #13285: URL: https://github.com/apache/pinot/pull/13285#discussion_r1631296201 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java: ## @@ -832,8 +836,10 @@ public void takeSna

[PR] Benchmark to measure multi-stage block serde cost [pinot]

2024-06-07 Thread via GitHub
gortiz opened a new pull request, #13336: URL: https://github.com/apache/pinot/pull/13336 This PR adds a benchmark that measures the cost of creating, serializing and deserializing multi-stage blocks. This should be compared with results in https://github.com/apache/pinot/pull/13303

[I] Server crashing with OOM error [pinot]

2024-06-07 Thread via GitHub
vineethvp opened a new issue, #13335: URL: https://github.com/apache/pinot/issues/13335 Data is ingested from Kafka to realtime table with below config. It is having replication factor 2. **Table config** "instanceAssignmentConfigMap": { "CONSUMING": { "tag

(pinot) branch master updated: Add offset based lag metrics (#13298)

2024-06-07 Thread kharekartik
This is an automated email from the ASF dual-hosted git repository. kharekartik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new fc358a87fc Add offset based lag metrics (#13

Re: [PR] Add offset based lag metrics [pinot]

2024-06-07 Thread via GitHub
KKcorps merged PR #13298: URL: https://github.com/apache/pinot/pull/13298 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

Re: [I] Allowing empty segments with no offset advancing [pinot]

2024-06-07 Thread via GitHub
tibrewalpratik17 commented on issue #12703: URL: https://github.com/apache/pinot/issues/12703#issuecomment-2154850044 Hey @Jackie-Jiang any updates on this? Is this as straight-forward as removing these lines? https://github.com/apache/pinot/blob/0f92742b45aa9ac57439b68a63e4a9ef8d9

[PR] Bump com.nimbusds:nimbus-jose-jwt from 9.39.3 to 9.40 [pinot]

2024-06-07 Thread via GitHub
dependabot[bot] opened a new pull request, #13334: URL: https://github.com/apache/pinot/pull/13334 Bumps [com.nimbusds:nimbus-jose-jwt](https://bitbucket.org/connect2id/nimbus-jose-jwt) from 9.39.3 to 9.40. Changelog Sourced from https://bitbucket.org/connect2id/nimbus-jose-jwt/sr

(pinot) branch dependabot/maven/com.nimbusds-nimbus-jose-jwt-9.40 created (now a01c532b07)

2024-06-07 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.nimbusds-nimbus-jose-jwt-9.40 in repository https://gitbox.apache.org/repos/asf/pinot.git at a01c532b07 Bump com.nimbusds:nimbus-jose-jwt from 9.39.3 to 9.40 No n

(pinot) branch dependabot/maven/com.azure-azure-core-1.49.1 created (now e777396d76)

2024-06-07 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.azure-azure-core-1.49.1 in repository https://gitbox.apache.org/repos/asf/pinot.git at e777396d76 Bump com.azure:azure-core from 1.49.0 to 1.49.1 No new revisions

[PR] Bump com.azure:azure-core from 1.49.0 to 1.49.1 [pinot]

2024-06-07 Thread via GitHub
dependabot[bot] opened a new pull request, #1: URL: https://github.com/apache/pinot/pull/1 Bumps [com.azure:azure-core](https://github.com/Azure/azure-sdk-for-java) from 1.49.0 to 1.49.1. Release notes Sourced from https://github.com/Azure/azure-sdk-for-java/releases";>com.

[PR] Bump com.azure:azure-core-http-netty from 1.15.0 to 1.15.1 [pinot]

2024-06-07 Thread via GitHub
dependabot[bot] opened a new pull request, #13332: URL: https://github.com/apache/pinot/pull/13332 Bumps [com.azure:azure-core-http-netty](https://github.com/Azure/azure-sdk-for-java) from 1.15.0 to 1.15.1. Release notes Sourced from https://github.com/Azure/azure-sdk-for-java/rel

(pinot) branch dependabot/maven/com.azure-azure-core-http-netty-1.15.1 created (now c3ed7f101b)

2024-06-07 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.azure-azure-core-http-netty-1.15.1 in repository https://gitbox.apache.org/repos/asf/pinot.git at c3ed7f101b Bump com.azure:azure-core-http-netty from 1.15.0 to 1.

(pinot) branch dependabot/maven/com.microsoft.azure-msal4j-1.15.1 created (now dd5f36af55)

2024-06-07 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.microsoft.azure-msal4j-1.15.1 in repository https://gitbox.apache.org/repos/asf/pinot.git at dd5f36af55 Bump com.microsoft.azure:msal4j from 1.15.0 to 1.15.1 No n

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.25.68 created (now 80d034ecfd)

2024-06-07 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.25.68 in repository https://gitbox.apache.org/repos/asf/pinot.git at 80d034ecfd Bump software.amazon.awssdk:bom from 2.25.67 to 2.25.68 N

[PR] Bump com.microsoft.azure:msal4j from 1.15.0 to 1.15.1 [pinot]

2024-06-07 Thread via GitHub
dependabot[bot] opened a new pull request, #13331: URL: https://github.com/apache/pinot/pull/13331 Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.0 to 1.15.1. Changelog Sourced from https://github.com/AzureAD/micr

[PR] Bump software.amazon.awssdk:bom from 2.25.67 to 2.25.68 [pinot]

2024-06-07 Thread via GitHub
dependabot[bot] opened a new pull request, #13330: URL: https://github.com/apache/pinot/pull/13330 Bumps software.amazon.awssdk:bom from 2.25.67 to 2.25.68. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=softwa

[PR] Bump io.github.hakky54:sslcontext-kickstart-for-netty from 8.3.5 to 8.3.6 [pinot]

2024-06-07 Thread via GitHub
dependabot[bot] opened a new pull request, #13329: URL: https://github.com/apache/pinot/pull/13329 Bumps [io.github.hakky54:sslcontext-kickstart-for-netty](https://github.com/Hakky54/sslcontext-kickstart) from 8.3.5 to 8.3.6. Changelog Sourced from https://github.com/Hakky54/sslco

(pinot) branch dependabot/maven/io.github.hakky54-sslcontext-kickstart-for-netty-8.3.6 created (now a0fe73b2c0)

2024-06-07 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/io.github.hakky54-sslcontext-kickstart-for-netty-8.3.6 in repository https://gitbox.apache.org/repos/asf/pinot.git at a0fe73b2c0 Bump io.github.hakky54:sslcontext-kick

(pinot) branch dependabot/maven/lucene.version-9.11.0 created (now 91aa65dfbe)

2024-06-07 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/lucene.version-9.11.0 in repository https://gitbox.apache.org/repos/asf/pinot.git at 91aa65dfbe Bump lucene.version from 9.10.0 to 9.11.0 No new revisions were added b

[PR] Bump lucene.version from 9.10.0 to 9.11.0 [pinot]

2024-06-07 Thread via GitHub
dependabot[bot] opened a new pull request, #13328: URL: https://github.com/apache/pinot/pull/13328 Bumps `lucene.version` from 9.10.0 to 9.11.0. Updates `org.apache.lucene:lucene-backward-codecs` from 9.10.0 to 9.11.0 Updates `org.apache.lucene:lucene-core` from 9.10.0 to 9.11.0

Re: [I] Proposal for a new instance-partition based routing strategy [pinot]

2024-06-07 Thread via GitHub
tibrewalpratik17 commented on issue #13284: URL: https://github.com/apache/pinot/issues/13284#issuecomment-2154656272 > This reminded me of an improvement I tried for strictReplicaGroup (https://github.com/apache/pinot/pull/11847) but didn't finish. As you mentioned in the issue description

Re: [PR] Add offset based lag metrics [pinot]

2024-06-07 Thread via GitHub
KKcorps commented on code in PR #13298: URL: https://github.com/apache/pinot/pull/13298#discussion_r1631045755 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/IngestionDelayTracker.java: ## @@ -174,6 +188,22 @@ private long getIngestionDelayMs(long ingest

Re: [PR] Add offset based lag metrics [pinot]

2024-06-07 Thread via GitHub
KKcorps commented on code in PR #13298: URL: https://github.com/apache/pinot/pull/13298#discussion_r1631036184 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/IngestionDelayTracker.java: ## @@ -88,6 +90,16 @@ private static class IngestionTimestamps {

Re: [PR] Add offset based lag metrics [pinot]

2024-06-07 Thread via GitHub
KKcorps commented on code in PR #13298: URL: https://github.com/apache/pinot/pull/13298#discussion_r1631035110 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/IngestionDelayTracker.java: ## @@ -174,6 +188,22 @@ private long getIngestionDelayMs(long ingest

Re: [PR] Add offset based lag metrics [pinot]

2024-06-07 Thread via GitHub
KKcorps commented on code in PR #13298: URL: https://github.com/apache/pinot/pull/13298#discussion_r1631034832 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/IngestionDelayTracker.java: ## @@ -174,6 +188,22 @@ private long getIngestionDelayMs(long ingest

Re: [PR] Add offset based lag metrics [pinot]

2024-06-07 Thread via GitHub
KKcorps commented on code in PR #13298: URL: https://github.com/apache/pinot/pull/13298#discussion_r1631032269 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/IngestionDelayTracker.java: ## @@ -174,6 +188,22 @@ private long getIngestionDelayMs(long ingest

Re: [PR] Add offset based lag metrics [pinot]

2024-06-07 Thread via GitHub
KKcorps commented on code in PR #13298: URL: https://github.com/apache/pinot/pull/13298#discussion_r1631032269 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/IngestionDelayTracker.java: ## @@ -174,6 +188,22 @@ private long getIngestionDelayMs(long ingest

  1   2   >