Re: [PR] Enhancing metadata API to return upsert partition to primary key count map for both controller and server APIs [pinot]

2024-02-07 Thread via GitHub
KKcorps commented on code in PR #12334: URL: https://github.com/apache/pinot/pull/12334#discussion_r1482466264 ## pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java: ## @@ -283,9 +284,23 @@ public String getSegmentMetadata( } } +

Re: [PR] [Multi-stage] Optimize mailbox info in query plan [pinot]

2024-02-07 Thread via GitHub
codecov-commenter commented on PR #12382: URL: https://github.com/apache/pinot/pull/12382#issuecomment-1933264628 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12382?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] skip invalid json string rather than throwing error during json indexing [pinot]

2024-02-07 Thread via GitHub
wirybeaver commented on PR #12238: URL: https://github.com/apache/pinot/pull/12238#issuecomment-1933262709 @walterddr Could you review again to see if there's any objection? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[PR] [Multi-stage] Optimize mailbox info in query plan [pinot]

2024-02-07 Thread via GitHub
Jackie-Jiang opened a new pull request, #12382: URL: https://github.com/apache/pinot/pull/12382 Extended optimization over #12370 Further reduce the footprint of mailbox info by: - Only keep the worker id instead of the full virtual server address - Group multiple workers on the same

Re: [I] FreshnessBasedConsumptionStatusChecker never pass when a table got deleted during the restart. [pinot]

2024-02-07 Thread via GitHub
xiangfu0 commented on issue #12381: URL: https://github.com/apache/pinot/issues/12381#issuecomment-1933224799 cc: @Jackie-Jiang @mayankshriv -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[I] FreshnessBasedConsumptionStatusChecker never pass when a table got deleted during the restart. [pinot]

2024-02-07 Thread via GitHub
xiangfu0 opened a new issue, #12381: URL: https://github.com/apache/pinot/issues/12381 Below are the log reference: ``` 2024/02/08 01:19:09.149 INFO [BaseServerStarter] [Start a Pinot [SERVER]] Sleep for 1ms as service status has not turned GOOD: MultipleCallbackServiceStatusCallb

Re: [PR] Add round-robin logic during downloadSegmentFromPeer [pinot]

2024-02-07 Thread via GitHub
ankitsultana commented on code in PR #12353: URL: https://github.com/apache/pinot/pull/12353#discussion_r1482243905 ## pinot-common/src/main/java/org/apache/pinot/common/utils/RoundRobinURIProvider.java: ## @@ -63,4 +78,33 @@ public URI next() { _index = (_index + 1) % _uri

Re: [PR] Add round-robin logic during downloadSegmentFromPeer [pinot]

2024-02-07 Thread via GitHub
ankitsultana commented on code in PR #12353: URL: https://github.com/apache/pinot/pull/12353#discussion_r1482240176 ## pinot-common/src/main/java/org/apache/pinot/common/utils/RoundRobinURIProvider.java: ## @@ -63,4 +78,33 @@ public URI next() { _index = (_index + 1) % _uri

Re: [PR] skip invalid json string rather than throwing error during json indexing [pinot]

2024-02-07 Thread via GitHub
wirybeaver commented on code in PR #12238: URL: https://github.com/apache/pinot/pull/12238#discussion_r1482237810 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/JsonUtils.java: ## @@ -719,4 +721,19 @@ private static void addFieldToPinotSchema(Schema pinotSchema, DataType

Re: [PR] skip invalid json string rather than throwing error during json indexing [pinot]

2024-02-07 Thread via GitHub
wirybeaver commented on PR #12238: URL: https://github.com/apache/pinot/pull/12238#issuecomment-1933117409 > Apologies. Had stamped but had a question. @wirybeaver can you take a look? no worries -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Make segment download from Peer servers more robust by retrying both peer discovery and download. [pinot]

2024-02-07 Thread via GitHub
ankitsultana commented on code in PR #12317: URL: https://github.com/apache/pinot/pull/12317#discussion_r1482212043 ## pinot-common/src/main/java/org/apache/pinot/common/utils/fetcher/BaseSegmentFetcher.java: ## @@ -109,6 +110,40 @@ public File fetchUntarSegmentToLocalStreamed(U

Re: [PR] skip invalid json string rather than throwing error during json indexing [pinot]

2024-02-07 Thread via GitHub
ankitsultana commented on code in PR #12238: URL: https://github.com/apache/pinot/pull/12238#discussion_r1482203477 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/JsonUtils.java: ## @@ -719,4 +721,19 @@ private static void addFieldToPinotSchema(Schema pinotSchema, DataTy

Re: [PR] make thread local variable static to avoid the memory leaking issue [pinot]

2024-02-07 Thread via GitHub
wirybeaver commented on PR #12242: URL: https://github.com/apache/pinot/pull/12242#issuecomment-1932714219 @gortiz Could you review the code? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] [Multi-stage] Optimize query plan serialization [pinot]

2024-02-07 Thread via GitHub
Jackie-Jiang merged PR #12370: URL: https://github.com/apache/pinot/pull/12370 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] [multistage][feature] support RelDistribution trait planning [pinot]

2024-02-07 Thread via GitHub
ankitsultana commented on code in PR #11976: URL: https://github.com/apache/pinot/pull/11976#discussion_r1481959847 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotRelDistributionTraitRule.java: ## @@ -0,0 +1,178 @@ +/** + * Licensed to the Apache Software

Re: [PR] make thread local variable static to avoid the memory leaking issue [pinot]

2024-02-07 Thread via GitHub
wirybeaver commented on PR #12242: URL: https://github.com/apache/pinot/pull/12242#issuecomment-1932695513 I made an A/B testing to compare the performance option 1(static thread local) vs option 2(move bytes array to context) in two clusters with same size and configuration. It turns out t

(pinot) branch master updated: Fixing the bug for Upsert compaction task generator (#12380)

2024-02-07 Thread snlee
This is an automated email from the ASF dual-hosted git repository. snlee pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new f6ae06f499 Fixing the bug for Upsert compaction ta

Re: [PR] Fixing the bug for Upsert compaction task generator [pinot]

2024-02-07 Thread via GitHub
snleee merged PR #12380: URL: https://github.com/apache/pinot/pull/12380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

Re: [PR] [multistage][feature] support RelDistribution trait planning [pinot]

2024-02-07 Thread via GitHub
walterddr commented on code in PR #11976: URL: https://github.com/apache/pinot/pull/11976#discussion_r1481704588 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotRelDistributionTraitRule.java: ## @@ -0,0 +1,178 @@ +/** + * Licensed to the Apache Software Fo

Re: [PR] [multistage][feature] support RelDistribution trait planning [pinot]

2024-02-07 Thread via GitHub
walterddr commented on code in PR #11976: URL: https://github.com/apache/pinot/pull/11976#discussion_r1481704588 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotRelDistributionTraitRule.java: ## @@ -0,0 +1,178 @@ +/** + * Licensed to the Apache Software Fo

Re: [PR] Adding support for configuration through environment variables [pinot]

2024-02-07 Thread via GitHub
t0mpere commented on PR #12307: URL: https://github.com/apache/pinot/pull/12307#issuecomment-1932207192 Doc added -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [PR] [multistage][feature] support RelDistribution trait planning [pinot]

2024-02-07 Thread via GitHub
gortiz commented on code in PR #11976: URL: https://github.com/apache/pinot/pull/11976#discussion_r1481445739 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotRelDistributionTraitRule.java: ## @@ -0,0 +1,178 @@ +/** + * Licensed to the Apache Software Found

Re: [PR] Add a test that verifies how mode works right now with null [pinot]

2024-02-07 Thread via GitHub
gortiz commented on code in PR #12226: URL: https://github.com/apache/pinot/pull/12226#discussion_r1481122398 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/NullableSingleInputAggregationFunction.java: ## @@ -0,0 +1,368 @@ +/** + * Licensed to the A

Re: [PR] [null-aggr] Add null handling support in `mode` aggregation [pinot]

2024-02-07 Thread via GitHub
gortiz commented on code in PR #12227: URL: https://github.com/apache/pinot/pull/12227#discussion_r1481119765 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/NullableSingleInputAggregationFunction.java: ## @@ -0,0 +1,368 @@ +/** + * Licensed to the A

Re: [PR] Fixing the bug for Upsert compaction task generator [pinot]

2024-02-07 Thread via GitHub
codecov-commenter commented on PR #12380: URL: https://github.com/apache/pinot/pull/12380#issuecomment-1931541361 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12380?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R