[GitHub] [pinot] ankitsultana commented on a diff in pull request #10463: Segment compaction for upsert real-time tables

2023-05-19 Thread via GitHub
ankitsultana commented on code in PR #10463: URL: https://github.com/apache/pinot/pull/10463#discussion_r1199518455 ## pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/upsertcompaction/UpsertCompactionTaskExecutor.java: #

[GitHub] [pinot] siddharthteotia commented on issue #10783: Extensible PinotDataBuffer SPI

2023-05-19 Thread via GitHub
siddharthteotia commented on issue #10783: URL: https://github.com/apache/pinot/issues/10783#issuecomment-1555401853 +1. Thanks @gortiz So I am guessing the next plan of action is to: - Review the SPI and Unsafe based implementation in https://github.com/apache/pinot/pull/1052

[GitHub] [pinot] somandal commented on issue #10745: [multistage] The multistage aggregate planner and runtime do not work with all v1 aggregation functions

2023-05-19 Thread via GitHub
somandal commented on issue #10745: URL: https://github.com/apache/pinot/issues/10745#issuecomment-1555378367 Please review this proposal written by me and @vvivekiyer : https://docs.google.com/document/d/1Us6aBvTpNLMEy0ODo34OgTk73h_LVFFAH6q17689h1M/edit?usp=sharing -- This is an automa

[GitHub] [pinot] xiangfu0 commented on pull request #10776: upgrade guava version to 30.1.1-jre for CVE fixes

2023-05-19 Thread via GitHub
xiangfu0 commented on PR #10776: URL: https://github.com/apache/pinot/pull/10776#issuecomment-1555369354 > guava is widely used. should we do a perf regression test before this major upgrade? This is the lowest version we need to get due to CVE, perf test won't block this upgrade.

[pinot] branch master updated (4a18a3d28b -> 3851f33094)

2023-05-19 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 4a18a3d28b update pinot helm readme (#10778) add 3851f33094 upgrade guava version to 30.1.1-jre for CVE fixes (#1

[GitHub] [pinot] xiangfu0 merged pull request #10776: upgrade guava version to 30.1.1-jre for CVE fixes

2023-05-19 Thread via GitHub
xiangfu0 merged PR #10776: URL: https://github.com/apache/pinot/pull/10776 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[GitHub] [pinot] deemoliu commented on a diff in pull request #10047: Add upsert ttl configs for Pinot upsert optimizations

2023-05-19 Thread via GitHub
deemoliu commented on code in PR #10047: URL: https://github.com/apache/pinot/pull/10047#discussion_r1199493215 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java: ## @@ -181,6 +198,55 @@ protected void rem

[GitHub] [pinot] deemoliu commented on a diff in pull request #10047: Add upsert ttl configs for Pinot upsert optimizations

2023-05-19 Thread via GitHub
deemoliu commented on code in PR #10047: URL: https://github.com/apache/pinot/pull/10047#discussion_r1199492928 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/PartitionUpsertMetadataManager.java: ## @@ -84,6 +84,16 @@ public interface PartitionUpsertM

[GitHub] [pinot] siddharthteotia commented on pull request #10772: [multistage] hintable option framework & dynamic broadcast hint

2023-05-19 Thread via GitHub
siddharthteotia commented on PR #10772: URL: https://github.com/apache/pinot/pull/10772#issuecomment-1555363369 I am also reviewing this today -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [pinot] xiangfu0 commented on issue #10783: Extensible PinotDataBuffer SPI

2023-05-19 Thread via GitHub
xiangfu0 commented on issue #10783: URL: https://github.com/apache/pinot/issues/10783#issuecomment-1555340395 +1, awesome work! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[GitHub] [pinot] ankitsultana opened a new issue, #10788: Rebalance with Downtime Can Cause Data Loss

2023-05-19 Thread via GitHub
ankitsultana opened a new issue, #10788: URL: https://github.com/apache/pinot/issues/10788 Taking an example of a realtime table, if the segment download url link for some segments are missing, issuing a Table Rebalance with Downtime can cause data loss with very high likelihood (even with

[GitHub] [pinot] klsince commented on a diff in pull request #10785: [WIP] allow env var substibution for Pinot configs

2023-05-19 Thread via GitHub
klsince commented on code in PR #10785: URL: https://github.com/apache/pinot/pull/10785#discussion_r1199417840 ## pinot-spi/src/main/java/org/apache/pinot/spi/env/PinotConfiguration.java: ## @@ -357,6 +358,9 @@ public long getProperty(String name, long defaultValue) { */

[GitHub] [pinot] dang-stripe opened a new issue, #10787: Failed queries due to servers shutting down before brokers finish routing table changes

2023-05-19 Thread via GitHub
dang-stripe opened a new issue, #10787: URL: https://github.com/apache/pinot/issues/10787 We've observed some 425 error query failures during rolling restarts on a relatively low QPS cluster. Looking at logs, we noticed that the server shutdown before the broker finished processing the rout

[GitHub] [pinot] dd-willgan commented on a diff in pull request #10785: [WIP] allow env var substibution for Pinot configs

2023-05-19 Thread via GitHub
dd-willgan commented on code in PR #10785: URL: https://github.com/apache/pinot/pull/10785#discussion_r1199344351 ## pinot-spi/src/main/java/org/apache/pinot/spi/env/PinotConfiguration.java: ## @@ -357,6 +358,9 @@ public long getProperty(String name, long defaultValue) { */

[GitHub] [pinot] PeterCorless opened a new issue, #10786: [Release Notes] Add Dates and Link to Github

2023-05-19 Thread via GitHub
PeterCorless opened a new issue, #10786: URL: https://github.com/apache/pinot/issues/10786 - Can dates of publication be added to the release notes? - Can a link to the GitHub for the release be added? Example: On the release notes for [0.12.0](https://docs.pinot.apache.org/basi

[GitHub] [pinot] codecov-commenter commented on pull request #10785: [WIP] allow env var substibution for Pinot configs

2023-05-19 Thread via GitHub
codecov-commenter commented on PR #10785: URL: https://github.com/apache/pinot/pull/10785#issuecomment-1555127341 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10785?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] klsince commented on a diff in pull request #10726: [bugfix] Pinot config env substitution

2023-05-19 Thread via GitHub
klsince commented on code in PR #10726: URL: https://github.com/apache/pinot/pull/10726#discussion_r1199273342 ## pinot-spi/src/main/java/org/apache/pinot/spi/env/PinotConfiguration.java: ## @@ -356,15 +354,7 @@ public long getProperty(String name, long defaultValue) { * @re

[GitHub] [pinot] klsince opened a new pull request, #10785: [WIP] allow env var substibution for Pinot configs

2023-05-19 Thread via GitHub
klsince opened a new pull request, #10785: URL: https://github.com/apache/pinot/pull/10785 Testing some ideas to make env var substitution work for Pinot configs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [pinot] walterddr commented on a diff in pull request #10726: [bugfix] Pinot config env substitution

2023-05-19 Thread via GitHub
walterddr commented on code in PR #10726: URL: https://github.com/apache/pinot/pull/10726#discussion_r1199242904 ## pinot-spi/src/main/java/org/apache/pinot/spi/env/PinotConfiguration.java: ## @@ -356,15 +354,7 @@ public long getProperty(String name, long defaultValue) { * @

[GitHub] [pinot] dang-stripe opened a new pull request, #10784: Ensure servers wait the full `noQueryTimeout` before shutting down

2023-05-19 Thread via GitHub
dang-stripe opened a new pull request, #10784: URL: https://github.com/apache/pinot/pull/10784 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [pinot] mayankshriv commented on issue #10783: Extensible PinotDataBuffer SPI

2023-05-19 Thread via GitHub
mayankshriv commented on issue #10783: URL: https://github.com/apache/pinot/issues/10783#issuecomment-1555012258 +1 from me as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[pinot] branch master updated (e302318d33 -> 4a18a3d28b)

2023-05-19 Thread rongr
This is an automated email from the ASF dual-hosted git repository. rongr pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from e302318d33 bug fix: add @JsonProperty to SegmentAssignmentConfig (#10759) add 4a18a3d28b update pinot helm readme

[GitHub] [pinot] walterddr merged pull request #10778: update pinot helm readme

2023-05-19 Thread via GitHub
walterddr merged PR #10778: URL: https://github.com/apache/pinot/pull/10778 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

[GitHub] [pinot] walterddr commented on pull request #10771: Enable case insensitivity by default

2023-05-19 Thread via GitHub
walterddr commented on PR #10771: URL: https://github.com/apache/pinot/pull/10771#issuecomment-1554801098 yes I agree that we should follow the case insensitivity goal. in fact PostgresQL does it more specifically: - unquoted names are case-insensitive. Thus `SELECT * FROM hello` and `SE

[GitHub] [pinot] walterddr commented on a diff in pull request #10772: [multistage] hintable option framework & dynamic broadcast hint

2023-05-19 Thread via GitHub
walterddr commented on code in PR #10772: URL: https://github.com/apache/pinot/pull/10772#discussion_r1199071543 ## pinot-query-planner/src/main/java/org/apache/calcite/rel/hint/PinotHintOptions.java: ## @@ -0,0 +1,44 @@ +/** + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [pinot] kishoreg commented on issue #10783: Extensible PinotDataBuffer SPI

2023-05-19 Thread via GitHub
kishoreg commented on issue #10783: URL: https://github.com/apache/pinot/issues/10783#issuecomment-1554671555 This is amazing @gortiz . I am +1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [pinot] gortiz opened a new issue, #10783: Extensible PinotDataBuffer SPI

2023-05-19 Thread via GitHub
gortiz opened a new issue, #10783: URL: https://github.com/apache/pinot/issues/10783 This is a PEP-Request. I propose to create a extensible SPI that can be used to change the PinotDataBuffer implementation used at runtime. Treat this issue as a clone of https://github.com/apache/pinot/issu

[pinot] branch master updated (8293facfa0 -> e302318d33)

2023-05-19 Thread kharekartik
This is an automated email from the ASF dual-hosted git repository. kharekartik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 8293facfa0 Keep last completed segment for retention (#10754) add e302318d33 bug fix: add @JsonProperty to Se

[GitHub] [pinot] KKcorps merged pull request #10759: bug fix: add @JsonProperty for variable _assignmentStrategy in SegmentAssignmentConfig

2023-05-19 Thread via GitHub
KKcorps merged PR #10759: URL: https://github.com/apache/pinot/pull/10759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

[GitHub] [pinot] KKcorps commented on pull request #10759: bug fix: add @JsonProperty for variable _assignmentStrategy in SegmentAssignmentConfig

2023-05-19 Thread via GitHub
KKcorps commented on PR #10759: URL: https://github.com/apache/pinot/pull/10759#issuecomment-1554177933 Thank you for the contribution! Waiting for the checks to complete. Will merge as soon as they are done. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [pinot] INNOCENT-BOY commented on pull request #10759: bug fix: add @JsonProperty for variable _assignmentStrategy in SegmentAssignmentConfig

2023-05-19 Thread via GitHub
INNOCENT-BOY commented on PR #10759: URL: https://github.com/apache/pinot/pull/10759#issuecomment-1554126121 Hi Team, please help me to merge this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g