Re: [PR] Fix a typo when calculating query freshness [pinot]

2024-04-16 Thread via GitHub
codecov-commenter commented on PR #12947: URL: https://github.com/apache/pinot/pull/12947#issuecomment-2060512959 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12947?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] TLS Port for Minion [pinot]

2024-04-16 Thread via GitHub
suddendust commented on code in PR #12943: URL: https://github.com/apache/pinot/pull/12943#discussion_r1568292834 ## pinot-minion/src/main/java/org/apache/pinot/minion/BaseMinionStarter.java: ## @@ -116,6 +117,7 @@ public void init(PinotConfiguration config) _instanceId =

Re: [PR] Replace the usage of commons-collections to commons-collections4 [pinot]

2024-04-16 Thread via GitHub
xiangfu0 closed pull request #12711: Replace the usage of commons-collections to commons-collections4 URL: https://github.com/apache/pinot/pull/12711 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] TLS Port for Minion [pinot]

2024-04-16 Thread via GitHub
xiangfu0 commented on code in PR #12943: URL: https://github.com/apache/pinot/pull/12943#discussion_r1568285323 ## pinot-minion/src/main/java/org/apache/pinot/minion/BaseMinionStarter.java: ## @@ -116,6 +117,7 @@ public void init(PinotConfiguration config) _instanceId = C

[PR] Fix a typo when calculating query freshness [pinot]

2024-04-16 Thread via GitHub
Jackie-Jiang opened a new pull request, #12947: URL: https://github.com/apache/pinot/pull/12947 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Move PinotRelExchangeType back to the original package to fix backward compatibility [pinot]

2024-04-16 Thread via GitHub
Jackie-Jiang merged PR #12944: URL: https://github.com/apache/pinot/pull/12944 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated (1d807df401 -> 7dbc345929)

2024-04-16 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 1d807df401 Add validation check for forward index disabled if it's a REALTIME table (#12838) add 7dbc345929 Move

Re: [PR] Allowing users to pass minionInstanceTag as a param in /tasks/schedule API [pinot]

2024-04-16 Thread via GitHub
tibrewalpratik17 commented on PR #12786: URL: https://github.com/apache/pinot/pull/12786#issuecomment-2060387086 @siddharthteotia @vvivekiyer rebased with master. Sorry for the delay as I was on PTO. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] TLS Port for Minion [pinot]

2024-04-16 Thread via GitHub
codecov-commenter commented on PR #12943: URL: https://github.com/apache/pinot/pull/12943#issuecomment-2060314973 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12943?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] dynamic chunk sizing for v4 raw forward index [pinot]

2024-04-16 Thread via GitHub
itschrispeck commented on code in PR #12945: URL: https://github.com/apache/pinot/pull/12945#discussion_r1568163690 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/fwd/SingleValueVarByteRawIndexCreator.java: ## @@ -38,6 +38,7 @@ */ pub

Re: [PR] dynamic chunk sizing for v4 raw forward index [pinot]

2024-04-16 Thread via GitHub
itschrispeck commented on code in PR #12945: URL: https://github.com/apache/pinot/pull/12945#discussion_r1568163690 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/fwd/SingleValueVarByteRawIndexCreator.java: ## @@ -38,6 +38,7 @@ */ pub

Re: [PR] TLS Port for Minion [pinot]

2024-04-16 Thread via GitHub
xiangfu0 commented on code in PR #12943: URL: https://github.com/apache/pinot/pull/12943#discussion_r1568134164 ## pinot-minion/src/main/java/org/apache/pinot/minion/BaseMinionStarter.java: ## @@ -125,6 +127,20 @@ public void init(PinotConfiguration config) MinionEventObser

Re: [I] `createDictionaryForColumn` does not take into account inverted index [pinot]

2024-04-16 Thread via GitHub
deepthi-912 commented on issue #12635: URL: https://github.com/apache/pinot/issues/12635#issuecomment-2060229409 Sure, I will check it out. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] Support ORC 2.0+ [pinot]

2024-04-16 Thread via GitHub
abhioncbr commented on issue #12858: URL: https://github.com/apache/pinot/issues/12858#issuecomment-2060222463 orc version 2.0.0 drops support for java 8/11. For reference - https://github.com/apache/orc/releases/tag/v2.0.0, [ORC-1512](https://issues.apache.org/jira/browse/ORC-1512

Re: [PR] Enhance ProtoSerializationUtils to handle class move [pinot]

2024-04-16 Thread via GitHub
codecov-commenter commented on PR #12946: URL: https://github.com/apache/pinot/pull/12946#issuecomment-2060195130 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12946?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Enhance ProtoSerializationUtils to handle class move [pinot]

2024-04-16 Thread via GitHub
Jackie-Jiang opened a new pull request, #12946: URL: https://github.com/apache/pinot/pull/12946 With this change, we can handle class movement of regular classes and enums. For interface, we still need to rely on the full class name. This can prevent the problem introduced in #12837 --

Re: [PR] dynamic chunk sizing for v4 raw forward index [pinot]

2024-04-16 Thread via GitHub
codecov-commenter commented on PR #12945: URL: https://github.com/apache/pinot/pull/12945#issuecomment-2060112336 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12945?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

(pinot) branch master updated (67cb52c04b -> 1d807df401)

2024-04-16 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 67cb52c04b Refine SegmentFetcherFactory (#12936) add 1d807df401 Add validation check for forward index disabled if i

(pinot) branch add-validation-check-for-forward-index-disabled-when-enabling-columnar-segment-creation deleted (was 7dfd5c619e)

2024-04-16 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch add-validation-check-for-forward-index-disabled-when-enabling-columnar-segment-creation in repository https://gitbox.apache.org/repos/asf/pinot.git was 7dfd5c619e Add validation check for for

Re: [PR] Add validation check for forward index disabled when enabling columnar segment creation [pinot]

2024-04-16 Thread via GitHub
jackjlli merged PR #12838: URL: https://github.com/apache/pinot/pull/12838 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[PR] dynamic chunk sizing for v4 raw forward index [pinot]

2024-04-16 Thread via GitHub
itschrispeck opened a new pull request, #12945: URL: https://github.com/apache/pinot/pull/12945 **Background** - V4 format was introduced to better handle variable length data chunk size by reducing the potential for large allocations https://github.com/apache/pinot/pull/7661 - V3 fo

Re: [PR] Move PinotRelExchangeType back to the original package to fix backward compatibility [pinot]

2024-04-16 Thread via GitHub
codecov-commenter commented on PR #12944: URL: https://github.com/apache/pinot/pull/12944#issuecomment-2060023905 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12944?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [I] Enable consistentDataPush for Spark execution framework [pinot]

2024-04-16 Thread via GitHub
swaminathanmanish commented on issue #12941: URL: https://github.com/apache/pinot/issues/12941#issuecomment-2059995417 @Jackie-Jiang - Please assign this to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[PR] Move PinotRelExchangeType back to the original package to fix backward compatibility [pinot]

2024-04-16 Thread via GitHub
Jackie-Jiang opened a new pull request, #12944: URL: https://github.com/apache/pinot/pull/12944 For the reason described in #12934, we cannot move the class serialized in the query plan Revert the package move of `PinotRelExchangeType` in #12837 -- This is an automated message from the

Re: [I] `createDictionaryForColumn` does not take into account inverted index [pinot]

2024-04-16 Thread via GitHub
npawar commented on issue #12635: URL: https://github.com/apache/pinot/issues/12635#issuecomment-2059969650 @deepthi912 want to give this a shot? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] (WIP) Enable complexType handling in SegmentProcessFramework [pinot]

2024-04-16 Thread via GitHub
codecov-commenter commented on PR #12942: URL: https://github.com/apache/pinot/pull/12942#issuecomment-2059933408 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12942?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] TLS Port for Minion [pinot]

2024-04-16 Thread via GitHub
suddendust opened a new pull request, #12943: URL: https://github.com/apache/pinot/pull/12943 This PR adds a TLS port to minion, just like other components. With this, we can communicate with minion using `https`. In addition to this, we should add this config to minions: ``` pino

Re: [PR] Refine SegmentFetcherFactory [pinot]

2024-04-16 Thread via GitHub
Jackie-Jiang merged PR #12936: URL: https://github.com/apache/pinot/pull/12936 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: Refine SegmentFetcherFactory (#12936)

2024-04-16 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 67cb52c04b Refine SegmentFetcherFactory (#12936)

Re: [PR] Refine SegmentFetcherFactory [pinot]

2024-04-16 Thread via GitHub
Jackie-Jiang commented on code in PR #12936: URL: https://github.com/apache/pinot/pull/12936#discussion_r1567944026 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java: ## @@ -693,27 +693,22 @@ File downloadAndDecrypt(String segmentName, Seg

Re: [PR] Add Prefix, Suffix and Ngram UDFs [pinot]

2024-04-16 Thread via GitHub
deemoliu commented on code in PR #12392: URL: https://github.com/apache/pinot/pull/12392#discussion_r1567925821 ## pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java: ## @@ -581,6 +586,107 @@ public static String[] split(String input, String

Re: [PR] Add Prefix, Suffix and Ngram UDFs [pinot]

2024-04-16 Thread via GitHub
deemoliu commented on code in PR #12392: URL: https://github.com/apache/pinot/pull/12392#discussion_r1567926498 ## pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java: ## @@ -18,6 +18,10 @@ */ package org.apache.pinot.common.function.scalar

Re: [PR] Add Prefix, Suffix and Ngram UDFs [pinot]

2024-04-16 Thread via GitHub
deemoliu commented on code in PR #12392: URL: https://github.com/apache/pinot/pull/12392#discussion_r1567925396 ## pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java: ## @@ -570,6 +572,107 @@ public static String[] split(String input, String

Re: [PR] Add validation check for forward index disabled when enabling columnar segment creation [pinot]

2024-04-16 Thread via GitHub
jackjlli commented on code in PR #12838: URL: https://github.com/apache/pinot/pull/12838#discussion_r1567924010 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/TableConfigUtils.java: ## @@ -171,7 +171,8 @@ public static void validate(TableConfig tableCo

(pinot) branch add-validation-check-for-forward-index-disabled-when-enabling-columnar-segment-creation updated (75498ba651 -> 7dfd5c619e)

2024-04-16 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch add-validation-check-for-forward-index-disabled-when-enabling-columnar-segment-creation in repository https://gitbox.apache.org/repos/asf/pinot.git discard 75498ba651 Add validation check for for

Re: [PR] (WIP) Enable complexType handling in SegmentProcessFramework [pinot]

2024-04-16 Thread via GitHub
swaminathanmanish commented on PR #12942: URL: https://github.com/apache/pinot/pull/12942#issuecomment-2059895522 @klsince - Could you take a look ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] (WIP) Enable complexType handling in SegmentProcessFramework [pinot]

2024-04-16 Thread via GitHub
swaminathanmanish opened a new pull request, #12942: URL: https://github.com/apache/pinot/pull/12942 **Whats in the PR**: Enabling complexType (ingestion config) handling in SegmentProcessorFramework **Why its needed**: For batch/minion based ingestion SegmentProcessorFramework i

Re: [PR] Refine SegmentFetcherFactory [pinot]

2024-04-16 Thread via GitHub
klsince commented on code in PR #12936: URL: https://github.com/apache/pinot/pull/12936#discussion_r1567833872 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java: ## @@ -693,27 +693,22 @@ File downloadAndDecrypt(String segmentName, SegmentZ

Re: [PR] Fix metric rule pattern regex [pinot]

2024-04-16 Thread via GitHub
Jackie-Jiang merged PR #12856: URL: https://github.com/apache/pinot/pull/12856 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: Fix metric rule pattern regex (#12856)

2024-04-16 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new d4cb93dbda Fix metric rule pattern regex (#12856)

Re: [PR] Skip node level throttle during server bootstrap - Enhancement [pinot]

2024-04-16 Thread via GitHub
deepthi912 commented on PR #12930: URL: https://github.com/apache/pinot/pull/12930#issuecomment-2059586956 I totally missed enableThrottling flag. I can try adding more logs locally and see when the throttling is kicking in to identify the grey area if present. -- This is an automated mes

(pinot) branch master updated (9b4ec33d68 -> b264512c0c)

2024-04-16 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 9b4ec33d68 Bump org.scala-lang.modules:scala-xml_2.12 from 1.3.0 to 2.3.0 (#12939) add b264512c0c Bump org.codeha

Re: [PR] Bump org.codehaus.mojo:buildnumber-maven-plugin from 1.3 to 3.2.0 [pinot]

2024-04-16 Thread via GitHub
Jackie-Jiang merged PR #12937: URL: https://github.com/apache/pinot/pull/12937 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Bump org.scala-lang.modules:scala-xml_2.12 from 1.3.0 to 2.3.0 [pinot]

2024-04-16 Thread via GitHub
Jackie-Jiang merged PR #12939: URL: https://github.com/apache/pinot/pull/12939 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated (1393750622 -> 9b4ec33d68)

2024-04-16 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 1393750622 Allow Server throttling just before executing queries on server to allow max CPU and disk utilization (#129

(pinot) branch dependabot/maven/org.scala-lang.modules-scala-xml_2.12-2.3.0 deleted (was eaade464ee)

2024-04-16 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/org.scala-lang.modules-scala-xml_2.12-2.3.0 in repository https://gitbox.apache.org/repos/asf/pinot.git was eaade464ee Bump org.scala-lang.modules:scala-xml_2.12 from 1.3.0

Re: [PR] Skip node level throttle during server bootstrap - Enhancement [pinot]

2024-04-16 Thread via GitHub
npawar commented on PR #12930: URL: https://github.com/apache/pinot/pull/12930#issuecomment-2059500312 This doesn't seem to be the right fix.. If you look at line https://github.com/apache/pinot/pull/12930/files#diff-f99e2a5e74a22eaa5a052fce00e6b265418fb6b419969de16976bb405ec11a65R649, the

(pinot-site) branch new-site-dev updated: - Remove ReactPlayer, -Code Cleanup, - Remove testing code for debugging (#116)

2024-04-16 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch new-site-dev in repository https://gitbox.apache.org/repos/asf/pinot-site.git The following commit(s) were added to refs/heads/new-site-dev by this push: new 37980a50 - Remove ReactPlayer,

Re: [PR] - Remove ReactPlayer, -Code Cleanup, - Remove testing code for debugging [pinot-site]

2024-04-16 Thread via GitHub
xiangfu0 merged PR #116: URL: https://github.com/apache/pinot-site/pull/116 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

[PR] - Remove ReactPlayer, -Code Cleanup, - Remove testing code for debugging [pinot-site]

2024-04-16 Thread via GitHub
gio-startree opened a new pull request, #116: URL: https://github.com/apache/pinot-site/pull/116 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [I] Proposal to Harmonize Pinot Dependency Management [pinot]

2024-04-16 Thread via GitHub
gortiz commented on issue #12676: URL: https://github.com/apache/pinot/issues/12676#issuecomment-2059105994 > FYI @gortiz I attempted to incorporate your BOM subproject idea and import it from the Pinot POM, but that is explicitly disallowed by [Maven's dependency management](https://maven.

Re: [PR] Metric for count of tables configured with various tier backends [pinot]

2024-04-16 Thread via GitHub
codecov-commenter commented on PR #12940: URL: https://github.com/apache/pinot/pull/12940#issuecomment-2059016175 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12940?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Metric for count of tables configured with various tier backends [pinot]

2024-04-16 Thread via GitHub
shounakmk219 opened a new pull request, #12940: URL: https://github.com/apache/pinot/pull/12940 Metric for - # of tables using each of the tier backend : `pinot.controller.XYZtierBackendTableCount` - total # of tables having tier backend configured : `pinot.controller.tierBackend

(pinot) branch dependabot/maven/org.scala-lang.modules-scala-xml_2.12-2.3.0 created (now eaade464ee)

2024-04-16 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.scala-lang.modules-scala-xml_2.12-2.3.0 in repository https://gitbox.apache.org/repos/asf/pinot.git at eaade464ee Bump org.scala-lang.modules:scala-xml_2.12 from 1

[PR] Bump org.scala-lang.modules:scala-xml_2.12 from 1.3.0 to 2.3.0 [pinot]

2024-04-16 Thread via GitHub
dependabot[bot] opened a new pull request, #12939: URL: https://github.com/apache/pinot/pull/12939 Bumps [org.scala-lang.modules:scala-xml_2.12](https://github.com/scala/scala-xml) from 1.3.0 to 2.3.0. Release notes Sourced from https://github.com/scala/scala-xml/releases";>org.sc

Re: [I] Support API for checking if segments need to be reloaded for a table [pinot]

2024-04-16 Thread via GitHub
nitin-startree commented on issue #12117: URL: https://github.com/apache/pinot/issues/12117#issuecomment-2058802380 @Jackie-Jiang - Could you provide an estimated timeline or any updates regarding this Feature request? I'm currently reviewing all tickets labelled as "Waiting on Feature" to

(pinot-site) branch new-static-prod updated: Update .htaccess

2024-04-16 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch new-static-prod in repository https://gitbox.apache.org/repos/asf/pinot-site.git The following commit(s) were added to refs/heads/new-static-prod by this push: new 54094b76 Update .htaccess

Re: [PR] Multi stage stats [pinot]

2024-04-16 Thread via GitHub
gortiz commented on code in PR #12704: URL: https://github.com/apache/pinot/pull/12704#discussion_r1566907287 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/plan/MultiStageQueryStats.java: ## @@ -0,0 +1,668 @@ +/** + * Licensed to the Apache Software Foundat

Re: [PR] Multi stage stats [pinot]

2024-04-16 Thread via GitHub
gortiz commented on code in PR #12704: URL: https://github.com/apache/pinot/pull/12704#discussion_r1566905246 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/MetadataBlock.java: ## @@ -36,108 +39,129 @@ */ public class MetadataBlock extends BaseDataBlock { -