Re: [PR] [null-aggr] Add null handling support in `mode` aggregation [pinot]

2024-02-29 Thread via GitHub
gortiz commented on code in PR #12227: URL: https://github.com/apache/pinot/pull/12227#discussion_r1507160348 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/ModeAggregationFunction.java: ## @@ -467,7 +501,11 @@ public ColumnDataType getFinalResultCo

Re: [PR] [null-aggr] Add null handling support in `mode` aggregation [pinot]

2024-02-29 Thread via GitHub
gortiz commented on code in PR #12227: URL: https://github.com/apache/pinot/pull/12227#discussion_r1507164446 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/ModeAggregationFunction.java: ## @@ -467,7 +501,11 @@ public ColumnDataType getFinalResultCo

Re: [PR] [Multi-stage] Reduce the stats transferred [pinot]

2024-02-29 Thread via GitHub
Jackie-Jiang merged PR #12517: URL: https://github.com/apache/pinot/pull/12517 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: [Multi-stage] Reduce the stats transfered (#12517)

2024-02-29 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 35c89c87a5 [Multi-stage] Reduce the stats transfe

Re: [PR] [null-aggr] Add null handling support in `mode` aggregation [pinot]

2024-02-29 Thread via GitHub
gortiz commented on code in PR #12227: URL: https://github.com/apache/pinot/pull/12227#discussion_r1507172781 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/NullableSingleInputAggregationFunction.java: ## @@ -0,0 +1,263 @@ +/** + * Licensed to the A

Re: [PR] [null-aggr] Add null handling support in `mode` aggregation [pinot]

2024-02-29 Thread via GitHub
gortiz commented on code in PR #12227: URL: https://github.com/apache/pinot/pull/12227#discussion_r1507175271 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/NullableSingleInputAggregationFunction.java: ## @@ -0,0 +1,263 @@ +/** + * Licensed to the A

Re: [PR] Add a post-validator visitor that verifies there are no cast to bytes [pinot]

2024-02-29 Thread via GitHub
gortiz commented on code in PR #12475: URL: https://github.com/apache/pinot/pull/12475#discussion_r1507181049 ## pinot-query-planner/src/main/java/org/apache/pinot/query/validate/BytesCastVisitor.java: ## @@ -0,0 +1,73 @@ +/** + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [null-aggr] Add null handling support in `mode` aggregation [pinot]

2024-02-29 Thread via GitHub
gortiz commented on code in PR #12227: URL: https://github.com/apache/pinot/pull/12227#discussion_r1507178148 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/FieldConfig.java: ## @@ -254,6 +256,24 @@ public Builder withTierOverwrites(JsonNode tierOverwrites) {

Re: [PR] Add a post-validator visitor that verifies there are no cast to bytes [pinot]

2024-02-29 Thread via GitHub
gortiz commented on code in PR #12475: URL: https://github.com/apache/pinot/pull/12475#discussion_r1507187948 ## pinot-query-planner/src/main/java/org/apache/pinot/query/validate/BytesCastVisitor.java: ## @@ -0,0 +1,73 @@ +/** + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Add a post-validator visitor that verifies there are no cast to bytes [pinot]

2024-02-29 Thread via GitHub
gortiz commented on code in PR #12475: URL: https://github.com/apache/pinot/pull/12475#discussion_r1507194978 ## pinot-query-planner/src/main/java/org/apache/pinot/query/validate/BytesCastVisitor.java: ## @@ -0,0 +1,73 @@ +/** + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Record enricher [pinot]

2024-02-29 Thread via GitHub
gortiz commented on PR #12243: URL: https://github.com/apache/pinot/pull/12243#issuecomment-1970696484 Do you suggest to rollback the change and modify `TransformPipeline` to support the same semantics defined in `ExpressionTransformer`? We are still in time to do that. -- This is an aut

Re: [PR] Attempt to apply timestamp expr override in V2 [pinot]

2024-02-29 Thread via GitHub
gortiz closed pull request #12421: Attempt to apply timestamp expr override in V2 URL: https://github.com/apache/pinot/pull/12421 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] [draft] Null mode at query time [pinot]

2024-02-29 Thread via GitHub
gortiz closed pull request #11935: [draft] Null mode at query time URL: https://github.com/apache/pinot/pull/11935 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

[PR] Metrics for Realtime Rows Fetched and Stream Consumer Create Exceptions [pinot]

2024-02-29 Thread via GitHub
suddendust opened a new pull request, #12522: URL: https://github.com/apache/pinot/pull/12522 This PR adds two metrics in `RealtimeSegmentDataManager`: 1. `REALTIME_LAST_FETCHED_BATCH_SIZE`: This is a new gauge that can assume the following values: - 0: When there were no messages

Re: [PR] Metrics for Realtime Rows Fetched and Stream Consumer Create Exceptions [pinot]

2024-02-29 Thread via GitHub
codecov-commenter commented on PR #12522: URL: https://github.com/apache/pinot/pull/12522#issuecomment-1970764635 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12522?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [I] Timeout expired while fetching topic metadata [pinot]

2024-02-29 Thread via GitHub
sandeepjain98 commented on issue #8396: URL: https://github.com/apache/pinot/issues/8396#issuecomment-1970777144 Hi @xiangfu0 , is it solved . even I am facing same issue. " Caused by: org.apache.pinot.shaded.org.apache.kafka.common.errors.TimeoutException: Timeout expired while

[PR] Modify consumingSegmentsInfo endpoint to indicate how many servers failed [pinot]

2024-02-29 Thread via GitHub
gortiz opened a new pull request, #12523: URL: https://github.com/apache/pinot/pull/12523 Controller endpoint /table/consumingSegmentsInfo may return partial information when servers timeout or there is a parsing error dealing with their response. These cases registered as a log, but there

Re: [PR] Metrics for Realtime Rows Fetched and Stream Consumer Create Exceptions [pinot]

2024-02-29 Thread via GitHub
gortiz commented on PR #12522: URL: https://github.com/apache/pinot/pull/12522#issuecomment-1970846422 I like the idea, but I'm not sure if the current implementation is correct. Specifically, the scenario that concerns me is the one where: - we fetch batches far often than the Prometheus

Re: [I] Configurable heap size for query and realtime segment persistence. [pinot]

2024-02-29 Thread via GitHub
Lvnszn commented on issue #12503: URL: https://github.com/apache/pinot/issues/12503#issuecomment-1970895940 > Do you have access to the server node? K8s log might not be enough because it won't log the actual stack trace, and it doesn't contain the non-error logs. The log has been rec

Re: [PR] Metrics for Realtime Rows Fetched and Stream Consumer Create Exceptions [pinot]

2024-02-29 Thread via GitHub
suddendust commented on PR #12522: URL: https://github.com/apache/pinot/pull/12522#issuecomment-1970950356 That's a valid case. >If it doesn't increase in a while, we can be sure that there are no data in the source and therefore we can skip some alerts. We can't differentiate

Re: [PR] adding support of mega_bytes when configuring broker response size [pinot]

2024-02-29 Thread via GitHub
gortiz commented on PR #12510: URL: https://github.com/apache/pinot/pull/12510#issuecomment-1970892525 +1 to Jackie suggestion -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Consistency in API response for live broker [pinot]

2024-02-29 Thread via GitHub
cypherean commented on PR #12201: URL: https://github.com/apache/pinot/pull/12201#issuecomment-1971105562 @Jackie-Jiang @walterddr please take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Metrics for Realtime Rows Fetched and Stream Consumer Create Exceptions [pinot]

2024-02-29 Thread via GitHub
ege-st commented on PR #12522: URL: https://github.com/apache/pinot/pull/12522#issuecomment-1971166374 This seems like a situation where a histogram would be right choice: it would let us do percentile breakdowns of rows read and I believe it also includes the counter that @gortiz suggested

[PR] Pass explicit TypeRef when evaluating MV jsonPath [pinot]

2024-02-29 Thread via GitHub
saurabhd336 opened a new pull request, #12524: URL: https://github.com/apache/pinot/pull/12524 For a json column `myJsonCol` with values like ``` {"values": [1, 2, 3, 4, 5]} ``` ``` select summv(json_extract_scalar(myJsonCol, '$.values[0:3]', 'INT_ARRAY')) from wxcc_agent_

Re: [PR] Metrics for Realtime Rows Fetched and Stream Consumer Create Exceptions [pinot]

2024-02-29 Thread via GitHub
gortiz commented on PR #12522: URL: https://github.com/apache/pinot/pull/12522#issuecomment-1971375101 Small tip, instead of rate you can also use [increase](https://prometheus.io/docs/prometheus/latest/querying/functions/#increase). I don't think the difference in performance would be impo

Re: [PR] Metrics for Realtime Rows Fetched and Stream Consumer Create Exceptions [pinot]

2024-02-29 Thread via GitHub
gortiz commented on PR #12522: URL: https://github.com/apache/pinot/pull/12522#issuecomment-1971379244 LGTM. Unless there is some urgency, I'll wait til tomorrow to merge it just in case other people want to review it. -- This is an automated message from the Apache Git Service. To respon

Re: [PR] Pass explicit TypeRef when evaluating MV jsonPath [pinot]

2024-02-29 Thread via GitHub
codecov-commenter commented on PR #12524: URL: https://github.com/apache/pinot/pull/12524#issuecomment-1971383094 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12524?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] Pass explicit TypeRef when evaluating MV jsonPath [pinot]

2024-02-29 Thread via GitHub
saurabhd336 merged PR #12524: URL: https://github.com/apache/pinot/pull/12524 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.

(pinot) branch master updated: Pass explicit TypeRef when evaluating MV jsonPath (#12524)

2024-02-29 Thread saurabhd336
This is an automated email from the ASF dual-hosted git repository. saurabhd336 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new bddd3612bc Pass explicit TypeRef when evalua

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-29 Thread via GitHub
suddendust commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1507874579 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +258,71 @@ public SuccessResponse putData( }

(pinot) branch release-1.1.0-rc updated (7cec963a88 -> ce94496e2e)

2024-02-29 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a change to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git discard 7cec963a88 [maven-release-plugin] prepare for next development iteration omit 17e33d6cbf [maven-rele

Re: [PR] Metrics for Realtime Rows Fetched and Stream Consumer Create Exceptions [pinot]

2024-02-29 Thread via GitHub
zhtaoxiang commented on code in PR #12522: URL: https://github.com/apache/pinot/pull/12522#discussion_r1507951724 ## pinot-common/src/main/java/org/apache/pinot/common/metrics/ServerMeter.java: ## @@ -35,12 +35,14 @@ public enum ServerMeter implements AbstractMetrics.Meter {

(pinot) branch release-1.1.0-rc updated: [maven-release-plugin] prepare release release-1.1.0-rc0

2024-02-29 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a commit to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-1.1.0-rc by this push: new 3bae032004 [maven-release

Re: [PR] Metrics for Realtime Rows Fetched and Stream Consumer Create Exceptions [pinot]

2024-02-29 Thread via GitHub
suddendust commented on code in PR #12522: URL: https://github.com/apache/pinot/pull/12522#discussion_r1507986026 ## pinot-common/src/main/java/org/apache/pinot/common/metrics/ServerMeter.java: ## @@ -35,12 +35,14 @@ public enum ServerMeter implements AbstractMetrics.Meter {

Re: [PR] Metrics for Realtime Rows Fetched and Stream Consumer Create Exceptions [pinot]

2024-02-29 Thread via GitHub
zhtaoxiang commented on code in PR #12522: URL: https://github.com/apache/pinot/pull/12522#discussion_r1508002971 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java: ## @@ -1661,22 +1666,35 @@ private void makeStreamConsumer(S

Re: [PR] Metrics for Realtime Rows Fetched and Stream Consumer Create Exceptions [pinot]

2024-02-29 Thread via GitHub
suddendust commented on code in PR #12522: URL: https://github.com/apache/pinot/pull/12522#discussion_r1508009397 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java: ## @@ -1661,22 +1666,35 @@ private void makeStreamConsumer(S

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-29 Thread via GitHub
Jackie-Jiang merged PR #12497: URL: https://github.com/apache/pinot/pull/12497 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: Rest Endpoint to Create ZNode (#12497)

2024-02-29 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 2249be357b Rest Endpoint to Create ZNode (#12497)

(pinot) branch release-1.1.0-rc updated (3bae032004 -> ce94496e2e)

2024-02-29 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a change to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git discard 3bae032004 [maven-release-plugin] prepare release release-1.1.0-rc0 This update removed existing revisio

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-29 Thread via GitHub
Jackie-Jiang commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1508023342 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +258,67 @@ public SuccessResponse putData( }

(pinot) annotated tag release-1.1.0-rc0 deleted (was d236eef9b2)

2024-02-29 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a change to annotated tag release-1.1.0-rc0 in repository https://gitbox.apache.org/repos/asf/pinot.git *** WARNING: tag release-1.1.0-rc0 was deleted! *** tag was d236eef9b2 The revisions that were on th

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-02-29 Thread via GitHub
Jackie-Jiang commented on code in PR #12417: URL: https://github.com/apache/pinot/pull/12417#discussion_r1508053974 ## pinot-broker/src/main/java/org/apache/pinot/broker/api/resources/PinotBrokerRouting.java: ## @@ -61,8 +69,23 @@ public class PinotBrokerRouting { @ApiRes

Re: [PR] Adding record reader config/context param to record transformer [pinot]

2024-02-29 Thread via GitHub
klsince commented on code in PR #12520: URL: https://github.com/apache/pinot/pull/12520#discussion_r1508078762 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/RecordTransformer.java: ## @@ -43,4 +44,15 @@ default boolean isNoOp() { */

Re: [PR] Adding record reader config/context param to record transformer [pinot]

2024-02-29 Thread via GitHub
swaminathanmanish commented on code in PR #12520: URL: https://github.com/apache/pinot/pull/12520#discussion_r1508091484 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/RecordTransformer.java: ## @@ -43,4 +44,15 @@ default boolean isNoOp() {

(pinot) branch release-1.1.0-rc updated: [maven-release-plugin] prepare release release-1.1.0-rc0

2024-02-29 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a commit to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-1.1.0-rc by this push: new 56fc5a6dfc [maven-release

(pinot) annotated tag release-1.1.0-rc0 updated (56fc5a6dfc -> 6ad905acf2)

2024-02-29 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a change to annotated tag release-1.1.0-rc0 in repository https://gitbox.apache.org/repos/asf/pinot.git *** WARNING: tag release-1.1.0-rc0 was modified! *** from 56fc5a6dfc (commit) to 6ad905acf2 (ta

(pinot) branch release-1.1.0-rc updated: [maven-release-plugin] prepare for next development iteration

2024-02-29 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a commit to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-1.1.0-rc by this push: new e75703bda3 [maven-release

Re: [PR] Adding record reader config/context param to record transformer [pinot]

2024-02-29 Thread via GitHub
klsince commented on code in PR #12520: URL: https://github.com/apache/pinot/pull/12520#discussion_r1508106925 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/RecordTransformer.java: ## @@ -43,4 +43,15 @@ default boolean isNoOp() { */

Re: [PR] Adding record reader config/context param to record transformer [pinot]

2024-02-29 Thread via GitHub
klsince merged PR #12520: URL: https://github.com/apache/pinot/pull/12520 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

(pinot) branch master updated (2249be357b -> f0fcbd8701)

2024-02-29 Thread xbli
This is an automated email from the ASF dual-hosted git repository. xbli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 2249be357b Rest Endpoint to Create ZNode (#12497) add f0fcbd8701 Adding record reader config/context param to recor

Re: [PR] adding support of mega_bytes when configuring broker response size [pinot]

2024-02-29 Thread via GitHub
Zhenyun20023 commented on PR #12510: URL: https://github.com/apache/pinot/pull/12510#issuecomment-1972037243 @Jackie-Jiang , I made changes per your suggestions. Can you take a look? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Modify consumingSegmentsInfo endpoint to indicate how many servers failed [pinot]

2024-02-29 Thread via GitHub
codecov-commenter commented on PR #12523: URL: https://github.com/apache/pinot/pull/12523#issuecomment-1972090121 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12523?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [I] Provide pinot schema when initializing StreamMessageDecoder [pinot]

2024-02-29 Thread via GitHub
Jackie-Jiang commented on issue #12521: URL: https://github.com/apache/pinot/issues/12521#issuecomment-1972111556 Yes. I think we can modify the javadoc, and it should apply to all decoders. It doesn't make sense to have a decoder that decodes nothing -- This is an automated message from

[PR] add insecure mode when Pinot uses TLS connections [pinot]

2024-02-29 Thread via GitHub
zhtaoxiang opened a new pull request, #12525: URL: https://github.com/apache/pinot/pull/12525 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

[PR] Revert " Adding record reader config/context param to record transformer (#12520)" [pinot]

2024-02-29 Thread via GitHub
klsince opened a new pull request, #12526: URL: https://github.com/apache/pinot/pull/12526 This reverts commit f0fcbd8701e025495a136e644262c83607e1eaa7. The interface RecordRecordConfig was mistaken as the TransformerContext, that's supposed to be added and passed to transform() to ac

Re: [PR] Revert " Adding record reader config/context param to record transformer (#12520)" [pinot]

2024-02-29 Thread via GitHub
Jackie-Jiang merged PR #12526: URL: https://github.com/apache/pinot/pull/12526 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated (f0fcbd8701 -> 8972bc6066)

2024-02-29 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from f0fcbd8701 Adding record reader config/context param to record transformer (#12520) add 8972bc6066 Revert " Addi

Re: [PR] [null-aggr] Add null handling support in `mode` aggregation [pinot]

2024-02-29 Thread via GitHub
Jackie-Jiang commented on code in PR #12227: URL: https://github.com/apache/pinot/pull/12227#discussion_r1508308471 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/ModeAggregationFunction.java: ## @@ -263,11 +264,14 @@ public void aggregate(int lengt

Re: [PR] Add a post-validator visitor that verifies there are no cast to bytes [pinot]

2024-02-29 Thread via GitHub
Jackie-Jiang commented on code in PR #12475: URL: https://github.com/apache/pinot/pull/12475#discussion_r1508311185 ## pinot-query-planner/src/main/java/org/apache/pinot/query/validate/BytesCastVisitor.java: ## @@ -0,0 +1,73 @@ +/** + * Licensed to the Apache Software Foundation

Re: [PR] Add a post-validator visitor that verifies there are no cast to bytes [pinot]

2024-02-29 Thread via GitHub
Jackie-Jiang commented on code in PR #12475: URL: https://github.com/apache/pinot/pull/12475#discussion_r1508312558 ## pinot-query-planner/src/main/java/org/apache/pinot/query/validate/BytesCastVisitor.java: ## @@ -0,0 +1,73 @@ +/** + * Licensed to the Apache Software Foundation

Re: [PR] Record enricher [pinot]

2024-02-29 Thread via GitHub
Jackie-Jiang commented on PR #12243: URL: https://github.com/apache/pinot/pull/12243#issuecomment-1972155063 Yes. I prefer keeping one single interface for enrichment/transformation on records, unless there are fundamental differences between them. @saurabhd336 what do you think? -- This

Re: [PR] Egalpin/skip indexes minor changes [pinot]

2024-02-29 Thread via GitHub
egalpin commented on PR #12514: URL: https://github.com/apache/pinot/pull/12514#issuecomment-1972157886 @Jackie-Jiang all set, I believe the first tests that failed were due to noisy remote test environment. I tried to make a change to account for the failures in https://github.com/apache

Re: [PR] add insecure mode when Pinot uses TLS connections [pinot]

2024-02-29 Thread via GitHub
codecov-commenter commented on PR #12525: URL: https://github.com/apache/pinot/pull/12525#issuecomment-1972164101 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12525?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [I] Auto-tuning Pinot real-time segment size based on actual stream data consumption [pinot]

2024-02-29 Thread via GitHub
chenboat commented on issue #12513: URL: https://github.com/apache/pinot/issues/12513#issuecomment-1972172243 My proposal is to use number of *bytes* consumed in the previous segment instead of the number of *rows* consumed to determine the current segment consumption target. In part

Re: [PR] Add minionInstanceTag config to minion-tasks for resource isolation [pinot]

2024-02-29 Thread via GitHub
Jackie-Jiang commented on code in PR #12459: URL: https://github.com/apache/pinot/pull/12459#discussion_r1508329145 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotTaskManager.java: ## @@ -534,66 +535,88 @@ private synchronized Map scheduleT

Re: [I] Configurable heap size for query and realtime segment persistence. [pinot]

2024-02-29 Thread via GitHub
Jackie-Jiang commented on issue #12503: URL: https://github.com/apache/pinot/issues/12503#issuecomment-1972254102 Is the problem resolved automatically? We'll need more context in order to debug it -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [PR] Consistency in API response for live broker [pinot]

2024-02-29 Thread via GitHub
Jackie-Jiang commented on code in PR #12201: URL: https://github.com/apache/pinot/pull/12201#discussion_r1508366878 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java: ## @@ -3921,7 +3921,16 @@ public TableStats getTableStats(

Re: [PR] adding support of mega_bytes when configuring broker response size [pinot]

2024-02-29 Thread via GitHub
Jackie-Jiang commented on code in PR #12510: URL: https://github.com/apache/pinot/pull/12510#discussion_r1508376156 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java: ## @@ -1773,17 +1774,23 @@ private void setMaxServerResponseSiz

Re: [PR] Added /uptime and /start-time for all components. Addressed comments and fixed tests from pull request 12389. [pinot]

2024-02-29 Thread via GitHub
suyashpatel98 commented on PR #12512: URL: https://github.com/apache/pinot/pull/12512#issuecomment-1972399286 > Do we want uptime in seconds, or in a more human readable format? @Jackie-Jiang Xiang in the old PR [here](https://github.com/apache/pinot/pull/12389) was inclined towards

[PR] enable Netty native transports in ClusterTest [pinot]

2024-02-29 Thread via GitHub
sullis opened a new pull request, #12527: URL: https://github.com/apache/pinot/pull/12527 Motiviation: by default NettyConfig does not use native transports. Modifications: - update Netty properties in ClusterTest.java Result: This change enables Netty nativ

[PR] maven: no transfer progress [pinot]

2024-02-29 Thread via GitHub
sullis opened a new pull request, #12528: URL: https://github.com/apache/pinot/pull/12528 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

Re: [PR] enable Netty native transports in ClusterTest [pinot]

2024-02-29 Thread via GitHub
codecov-commenter commented on PR #12527: URL: https://github.com/apache/pinot/pull/12527#issuecomment-1972467793 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12527?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] maven no transfer progress [pinot]

2024-02-29 Thread via GitHub
codecov-commenter commented on PR #12528: URL: https://github.com/apache/pinot/pull/12528#issuecomment-1972468907 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12528?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[PR] Allow passing custom record reader to be inited/closed in SegmentProcessorFramework [pinot]

2024-02-29 Thread via GitHub
swaminathanmanish opened a new pull request, #12529: URL: https://github.com/apache/pinot/pull/12529 **Whats in the PR:** Allow custom records to be inited/closed i.e managed within SegmentProcessorFramework **Why its needed:** We have custom readers (eg: where record needs to be en

Re: [PR] Allow passing custom record reader to be inited/closed in SegmentProcessorFramework [pinot]

2024-02-29 Thread via GitHub
swaminathanmanish commented on PR #12529: URL: https://github.com/apache/pinot/pull/12529#issuecomment-1972548026 @snleee - Please take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] [partial-upsert] Bug in Handling Comparison Column Ties When Replacing Segment [pinot]

2024-02-29 Thread via GitHub
ankitsultana closed issue #12398: [partial-upsert] Bug in Handling Comparison Column Ties When Replacing Segment URL: https://github.com/apache/pinot/issues/12398 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [I] [partial-upsert] Bug in Handling Comparison Column Ties When Replacing Segment [pinot]

2024-02-29 Thread via GitHub
ankitsultana commented on issue #12398: URL: https://github.com/apache/pinot/issues/12398#issuecomment-1972554920 Closing this since this was resolved by #12395. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Allow passing custom record reader to be inited/closed in SegmentProcessorFramework [pinot]

2024-02-29 Thread via GitHub
codecov-commenter commented on PR #12529: URL: https://github.com/apache/pinot/pull/12529#issuecomment-1972579030 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12529?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-02-29 Thread via GitHub
shounakmk219 commented on PR #12417: URL: https://github.com/apache/pinot/pull/12417#issuecomment-1972579675 @Jackie-Jiang I have reverted the v2 APIs and the request filter approach to achieve table translation. We can talk around what's the best way to handle the translation separately in

[PR] Prepare for next development iteration [pinot]

2024-02-29 Thread via GitHub
vvivekiyer opened a new pull request, #12530: URL: https://github.com/apache/pinot/pull/12530 Updating version for next release development iteration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] reformat ClusterTest.java [pinot]

2024-02-29 Thread via GitHub
sullis opened a new pull request, #12531: URL: https://github.com/apache/pinot/pull/12531 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

(pinot) annotated tag release-1.1.0-rc0 deleted (was 6ad905acf2)

2024-02-29 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a change to annotated tag release-1.1.0-rc0 in repository https://gitbox.apache.org/repos/asf/pinot.git *** WARNING: tag release-1.1.0-rc0 was deleted! *** tag was 6ad905acf2 The revisions that were on th

(pinot) branch release-1.1.0-rc updated (e75703bda3 -> ce94496e2e)

2024-02-29 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a change to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git discard e75703bda3 [maven-release-plugin] prepare for next development iteration discard 56fc5a6dfc [maven-rele

Re: [PR] Prepare for next development iteration [pinot]

2024-02-29 Thread via GitHub
codecov-commenter commented on PR #12530: URL: https://github.com/apache/pinot/pull/12530#issuecomment-1972656712 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12530?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] Add the support to extract json array elements from json index for the transform function jsonExtractIndex [pinot]

2024-02-29 Thread via GitHub
itschrispeck commented on code in PR #12466: URL: https://github.com/apache/pinot/pull/12466#discussion_r1508495654 ## pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/JsonExtractIndexTransformFunction.java: ## @@ -90,9 +90,13 @@ public void init(List a

Re: [PR] reformat ClusterTest.java [pinot]

2024-02-29 Thread via GitHub
codecov-commenter commented on PR #12531: URL: https://github.com/apache/pinot/pull/12531#issuecomment-1972679095 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12531?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

(pinot) branch release-1.1.0-rc updated: [maven-release-plugin] prepare release release-1.1.0-rc0

2024-02-29 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a commit to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-1.1.0-rc by this push: new c72b56a55c [maven-release

(pinot) branch release-1.1.0-rc updated (c72b56a55c -> ce94496e2e)

2024-02-29 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a change to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git discard c72b56a55c [maven-release-plugin] prepare release release-1.1.0-rc0 This update removed existing revisio