Re: [PR] [null-aggr] Add null handling support in `mode` aggregation [pinot]

2024-02-28 Thread via GitHub
gortiz commented on PR #12227: URL: https://github.com/apache/pinot/pull/12227#issuecomment-1970595131 > Mostly good. I saw you added mode into the AllNullQueriesTest, but somehow I didn't find where the null intermediate result is handled The change I made there was to create one tes

Re: [PR] (WIP...) Adding record reader config/context param to record transformer [pinot]

2024-02-28 Thread via GitHub
codecov-commenter commented on PR #12520: URL: https://github.com/apache/pinot/pull/12520#issuecomment-1970582482 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12520?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

(pinot) branch release-1.1.0-rc updated: [maven-release-plugin] prepare for next development iteration

2024-02-28 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a commit to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-1.1.0-rc by this push: new 7cec963a88 [maven-release

(pinot) annotated tag release-1.1.0-rc updated (17e33d6cbf -> b074f361cc)

2024-02-28 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a change to annotated tag release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git *** WARNING: tag release-1.1.0-rc was modified! *** from 17e33d6cbf (commit) to b074f361cc (tag)

(pinot) branch release-1.1.0-rc updated: [maven-release-plugin] prepare release release-1.1.0-rc

2024-02-28 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a commit to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-1.1.0-rc by this push: new 17e33d6cbf [maven-release

Re: [I] Provide pinot schema when initializing StreamMessageDecoder [pinot]

2024-02-28 Thread via GitHub
rseetham commented on issue #12521: URL: https://github.com/apache/pinot/issues/12521#issuecomment-1970554202 For clarification, > @param fieldsToRead The fields to read from the source stream. If blank, reads all fields (only for AVRO/JSON currently) So o

Re: [I] Provide pinot schema when initializing StreamMessageDecoder [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on issue #12521: URL: https://github.com/apache/pinot/issues/12521#issuecomment-1970547053 cc @snleee @swaminathanmanish -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] update ControllerJobType from enum to string [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang merged PR #12518: URL: https://github.com/apache/pinot/pull/12518 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: update ControllerJobType from enum to string (#12518)

2024-02-28 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 704f73d4f9 update ControllerJobType from enum to

[I] Provide pinot schema when initializing StreamMessageDecoder [pinot]

2024-02-28 Thread via GitHub
rseetham opened a new issue, #12521: URL: https://github.com/apache/pinot/issues/12521 [StreamMessageDecoder's](https://github.com/apache/pinot/blob/ac13a191b945a80084f0a2794391e4be2f463252/pinot-spi/src/main/java/org/apache/pinot/spi/stream/StreamMessageDecoder.java#L49) init is `void i

Re: [PR] update ControllerJobType from enum to string [pinot]

2024-02-28 Thread via GitHub
codecov-commenter commented on PR #12518: URL: https://github.com/apache/pinot/pull/12518#issuecomment-1970526687 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12518?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] (WIP...) Adding record reader config/context param to record transformer [pinot]

2024-02-28 Thread via GitHub
swaminathanmanish commented on PR #12520: URL: https://github.com/apache/pinot/pull/12520#issuecomment-1970507677 @snleee , @klsince - Based on our discussion, please take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

(pinot) branch release-1.1.0-rc updated (b9216f33b3 -> ce94496e2e)

2024-02-28 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a change to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git discard b9216f33b3 [maven-release-plugin] prepare for next development iteration This update removed existing r

[PR] (WIP...) Adding record reader config/context param to record transformer [pinot]

2024-02-28 Thread via GitHub
swaminathanmanish opened a new pull request, #12520: URL: https://github.com/apache/pinot/pull/12520 **Whats in the PR:** Adding Record reader config/context to transform api, for transformer to use it during transformation of record. **Why its needed:** Custom transformers need to

(pinot) branch release-1.1.0-rc updated: [maven-release-plugin] prepare for next development iteration

2024-02-28 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a commit to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-1.1.0-rc by this push: new b9216f33b3 [maven-release

(pinot) annotated tag release-1.1.0-rc0 updated (ce94496e2e -> d236eef9b2)

2024-02-28 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a change to annotated tag release-1.1.0-rc0 in repository https://gitbox.apache.org/repos/asf/pinot.git *** WARNING: tag release-1.1.0-rc0 was modified! *** from ce94496e2e (commit) to d236eef9b2 (ta

Re: [PR] [Multi-stage] Reduce the stats transferred [pinot]

2024-02-28 Thread via GitHub
codecov-commenter commented on PR #12517: URL: https://github.com/apache/pinot/pull/12517#issuecomment-1970501009 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12517?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] Propagate Disable User Agent Config to Http Client [pinot]

2024-02-28 Thread via GitHub
satishd commented on PR #12479: URL: https://github.com/apache/pinot/pull/12479#issuecomment-1970493679 Created a followup issue https://github.com/apache/pinot/issues/12519 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[PR] update ControllerJobType from enum to string [pinot]

2024-02-28 Thread via GitHub
zhtaoxiang opened a new pull request, #12518: URL: https://github.com/apache/pinot/pull/12518 update ControllerJobType from enum to string to make it flexible to extend -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

(pinot) branch release-1.1.0-rc updated (56c82a36b8 -> ce94496e2e)

2024-02-28 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a change to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git discard 56c82a36b8 [maven-release-plugin] prepare release release-1.1.0-rc0 This update removed existing revisio

(pinot) branch release-1.1.0-rc updated: [maven-release-plugin] prepare release release-1.1.0-rc0

2024-02-28 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a commit to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-1.1.0-rc by this push: new 56c82a36b8 [maven-release

[PR] [Multi-stage] Reduce the stats transferred [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang opened a new pull request, #12517: URL: https://github.com/apache/pinot/pull/12517 Partially solve #12426 - Send metadata only to one mailbox as all the metadata will be aggregated at the root level - When trace is disabled, do not collect stage stats except for the l

Re: [I] Http Response Headers for Error Codes [pinot]

2024-02-28 Thread via GitHub
ankitsultana closed issue #12190: Http Response Headers for Error Codes URL: https://github.com/apache/pinot/issues/12190 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

(pinot) branch release-1.1.0-rc updated: Update license & notice for release 1.1.0

2024-02-28 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a commit to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-1.1.0-rc by this push: new ce94496e2e Update license

(pinot) branch master updated: Added pinot-error-code header in query response (#12338)

2024-02-28 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new ac13a191b9 Added pinot-error-code header in query

Re: [PR] Added pinot-error-code header in query response [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang merged PR #12338: URL: https://github.com/apache/pinot/pull/12338 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Add the support to extract json array elements from json index for the transform function jsonExtractIndex [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on PR #12466: URL: https://github.com/apache/pinot/pull/12466#issuecomment-1970189921 @itschrispeck Can you please help take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[I] Handle OFFLINE -> ONLINE state transition for consuming segment [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang opened a new issue, #12516: URL: https://github.com/apache/pinot/issues/12516 More context in #12390 When a server lost ZK connection, it would reset the segment state from initial state (OFFLINE), and then re-do the state transition. If the segment was in CONSUMING sta

Re: [I] InterruptedException when acquiring partitonConsumerSemaphore [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on issue #12390: URL: https://github.com/apache/pinot/issues/12390#issuecomment-1970186099 This is great finding! Let me create an issue to track this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Propagate Disable User Agent Config to Http Client [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on PR #12479: URL: https://github.com/apache/pinot/pull/12479#issuecomment-1970184623 Since the fix is very straight forward, I'll merge it now. We can consider adding a test separately -- This is an automated message from the Apache Git Service. To respond to the m

(pinot) branch master updated: Propagate Disable User Agent Config to Http Client (#12479)

2024-02-28 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 689d7d712b Propagate Disable User Agent Config to

Re: [PR] Propagate Disable User Agent Config to Http Client [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang merged PR #12479: URL: https://github.com/apache/pinot/pull/12479 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Egalpin/skip indexes minor changes [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on PR #12514: URL: https://github.com/apache/pinot/pull/12514#issuecomment-1970181930 Can you please take a look at the test failures? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

(pinot) branch master updated (53b3bfa41c -> eb61827435)

2024-02-28 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 53b3bfa41c refactor TlsUtils class (#12515) add eb61827435 Update license & notice for release 1.1.0 (#12511) No

Re: [PR] Update license & notice for release 1.1.0 [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang merged PR #12511: URL: https://github.com/apache/pinot/pull/12511 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] adding support of mega_bytes when configuring broker response size [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on PR #12510: URL: https://github.com/apache/pinot/pull/12510#issuecomment-1970180726 Suggest reusing the same field but allow it to take values such as `128MB`. Take a look at `DataSizeUtils` -- This is an automated message from the Apache Git Service. To respond t

Re: [I] Auto-tuning Pinot real-time segment size based on actual stream data consumption [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on issue #12513: URL: https://github.com/apache/pinot/issues/12513#issuecomment-1970166086 The challenge here is how to estimate the segment size during consumption. Do you have a solution in mind? -- This is an automated message from the Apache Git Service. To resp

Re: [I] Auto-tuning Pinot real-time segment size based on actual stream data consumption [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on issue #12513: URL: https://github.com/apache/pinot/issues/12513#issuecomment-1970164750 Here is another related issue with this algorithm: #12509 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] [null-aggr] Add null handling support in `mode` aggregation [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on code in PR #12227: URL: https://github.com/apache/pinot/pull/12227#discussion_r1506835734 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/FieldConfig.java: ## @@ -254,6 +256,24 @@ public Builder withTierOverwrites(JsonNode tierOverwrites)

Re: [I] Support mass data export [pinot]

2024-02-28 Thread via GitHub
egalpin commented on issue #12315: URL: https://github.com/apache/pinot/issues/12315#issuecomment-1970163695 thanks @cbalci , I'll have a look at the Spark Connector and see if that might suit my needs as well 👍 -- This is an automated message from the Apache Git Service. To respond to t

(pinot) branch master updated: refactor TlsUtils class (#12515)

2024-02-28 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 53b3bfa41c refactor TlsUtils class (#12515) 53b3

Re: [PR] refactor TlsUtils class [pinot]

2024-02-28 Thread via GitHub
xiangfu0 merged PR #12515: URL: https://github.com/apache/pinot/pull/12515 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1506815769 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +258,71 @@ public SuccessResponse putData( }

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on code in PR #12417: URL: https://github.com/apache/pinot/pull/12417#discussion_r1506799194 ## pinot-common/src/main/java/org/apache/pinot/common/utils/DatabaseUtils.java: ## @@ -0,0 +1,91 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

Re: [I] [Flaky test] CaseTransformFunctionTest. testCaseTransformFunctionWithIntResults() [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on issue #12367: URL: https://github.com/apache/pinot/issues/12367#issuecomment-1970089446 Comparing a float column with a double value might never match when the implicit casting changes the float value. This is actually the standard SQL behavior, and the expected re

Re: [PR] refactor TlsUtils class [pinot]

2024-02-28 Thread via GitHub
codecov-commenter commented on PR #12515: URL: https://github.com/apache/pinot/pull/12515#issuecomment-1970087478 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12515?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] Add a post-validator visitor that verifies there are no cast to bytes [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on code in PR #12475: URL: https://github.com/apache/pinot/pull/12475#discussion_r1506782839 ## pinot-query-planner/src/main/java/org/apache/pinot/query/validate/BytesCastVisitor.java: ## @@ -0,0 +1,73 @@ +/** + * Licensed to the Apache Software Foundation

Re: [PR] Added /uptime and /start-time for all components. Addressed comments and fixed tests from pull request 12389. [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on PR #12512: URL: https://github.com/apache/pinot/pull/12512#issuecomment-1970067234 Do we want uptime in seconds, or in a more human readable format? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Servers can't consume messages due to permission denied error in the endpoint '/segmentConsumed' when ACL is enabled [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on issue #11407: URL: https://github.com/apache/pinot/issues/11407#issuecomment-1970061731 @zhtaoxiang Can you please take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Is pinot possible to support batch Ingestion with Upsert? [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on issue #10896: URL: https://github.com/apache/pinot/issues/10896#issuecomment-1970060654 Actually the exception was thrown by `RealtimeTableDataManager.downloadSegmentFromPeer()` which shouldn't be invoked at all. I guess what have happened is some other exception w

Re: [PR] Record enricher [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on PR #12243: URL: https://github.com/apache/pinot/pull/12243#issuecomment-1970056042 The interface for `RecordEnricher` is almost identical to `RecordTransformer`, so I believe we should be able to implement the enricher as a special transformer. I do see the limi

[PR] refactor TlsUtils class [pinot]

2024-02-28 Thread via GitHub
zhtaoxiang opened a new pull request, #12515: URL: https://github.com/apache/pinot/pull/12515 refactor the TlsUtils class to move recently added renewable tls logic to a separate class to make it easy to read -- This is an automated message from the Apache Git Service. To respond to the m

Re: [I] Configurable heap size for query and realtime segment persistence. [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang commented on issue #12503: URL: https://github.com/apache/pinot/issues/12503#issuecomment-1970033581 Do you have access to the server node? K8s log might not be enough because it won't log the actual stack trace, and it doesn't contain the non-error logs. -- This is an automa

(pinot) branch master updated (28e480e8d3 -> 7e6117f046)

2024-02-28 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 28e480e8d3 Fix ColumnMinMaxValueGenerator (#12502) add 7e6117f046 Add controller API to get allLiveInstances (#124

Re: [PR] Add controller API to get allLiveInstances [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang merged PR #12498: URL: https://github.com/apache/pinot/pull/12498 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] adding support of mega_bytes when configuring broker response size [pinot]

2024-02-28 Thread via GitHub
Zhenyun20023 commented on PR #12510: URL: https://github.com/apache/pinot/pull/12510#issuecomment-1970003646 @vvivekiyer @Jackie-Jiang , can you take a look? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] adding support of mega_bytes when configuring broker response size [pinot]

2024-02-28 Thread via GitHub
codecov-commenter commented on PR #12510: URL: https://github.com/apache/pinot/pull/12510#issuecomment-1969992553 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12510?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] Added /uptime and /start-time for all components. Addressed comments and fixed tests from pull request 12389. [pinot]

2024-02-28 Thread via GitHub
codecov-commenter commented on PR #12512: URL: https://github.com/apache/pinot/pull/12512#issuecomment-1969886330 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12512?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] Egalpin/skip indexes minor changes [pinot]

2024-02-28 Thread via GitHub
codecov-commenter commented on PR #12514: URL: https://github.com/apache/pinot/pull/12514#issuecomment-1969867631 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12514?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [I] [Adaptive Server Selector] support for strictReplicaGroup routing strategy [pinot]

2024-02-28 Thread via GitHub
ankitsultana commented on issue #12507: URL: https://github.com/apache/pinot/issues/12507#issuecomment-1969859207 @vvivekiyer if one enables adaptive server selector on a broker that serves Upsert tables, what would happen? Would it lead to incorrect results? -- This is an automated messa

Re: [I] Support mass data export [pinot]

2024-02-28 Thread via GitHub
kishoreg commented on issue #12315: URL: https://github.com/apache/pinot/issues/12315#issuecomment-1969769568 This is a great feature and we should have this on the roadmap. cc @npawar @mayankshriv -- This is an automated message from the Apache Git Service. To respond to the message, pl

[PR] Egalpin/skip indexes minor changes [pinot]

2024-02-28 Thread via GitHub
egalpin opened a new pull request, #12514: URL: https://github.com/apache/pinot/pull/12514 Relates to https://github.com/apache/pinot/pull/12414. Minor naming changes based on PR comments. cc/ @Jackie-Jiang @gortiz -- This is an automated message from the Apache Git Service. To

Re: [I] Support mass data export [pinot]

2024-02-28 Thread via GitHub
cbalci commented on issue #12315: URL: https://github.com/apache/pinot/issues/12315#issuecomment-1969574685 @egalpin have you looked into using [Spark Connector](https://github.com/apache/pinot/tree/master/pinot-connectors/pinot-spark-3-connector) for batch reading data from Pinot? It is mo

(pinot) branch master updated (bc07b8dd2c -> 28e480e8d3)

2024-02-28 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from bc07b8dd2c Metric for upsert tables count (#12505) add 28e480e8d3 Fix ColumnMinMaxValueGenerator (#12502) No new

Re: [PR] Fix ColumnMinMaxValueGenerator [pinot]

2024-02-28 Thread via GitHub
Jackie-Jiang merged PR #12502: URL: https://github.com/apache/pinot/pull/12502 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[I] Auto-tuning Pinot real-time segment size based on actual stream data consumption [pinot]

2024-02-28 Thread via GitHub
chenboat opened a new issue, #12513: URL: https://github.com/apache/pinot/issues/12513 Currently Pinot's adaptive realtime segment sizing algorithm (as documented [here](https://www.linkedin.com/blog/engineering/open-source/auto-tuning-pinot) makes the segment sizes converge to a target by

Re: [PR] [null-aggr] Add null handling support in `mode` aggregation [pinot]

2024-02-28 Thread via GitHub
gortiz commented on PR #12227: URL: https://github.com/apache/pinot/pull/12227#issuecomment-1969512968 @Jackie-Jiang this should be ready to merge. Can you take a look? As far as I can see the only discussion pending is whether the `NullableSingleInputAggregationFunction` should be merged w

Re: [PR] Adds per-column, query-time index skip option [pinot]

2024-02-28 Thread via GitHub
egalpin commented on PR #12414: URL: https://github.com/apache/pinot/pull/12414#issuecomment-1969505872 @Jackie-Jiang I'll make a follow-up PR to address these by end of week -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

(pinot) branch master updated: Metric for upsert tables count (#12505)

2024-02-28 Thread saurabhd336
This is an automated email from the ASF dual-hosted git repository. saurabhd336 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new bc07b8dd2c Metric for upsert tables count (#

Re: [PR] Metric for upsert tables count [pinot]

2024-02-28 Thread via GitHub
saurabhd336 merged PR #12505: URL: https://github.com/apache/pinot/pull/12505 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-02-28 Thread via GitHub
shounakmk219 commented on code in PR #12417: URL: https://github.com/apache/pinot/pull/12417#discussion_r1506102610 ## pinot-common/src/main/java/org/apache/pinot/common/utils/DatabaseUtils.java: ## @@ -0,0 +1,91 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-02-28 Thread via GitHub
shounakmk219 commented on code in PR #12417: URL: https://github.com/apache/pinot/pull/12417#discussion_r1506101339 ## pinot-common/src/main/java/org/apache/pinot/common/utils/DatabaseUtils.java: ## @@ -0,0 +1,91 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-02-28 Thread via GitHub
shounakmk219 commented on code in PR #12417: URL: https://github.com/apache/pinot/pull/12417#discussion_r1506100806 ## pinot-common/src/main/java/org/apache/pinot/common/utils/DatabaseUtils.java: ## @@ -0,0 +1,91 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

Re: [I] [Flaky test] CaseTransformFunctionTest. testCaseTransformFunctionWithIntResults() [pinot]

2024-02-28 Thread via GitHub
aditya0811 commented on issue #12367: URL: https://github.com/apache/pinot/issues/12367#issuecomment-1969084205 > @aditya0811 Thanks for taking this issue! I feel the problem is from the implicit cast from float to double within the double comparison. The literal is parsed as double, thus c

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-02-28 Thread via GitHub
shounakmk219 commented on code in PR #12417: URL: https://github.com/apache/pinot/pull/12417#discussion_r1505907187 ## pinot-common/src/main/java/org/apache/pinot/common/utils/DatabaseUtils.java: ## @@ -0,0 +1,91 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Add a post-validator visitor that verifies there are no cast to bytes [pinot]

2024-02-28 Thread via GitHub
codecov-commenter commented on PR #12475: URL: https://github.com/apache/pinot/pull/12475#issuecomment-1968677570 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12475?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [I] Support uptime API in all pinot components [pinot]

2024-02-28 Thread via GitHub
suyashpatel98 commented on issue #12298: URL: https://github.com/apache/pinot/issues/12298#issuecomment-1968480450 @mayankshriv @saurabhd336 @xiangfu0 Addressed comments on the first PR. Please review when you find time. I closed the old PR and created a new because the history of the old P

Re: [PR] Add config option for timezone [pinot]

2024-02-28 Thread via GitHub
dttung2905 commented on PR #12386: URL: https://github.com/apache/pinot/pull/12386#issuecomment-1968476338 Thanks. I have updated the PR description to include a release note -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Add a config similar to `realtime.segment.flush.threshold.rows` but independent of partitions consumed by server [pinot]

2024-02-28 Thread via GitHub
suyashpatel98 commented on issue #12508: URL: https://github.com/apache/pinot/issues/12508#issuecomment-1968467279 @Jackie-Jiang Can you please assign this to me? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[PR] Added /uptime and /start-time for all components. Addressed comments and fixed tests from pull request 12389. [pinot]

2024-02-28 Thread via GitHub
suyashpatel98 opened a new pull request, #12512: URL: https://github.com/apache/pinot/pull/12512 Added /uptime and /start-time for all components. Addressed comments and fixed tests from pull request [12389](https://github.com/apache/pinot/pull/12389). Tested /uptime and /start-time endpoi

Re: [I] Servers can't consume messages due to permission denied error in the endpoint '/segmentConsumed' when ACL is enabled [pinot]

2024-02-28 Thread via GitHub
vineethvp commented on issue #11407: URL: https://github.com/apache/pinot/issues/11407#issuecomment-1968443498 Hi, Facing the same issue in 1.0.0 version. Is there any ETA on when this be fixed or any workaround? This is a blocker as new segments are not getting created after threshold is m