[GitHub] [pinot] xiangfu0 commented on a diff in pull request #8343: Timestamp type index

2022-04-08 Thread GitBox
xiangfu0 commented on code in PR #8343: URL: https://github.com/apache/pinot/pull/8343#discussion_r846583089 ## pinot-common/src/main/java/org/apache/pinot/common/config/provider/TableCache.java: ## @@ -242,6 +252,23 @@ private void putTableConfig(ZNRecord znRecord) _tabl

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8343: Timestamp type index

2022-04-08 Thread GitBox
Jackie-Jiang commented on code in PR #8343: URL: https://github.com/apache/pinot/pull/8343#discussion_r846547501 ## pinot-common/src/main/java/org/apache/pinot/common/config/provider/TableCache.java: ## @@ -242,6 +252,23 @@ private void putTableConfig(ZNRecord znRecord) _

[GitHub] [pinot] codecov-commenter commented on pull request #8497: Bump moment from 2.29.1 to 2.29.2 in /pinot-controller/src/main/resources

2022-04-08 Thread GitBox
codecov-commenter commented on PR #8497: URL: https://github.com/apache/pinot/pull/8497#issuecomment-1093507945 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8497?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[pinot] branch dependabot/npm_and_yarn/pinot-controller/src/main/resources/moment-2.29.2 created (now 6ba54b91c6)

2022-04-08 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/npm_and_yarn/pinot-controller/src/main/resources/moment-2.29.2 in repository https://gitbox.apache.org/repos/asf/pinot.git at 6ba54b91c6 Bump moment in /pinot-controller/src

[GitHub] [pinot] dependabot[bot] opened a new pull request, #8497: Bump moment from 2.29.1 to 2.29.2 in /pinot-controller/src/main/resources

2022-04-08 Thread GitBox
dependabot[bot] opened a new pull request, #8497: URL: https://github.com/apache/pinot/pull/8497 Bumps [moment](https://github.com/moment/moment) from 2.29.1 to 2.29.2. Changelog Sourced from https://github.com/moment/moment/blob/develop/CHANGELOG.md";>moment's changelog. 2.

[GitHub] [pinot] Jackie-Jiang merged pull request #8495: fix netty-bom import

2022-04-08 Thread GitBox
Jackie-Jiang merged PR #8495: URL: https://github.com/apache/pinot/pull/8495 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

[pinot] branch master updated (8e21aec1c2 -> 7dd374f2c2)

2022-04-08 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 8e21aec1c2 Add endpoints for some finer control on minion tasks (#8486) add 7dd374f2c2 fix netty-bom import (#8495

[GitHub] [pinot] Jackie-Jiang commented on pull request #8118: Infer SDF for Timestamp in dimension field spec

2022-04-08 Thread GitBox
Jackie-Jiang commented on PR #8118: URL: https://github.com/apache/pinot/pull/8118#issuecomment-1093481291 > > Sorry I missed track of this PR. Checked the code and it should be safe to add the support without performance regression. @kbastani Could you please address the comments and we ca

[GitHub] [pinot] jasperjiaguo commented on pull request #8495: fix netty-bom import

2022-04-08 Thread GitBox
jasperjiaguo commented on PR #8495: URL: https://github.com/apache/pinot/pull/8495#issuecomment-1093412759 Yes LGTM. Thanks for fixing this! @walterddr -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8467: pinot-java-client controller based broker selector

2022-04-08 Thread GitBox
Jackie-Jiang commented on code in PR #8467: URL: https://github.com/apache/pinot/pull/8467#discussion_r846514461 ## pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/ControllerURLUtils.java: ## @@ -0,0 +1,35 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] [pinot] richardstartin commented on pull request #8118: Infer SDF for Timestamp in dimension field spec

2022-04-08 Thread GitBox
richardstartin commented on PR #8118: URL: https://github.com/apache/pinot/pull/8118#issuecomment-1093397314 > Sorry I missed track of this PR. Checked the code and it should be safe to add the support without performance regression. @kbastani Could you please address the comments and we ca

[GitHub] [pinot] Jackie-Jiang commented on issue #8492: Need ways to check whether segment reload/refresh is done on server side

2022-04-08 Thread GitBox
Jackie-Jiang commented on issue #8492: URL: https://github.com/apache/pinot/issues/8492#issuecomment-1093396504 Since the reload/refresh is done via the Helix messaging mechanism, we should also explore whether Helix can support reading response for the message. -- This is an automated me

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8118: Infer SDF for Timestamp in dimension field spec

2022-04-08 Thread GitBox
Jackie-Jiang commented on code in PR #8118: URL: https://github.com/apache/pinot/pull/8118#discussion_r846512405 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/TimestampUtils.java: ## @@ -19,19 +19,27 @@ package org.apache.pinot.spi.utils; import java.sql.Timestamp; +

[GitHub] [pinot] Jackie-Jiang commented on issue #8487: boolean function not recognized in filter clause

2022-04-08 Thread GitBox
Jackie-Jiang commented on issue #8487: URL: https://github.com/apache/pinot/issues/8487#issuecomment-1093361055 This and #8444 is the same issue and can be solved together -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8483: Minimize data movement between instances in pools

2022-04-08 Thread GitBox
Jackie-Jiang commented on code in PR #8483: URL: https://github.com/apache/pinot/pull/8483#discussion_r846476628 ## pinot-common/src/main/java/org/apache/pinot/common/assignment/InstancePartitions.java: ## @@ -58,24 +60,39 @@ @JsonIgnoreProperties(ignoreUnknown = true) public

[GitHub] [pinot] walterddr commented on pull request #8484: adding custome parser

2022-04-08 Thread GitBox
walterddr commented on PR #8484: URL: https://github.com/apache/pinot/pull/8484#issuecomment-1093347432 this is still in POC phase but we were able to incorporate a new syntax and a new SqlCall node during parsing phase. caveat. no longer using BABEL parser factory. let's see how man

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #8465: Add adhoc minion task creation endpoint

2022-04-08 Thread GitBox
xiangfu0 commented on code in PR #8465: URL: https://github.com/apache/pinot/pull/8465#discussion_r846458767 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTaskRestletResource.java: ## @@ -346,6 +347,29 @@ public StringResultResponse getTaskStat

[GitHub] [pinot] cbalci commented on pull request #8481: [pinot-spark-connector] Add option to connect using GRPC

2022-04-08 Thread GitBox
cbalci commented on PR #8481: URL: https://github.com/apache/pinot/pull/8481#issuecomment-1093292709 @yupeng9 ptal -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[GitHub] [pinot] Jackie-Jiang commented on issue #7978: Augmented Index Configuration for JSON data

2022-04-08 Thread GitBox
Jackie-Jiang commented on issue #7978: URL: https://github.com/apache/pinot/issues/7978#issuecomment-1093252987 @egalpin I see the point. Essentially we want to support the arbitrary nested array/map with fixed schema, and allow index on top of them. This is a very interesting and challengi

[GitHub] [pinot] richardstartin commented on pull request #8496: Reduce the size of arrays allocated in DataBlockCache

2022-04-08 Thread GitBox
richardstartin commented on PR #8496: URL: https://github.com/apache/pinot/pull/8496#issuecomment-1093247805 This is consistent with my own findings, but I was hoping to eliminate the need for the cache in the first place by preventing from planning duplicate work, and then store the array

[GitHub] [pinot] hristo-stripe commented on pull request #8496: Reduce the size of arrays allocated in DataBlockCache

2022-04-08 Thread GitBox
hristo-stripe commented on PR #8496: URL: https://github.com/apache/pinot/pull/8496#issuecomment-1093245499 I cannot apply labels, but this likely has to be under the `performance` label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [pinot] Jackie-Jiang commented on pull request #7916: FILTER Clauses for Aggregates

2022-04-08 Thread GitBox
Jackie-Jiang commented on PR #7916: URL: https://github.com/apache/pinot/pull/7916#issuecomment-1093234891 No it is not supported. For the example you give, `SELECT SUM(value) WHERE IN_SUBQUERY(entityId, 'SELECT ID_SET(entityId) FROM other_table WHERE cond = ') = 1 FROM table` should work

[GitHub] [pinot] hristo-stripe opened a new pull request, #8496: Reduce the size of arrays allocated in DataBlockCache

2022-04-08 Thread GitBox
hristo-stripe opened a new pull request, #8496: URL: https://github.com/apache/pinot/pull/8496 We are seeing ~50% of memory allocated in a given period from these arrays. Our use case is such that we have heavy filtering from a given segment, producing fewer than 100 docs in the vast m

[GitHub] [pinot] Jackie-Jiang commented on pull request #6719: Add TIMESTAMP and BOOLEAN data type support

2022-04-08 Thread GitBox
Jackie-Jiang commented on PR #6719: URL: https://github.com/apache/pinot/pull/6719#issuecomment-1093225949 @MrNeocore TIMESTAMP is internally stored as LONG, so it will be more efficiently performance wise. We will also build optimizations/indexes on TIMESTAMP type, which won't be applied t

[GitHub] [pinot] siddharthteotia commented on pull request #8485: Manual tracing

2022-04-08 Thread GitBox
siddharthteotia commented on PR #8485: URL: https://github.com/apache/pinot/pull/8485#issuecomment-1093194969 Thanks @richardstartin . Reviewing this today -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8465: Add adhoc minion task creation endpoint

2022-04-08 Thread GitBox
Jackie-Jiang commented on code in PR #8465: URL: https://github.com/apache/pinot/pull/8465#discussion_r846383207 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTaskRestletResource.java: ## @@ -346,6 +347,29 @@ public StringResultResponse getTask

[pinot] branch master updated: Add endpoints for some finer control on minion tasks (#8486)

2022-04-08 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 8e21aec1c2 Add endpoints for some finer control o

[GitHub] [pinot] Jackie-Jiang merged pull request #8486: Add endpoints for some finer control on minion tasks

2022-04-08 Thread GitBox
Jackie-Jiang merged PR #8486: URL: https://github.com/apache/pinot/pull/8486 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

[GitHub] [pinot] cedricbarbin commented on issue #4134: Error while trying to run quick-start-offline on windows

2022-04-08 Thread GitBox
cedricbarbin commented on issue #4134: URL: https://github.com/apache/pinot/issues/4134#issuecomment-1093156775 A working patch for 0.11 Class : org.apache.pinot.common.utils.URIUtils Line 41 : String path = "file:/" + getPath(basePath, parts).replace("\\", "/"); -- This i

[GitHub] [pinot] kkrugler commented on issue #8477: Airline dataset from the talk about the Star-Tree Index

2022-04-08 Thread GitBox
kkrugler commented on issue #8477: URL: https://github.com/apache/pinot/issues/8477#issuecomment-1093136558 Hi @ClouddJR - the best place to ask questions like this is in the [Pinot Slack workspace](https://apache-pinot.slack.com/), e.g. via the general channel. You could maybe use this inv

[pinot] branch master updated: Make the PR template simpler (#8455)

2022-04-08 Thread sajjad
This is an automated email from the ASF dual-hosted git repository. sajjad pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new e986d00548 Make the PR template simpler (#8455) e

[GitHub] [pinot] sajjad-moradi merged pull request #8455: Make the PR template simpler

2022-04-08 Thread GitBox
sajjad-moradi merged PR #8455: URL: https://github.com/apache/pinot/pull/8455 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.

[GitHub] [pinot] codecov-commenter commented on pull request #8495: fix netty-bom import

2022-04-08 Thread GitBox
codecov-commenter commented on PR #8495: URL: https://github.com/apache/pinot/pull/8495#issuecomment-1093115087 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8495?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[GitHub] [pinot] richardstartin commented on pull request #8485: Manual tracing

2022-04-08 Thread GitBox
richardstartin commented on PR #8485: URL: https://github.com/apache/pinot/pull/8485#issuecomment-1093114977 I’ve measured the overhead of this manual instrumentation and it is within the noise when tracing is disabled. The will however augment default tracing, adding an entry per tra

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #8343: Timestamp type index

2022-04-08 Thread GitBox
xiangfu0 commented on code in PR #8343: URL: https://github.com/apache/pinot/pull/8343#discussion_r846339619 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/creator/SegmentGeneratorConfig.java: ## @@ -202,6 +207,33 @@ public SegmentGeneratorConfig(TableConfig tab

[GitHub] [pinot] walterddr commented on a diff in pull request #8365: Use netty-bom to keep the netty-related libraries sync

2022-04-08 Thread GitBox
walterddr commented on code in PR #8365: URL: https://github.com/apache/pinot/pull/8365#discussion_r846310352 ## pinot-connectors/prestodb-pinot-dependencies/pinot-core-jdk8/pom.xml: ## @@ -112,8 +112,10 @@ --> io.netty - netty-tcnative-boringssl-static -

[GitHub] [pinot] walterddr opened a new pull request, #8495: fix netty-bom import

2022-04-08 Thread GitBox
walterddr opened a new pull request, #8495: URL: https://github.com/apache/pinot/pull/8495 import scope should be defined in DependencyManagement section -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [pinot] walterddr commented on a diff in pull request #8365: Use netty-bom to keep the netty-related libraries sync

2022-04-08 Thread GitBox
walterddr commented on code in PR #8365: URL: https://github.com/apache/pinot/pull/8365#discussion_r846307100 ## pinot-connectors/prestodb-pinot-dependencies/pinot-core-jdk8/pom.xml: ## @@ -112,8 +112,10 @@ --> io.netty - netty-tcnative-boringssl-static -

[GitHub] [pinot] walterddr commented on a diff in pull request #7358: [issue-7357] Add support for Avro logical types in realtime ingestion

2022-04-08 Thread GitBox
walterddr commented on code in PR #7358: URL: https://github.com/apache/pinot/pull/7358#discussion_r846215393 ## pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroRecordReaderConfig.java: ## @@ -0,0 +1,36 @@ +/** + * Lice

[GitHub] [pinot] codecov-commenter commented on pull request #8494: Upgrade snappy version to 1.1.8.2

2022-04-08 Thread GitBox
codecov-commenter commented on PR #8494: URL: https://github.com/apache/pinot/pull/8494#issuecomment-1092776560 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8494?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[GitHub] [pinot] KKcorps opened a new pull request, #8494: Upgrade snappy version to 1.1.8.2

2022-04-08 Thread GitBox
KKcorps opened a new pull request, #8494: URL: https://github.com/apache/pinot/pull/8494 Current snappy version doesn't work for M1 Mac devices. Need to upgrade to 1.1.8.2 since that has M1 binaries. -- This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [pinot] ddcprg commented on a diff in pull request #7358: [issue-7357] Add support for Avro logical types in realtime ingestion

2022-04-08 Thread GitBox
ddcprg commented on code in PR #7358: URL: https://github.com/apache/pinot/pull/7358#discussion_r845935916 ## pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroRecordReaderConfig.java: ## @@ -0,0 +1,36 @@ +/** + * License

[GitHub] [pinot] ddcprg commented on a diff in pull request #7358: [issue-7357] Add support for Avro logical types in realtime ingestion

2022-04-08 Thread GitBox
ddcprg commented on code in PR #7358: URL: https://github.com/apache/pinot/pull/7358#discussion_r845935916 ## pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroRecordReaderConfig.java: ## @@ -0,0 +1,36 @@ +/** + * License

[GitHub] [pinot] nizarhejazi closed pull request #8468: Support Single-valued BigDecimal columns across different SQL statements, and Pinot transforms and operators

2022-04-08 Thread GitBox
nizarhejazi closed pull request #8468: Support Single-valued BigDecimal columns across different SQL statements, and Pinot transforms and operators URL: https://github.com/apache/pinot/pull/8468 -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [pinot] nizarhejazi commented on pull request #8468: Support Single-valued BigDecimal columns across different SQL statements, and Pinot transforms and operators

2022-04-08 Thread GitBox
nizarhejazi commented on PR #8468: URL: https://github.com/apache/pinot/pull/8468#issuecomment-1092618283 Agreed w/ @Jackie-Jiang to split the PR into multiple PRs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [pinot] ClouddJR commented on issue #8477: Airline dataset from the talk about the Star-Tree Index

2022-04-08 Thread GitBox
ClouddJR commented on issue #8477: URL: https://github.com/apache/pinot/issues/8477#issuecomment-1092607076 Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [pinot] ClouddJR closed issue #8477: Airline dataset from the talk about the Star-Tree Index

2022-04-08 Thread GitBox
ClouddJR closed issue #8477: Airline dataset from the talk about the Star-Tree Index URL: https://github.com/apache/pinot/issues/8477 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com