[GitHub] [pinot] 61yao commented on a diff in pull request #10613: [feature] [null support # 10] Add null support in all transform function and pass the bitmap to upstream

61yao commented on code in PR #10613: URL: https://github.com/apache/pinot/pull/10613#discussion_r1220938885 ## pinot-core/src/main/java/org/apache/pinot/core/operator/docvalsets/TransformBlockValSet.java: ## @@ -46,47 +45,23 @@ public class TransformBlockValSet implements Block

[GitHub] [pinot] eaugene commented on issue #10832: option(skipUpsert=true) is not case inSensitive

eaugene commented on issue #10832: URL: https://github.com/apache/pinot/issues/10832#issuecomment-1579904691 Sure @Jackie-Jiang . Please assign to me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [pinot] npawar commented on issue #6083: Support Array function for multi-value columns

npawar commented on issue #6083: URL: https://github.com/apache/pinot/issues/6083#issuecomment-1579764245 > @npawar Hi, thank you! I also find this but it seems that we don't have query support: https://github.com/apache/pinot/blob/master/pinot-common/src/main/java/org/apache/pinot/common/f

[GitHub] [pinot] monicaluodialpad commented on issue #6083: Support Array function for multi-value columns

monicaluodialpad commented on issue #6083: URL: https://github.com/apache/pinot/issues/6083#issuecomment-1579672175 @npawar Hi, thank you! I also find this but it seems that we don't have query support: https://github.com/apache/pinot/blob/master/pinot-common/src/main/java/org/apache/pinot/

[GitHub] [pinot] codecov-commenter commented on pull request #10859: Refactor single column distinct executors.

codecov-commenter commented on PR #10859: URL: https://github.com/apache/pinot/pull/10859#issuecomment-1579657424 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10859?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] gortiz commented on pull request #10851: Fix #10713 by giving metainfo more priority than config

gortiz commented on PR #10851: URL: https://github.com/apache/pinot/pull/10851#issuecomment-1579646593 I'm running the tests again because last time presto driver tests failed due to some network problem. Apart form that, it is ready to merge. -- This is an automated message from the Apac

[GitHub] [pinot] gortiz commented on pull request #10528: PinotBufferFactory and a buffer implementation that uses Unsafe

gortiz commented on PR #10528: URL: https://github.com/apache/pinot/pull/10528#issuecomment-1579642976 I think we can merge this now. The tests that are failing are the flaky ones. I've also created a branch this morning where all tests were executed using Unsafe buffer and again it only fa

[GitHub] [pinot] gortiz closed pull request #10857: [Do not merge] This is just a fake PR to run the tests

gortiz closed pull request #10857: [Do not merge] This is just a fake PR to run the tests URL: https://github.com/apache/pinot/pull/10857 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [pinot] egalpin commented on a diff in pull request #10841: [feature] multi-value datetime transform variants

egalpin commented on code in PR #10841: URL: https://github.com/apache/pinot/pull/10841#discussion_r1220487877 ## pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java: ## @@ -642,4 +1138,30 @@ public static long timestampDiff(String unit, lon

[GitHub] [pinot] ankitsultana commented on pull request #10815: [Draft] Periodical Tmp Segment file deletion

ankitsultana commented on PR #10815: URL: https://github.com/apache/pinot/pull/10815#issuecomment-1579462537 @wirybeaver Let's follow-up offline -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [pinot] ankitsultana commented on pull request #10758: [Draft] Add a new controller endpoint for segment deletion with a time window

ankitsultana commented on PR #10758: URL: https://github.com/apache/pinot/pull/10758#issuecomment-1579460172 Thanks @wirybeaver. Let us know when this is ready for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [pinot] ankitsultana commented on issue #10853: Add API to Broker and Server to Return Tenant Tags

ankitsultana commented on issue #10853: URL: https://github.com/apache/pinot/issues/10853#issuecomment-1579459559 We have some use-cases where we have sidecars running alongside Pinot servers and brokers. Those sidecars do some obs and automation work. With this we'll be able to make the si

[GitHub] [pinot] snleee commented on a diff in pull request #10856: Add the support for filling the default header if the header is missing

snleee commented on code in PR #10856: URL: https://github.com/apache/pinot/pull/10856#discussion_r1220317462 ## pinot-plugins/pinot-input-format/pinot-csv/src/main/java/org/apache/pinot/plugin/inputformat/csv/CSVDefaultHeaderUtil.java: ## @@ -0,0 +1,195 @@ +/** + * Licensed to

[GitHub] [pinot] codecov-commenter commented on pull request #10860: Select distinct multi-column NULL support.

codecov-commenter commented on PR #10860: URL: https://github.com/apache/pinot/pull/10860#issuecomment-1579416295 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10860?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] snleee commented on a diff in pull request #10856: Add the support for filling the default header if the header is missing

snleee commented on code in PR #10856: URL: https://github.com/apache/pinot/pull/10856#discussion_r1220279900 ## pinot-plugins/pinot-input-format/pinot-csv/src/main/java/org/apache/pinot/plugin/inputformat/csv/CSVDefaultHeaderUtil.java: ## @@ -0,0 +1,195 @@ +/** + * Licensed to

[GitHub] [pinot] snleee commented on a diff in pull request #10856: Add the support for filling the default header if the header is missing

snleee commented on code in PR #10856: URL: https://github.com/apache/pinot/pull/10856#discussion_r1220277626 ## pinot-plugins/pinot-input-format/pinot-csv/src/main/java/org/apache/pinot/plugin/inputformat/csv/CSVDefaultHeaderUtil.java: ## @@ -0,0 +1,195 @@ +/** + * Licensed to

[GitHub] [pinot] swaminathanmanish commented on pull request #10856: Add the support for filling the default header if the header is missing

swaminathanmanish commented on PR #10856: URL: https://github.com/apache/pinot/pull/10856#issuecomment-1579337132 Can we also document user experience when this feature is enabled ? 1. What happens if proper headers are there already ? Here the algorithm should not get this wrong and ove

[GitHub] [pinot] swaminathanmanish commented on a diff in pull request #10856: Add the support for filling the default header if the header is missing

swaminathanmanish commented on code in PR #10856: URL: https://github.com/apache/pinot/pull/10856#discussion_r1220186303 ## pinot-plugins/pinot-input-format/pinot-csv/src/main/java/org/apache/pinot/plugin/inputformat/csv/CSVDefaultHeaderUtil.java: ## @@ -0,0 +1,195 @@ +/** + * L

[GitHub] [pinot] shenyu0127 opened a new pull request, #10860: Select distinct multi-column NULL support.

shenyu0127 opened a new pull request, #10860: URL: https://github.com/apache/pinot/pull/10860 This PR enables NULL support for queries like `SELECT DISTINCT col1, col2 from testTable ORDER BY col1, col2`. Tested in unit tests. -- This is an automated message from the Apache Git Ser

[pinot] branch master updated (9fae8ad72e -> 3b2e31cfad)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 9fae8ad72e Make DistinctTable consume OrderByExpressionContext::isNullsLast (#10839) add 3b2e31cfad [hotfix] use

[GitHub] [pinot] swaminathanmanish commented on a diff in pull request #10856: Add the support for filling the default header if the header is missing

swaminathanmanish commented on code in PR #10856: URL: https://github.com/apache/pinot/pull/10856#discussion_r1220174979 ## pinot-plugins/pinot-input-format/pinot-csv/src/main/java/org/apache/pinot/plugin/inputformat/csv/CSVDefaultHeaderUtil.java: ## @@ -0,0 +1,195 @@ +/** + * L

[GitHub] [pinot] Jackie-Jiang closed issue #10829: [flaky test] query with pipeline breaker seems unstable

Jackie-Jiang closed issue #10829: [flaky test] query with pipeline breaker seems unstable URL: https://github.com/apache/pinot/issues/10829 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [pinot] Jackie-Jiang merged pull request #10858: [hotfix] use proper threadpool for test

Jackie-Jiang merged PR #10858: URL: https://github.com/apache/pinot/pull/10858 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] shenyu0127 opened a new pull request, #10859: Refactor single column distinct executors.

shenyu0127 opened a new pull request, #10859: URL: https://github.com/apache/pinot/pull/10859 This PR moves the common logic `process` into the base single column distinct executor class. Tested in unit tests. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [pinot] Jackie-Jiang commented on issue #10832: option(skipUpsert=true) is not case inSensitive

Jackie-Jiang commented on issue #10832: URL: https://github.com/apache/pinot/issues/10832#issuecomment-1579318320 Yes, I think it is a fair ask. What we can do is to always use lower case to store and look up query options. Do you want to raise a PR to add this support? -- This is an auto

[GitHub] [pinot] Jackie-Jiang commented on pull request #10836: refactor ConcurrentMapPartitionUpsertMetadataManager constructor

Jackie-Jiang commented on PR #10836: URL: https://github.com/apache/pinot/pull/10836#issuecomment-1579313827 > > One way is to create a different wrapper class to hold the modified configs > > Why do you need a wrapper class? wouldn't a builder pattern like [this](https://github.com/

[pinot] branch master updated: Make DistinctTable consume OrderByExpressionContext::isNullsLast (#10839)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 9fae8ad72e Make DistinctTable consume OrderByExp

[GitHub] [pinot] Jackie-Jiang merged pull request #10839: Make DistinctTable consume OrderByExpressionContext::isNullsLast.

Jackie-Jiang merged PR #10839: URL: https://github.com/apache/pinot/pull/10839 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] jackjlli commented on a diff in pull request #9416: Compatible dropwizard metrics

jackjlli commented on code in PR #9416: URL: https://github.com/apache/pinot/pull/9416#discussion_r1220152805 ## pinot-plugins/pinot-metrics/pinot-compound-metrics/src/main/java/org/apache/pinot/plugin/metrics/compound/CompoundPinotMetricsFactory.java: ## @@ -0,0 +1,170 @@ +/**

[GitHub] [pinot] codecov-commenter commented on pull request #10858: [hotfix] use proper threadpool for test

codecov-commenter commented on PR #10858: URL: https://github.com/apache/pinot/pull/10858#issuecomment-1579284373 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10858?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] codecov-commenter commented on pull request #10857: [Do not merge] This is just a fake PR to run the tests

codecov-commenter commented on PR #10857: URL: https://github.com/apache/pinot/pull/10857#issuecomment-1579281746 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] walterddr commented on pull request #10858: [hotfix] use proper threadpool for test

walterddr commented on PR #10858: URL: https://github.com/apache/pinot/pull/10858#issuecomment-1579278441 actually it doesn't really matter b/c the test using the 2 worker/runner theadpool is not important. will land. -- This is an automated message from the Apache Git Service. To respon

[GitHub] [pinot] walterddr commented on pull request #10858: [hotfix] use proper threadpool for test

walterddr commented on PR #10858: URL: https://github.com/apache/pinot/pull/10858#issuecomment-1579274631 > so basically, there are dependencies between tasks from different stages, and a fixed thread pool might have led to dead lock among them, like those taking threads and running might a

[GitHub] [pinot] navina commented on pull request #10836: refactor ConcurrentMapPartitionUpsertMetadataManager constructor

navina commented on PR #10836: URL: https://github.com/apache/pinot/pull/10836#issuecomment-1579253340 > One way is to create a different wrapper class to hold the modified configs Why do you need a wrapper class? wouldn't a builder pattern like [this](https://github.com/apache/pinot

[pinot] branch master updated (f8766129f0 -> 0f244ecf21)

This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from f8766129f0 Support Off Heap for Native Text Indices (#10842) add 0f244ecf21 [feature] Add flags to indicate if

[GitHub] [pinot] npawar merged pull request #10835: [feature] Add flags to indicate if FST, H3, and Text indexes are on a segment

npawar merged PR #10835: URL: https://github.com/apache/pinot/pull/10835 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

[GitHub] [pinot] somandal commented on a diff in pull request #10846: [multistage] Initial planner changes to support the v1 aggregation functions in multistage

somandal commented on code in PR #10846: URL: https://github.com/apache/pinot/pull/10846#discussion_r1219868444 ## pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java: ## @@ -103,6 +98,61 @@ public final void initNoDuplicate() { throw Ut

[GitHub] [pinot] somandal commented on a diff in pull request #10846: [multistage] Initial planner changes to support the v1 aggregation functions in multistage

somandal commented on code in PR #10846: URL: https://github.com/apache/pinot/pull/10846#discussion_r1220101094 ## pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java: ## @@ -103,6 +98,61 @@ public final void initNoDuplicate() { throw Ut

[pinot] branch native_mmap deleted (was 700509e9fc)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch native_mmap in repository https://gitbox.apache.org/repos/asf/pinot.git was 700509e9fc Add a TODO for Load Mode The revisions that were on this branch are still contained in other reference

[GitHub] [pinot] Jackie-Jiang commented on pull request #10836: refactor ConcurrentMapPartitionUpsertMetadataManager constructor

Jackie-Jiang commented on PR #10836: URL: https://github.com/apache/pinot/pull/10836#issuecomment-1579206540 > > > @Jackie-Jiang Can I assign the `comparisonColumns` value to upsertConfig and pass it to PartitionUpsertMetadataManager? then we don't need to repeat the same steps in partition

[GitHub] [pinot] klsince commented on pull request #10858: [hotfix] use proper threadpool for test

klsince commented on PR #10858: URL: https://github.com/apache/pinot/pull/10858#issuecomment-1579204387 > ... thus all threads within the threadpool are blocked and no more tasks can be scheduled into it so basically, there are dependencies between tasks from different stages, and a

[GitHub] [pinot] walterddr commented on a diff in pull request #10858: [hotfix] use proper threadpool for test

walterddr commented on code in PR #10858: URL: https://github.com/apache/pinot/pull/10858#discussion_r1220061034 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/QueryServerEnclosure.java: ## @@ -69,7 +68,7 @@ public class QueryServerEnclosure { private final Helix

[GitHub] [pinot] Jackie-Jiang commented on pull request #10758: [Draft] Add a new controller endpoint for segment deletion with a time window

Jackie-Jiang commented on PR #10758: URL: https://github.com/apache/pinot/pull/10758#issuecomment-1579187687 cc @snleee -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [pinot] walterddr commented on pull request #10858: [hotfix] use proper threadpool for test

walterddr commented on PR #10858: URL: https://github.com/apache/pinot/pull/10858#issuecomment-1579187444 > Why did the unit tests not using cached thread pool lead to the tests occasionally failing? GHA servers only has 2 virtual core, and in the pipeline breaker scenario we will ha

[GitHub] [pinot] ege-st commented on pull request #10858: [hotfix] use proper threadpool for test

ege-st commented on PR #10858: URL: https://github.com/apache/pinot/pull/10858#issuecomment-1579184306 Why did the unit tests not using cached thread pool lead to the tests occasionally failing? -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [pinot] walterddr opened a new pull request, #10858: [hotfix] use proper threadpool for test

walterddr opened a new pull request, #10858: URL: https://github.com/apache/pinot/pull/10858 we upgraded the intermediate stage threadpool to cached threadpool already. but forgot to change the one in test. -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [pinot] gortiz opened a new pull request, #10857: [Do not merge] This is just a fake PR to run the tests

gortiz opened a new pull request, #10857: URL: https://github.com/apache/pinot/pull/10857 This is a test for #10528. The code is the same, but it always uses the unsafe buffer library -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [pinot] npawar commented on a diff in pull request #10855: jsonPathString should return null instead of string literal "null"

npawar commented on code in PR #10855: URL: https://github.com/apache/pinot/pull/10855#discussion_r1219984101 ## pinot-common/src/main/java/org/apache/pinot/common/function/scalar/JsonFunctions.java: ## @@ -135,7 +135,7 @@ public static String jsonPathString(Object object, Strin

[GitHub] [pinot] ege-st commented on a diff in pull request #10852: compress nullvector bitmap upon seal

ege-st commented on code in PR #10852: URL: https://github.com/apache/pinot/pull/10852#discussion_r1219975220 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/nullvalue/NullValueVectorCreator.java: ## @@ -42,7 +42,7 @@ * null, which is t

[GitHub] [pinot] vvivekiyer commented on a diff in pull request #10845: [Multistage] Runtime changes for leveraging V1 Aggregation Functions

vvivekiyer commented on code in PR #10845: URL: https://github.com/apache/pinot/pull/10845#discussion_r1219905051 ## pinot-core/src/main/java/org/apache/pinot/core/common/IntermediateStageBlockValSet.java: ## @@ -0,0 +1,250 @@ +/** + * Licensed to the Apache Software Foundation

[GitHub] [pinot] abhioncbr commented on a diff in pull request #10845: [Multistage] Runtime changes for leveraging V1 Aggregation Functions

abhioncbr commented on code in PR #10845: URL: https://github.com/apache/pinot/pull/10845#discussion_r1219879274 ## pinot-core/src/main/java/org/apache/pinot/core/common/IntermediateStageBlockValSet.java: ## @@ -0,0 +1,250 @@ +/** + * Licensed to the Apache Software Foundation (

[GitHub] [pinot] abhioncbr commented on a diff in pull request #10845: [Multistage] Runtime changes for leveraging V1 Aggregation Functions

abhioncbr commented on code in PR #10845: URL: https://github.com/apache/pinot/pull/10845#discussion_r1219879274 ## pinot-core/src/main/java/org/apache/pinot/core/common/IntermediateStageBlockValSet.java: ## @@ -0,0 +1,250 @@ +/** + * Licensed to the Apache Software Foundation (

[GitHub] [pinot] somandal commented on a diff in pull request #10846: [multistage] Initial planner changes to support the v1 aggregation functions in multistage

somandal commented on code in PR #10846: URL: https://github.com/apache/pinot/pull/10846#discussion_r1219868444 ## pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java: ## @@ -103,6 +98,61 @@ public final void initNoDuplicate() { throw Ut

[GitHub] [pinot] kishoreg commented on a diff in pull request #10846: [multistage] Initial planner changes to support the v1 aggregation functions in multistage

kishoreg commented on code in PR #10846: URL: https://github.com/apache/pinot/pull/10846#discussion_r1219852675 ## pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java: ## @@ -103,6 +98,61 @@ public final void initNoDuplicate() { throw Ut

[GitHub] [pinot] somandal commented on a diff in pull request #10846: [multistage] Initial planner changes to support the v1 aggregation functions in multistage

somandal commented on code in PR #10846: URL: https://github.com/apache/pinot/pull/10846#discussion_r1219834738 ## pinot-query-planner/src/main/java/org/apache/calcite/sql/fun/PinotOperatorTable.java: ## @@ -103,6 +98,61 @@ public final void initNoDuplicate() { throw Ut

[GitHub] [pinot] codecov-commenter commented on pull request #10856: Add the support for filling the default header if the header is missing

codecov-commenter commented on PR #10856: URL: https://github.com/apache/pinot/pull/10856#issuecomment-1578071935 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10856?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R