[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11241: [multistage] support multi-tenant lookup for v2 engine

Jackie-Jiang commented on code in PR #11241: URL: https://github.com/apache/pinot/pull/11241#discussion_r1281460234 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java: ## @@ -214,31 +215,19 @@ private String getMultiStageQueryResp

[GitHub] [pinot] gortiz commented on a diff in pull request #11205: [Draft] Remove noop

gortiz commented on code in PR #11205: URL: https://github.com/apache/pinot/pull/11205#discussion_r1281459982 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/OpChainExecutor.java: ## @@ -0,0 +1,129 @@ +/** + * Licensed to the Apache Software Foundation (ASF)

[pinot] branch master updated (42897ad448 -> 044588f94f)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 42897ad448 Remove support for High level consumers in Apache Pinot (#11017) add 044588f94f HAVING filter IS NULL

[GitHub] [pinot] Jackie-Jiang merged pull request #11242: HAVING filter IS NULL and IS NOT NULL support.

Jackie-Jiang merged PR #11242: URL: https://github.com/apache/pinot/pull/11242 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] Jackie-Jiang commented on pull request #11206: Bump io.grpc:grpc-context from 1.14.0 to 1.57.0

Jackie-Jiang commented on PR #11206: URL: https://github.com/apache/pinot/pull/11206#issuecomment-1661583504 @walterddr This is added to pulsar module, and seems we don't have enough coverage on pulsar. @KKcorps do you think we can merge this? -- This is an automated message from the Apac

[GitHub] [pinot] gortiz commented on a diff in pull request #11205: [Draft] Remove noop

gortiz commented on code in PR #11205: URL: https://github.com/apache/pinot/pull/11205#discussion_r1281457889 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/ReceivingMailbox.java: ## @@ -51,16 +52,30 @@ public class ReceivingMailbox { // TODO: Revisit if

[pinot] branch master updated: Remove support for High level consumers in Apache Pinot (#11017)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 42897ad448 Remove support for High level consumer

[GitHub] [pinot] Jackie-Jiang merged pull request #11017: Remove support for High level consumers in Apache Pinot

Jackie-Jiang merged PR #11017: URL: https://github.com/apache/pinot/pull/11017 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11017: Remove support for High level consumers in Apache Pinot

Jackie-Jiang commented on code in PR #11017: URL: https://github.com/apache/pinot/pull/11017#discussion_r1281456311 ## pinot-spi/src/main/java/org/apache/pinot/spi/stream/StreamConfig.java: ## @@ -223,6 +212,17 @@ public StreamConfig(String tableNameWithType, Map streamConfigMa

[pinot] branch master updated: Support distinctCountBitmap and *TupleSketch functions in v2 (#11245)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 659fe7c68a Support distinctCountBitmap and *Tupl

[GitHub] [pinot] xiangfu0 merged pull request #11245: [multistage] Support distinctCountBitmap and *TupleSketch functions

xiangfu0 merged PR #11245: URL: https://github.com/apache/pinot/pull/11245 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[GitHub] [pinot] navina commented on a diff in pull request #11017: Remove support for High level consumers in Apache Pinot

navina commented on code in PR #11017: URL: https://github.com/apache/pinot/pull/11017#discussion_r1281431147 ## pinot-controller/src/main/java/org/apache/pinot/controller/BaseControllerStarter.java: ## @@ -502,6 +495,24 @@ protected void configure() { LOGGER.info("Starti

[GitHub] [pinot] navina commented on a diff in pull request #11017: Remove support for High level consumers in Apache Pinot

navina commented on code in PR #11017: URL: https://github.com/apache/pinot/pull/11017#discussion_r1281425878 ## pinot-spi/src/main/java/org/apache/pinot/spi/stream/StreamConfig.java: ## @@ -223,6 +212,17 @@ public StreamConfig(String tableNameWithType, Map streamConfigMa _

[GitHub] [pinot] navina commented on issue #11211: Make segment upload to deep store misconfiguration fail fast

navina commented on issue #11211: URL: https://github.com/apache/pinot/issues/11211#issuecomment-1661534323 @jadami10 Yeah. ideally, the segment store uri should never be null in the server. However, for whatever reason, we allow that to be null in the server instance. Falling back to c

[GitHub] [pinot] Jackie-Jiang commented on issue #11211: Make segment upload to deep store misconfiguration fail fast

Jackie-Jiang commented on issue #11211: URL: https://github.com/apache/pinot/issues/11211#issuecomment-1661521773 @jadami10 You mean falling back to regular controller upload? That will be more smooth (no failure), but can potentially overwhelm controller because that is the reason why we i

[pinot] branch master updated: add multi-value support for native text index (#11204)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 83ed397210 add multi-value support for native tex

[GitHub] [pinot] Jackie-Jiang merged pull request #11204: add multi-value support for native text index

Jackie-Jiang merged PR #11204: URL: https://github.com/apache/pinot/pull/11204 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11183: Reload Table API to reload table data

Jackie-Jiang commented on code in PR #11183: URL: https://github.com/apache/pinot/pull/11183#discussion_r1281411894 ## pinot-server/src/main/java/org/apache/pinot/server/starter/helix/HelixInstanceDataManager.java: ## @@ -242,6 +242,30 @@ public void addRealtimeSegment(String re

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11183: Reload Table API to reload table data

Jackie-Jiang commented on code in PR #11183: URL: https://github.com/apache/pinot/pull/11183#discussion_r1281409809 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java: ## @@ -696,44 +696,9 @@ public ServerReloadController

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11183: Reload Table API to reload table data

Jackie-Jiang commented on code in PR #11183: URL: https://github.com/apache/pinot/pull/11183#discussion_r1281409133 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ServerReloadControllerJobStatusResponse.java: ## @@ -20,15 +20,25 @@ import java.uti

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11205: [Draft] Remove noop

Jackie-Jiang commented on code in PR #11205: URL: https://github.com/apache/pinot/pull/11205#discussion_r1281397920 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/executor/OpChainSchedulerService.java: ## @@ -31,151 +33,84 @@ import org.slf4j.LoggerFactory;

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11205: [Draft] Remove noop

Jackie-Jiang commented on code in PR #11205: URL: https://github.com/apache/pinot/pull/11205#discussion_r1281390603 ## pinot-core/src/main/java/org/apache/pinot/core/common/Operator.java: ## @@ -39,6 +39,7 @@ public interface Operator { * @throws EarlyTerminationException if

[pinot] branch master updated: [bugfix] Do not move real-time segments to working dir on restart (#11226)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 1e58df7713 [bugfix] Do not move real-time segment

[GitHub] [pinot] Jackie-Jiang merged pull request #11226: [bugfix] Do not move real-time segments to working dir on restart

Jackie-Jiang merged PR #11226: URL: https://github.com/apache/pinot/pull/11226 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11226: [bugfix] Do not move real-time segments to working dir on restart

Jackie-Jiang commented on code in PR #11226: URL: https://github.com/apache/pinot/pull/11226#discussion_r1281388735 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java: ## @@ -232,7 +232,9 @@ public void addSegment(ImmutableSegment immutableS

[GitHub] [pinot] Jackie-Jiang merged pull request #11220: ExpressionFilterOperator NULL support.

Jackie-Jiang merged PR #11220: URL: https://github.com/apache/pinot/pull/11220 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[pinot] branch master updated: ExpressionFilterOperator NULL support. (#11220)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 103b13e39a ExpressionFilterOperator NULL support.

[GitHub] [pinot] Aravind-Suresh commented on a diff in pull request #11183: Reload Table API to reload table data

Aravind-Suresh commented on code in PR #11183: URL: https://github.com/apache/pinot/pull/11183#discussion_r1281376726 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java: ## @@ -696,44 +696,9 @@ public ServerReloadControll

[GitHub] [pinot] Aravind-Suresh commented on a diff in pull request #11183: Reload Table API to reload table data

Aravind-Suresh commented on code in PR #11183: URL: https://github.com/apache/pinot/pull/11183#discussion_r1281375112 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotRealtimeTableResource.java: ## @@ -104,6 +110,70 @@ public Response pauseConsump

[GitHub] [pinot] Aravind-Suresh commented on a diff in pull request #11183: Reload Table API to reload table data

Aravind-Suresh commented on code in PR #11183: URL: https://github.com/apache/pinot/pull/11183#discussion_r1281374635 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotRealtimeTableResource.java: ## @@ -104,6 +110,70 @@ public Response pauseConsump

[GitHub] [pinot] itschrispeck closed issue #11238: Emit LL consumer current offset metrics

itschrispeck closed issue #11238: Emit LL consumer current offset metrics URL: https://github.com/apache/pinot/issues/11238 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [pinot] itschrispeck commented on issue #11238: Emit LL consumer current offset metrics

itschrispeck commented on issue #11238: URL: https://github.com/apache/pinot/issues/11238#issuecomment-1661460705 Thank you for the context! On closer inspection it looks like this metric is already emitted, but only for long offsets due to the issue you refer to https://github.com/a

[GitHub] [pinot] abhioncbr opened a new pull request, #11246: [WIP]: Remove PinotProtocolSocketFactory to reduce the usage of old commons-httpclient

abhioncbr opened a new pull request, #11246: URL: https://github.com/apache/pinot/pull/11246 - Deleting `PinotProtocolSocketFactory` as the first step to cut the usage of old `commons-httpclient` -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [pinot] chenboat commented on a diff in pull request #11098: Introduce FrequentStringsSketch and FrequentLongsSketch aggregation functions

chenboat commented on code in PR #11098: URL: https://github.com/apache/pinot/pull/11098#discussion_r1281307520 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/FrequentLongsSketchAggregationFunction.java: ## @@ -0,0 +1,251 @@ +/** + * Licensed to the

[GitHub] [pinot] chenboat commented on a diff in pull request #11098: Introduce FrequentStringsSketch and FrequentLongsSketch aggregation functions

chenboat commented on code in PR #11098: URL: https://github.com/apache/pinot/pull/11098#discussion_r1281306126 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/FrequentLongsSketchAggregationFunction.java: ## @@ -0,0 +1,251 @@ +/** + * Licensed to the

[GitHub] [pinot] chenboat commented on a diff in pull request #11098: Introduce FrequentStringsSketch and FrequentLongsSketch aggregation functions

chenboat commented on code in PR #11098: URL: https://github.com/apache/pinot/pull/11098#discussion_r1281305652 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/FrequentLongsSketchAggregationFunction.java: ## @@ -0,0 +1,251 @@ +/** + * Licensed to the

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11241: [multistage] support multi-tenant lookup for v2 engine

Jackie-Jiang commented on code in PR #11241: URL: https://github.com/apache/pinot/pull/11241#discussion_r1281301707 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java: ## @@ -299,16 +277,40 @@ private List getListTableConfigs(Lis

[GitHub] [pinot] shenyu0127 commented on issue #11099: OfflineClusterMemBasedBrokerQueryKillingTest.testDigestOOMMultipleQueries is flaky

shenyu0127 commented on issue #11099: URL: https://github.com/apache/pinot/issues/11099#issuecomment-1661328420 @jasperjiaguo Could you take a look at the flaky test? If it is hard to fix, how about deleting the test because we have tested the same behavior in `OfflineClusterM

[GitHub] [pinot] shenyu0127 commented on issue #11099: OfflineClusterMemBasedBrokerQueryKillingTest.testDigestOOMMultipleQueries is flaky

shenyu0127 commented on issue #11099: URL: https://github.com/apache/pinot/issues/11099#issuecomment-1661325913 The test is still flaky. In https://github.com/apache/pinot/pull/11121, I tried to fix the test by - reducing the data size so the servers do not go OOM. - reducing th

[GitHub] [pinot] Aravind-Suresh commented on a diff in pull request #11183: Reload Table API to reload table data

Aravind-Suresh commented on code in PR #11183: URL: https://github.com/apache/pinot/pull/11183#discussion_r1281278417 ## pinot-server/src/main/java/org/apache/pinot/server/starter/helix/HelixInstanceDataManager.java: ## @@ -242,6 +242,30 @@ public void addRealtimeSegment(String

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11017: Remove support for High level consumers in Apache Pinot

Jackie-Jiang commented on code in PR #11017: URL: https://github.com/apache/pinot/pull/11017#discussion_r1281271373 ## pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/src/main/java/org/apache/pinot/plugin/stream/kafka20/KafkaStreamLevelConsumer.java: ## @@ -41,6 +41,7 @@ /

[GitHub] [pinot] Aravind-Suresh commented on a diff in pull request #11183: Reload Table API to reload table data

Aravind-Suresh commented on code in PR #11183: URL: https://github.com/apache/pinot/pull/11183#discussion_r1281277544 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java: ## @@ -170,11 +172,13 @@ public class Pino

[GitHub] [pinot] codecov-commenter commented on pull request #11245: [multistage] Support distinctCountBitmap and *TupleSketch functions

codecov-commenter commented on PR #11245: URL: https://github.com/apache/pinot/pull/11245#issuecomment-1661305092 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11245?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] Aravind-Suresh commented on a diff in pull request #11183: Reload Table API to reload table data

Aravind-Suresh commented on code in PR #11183: URL: https://github.com/apache/pinot/pull/11183#discussion_r1281277201 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ServerReloadControllerJobStatusResponse.java: ## @@ -20,15 +20,25 @@ import java.u

[GitHub] [pinot] Jackie-Jiang commented on issue #11236: No FileSystem for scheme "hdfs"

Jackie-Jiang commented on issue #11236: URL: https://github.com/apache/pinot/issues/11236#issuecomment-1661291566 You should be able to find some answers searching for `Caused by: org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "hdfs"`. I guess it is related to

[pinot] branch master updated: Rebalance unit test and integration test time for github action (#11230)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 98803b786a Rebalance unit test and integration t

[GitHub] [pinot] xiangfu0 merged pull request #11230: Rebalance unit test and integration test time for github action

xiangfu0 merged PR #11230: URL: https://github.com/apache/pinot/pull/11230 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[GitHub] [pinot] xiangfu0 commented on pull request #11230: Rebalance unit test and integration test time for github action

xiangfu0 commented on PR #11230: URL: https://github.com/apache/pinot/pull/11230#issuecomment-1661257346 Another run: https://github.com/apache/pinot/actions/runs/5730948007?pr=11230 Better balanced than before. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #11230: Rebalance unit test and integration test time for github action

xiangfu0 commented on code in PR #11230: URL: https://github.com/apache/pinot/pull/11230#discussion_r1281250638 ## .github/workflows/scripts/.pinot_test.sh: ## @@ -56,18 +56,11 @@ else -pl 'pinot-core' \ -pl 'pinot-query-planner' \ -pl 'pinot-query-run

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #11230: Rebalance unit test and integration test time for github action

xiangfu0 commented on code in PR #11230: URL: https://github.com/apache/pinot/pull/11230#discussion_r1281250432 ## .github/workflows/scripts/.pinot_test.sh: ## @@ -56,18 +56,11 @@ else -pl 'pinot-core' \ -pl 'pinot-query-planner' \ -pl 'pinot-query-run

[GitHub] [pinot] shenyu0127 commented on a diff in pull request #11220: ExpressionFilterOperator NULL support.

shenyu0127 commented on code in PR #11220: URL: https://github.com/apache/pinot/pull/11220#discussion_r1281246684 ## pinot-core/src/main/java/org/apache/pinot/core/operator/dociditerators/ExpressionScanDocIdIterator.java: ## @@ -212,8 +215,27 @@ private void processProjectionBlo

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #11230: Rebalance unit test and integration test time for github action

xiangfu0 commented on code in PR #11230: URL: https://github.com/apache/pinot/pull/11230#discussion_r1281249830 ## .github/workflows/scripts/.pinot_test.sh: ## @@ -56,18 +56,11 @@ else -pl 'pinot-core' \ -pl 'pinot-query-planner' \ -pl 'pinot-query-run

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11183: Reload Table API to reload table data

Jackie-Jiang commented on code in PR #11183: URL: https://github.com/apache/pinot/pull/11183#discussion_r1281223068 ## pinot-common/src/main/java/org/apache/pinot/common/messages/TableReloadMessage.java: ## @@ -0,0 +1,70 @@ +/** + * Licensed to the Apache Software Foundation (AS

[GitHub] [pinot] xiangfu0 opened a new pull request, #11245: Support distinctCountBitmap and *TupleSketch functions in v2

xiangfu0 opened a new pull request, #11245: URL: https://github.com/apache/pinot/pull/11245 Support v2 functions: - distinctCountBitmap - *TupleSketch -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [pinot] walterddr commented on a diff in pull request #11234: [Multi-stage] Support partition based leaf stage processing

walterddr commented on code in PR #11234: URL: https://github.com/apache/pinot/pull/11234#discussion_r1281239345 ## pinot-query-planner/src/main/java/org/apache/pinot/query/routing/WorkerManager.java: ## @@ -96,19 +97,66 @@ private static boolean isLeafPlan(DispatchablePlanMetad

[GitHub] [pinot] jadami10 commented on issue #11244: There's no non-deprecated way to validate a table config without the schema

jadami10 commented on issue #11244: URL: https://github.com/apache/pinot/issues/11244#issuecomment-1661213043 never mind i read this wrong. `/tableConfigs/validate` does exactly this. apologies. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [pinot] jadami10 closed issue #11244: There's no non-deprecated way to validate a table config without the schema

jadami10 closed issue #11244: There's no non-deprecated way to validate a table config without the schema URL: https://github.com/apache/pinot/issues/11244 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[pinot] branch master updated: Update the pinot tenants tables api to support returning broker tagged tables (#11184)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 6bce5faa54 Update the pinot tenants tables api to

[GitHub] [pinot] Jackie-Jiang merged pull request #11184: Update the pinot tenants tables api to support returning broker tagged tables

Jackie-Jiang merged PR #11184: URL: https://github.com/apache/pinot/pull/11184 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] Jackie-Jiang commented on issue #10719: Environment variables not substituted in configuration

Jackie-Jiang commented on issue #10719: URL: https://github.com/apache/pinot/issues/10719#issuecomment-1661201735 This is fixed with #10785 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [pinot] Jackie-Jiang closed issue #10719: Environment variables not substituted in configuration

Jackie-Jiang closed issue #10719: Environment variables not substituted in configuration URL: https://github.com/apache/pinot/issues/10719 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [pinot] Jackie-Jiang commented on pull request #11206: Bump io.grpc:grpc-context from 1.14.0 to 1.57.0

Jackie-Jiang commented on PR #11206: URL: https://github.com/apache/pinot/pull/11206#issuecomment-1661200648 cc @KKcorps -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11220: ExpressionFilterOperator NULL support.

Jackie-Jiang commented on code in PR #11220: URL: https://github.com/apache/pinot/pull/11220#discussion_r1281201506 ## pinot-core/src/main/java/org/apache/pinot/core/operator/dociditerators/ExpressionScanDocIdIterator.java: ## @@ -85,7 +88,7 @@ public int next() { Project

[GitHub] [pinot] codecov-commenter commented on pull request #11242: HAVING filter IS NULL and IS NOT NULL support.

codecov-commenter commented on PR #11242: URL: https://github.com/apache/pinot/pull/11242#issuecomment-1661168992 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11242?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11200: Add instrumentation for dcount merge. Release bitmaps for and/or doc id set to save memory. `performance`

Jackie-Jiang commented on code in PR #11200: URL: https://github.com/apache/pinot/pull/11200#discussion_r1281199310 ## pinot-core/src/main/java/org/apache/pinot/core/operator/docidsets/AndDocIdSet.java: ## @@ -75,13 +77,21 @@ public BlockDocIdIterator iterator() { List bitm

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11200: Add instrumentation for dcount merge. Release bitmaps for and/or doc id set to save memory. `performance`

Jackie-Jiang commented on code in PR #11200: URL: https://github.com/apache/pinot/pull/11200#discussion_r1281197936 ## pinot-core/src/main/java/org/apache/pinot/core/operator/docidsets/AndDocIdSet.java: ## @@ -75,13 +77,22 @@ public BlockDocIdIterator iterator() { List bitm

[GitHub] [pinot] jadami10 opened a new issue, #11244: There's no non-deprecated way to validate a table config without the schema

jadami10 opened a new issue, #11244: URL: https://github.com/apache/pinot/issues/11244 #6840 marked the `/tables/validateTableAndSchema` endpoint as deprecated without offering the same behavior elsewhere. That PR mentions validating a table should require the schema already exist, but that

[GitHub] [pinot] zhouxiz9 commented on pull request #10979: Skip generating rollup task with all merged segments when "skipAllMerged" mode is enabled

zhouxiz9 commented on PR #10979: URL: https://github.com/apache/pinot/pull/10979#issuecomment-1661155624 > @jtao15 @zhouxiz9 I synced up with @jtao15 yesterday. It looks that the issue that we need to address 2 things: > > 1. detect the failure issue earlier than 24 hours. > 2. opt

[GitHub] [pinot] zhouxiz9 commented on a diff in pull request #11243: Make task MaxAttemptsPerTask configurable

zhouxiz9 commented on code in PR #11243: URL: https://github.com/apache/pinot/pull/11243#discussion_r1281188316 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/generator/PinotTaskGenerator.java: ## @@ -66,6 +66,13 @@ default int getNumConcurrentTa

[GitHub] [pinot] zhouxiz9 opened a new pull request, #11243: Make task MaxAttemptsPerTask configurable

zhouxiz9 opened a new pull request, #11243: URL: https://github.com/apache/pinot/pull/11243 This PR makes task MaxAttemptsPerTask configurable. Right now the `MaxAttemptsPerTask` setting is hard coded as 1. This change would allow applications to configure different MaxAttemptsPerTask for t

[GitHub] [pinot] mcvsubbu commented on issue #11238: Emit LL consumer current offset metrics

mcvsubbu commented on issue #11238: URL: https://github.com/apache/pinot/issues/11238#issuecomment-1661141448 There are a few other features we have that may be of help: - Ingestion rate limiter - Data freshness metric - Ingestion based consumption status checker -- This is an

[GitHub] [pinot] mcvsubbu commented on issue #11238: Emit LL consumer current offset metrics

mcvsubbu commented on issue #11238: URL: https://github.com/apache/pinot/issues/11238#issuecomment-1661137431 The LLRealtimeSegmentDataManager is a stream-agnostic class. The long offset is a Kafka-specific way of storing offsets. Other strings have strings, so emitting a metric from this c

[GitHub] [pinot] walterddr commented on a diff in pull request #11230: Rebalance unit test and integration test time for github action

walterddr commented on code in PR #11230: URL: https://github.com/apache/pinot/pull/11230#discussion_r1281178757 ## .github/workflows/scripts/.pinot_test.sh: ## @@ -56,18 +56,11 @@ else -pl 'pinot-core' \ -pl 'pinot-query-planner' \ -pl 'pinot-query-ru

[GitHub] [pinot] shenyu0127 opened a new pull request, #11242: HAVING filter IS NULL and IS NOT NULL support.

shenyu0127 opened a new pull request, #11242: URL: https://github.com/apache/pinot/pull/11242 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

[GitHub] [pinot] codecov-commenter commented on pull request #11241: [multistage] support multi-tenant lookup for v2 engine

codecov-commenter commented on PR #11241: URL: https://github.com/apache/pinot/pull/11241#issuecomment-1661129631 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11241?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] walterddr commented on pull request #11229: [draft][test] no-op removal with yield

walterddr commented on PR #11229: URL: https://github.com/apache/pinot/pull/11229#issuecomment-1661096512 closing in favor of new changes in #11205 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [pinot] walterddr closed pull request #11229: [draft][test] no-op removal with yield

walterddr closed pull request #11229: [draft][test] no-op removal with yield URL: https://github.com/apache/pinot/pull/11229 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [pinot] walterddr commented on pull request #11213: [multistage] enhance submission reduce on dispatch as well

walterddr commented on PR #11213: URL: https://github.com/apache/pinot/pull/11213#issuecomment-1661091683 this PR still incurs race-condition on broker reduce that i don't know why. since it is no longer urgent to fix after #11173 i will shelf it until necessary to revisit -- This is an

[GitHub] [pinot] xiangfu0 commented on pull request #11230: Rebalance unit test and integration test time for github action

xiangfu0 commented on PR #11230: URL: https://github.com/apache/pinot/pull/11230#issuecomment-1661078614 Current setup seems pretty balanced on the runtime for unit test and integration tests runtime. https://github.com/apache/pinot/actions/runs/5730948007/job/15530794730?pr=11230 --

[GitHub] [pinot] walterddr opened a new pull request, #11241: [multistage] support multi-tenant lookup for v2 engine

walterddr opened a new pull request, #11241: URL: https://github.com/apache/pinot/pull/11241 this fixes #11237 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

[pinot] branch master updated: Make test Java 8 compatible (#11239)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new e7e2affa8f Make test Java 8 compatible (#11239) e

[GitHub] [pinot] Jackie-Jiang merged pull request #11239: Make test Java 8 compatible

Jackie-Jiang merged PR #11239: URL: https://github.com/apache/pinot/pull/11239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] tibrewalpratik17 opened a new issue, #11240: Tenant isolation / better task schedule prioritisation in minions

tibrewalpratik17 opened a new issue, #11240: URL: https://github.com/apache/pinot/issues/11240 At present, there is no way to associate specific minion hosts to execute tasks for a set of tables or prioritise task for those tables. This is similar to tenancy model we have for broker and s

[GitHub] [pinot] codecov-commenter commented on pull request #11239: Make test Java 8 compatible

codecov-commenter commented on PR #11239: URL: https://github.com/apache/pinot/pull/11239#issuecomment-1660924210 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11239?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] jadami10 commented on issue #11211: Make segment upload to deep store misconfiguration fail fast

jadami10 commented on issue #11211: URL: https://github.com/apache/pinot/issues/11211#issuecomment-1660865136 for another of our clusters that we set this table config but the servers didn't have https://github.com/apache/pinot/pull/10136, it was just ignored. so that also seems like accept

[GitHub] [pinot] yuanbenson opened a new pull request, #11239: Make test Java 8 compatible

yuanbenson opened a new pull request, #11239: URL: https://github.com/apache/pinot/pull/11239 Removes use of `Arrays.equals(boolean[] a, int aFromIndex, int aToIndex, boolean[] b, int bFromIndex, int bToIndex)` that is only available in https://docs.oracle.com/javase%2F9%2Fdocs%2Fapi%2F%2F

[GitHub] [pinot] walterddr opened a new issue, #11237: [multistage] support common broker tenant

walterddr opened a new issue, #11237: URL: https://github.com/apache/pinot/issues/11237 #10336 supported tenant-based v2 queries. however it doesn't handle cross-tenant access when tables are residing in different tenant but the broker has knowledge over all tables the following t

[GitHub] [pinot] isonusingh opened a new issue, #11236: No FileSystem for scheme "hdfs"

isonusingh opened a new issue, #11236: URL: https://github.com/apache/pinot/issues/11236 Hi Experts, I am trying to setup HDFS as deep storage for Apache Pinot. Below is the error I am facing: Did not find any fs classes in the configuration Got scheme hdfs, initializing class

[GitHub] [pinot] sylph-eu commented on pull request #11226: [bugfix] Do not move real-time segments to working dir on restart

sylph-eu commented on PR #11226: URL: https://github.com/apache/pinot/pull/11226#issuecomment-1660752656 Rebased -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

[GitHub] [pinot] walterddr commented on a diff in pull request #11205: [Draft] Remove noop

walterddr commented on code in PR #11205: URL: https://github.com/apache/pinot/pull/11205#discussion_r1280900873 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/ReceivingMailbox.java: ## @@ -51,16 +52,30 @@ public class ReceivingMailbox { // TODO: Revisit

[GitHub] [pinot] snleee commented on pull request #10979: Skip generating rollup task with all merged segments when "skipAllMerged" mode is enabled

snleee commented on PR #10979: URL: https://github.com/apache/pinot/pull/10979#issuecomment-1660648865 @jtao15 @zhouxiz9 I synced up with @jtao15 yesterday. It looks that the issue that we need to decouple 3 things: 1. detect the failure issue earlier than 24 hours. 2. optimize the

[pinot] branch master updated: [multistage] support more agg functions currently implementable (#11208)

This is an automated email from the ASF dual-hosted git repository. rongr pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new c68b3be360 [multistage] support more agg functions

[GitHub] [pinot] walterddr merged pull request #11208: [multistage] support the rest of the agg functions currently implementable

walterddr merged PR #11208: URL: https://github.com/apache/pinot/pull/11208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

[pinot] branch master updated: Support st-union function in v2 (#11235)

This is an automated email from the ASF dual-hosted git repository. yupeng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new d2789319d5 Support st-union function in v2 (#1123

[GitHub] [pinot] yupeng9 merged pull request #11235: [multistage] Support st-union function

yupeng9 merged PR #11235: URL: https://github.com/apache/pinot/pull/11235 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

[GitHub] [pinot] walterddr commented on a diff in pull request #11205: [Draft] Remove noop

walterddr commented on code in PR #11205: URL: https://github.com/apache/pinot/pull/11205#discussion_r1280818049 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/ReceivingMailbox.java: ## @@ -51,16 +52,30 @@ public class ReceivingMailbox { // TODO: Revisit

[GitHub] [pinot] walterddr commented on a diff in pull request #11205: [Draft] Remove noop

walterddr commented on code in PR #11205: URL: https://github.com/apache/pinot/pull/11205#discussion_r1280755616 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/ReceivingMailbox.java: ## @@ -79,7 +94,11 @@ public boolean offer(TransferableBlock block, long t

[GitHub] [pinot] gortiz commented on a diff in pull request #11205: [Draft] Remove noop

gortiz commented on code in PR #11205: URL: https://github.com/apache/pinot/pull/11205#discussion_r1280795098 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/ReceivingMailbox.java: ## @@ -133,4 +152,15 @@ public void cancel() { public int getNumPendingBloc

[GitHub] [pinot] walterddr commented on a diff in pull request #11205: [Draft] Remove noop

walterddr commented on code in PR #11205: URL: https://github.com/apache/pinot/pull/11205#discussion_r1280753470 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/ReceivingMailbox.java: ## @@ -51,16 +52,30 @@ public class ReceivingMailbox { // TODO: Revisit

[GitHub] [pinot] walterddr commented on a diff in pull request #11205: [Draft] Remove noop

walterddr commented on code in PR #11205: URL: https://github.com/apache/pinot/pull/11205#discussion_r1280085055 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/plan/OpChainExecutionContext.java: ## @@ -62,10 +65,10 @@ public OpChainExecutionContext(MailboxSe

  1   2   >