[GitHub] [pinot] sajjad-moradi opened a new issue, #10060: RT segment size does NOT follow "realtime.segment.flush.desired.size" parameter

2023-01-04 Thread GitBox
sajjad-moradi opened a new issue, #10060: URL: https://github.com/apache/pinot/issues/10060 Segments of realtime tables which have "flush.desired.size" configured do not honor the specified value (reported [here](https://apache-pinot.slack.com/archives/C011C9JHN7R/p1672836315569889)). Here

[pinot] branch master updated: Create metadata only tarball for metadata push job (#10034)

2023-01-04 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new b58514c799 Create metadata only tarball for meta

[GitHub] [pinot] xiangfu0 merged pull request #10034: Create metadata only tarball for metadata push job

2023-01-04 Thread GitBox
xiangfu0 merged PR #10034: URL: https://github.com/apache/pinot/pull/10034 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[GitHub] [pinot] tanmesh commented on pull request #10042: [WIP] Adding support for broker short-circuit for point queries w/ a limit clause

2023-01-04 Thread GitBox
tanmesh commented on PR #10042: URL: https://github.com/apache/pinot/pull/10042#issuecomment-1371818382 I am working on adding Test by taking inspiration from [testCancelQuery()](https://github.com/apache/pinot/blob/master/pinot-broker/src/test/java/org/apache/pinot/broker/requesthandler/Bas

[GitHub] [pinot] xiangfu0 commented on pull request #10027: Repurpose MinionInstancesCleanupTask to StaleInstancesCleanupTask to remove stale broker/server instances

2023-01-04 Thread GitBox
xiangfu0 commented on PR #10027: URL: https://github.com/apache/pinot/pull/10027#issuecomment-1371815209 Updated to https://docs.pinot.apache.org/basics/components/controller#staleinstancescleanuptask -- This is an automated message from the Apache Git Service. To respond to the message,

[pinot] branch master updated: Repurpose MinionInstancesCleanupTask to StaleInstancesCleanupTask to remove stale broker and server instances as well. (#10027)

2023-01-04 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 98c39a5f71 Repurpose MinionInstancesCleanupTask

[GitHub] [pinot] xiangfu0 merged pull request #10027: Repurpose MinionInstancesCleanupTask to StaleInstancesCleanupTask to remove stale broker/server instances

2023-01-04 Thread GitBox
xiangfu0 merged PR #10027: URL: https://github.com/apache/pinot/pull/10027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[GitHub] [pinot] tanmesh commented on a diff in pull request #10042: [WIP] Adding support for broker short-circuit for point queries w/ a limit clause

2023-01-04 Thread GitBox
tanmesh commented on code in PR #10042: URL: https://github.com/apache/pinot/pull/10042#discussion_r1062093669 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/SingleConnectionBrokerRequestHandler.java: ## @@ -106,13 +106,36 @@ protected BrokerResponseNative

[GitHub] [pinot] codecov-commenter commented on pull request #10059: Add pinot Prefix to Server Thread Pool Config

2023-01-04 Thread GitBox
codecov-commenter commented on PR #10059: URL: https://github.com/apache/pinot/pull/10059#issuecomment-1371737001 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10059?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] kishoreg commented on pull request #10000: [Issue 7519] Adds support for multiple filtered/unfiltered aggregations with GROUP BY

2023-01-04 Thread GitBox
kishoreg commented on PR #1: URL: https://github.com/apache/pinot/pull/1#issuecomment-1371730973 Thanks @egalpin for your contribution. Please add docs when you get a chance if its not done already -- This is an automated message from the Apache Git Service. To respond to the mes

[pinot] branch master updated: [Issue 7519] Adds support for multiple filtered/unfiltered aggregations with GROUP BY (#10000)

2023-01-04 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 303b1a7cbe [Issue 7519] Adds support for multiple

[GitHub] [pinot] Jackie-Jiang closed issue #7519: FILTER Clause Support

2023-01-04 Thread GitBox
Jackie-Jiang closed issue #7519: FILTER Clause Support URL: https://github.com/apache/pinot/issues/7519 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [pinot] Jackie-Jiang merged pull request #10000: [Issue 7519] Adds support for multiple filtered/unfiltered aggregations with GROUP BY

2023-01-04 Thread GitBox
Jackie-Jiang merged PR #1: URL: https://github.com/apache/pinot/pull/1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] ankitsultana opened a new pull request, #10059: Add pinot Prefix to Server Thread Pool Config

2023-01-04 Thread GitBox
ankitsultana opened a new pull request, #10059: URL: https://github.com/apache/pinot/pull/10059 In this [PR](https://github.com/apache/pinot/pull/10001) the configs looked like so: ``` controller.http.server.thread.pool.corePoolSize: 10 controller.http.server.thread.pool.maxPool

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

2023-01-04 Thread GitBox
Jackie-Jiang commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1062020489 ## pinot-common/src/main/java/org/apache/pinot/common/metrics/ServerGauge.java: ## @@ -45,8 +45,10 @@ public enum ServerGauge implements AbstractMetrics.Gauge { /

[GitHub] [pinot] Jackie-Jiang commented on pull request #10027: Repurpose MinionInstancesCleanupTask to StaleInstancesCleanupTask to remove stale broker/server instances

2023-01-04 Thread GitBox
Jackie-Jiang commented on PR #10027: URL: https://github.com/apache/pinot/pull/10027#issuecomment-1371628736 Please update the PR description with some release note, and also update the pinot doc about the new configs -- This is an automated message from the Apache Git Service. To respond

[GitHub] [pinot] vvivekiyer commented on pull request #10054: Add DistinctSum aggregation function

2023-01-04 Thread GitBox
vvivekiyer commented on PR #10054: URL: https://github.com/apache/pinot/pull/10054#issuecomment-1371608356 @Jackie-Jiang Yes was thinking about adding DistinctAvg in a future PR. Good point though. Let me rework on this PR by adding that functionality as well. I'll extract out the co

[GitHub] [pinot] codecov-commenter commented on pull request #10043: evaluate EQ queries against range index when available

2023-01-04 Thread GitBox
codecov-commenter commented on PR #10043: URL: https://github.com/apache/pinot/pull/10043#issuecomment-1371598926 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10043?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] jugomezv commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

2023-01-04 Thread GitBox
jugomezv commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1061991555 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/IngestionDelayTracker.java: ## @@ -0,0 +1,350 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] [pinot] Jackie-Jiang commented on pull request #10054: Add DistinctSum aggregation function

2023-01-04 Thread GitBox
Jackie-Jiang commented on PR #10054: URL: https://github.com/apache/pinot/pull/10054#issuecomment-1371561789 Are you planning to add `DistinctAvg` as well? The difference between `DistinctCount`, `DistinctSum` and `DistinctAvg` is just the broker merging, so should we consider extracting th

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #10034: Create metadata only tarball for metadata push job

2023-01-04 Thread GitBox
xiangfu0 commented on code in PR #10034: URL: https://github.com/apache/pinot/pull/10034#discussion_r1061980199 ## pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-hadoop/src/main/java/org/apache/pinot/plugin/ingestion/batch/hadoop/HadoopSegmentCreationMapper.java: ## @

[GitHub] [pinot] Jackie-Jiang commented on pull request #10055: Fix issue with S3 FS not working with segments with spaces in the name

2023-01-04 Thread GitBox
Jackie-Jiang commented on PR #10055: URL: https://github.com/apache/pinot/pull/10055#issuecomment-1371557458 Can we also add a test to guard the behavior? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10055: Fix issue with S3 FS not working with segments with spaces in the name

2023-01-04 Thread GitBox
Jackie-Jiang commented on code in PR #10055: URL: https://github.com/apache/pinot/pull/10055#discussion_r1061978669 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/store/SegmentDirectory.java: ## @@ -98,6 +98,8 @@ */ public abstract class SegmentDirectory impl

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10044: push utf8 comparison down into ValueReader

2023-01-04 Thread GitBox
Jackie-Jiang commented on code in PR #10044: URL: https://github.com/apache/pinot/pull/10044#discussion_r1061976752 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/util/ValueReaderComparisons.java: ## @@ -0,0 +1,156 @@ +/** + * Licensed to the Apache Softw

[pinot] branch range-index-equals-queries updated (ab2804b9e6 -> f72f22eac2)

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a change to branch range-index-equals-queries in repository https://gitbox.apache.org/repos/asf/pinot.git discard ab2804b9e6 evaluate EQ queries against range index when available add f72f22eac2 evalu

[GitHub] [pinot] jugomezv commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

2023-01-04 Thread GitBox
jugomezv commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1061974248 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/IngestionDelayTracker.java: ## @@ -0,0 +1,350 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] [pinot] jugomezv commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

2023-01-04 Thread GitBox
jugomezv commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1061972872 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/data/manager/TableDataManager.java: ## @@ -201,4 +201,18 @@ void addOrReplaceSegment(String segmentNa

[GitHub] [pinot] richardstartin commented on a diff in pull request #10044: push utf8 comparison down into ValueReader

2023-01-04 Thread GitBox
richardstartin commented on code in PR #10044: URL: https://github.com/apache/pinot/pull/10044#discussion_r1061970177 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/util/ValueReaderComparisons.java: ## @@ -0,0 +1,156 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] jugomezv commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

2023-01-04 Thread GitBox
jugomezv commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1061968962 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/data/manager/TableDataManager.java: ## @@ -44,6 +44,12 @@ @ThreadSafe public interface TableDataMan

[GitHub] [pinot] jugomezv commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

2023-01-04 Thread GitBox
jugomezv commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1061968584 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -212,6 +216,83 @@ protected void doShutdown() { if (_le

[GitHub] [pinot] snleee commented on a diff in pull request #10045: add checkpointing support to flink connector

2023-01-04 Thread GitBox
snleee commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1061952606 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -151,4 +148,18 @@ private void flush()

[GitHub] [pinot] snleee commented on a diff in pull request #10045: add checkpointing support to flink connector

2023-01-04 Thread GitBox
snleee commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1061952606 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -151,4 +148,18 @@ private void flush()

[GitHub] [pinot] snleee commented on a diff in pull request #10045: add checkpointing support to flink connector

2023-01-04 Thread GitBox
snleee commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1061952606 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -151,4 +148,18 @@ private void flush()

[GitHub] [pinot] snleee commented on a diff in pull request #10045: add checkpointing support to flink connector

2023-01-04 Thread GitBox
snleee commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1061952606 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -151,4 +148,18 @@ private void flush()

[GitHub] [pinot] snleee commented on a diff in pull request #10045: add checkpointing support to flink connector

2023-01-04 Thread GitBox
snleee commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1061952606 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -151,4 +148,18 @@ private void flush()

[GitHub] [pinot] walterddr commented on a diff in pull request #10006: [refactor] combine operator API changes

2023-01-04 Thread GitBox
walterddr commented on code in PR #10006: URL: https://github.com/apache/pinot/pull/10006#discussion_r1061941643 ## pinot-core/src/test/java/org/apache/pinot/core/operator/combine/CombineSlowOperatorsTest.java: ## @@ -194,17 +197,19 @@ private void testCancelCombineOperator(Bas

[GitHub] [pinot] snleee commented on a diff in pull request #10045: add checkpointing support to flink connector

2023-01-04 Thread GitBox
snleee commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1061939317 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -96,7 +102,7 @@ public void close() th

[GitHub] [pinot] snleee commented on a diff in pull request #10045: add checkpointing support to flink connector

2023-01-04 Thread GitBox
snleee commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1061937252 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -117,31 +123,22 @@ public void close() @Ov

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10058: [refactor] move min/max combine out

2023-01-04 Thread GitBox
Jackie-Jiang commented on code in PR #10058: URL: https://github.com/apache/pinot/pull/10058#discussion_r1061936474 ## pinot-core/src/main/java/org/apache/pinot/core/plan/CombinePlanNode.java: ## @@ -190,6 +194,13 @@ public List callJob() { return new SelectionOnlyCombi

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #10027: Repurpose MinionInstancesCleanupTask to StaleInstancesCleanupTask to remove stale broker/server instances

2023-01-04 Thread GitBox
xiangfu0 commented on code in PR #10027: URL: https://github.com/apache/pinot/pull/10027#discussion_r1061933883 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/cleanup/StaleInstancesCleanupTask.java: ## @@ -82,7 +95,47 @@ protected void runTask(Propertie

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #10027: Repurpose MinionInstancesCleanupTask to StaleInstancesCleanupTask to remove stale broker/server instances

2023-01-04 Thread GitBox
xiangfu0 commented on code in PR #10027: URL: https://github.com/apache/pinot/pull/10027#discussion_r1061933761 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/cleanup/StaleInstancesCleanupTask.java: ## @@ -34,23 +36,30 @@ /** - * A periodic task to

[GitHub] [pinot] mcvsubbu commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

2023-01-04 Thread GitBox
mcvsubbu commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1061906746 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/LLRealtimeSegmentDataManager.java: ## @@ -1594,4 +1616,9 @@ public String getSegmentName() { p

[GitHub] [pinot] klsince commented on a diff in pull request #10044: push utf8 comparison down into ValueReader

2023-01-04 Thread GitBox
klsince commented on code in PR #10044: URL: https://github.com/apache/pinot/pull/10044#discussion_r1061898335 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/util/ValueReaderComparisons.java: ## @@ -0,0 +1,156 @@ +/** + * Licensed to the Apache Software F

[GitHub] [pinot] jugomezv commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

2023-01-04 Thread GitBox
jugomezv commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1061886497 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -212,6 +216,83 @@ protected void doShutdown() { if (_le

[GitHub] [pinot] codecov-commenter commented on pull request #10058: [refactor] move min/max combine out

2023-01-04 Thread GitBox
codecov-commenter commented on PR #10058: URL: https://github.com/apache/pinot/pull/10058#issuecomment-1371411106 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10058?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10044: push utf8 comparison down into ValueReader

2023-01-04 Thread GitBox
Jackie-Jiang commented on code in PR #10044: URL: https://github.com/apache/pinot/pull/10044#discussion_r1061869181 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/util/ValueReaderComparisons.java: ## @@ -0,0 +1,174 @@ +/** + * Licensed to the Apache Softw

[GitHub] [pinot] Jackie-Jiang commented on issue #9989: Derivated column not reloaded when formula changes

2023-01-04 Thread GitBox
Jackie-Jiang commented on issue #9989: URL: https://github.com/apache/pinot/issues/9989#issuecomment-1371395434 The force update flag for schema update is introduced in #9382 which will be available in release 12.0 (we are currently in the progress of releasing it). Without that you can try

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10042: [WIP] Adding support for broker short-circuit for point queries w/ a limit clause

2023-01-04 Thread GitBox
Jackie-Jiang commented on code in PR #10042: URL: https://github.com/apache/pinot/pull/10042#discussion_r1061857493 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/SingleConnectionBrokerRequestHandler.java: ## @@ -106,13 +106,36 @@ protected BrokerResponseNa

[pinot] branch range-index-equals-queries updated (d42589a934 -> ab2804b9e6)

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a change to branch range-index-equals-queries in repository https://gitbox.apache.org/repos/asf/pinot.git omit d42589a934 evaluate EQ queries against range index when available add 880a5c779f emit

[pinot] branch propagate-inversion updated (897a5a224e -> 4307f59bf5)

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a change to branch propagate-inversion in repository https://gitbox.apache.org/repos/asf/pinot.git omit 897a5a224e print explain plan when integration test query fails to ease debugging add 4307f5

[pinot] branch explain-plan-on-error updated (c045a58ed7 -> 0d75855627)

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a change to branch explain-plan-on-error in repository https://gitbox.apache.org/repos/asf/pinot.git omit c045a58ed7 print explain plan when integration test query fails to ease debugging add 0d75

[pinot] branch explain-plan-on-error updated (54e58e1971 -> c045a58ed7)

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a change to branch explain-plan-on-error in repository https://gitbox.apache.org/repos/asf/pinot.git omit 54e58e1971 print explain plan when integration test query fails to ease debugging add c045

[pinot] branch propagate-inversion updated (ad9af0595d -> 897a5a224e)

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a change to branch propagate-inversion in repository https://gitbox.apache.org/repos/asf/pinot.git discard ad9af0595d print explain plan when integration test query fails to ease debugging add 897a5a

[pinot] branch propagate-inversion updated (d90d9232b8 -> ad9af0595d)

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a change to branch propagate-inversion in repository https://gitbox.apache.org/repos/asf/pinot.git omit d90d9232b8 print explain plan when integration test query fails to ease debugging add ad9af0

[GitHub] [pinot] Jackie-Jiang commented on pull request #10052: Add an API to download validDocIdsSnapshots from peer servers

2023-01-04 Thread GitBox
Jackie-Jiang commented on PR #10052: URL: https://github.com/apache/pinot/pull/10052#issuecomment-1371370149 Let's make the API download the current validDocIds instead of reading the snapshot file. We want the up-to-date validDocIds -- This is an automated message from the Apache Git Ser

[GitHub] [pinot] Jackie-Jiang commented on pull request #10001: Add Support for Tuning HTTP Server Thread Pool

2023-01-04 Thread GitBox
Jackie-Jiang commented on PR #10001: URL: https://github.com/apache/pinot/pull/10001#issuecomment-1371367160 Thanks! The doc is backed by this repo: https://github.com/pinot-contrib/pinot-docs -- This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [pinot] egalpin commented on pull request #10000: [Issue 7519] Adds support for multiple filtered/unfiltered aggregations with GROUP BY

2023-01-04 Thread GitBox
egalpin commented on PR #1: URL: https://github.com/apache/pinot/pull/1#issuecomment-1371358983 @Jackie-Jiang the integration test failure looks unrelated to me as the same tests are failing in the same way on other PRs / the failures don't seem to point to the changed areas of code

[GitHub] [pinot] codecov-commenter commented on pull request #10057: print explain plan when integration test query fails to ease debugging

2023-01-04 Thread GitBox
codecov-commenter commented on PR #10057: URL: https://github.com/apache/pinot/pull/10057#issuecomment-1371356122 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10057?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] navina commented on a diff in pull request #10047: [WIP] Add upsert ttl configs for Pinot upsert optimizations

2023-01-04 Thread GitBox
navina commented on code in PR #10047: URL: https://github.com/apache/pinot/pull/10047#discussion_r1061832162 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/TTLPartitionUpsertMetadataManager.java: ## @@ -0,0 +1,68 @@ +/** + * Licensed to the Apache So

[GitHub] [pinot] navina commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

2023-01-04 Thread GitBox
navina commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1061817534 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/LLRealtimeSegmentDataManager.java: ## @@ -1594,4 +1616,9 @@ public String getSegmentName() { pub

[GitHub] [pinot] navina commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

2023-01-04 Thread GitBox
navina commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1061812908 ## pinot-server/src/main/java/org/apache/pinot/server/starter/helix/IngestionBasedConsumptionStatusChecker.java: ## @@ -77,6 +77,9 @@ public int getNumConsumingSegmentsNo

[GitHub] [pinot] navina commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

2023-01-04 Thread GitBox
navina commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1061806303 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -212,6 +216,83 @@ protected void doShutdown() { if (_leas

[GitHub] [pinot] richardstartin opened a new pull request, #10057: print explain plan when integration test query fails to ease debugging

2023-01-04 Thread GitBox
richardstartin opened a new pull request, #10057: URL: https://github.com/apache/pinot/pull/10057 Makes it possible to diagnose cause of failed query integration test, which is otherwise completely opaque. The new error message looks like this: ``` Value: 1 does not match, expected

[pinot] 01/01: print explain plan when integration test query fails to ease debugging

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a commit to branch explain-plan-on-error in repository https://gitbox.apache.org/repos/asf/pinot.git commit 54e58e1971e56fd57adb5a9fd81bb0b9265e29ff Author: Richard Startin AuthorDate: Wed Jan 4 18:44:47 20

[pinot] branch explain-plan-on-error created (now 54e58e1971)

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a change to branch explain-plan-on-error in repository https://gitbox.apache.org/repos/asf/pinot.git at 54e58e1971 print explain plan when integration test query fails to ease debugging This branch

[pinot] branch propagate-inversion updated (b403f262ea -> d90d9232b8)

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a change to branch propagate-inversion in repository https://gitbox.apache.org/repos/asf/pinot.git from b403f262ea formatting add d90d9232b8 print explain plan when integration test query fails to

[GitHub] [pinot] walterddr commented on issue #10012: Flaky Test for `ForwardIndexHandlerTest`

2023-01-04 Thread GitBox
walterddr commented on issue #10012: URL: https://github.com/apache/pinot/issues/10012#issuecomment-1371288090 duplicate of https://github.com/apache/pinot/issues/9831 ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [pinot] navina commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

2023-01-04 Thread GitBox
navina commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1061765187 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/LLRealtimeSegmentDataManager.java: ## @@ -611,6 +616,8 @@ private boolean processStreamEvents(Messa

[GitHub] [pinot] navina commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

2023-01-04 Thread GitBox
navina commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1061765187 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/LLRealtimeSegmentDataManager.java: ## @@ -611,6 +616,8 @@ private boolean processStreamEvents(Messa

[GitHub] [pinot] saurabhd336 closed pull request #10023: Use inputstream when parsing http responses in CompletionService

2023-01-04 Thread GitBox
saurabhd336 closed pull request #10023: Use inputstream when parsing http responses in CompletionService URL: https://github.com/apache/pinot/pull/10023 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [pinot] walterddr commented on a diff in pull request #10006: [refactor] combine operator API changes

2023-01-04 Thread GitBox
walterddr commented on code in PR #10006: URL: https://github.com/apache/pinot/pull/10006#discussion_r1061736016 ## pinot-core/src/main/java/org/apache/pinot/core/operator/streaming/StreamingInstanceResponseOperator.java: ## @@ -34,25 +41,53 @@ public class StreamingInstanceRes

[pinot] branch propagate-inversion updated (304b15b807 -> b403f262ea)

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a change to branch propagate-inversion in repository https://gitbox.apache.org/repos/asf/pinot.git from 304b15b807 propagate inverted status with BitmapDocIdIterator to avoid expensive flipping of bitm

[pinot] branch propagate-inversion updated (06a7753e54 -> 304b15b807)

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a change to branch propagate-inversion in repository https://gitbox.apache.org/repos/asf/pinot.git omit 06a7753e54 propagate inverted status with BitmapDocIdIterator to avoid expensive flipping of bitm

[GitHub] [pinot] richardstartin opened a new pull request, #10056: propagate inverted status with BitmapDocIdIterator to avoid expensive flipping of bitmaps

2023-01-04 Thread GitBox
richardstartin opened a new pull request, #10056: URL: https://github.com/apache/pinot/pull/10056 This supersedes #8546 which addressed a bottleneck clearly visible in queries with negations -- This is an automated message from the Apache Git Service. To respond to the message, please log

[pinot] branch propagate-inversion updated (5dbc01fb00 -> 06a7753e54)

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a change to branch propagate-inversion in repository https://gitbox.apache.org/repos/asf/pinot.git discard 5dbc01fb00 propagate inverted status with BitmapDocIdIterator to avoid expensive flipping of bitm

[pinot] 01/01: propagate inverted status with BitmapDocIdIterator to avoid expensive flipping of bitmaps

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a commit to branch propagate-inversion in repository https://gitbox.apache.org/repos/asf/pinot.git commit 06a7753e54f41db9ae87c0d9944caba6affb8667 Author: Richard Startin AuthorDate: Wed Jan 4 09:00:31 2023

[pinot] 01/01: propagate inverted status with BitmapDocIdIterator to avoid expensive flipping of bitmaps

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a commit to branch propagate-inversion in repository https://gitbox.apache.org/repos/asf/pinot.git commit 5dbc01fb000c0f05e9e589da1db75ee3312ff650 Author: Richard Startin AuthorDate: Wed Jan 4 09:00:31 2023

[pinot] branch propagate-inversion created (now 5dbc01fb00)

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a change to branch propagate-inversion in repository https://gitbox.apache.org/repos/asf/pinot.git at 5dbc01fb00 propagate inverted status with BitmapDocIdIterator to avoid expensive flipping of bitm

[GitHub] [pinot] codecov-commenter commented on pull request #10055: Fix issue with S3 FS not working with segments with spaces in the name

2023-01-04 Thread GitBox
codecov-commenter commented on PR #10055: URL: https://github.com/apache/pinot/pull/10055#issuecomment-1370886055 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10055?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] bobby-richard commented on a diff in pull request #10045: add checkpointing support to flink connector

2023-01-04 Thread GitBox
bobby-richard commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1061424930 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -117,31 +123,22 @@ public void close()

[GitHub] [pinot] bobby-richard commented on a diff in pull request #10045: add checkpointing support to flink connector

2023-01-04 Thread GitBox
bobby-richard commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1061423619 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -151,4 +148,18 @@ private void flush()

[GitHub] [pinot] bobby-richard commented on a diff in pull request #10045: add checkpointing support to flink connector

2023-01-04 Thread GitBox
bobby-richard commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1061421013 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -96,7 +102,7 @@ public void close()

[GitHub] [pinot] saurabhd336 opened a new pull request, #10055: Fix issue with S3 FS not working with segments with spaces in the name

2023-01-04 Thread GitBox
saurabhd336 opened a new pull request, #10055: URL: https://github.com/apache/pinot/pull/10055 S3PinotFS fails to process segments with spaces in their names. This is due to URI.getPath() returning decoded path while the `.prefix` function expects unescaped path. -- This is an automa

[GitHub] [pinot] richardstartin commented on a diff in pull request #10044: push utf8 comparison down into ValueReader

2023-01-04 Thread GitBox
richardstartin commented on code in PR #10044: URL: https://github.com/apache/pinot/pull/10044#discussion_r1061330332 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/util/ValueReaderComparisons.java: ## @@ -0,0 +1,174 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] richardstartin commented on a diff in pull request #10044: push utf8 comparison down into ValueReader

2023-01-04 Thread GitBox
richardstartin commented on code in PR #10044: URL: https://github.com/apache/pinot/pull/10044#discussion_r1061329509 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/util/ValueReaderComparisons.java: ## @@ -0,0 +1,174 @@ +/** + * Licensed to the Apache Sof

[pinot] branch pushdown-comparison updated (011b086da0 -> 7f9f1b5733)

2023-01-04 Thread richardstartin
This is an automated email from the ASF dual-hosted git repository. richardstartin pushed a change to branch pushdown-comparison in repository https://gitbox.apache.org/repos/asf/pinot.git omit 011b086da0 don't apply to BYTES dictionaries, refactor add 7f9f1b5733 don't apply to BYTES di

[GitHub] [pinot] richardstartin commented on a diff in pull request #10044: push utf8 comparison down into ValueReader

2023-01-04 Thread GitBox
richardstartin commented on code in PR #10044: URL: https://github.com/apache/pinot/pull/10044#discussion_r1061282306 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/util/ValueReaderComparisons.java: ## @@ -0,0 +1,174 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] richardstartin commented on a diff in pull request #10044: push utf8 comparison down into ValueReader

2023-01-04 Thread GitBox
richardstartin commented on code in PR #10044: URL: https://github.com/apache/pinot/pull/10044#discussion_r1061277160 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/util/ValueReaderComparisons.java: ## @@ -0,0 +1,174 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] richardstartin commented on a diff in pull request #10044: push utf8 comparison down into ValueReader

2023-01-04 Thread GitBox
richardstartin commented on code in PR #10044: URL: https://github.com/apache/pinot/pull/10044#discussion_r1061273375 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/util/ValueReaderComparisons.java: ## @@ -0,0 +1,174 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] richardstartin commented on a diff in pull request #10044: push utf8 comparison down into ValueReader

2023-01-04 Thread GitBox
richardstartin commented on code in PR #10044: URL: https://github.com/apache/pinot/pull/10044#discussion_r1061263982 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/util/ValueReaderComparisons.java: ## @@ -0,0 +1,174 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] richardstartin commented on a diff in pull request #10044: push utf8 comparison down into ValueReader

2023-01-04 Thread GitBox
richardstartin commented on code in PR #10044: URL: https://github.com/apache/pinot/pull/10044#discussion_r1061263198 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/util/ValueReaderComparisons.java: ## @@ -0,0 +1,174 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] richardstartin commented on a diff in pull request #10044: push utf8 comparison down into ValueReader

2023-01-04 Thread GitBox
richardstartin commented on code in PR #10044: URL: https://github.com/apache/pinot/pull/10044#discussion_r1061262121 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/util/ValueReaderComparisons.java: ## @@ -0,0 +1,174 @@ +/** + * Licensed to the Apache Sof

[GitHub] [pinot] richardstartin closed pull request #10046: bytes dictionaries don't work with variable length byte[]s

2023-01-04 Thread GitBox
richardstartin closed pull request #10046: bytes dictionaries don't work with variable length byte[]s URL: https://github.com/apache/pinot/pull/10046 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [pinot] Airliquide76 commented on issue #9989: Derivated column not reloaded when formula changes

2023-01-04 Thread GitBox
Airliquide76 commented on issue #9989: URL: https://github.com/apache/pinot/issues/9989#issuecomment-1370616700 I cannot delete colum from schema. Only allowed option add column. De : Xiaotian (Jackie) Jiang ***@***.***> Envoyé : mercredi 4 janvie