[GitHub] [incubator-pinot] codecov-io commented on pull request #6094: Implement the segment merge task generator

codecov-io commented on pull request #6094: URL: https://github.com/apache/incubator-pinot/pull/6094#issuecomment-709807920 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6094?src=pr&el=h1) Report > Merging [#6094](https://codecov.io/gh/apache/incubator-pinot/pull/6094?s

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6046: Deep Extraction Support for ORC, Thrift, and ProtoBuf Records

codecov-io edited a comment on pull request #6046: URL: https://github.com/apache/incubator-pinot/pull/6046#issuecomment-709805252 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6046?src=pr&el=h1) Report > Merging [#6046](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] codecov-io commented on pull request #6046: Deep Extraction Support for ORC, Thrift, and ProtoBuf Records

codecov-io commented on pull request #6046: URL: https://github.com/apache/incubator-pinot/pull/6046#issuecomment-709805252 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6046?src=pr&el=h1) Report > Merging [#6046](https://codecov.io/gh/apache/incubator-pinot/pull/6046?s

[GitHub] [incubator-pinot] snleee commented on a change in pull request #6094: Implement the segment merge task generator

snleee commented on a change in pull request #6094: URL: https://github.com/apache/incubator-pinot/pull/6094#discussion_r506053544 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/mergestrategy/MergeStrategy.java ## @@ -0,0 +1,28 @@ +/

[GitHub] [incubator-pinot] snleee commented on a change in pull request #6094: Implement the segment merge task generator

snleee commented on a change in pull request #6094: URL: https://github.com/apache/incubator-pinot/pull/6094#discussion_r506053443 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/mergestrategy/MergeStrategy.java ## @@ -0,0 +1,28 @@ +/

[GitHub] [incubator-pinot] timsants commented on a change in pull request #6046: Deep Extraction Support for ORC, Thrift, and ProtoBuf Records

timsants commented on a change in pull request #6046: URL: https://github.com/apache/incubator-pinot/pull/6046#discussion_r506052351 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/data/readers/BaseRecordExtractor.java ## @@ -0,0 +1,190 @@ +/** + * Licensed to the A

[GitHub] [incubator-pinot] snleee commented on a change in pull request #6094: Implement the segment merge task generator

snleee commented on a change in pull request #6094: URL: https://github.com/apache/incubator-pinot/pull/6094#discussion_r506050967 ## File path: pinot-core/src/test/java/org/apache/pinot/core/segment/processing/framework/SegmentProcessingFrameworkTest.java ## @@ -325,7 +325,7

[GitHub] [incubator-pinot] snleee commented on a change in pull request #6094: Implement the segment merge task generator

snleee commented on a change in pull request #6094: URL: https://github.com/apache/incubator-pinot/pull/6094#discussion_r506050739 ## File path: pinot-core/src/main/java/org/apache/pinot/core/segment/name/NormalizedDateSegmentNameGenerator.java ## @@ -88,9 +96,9 @@ public Stri

[GitHub] [incubator-pinot] snleee commented on a change in pull request #6094: Implement the segment merge task generator

snleee commented on a change in pull request #6094: URL: https://github.com/apache/incubator-pinot/pull/6094#discussion_r506050239 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/generator/SegmentMergeRollupTaskGenerator.java ## @@ -0

[GitHub] [incubator-pinot] timsants commented on a change in pull request #6046: Deep Extraction Support for ORC, Thrift, and ProtoBuf Records

timsants commented on a change in pull request #6046: URL: https://github.com/apache/incubator-pinot/pull/6046#discussion_r506046635 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/data/readers/BaseRecordExtractor.java ## @@ -0,0 +1,190 @@ +/** + * Licensed to the A

[GitHub] [incubator-pinot] snleee commented on a change in pull request #6094: Implement the segment merge task generator

snleee commented on a change in pull request #6094: URL: https://github.com/apache/incubator-pinot/pull/6094#discussion_r506046413 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/generator/SegmentMergeRollupTaskGenerator.java ## @@ -0

[GitHub] [incubator-pinot] timsants commented on a change in pull request #6046: Deep Extraction Support for ORC, Thrift, and ProtoBuf Records

timsants commented on a change in pull request #6046: URL: https://github.com/apache/incubator-pinot/pull/6046#discussion_r506045817 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/data/readers/BaseRecordExtractor.java ## @@ -0,0 +1,190 @@ +/** + * Licensed to the A

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6149: add validation for upsert tables

codecov-io edited a comment on pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#issuecomment-709566750 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6149?src=pr&el=h1) Report > Merging [#6149](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6149: add validation for upsert tables

codecov-io edited a comment on pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#issuecomment-709566750 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6149?src=pr&el=h1) Report > Merging [#6149](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] yupeng9 commented on a change in pull request #6149: add validation for upsert tables

yupeng9 commented on a change in pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#discussion_r506032003 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/TableConfigUtils.java ## @@ -230,6 +235,39 @@ private static void validateInges

[GitHub] [incubator-pinot] yupeng9 commented on a change in pull request #6149: add validation for upsert tables

yupeng9 commented on a change in pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#discussion_r506031794 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/TableConfigUtils.java ## @@ -230,6 +234,38 @@ private static void validateInges

[GitHub] [incubator-pinot] jackjlli commented on pull request #6139: Remove tyrus dependencies in pinot-tools module

jackjlli commented on pull request #6139: URL: https://github.com/apache/incubator-pinot/pull/6139#issuecomment-709667036 @Jackie-Jiang The root cause is that in some machine the `tyrus` jar appears before `jersey-container-grizzly2-http`. The reason why this issue cannot be reproduced

[GitHub] [incubator-pinot] Jackie-Jiang closed issue #6144: Querying an indexed field for distinct values is slow

Jackie-Jiang closed issue #6144: URL: https://github.com/apache/incubator-pinot/issues/6144 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6149: add validation for upsert tables

Jackie-Jiang commented on a change in pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#discussion_r505924807 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/TableConfigUtils.java ## @@ -230,6 +235,39 @@ private static void validate

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6149: add validation for upsert tables

Jackie-Jiang commented on a change in pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#discussion_r505924483 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/TableConfigUtils.java ## @@ -230,6 +234,38 @@ private static void validate

[incubator-pinot] branch master updated (05f1a89 -> 44fcf1e)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 05f1a89 [TE] Add thirdeye-dashboard dep to thirdeye-dist (#6135) add 44fcf1e add query option of disa

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #6141: add query option of disabling upsert during query

Jackie-Jiang merged pull request #6141: URL: https://github.com/apache/incubator-pinot/pull/6141 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6046: Deep Extraction Support for ORC, Thrift, and ProtoBuf Records

Jackie-Jiang commented on a change in pull request #6046: URL: https://github.com/apache/incubator-pinot/pull/6046#discussion_r505918629 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/data/readers/BaseRecordExtractor.java ## @@ -0,0 +1,190 @@ +/** + * Licensed to t

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6149: add validation for upsert tables

codecov-io edited a comment on pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#issuecomment-709566750 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6149?src=pr&el=h1) Report > Merging [#6149](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] Jackie-Jiang commented on pull request #6139: Remove tyrus dependencies in pinot-tools module

Jackie-Jiang commented on pull request #6139: URL: https://github.com/apache/incubator-pinot/pull/6139#issuecomment-709638616 I'm against changing the data to a static file. The purpose of the quick start is letting the user play with Pinot and they should be able to see the latest events

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6149: add validation for upsert tables

codecov-io edited a comment on pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#issuecomment-709566750 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6149?src=pr&el=h1) Report > Merging [#6149](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] icefury71 commented on a change in pull request #6149: add validation for upsert tables

icefury71 commented on a change in pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#discussion_r505909024 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/TableConfigUtils.java ## @@ -230,6 +234,38 @@ private static void validateIng

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6124: RealtimeToOfflineSegments task generator

Jackie-Jiang commented on a change in pull request #6124: URL: https://github.com/apache/incubator-pinot/pull/6124#discussion_r505756502 ## File path: pinot-common/src/main/java/org/apache/pinot/common/minion/RealtimeToOfflineSegmentsTaskMetadata.java ## @@ -0,0 +1,106 @@ +/**

[GitHub] [incubator-pinot] yupeng9 commented on pull request #6149: add validation for upsert tables

yupeng9 commented on pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#issuecomment-709626207 @icefury71 @Jackie-Jiang thanks for the review. Comments addressed. This is an automated message from th

[GitHub] [incubator-pinot] lgo commented on issue #6145: Querying a partitioned column for a non-existent value times out

lgo commented on issue #6145: URL: https://github.com/apache/incubator-pinot/issues/6145#issuecomment-709623706 Gathering more details from folks who had originally noticed this issue, will circle back when I have specifics. I might have mispoken about the issue since just trying what I d

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6141: add query option of disabling upsert during query

codecov-io edited a comment on pull request #6141: URL: https://github.com/apache/incubator-pinot/pull/6141#issuecomment-708169596 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6141?src=pr&el=h1) Report > Merging [#6141](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] lgo commented on issue #6144: Querying an indexed field for distinct values is slow

lgo commented on issue #6144: URL: https://github.com/apache/incubator-pinot/issues/6144#issuecomment-709619746 Ah I did not find that original ticket! Feel free to close this in favor of that one since it sounds like the same thing (:

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6141: add query option of disabling upsert during query

codecov-io edited a comment on pull request #6141: URL: https://github.com/apache/incubator-pinot/pull/6141#issuecomment-708169596 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6141?src=pr&el=h1) Report > Merging [#6141](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] yupeng9 commented on a change in pull request #6149: add validation for upsert tables

yupeng9 commented on a change in pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#discussion_r505885977 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/TableConfigUtils.java ## @@ -230,6 +234,38 @@ private static void validateInges

[GitHub] [incubator-pinot] yupeng9 commented on a change in pull request #6149: add validation for upsert tables

yupeng9 commented on a change in pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#discussion_r505884895 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/TableConfigUtils.java ## @@ -230,6 +234,38 @@ private static void validateInges

[GitHub] [incubator-pinot] yupeng9 commented on a change in pull request #6149: add validation for upsert tables

yupeng9 commented on a change in pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#discussion_r505877388 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/TableConfigUtils.java ## @@ -230,6 +234,38 @@ private static void validateInges

[GitHub] [incubator-pinot] icefury71 commented on a change in pull request #6149: add validation for upsert tables

icefury71 commented on a change in pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#discussion_r505828804 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/TableConfigUtils.java ## @@ -230,6 +234,38 @@ private static void validateIng

[GitHub] [incubator-pinot] icefury71 commented on pull request #6149: add validation for upsert tables

icefury71 commented on pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#issuecomment-709577612 Can you add unit test for this as well please ? Look at TableConfigUtilsTest This is an automated mess

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6149: add validation for upsert tables

Jackie-Jiang commented on a change in pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#discussion_r505814806 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/TableConfigUtils.java ## @@ -230,6 +234,38 @@ private static void validate

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6141: add query option of disabling upsert during query

Jackie-Jiang commented on a change in pull request #6141: URL: https://github.com/apache/incubator-pinot/pull/6141#discussion_r505812869 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/QueryOptions.java ## @@ -64,6 +67,10 @@ public boolean isPreserveType() {

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6149: add validation for upsert tables

codecov-io edited a comment on pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#issuecomment-709566750 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6149?src=pr&el=h1) Report > Merging [#6149](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] codecov-io commented on pull request #6149: add validation for upsert tables

codecov-io commented on pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149#issuecomment-709566750 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6149?src=pr&el=h1) Report > Merging [#6149](https://codecov.io/gh/apache/incubator-pinot/pull/6149?s

[incubator-pinot] branch te-remove-shade-plugin created (now 05f1a89)

This is an automated email from the ASF dual-hosted git repository. jihao pushed a change to branch te-remove-shade-plugin in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at 05f1a89 [TE] Add thirdeye-dashboard dep to thirdeye-dist (#6135) No new revisions were adde

[incubator-pinot] branch master updated (30bed9b -> 05f1a89)

This is an automated email from the ASF dual-hosted git repository. jihao pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 30bed9b add query runner support for query file resampling (#6136) add 05f1a89 [TE] Add thirdeye-dashb

[GitHub] [incubator-pinot] jihaozh merged pull request #6135: [TE] Add thirdeye-dashboard dep to thirdeye-dist

jihaozh merged pull request #6135: URL: https://github.com/apache/incubator-pinot/pull/6135 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[incubator-pinot] branch master updated (a71b960 -> 30bed9b)

This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from a71b960 Add OnHeapGuavaBloomFilterReader (#6147) add 30bed9b add query runner support for query file

[incubator-pinot] branch master updated (a71b960 -> 30bed9b)

This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from a71b960 Add OnHeapGuavaBloomFilterReader (#6147) add 30bed9b add query runner support for query file

[GitHub] [incubator-pinot] apucher merged pull request #6136: add query runner support for query file resampling

apucher merged pull request #6136: URL: https://github.com/apache/incubator-pinot/pull/6136 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [incubator-pinot] yupeng9 opened a new pull request #6149: add validation for upsert tables

yupeng9 opened a new pull request #6149: URL: https://github.com/apache/incubator-pinot/pull/6149 ## Description Part of a series of PRs for #4261 Check this [doc](https://docs.google.com/document/d/1qljEMndPMxbbKtjlVn9mn2toz7Qrk0TGQsHLfI--7h8/edit#heading=h.lsfmyoyyxtgt) out for the

[incubator-pinot] branch query-runner-sampling-mode updated (0546110 -> 7737d15)

This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch query-runner-sampling-mode in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 0546110 track exception counts add 7737d15 increase targetQPS accuracy No new r

[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6145: Querying a partitioned column for a non-existent value times out

Jackie-Jiang commented on issue #6145: URL: https://github.com/apache/incubator-pinot/issues/6145#issuecomment-709507451 This is unexpected. This query should be at lease as fast as `SELECT * FROM table WHERE type = 'foo'`. Is this query always timing out? If it can return result sometime

[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6144: Querying an indexed field for distinct values is slow

Jackie-Jiang commented on issue #6144: URL: https://github.com/apache/incubator-pinot/issues/6144#issuecomment-709505265 Inverted index only helps during the filtering. The performance difference between these 2 queries should mainly coming from the group-by using the dictionary id instea

[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6081: Optimize the performance of DISTINCT queries

Jackie-Jiang commented on issue #6081: URL: https://github.com/apache/incubator-pinot/issues/6081#issuecomment-709502802 Use dictionary ids instead of fetching actual values inner segment This is an automated message from th

[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6144: Querying an indexed field for distinct values is slow

Jackie-Jiang commented on issue #6144: URL: https://github.com/apache/incubator-pinot/issues/6144#issuecomment-709502945 This is related to #6081 This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #6147: Add OnHeapGuavaBloomFilterReader

Jackie-Jiang merged pull request #6147: URL: https://github.com/apache/incubator-pinot/pull/6147 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[incubator-pinot] branch master updated: Add OnHeapGuavaBloomFilterReader (#6147)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new a71b960 Add OnHeapGuavaBloomFilterRead

[GitHub] [incubator-pinot] ianvkoeppe commented on issue #5751: Support Serving Latest Offline Segments Immediately

ianvkoeppe commented on issue #5751: URL: https://github.com/apache/incubator-pinot/issues/5751#issuecomment-709426214 @mayankshriv, I would like to still get this done, but it's not something I can prioritize right now. Th

[GitHub] [incubator-pinot] shahsank3t opened a new pull request #6148: UI integration of instance and segment operations

shahsank3t opened a new pull request #6148: URL: https://github.com/apache/incubator-pinot/pull/6148 ## Description Adding Operations for Instances (Server & Broker)  ![ima

[GitHub] [incubator-pinot] snleee commented on a change in pull request #6124: RealtimeToOfflineSegments task generator

snleee commented on a change in pull request #6124: URL: https://github.com/apache/incubator-pinot/pull/6124#discussion_r505190192 ## File path: pinot-common/src/main/java/org/apache/pinot/common/minion/RealtimeToOfflineSegmentsTaskMetadata.java ## @@ -0,0 +1,106 @@ +/** + * L