[GitHub] [pinot] gortiz commented on a diff in pull request #10977: allow to add custom context into query trace

2023-06-27 Thread via GitHub
gortiz commented on code in PR #10977: URL: https://github.com/apache/pinot/pull/10977#discussion_r1244761724 ## pinot-core/src/main/java/org/apache/pinot/core/util/trace/BuiltInTracer.java: ## @@ -50,6 +50,15 @@ public void close() { } TraceContext.logTime(operato

[GitHub] [pinot] robertzych commented on a diff in pull request #10463: Segment compaction for upsert real-time tables

2023-06-27 Thread via GitHub
robertzych commented on code in PR #10463: URL: https://github.com/apache/pinot/pull/10463#discussion_r1244647288 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -489,14 +489,16 @@ static void setDefaultTimeValueIfInva

[GitHub] [pinot] codecov-commenter commented on pull request #10990: Updated code for setting value of segment min/max property.

2023-06-27 Thread via GitHub
codecov-commenter commented on PR #10990: URL: https://github.com/apache/pinot/pull/10990#issuecomment-1610632503 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10990?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] abhioncbr commented on a diff in pull request #10990: Updated code to getting valid value for segment min/max property.

2023-06-27 Thread via GitHub
abhioncbr commented on code in PR #10990: URL: https://github.com/apache/pinot/pull/10990#discussion_r1244585380 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreatorTest.java: ## @@ -174,21 +202,52 @@ private static

[GitHub] [pinot] abhioncbr opened a new pull request, #10990: Updated code to getting valid value for segment min/max property.

2023-06-27 Thread via GitHub
abhioncbr opened a new pull request, #10990: URL: https://github.com/apache/pinot/pull/10990 Previously, we skipped to set the value for min/max if the value was invalid. We were determining invalid values based on the following checks - If value contains leading or trailing whitespace c

[GitHub] [pinot] itschrispeck opened a new pull request, #10989: do not readd lucene readers to queue if segment is destroyed

2023-06-27 Thread via GitHub
itschrispeck opened a new pull request, #10989: URL: https://github.com/apache/pinot/pull/10989 Issue described in: https://github.com/apache/pinot/issues/10976 Tested the cherry-picked commit on an internal cluster, verified via logs that `_luceneRealtimeReaders` no longer constantly

[GitHub] [pinot] Jackie-Jiang commented on pull request #10985: [Clean up] Do not count DISTINCT as aggregation

2023-06-27 Thread via GitHub
Jackie-Jiang commented on PR #10985: URL: https://github.com/apache/pinot/pull/10985#issuecomment-1610421570 cc @vvivekiyer @somandal This should simplify the aggregation function creation on V2 engine -- This is an automated message from the Apache Git Service. To respond to the message,

[pinot] branch master updated: Enhance early terminate for combine operator (#10988)

2023-06-27 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 04bdb9a397 Enhance early terminate for combine op

[GitHub] [pinot] Jackie-Jiang merged pull request #10988: Enhance early terminate for combine operator

2023-06-27 Thread via GitHub
Jackie-Jiang merged PR #10988: URL: https://github.com/apache/pinot/pull/10988 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10969: Instance drop operation validation check API

2023-06-27 Thread via GitHub
Jackie-Jiang commented on code in PR #10969: URL: https://github.com/apache/pinot/pull/10969#discussion_r1244528990 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotInstanceRestletResource.java: ## @@ -390,4 +391,33 @@ public SuccessResponse updat

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10926: Realtime pre-aggregation for Distinct Count HLL & Big Decimal

2023-06-27 Thread via GitHub
Jackie-Jiang commented on code in PR #10926: URL: https://github.com/apache/pinot/pull/10926#discussion_r1244507824 ## pinot-core/src/test/java/org/apache/pinot/core/common/ObjectSerDeUtilsTest.java: ## @@ -374,4 +378,38 @@ public void testDouble2LongMap() { assertEquals(

[GitHub] [pinot] codecov-commenter commented on pull request #10988: Enhance early terminate for combine operator

2023-06-27 Thread via GitHub
codecov-commenter commented on PR #10988: URL: https://github.com/apache/pinot/pull/10988#issuecomment-1610407375 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10988?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] abhioncbr commented on a diff in pull request #10891: Ensure min/max value generation in the segment metadata.

2023-06-27 Thread via GitHub
abhioncbr commented on code in PR #10891: URL: https://github.com/apache/pinot/pull/10891#discussion_r1244511327 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java: ## @@ -570,12 +570,14 @@ public static void

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10463: Segment compaction for upsert real-time tables

2023-06-27 Thread via GitHub
Jackie-Jiang commented on code in PR #10463: URL: https://github.com/apache/pinot/pull/10463#discussion_r1244504059 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -489,14 +489,16 @@ static void setDefaultTimeValueIfIn

[GitHub] [pinot] codecov-commenter commented on pull request #10985: [Clean up] Do not count DISTINCT as aggregation

2023-06-27 Thread via GitHub
codecov-commenter commented on PR #10985: URL: https://github.com/apache/pinot/pull/10985#issuecomment-1610386681 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10985?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] codecov-commenter commented on pull request #10987: (Still WIP) Adding segmentRowComputer interface for SegmentProcessorFramework to use

2023-06-27 Thread via GitHub
codecov-commenter commented on PR #10987: URL: https://github.com/apache/pinot/pull/10987#issuecomment-1610382874 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10987?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] walterddr closed pull request #10984: [bugfix] check exception before create group-by result block

2023-06-27 Thread via GitHub
walterddr closed pull request #10984: [bugfix] check exception before create group-by result block URL: https://github.com/apache/pinot/pull/10984 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [pinot] walterddr commented on pull request #10984: [bugfix] check exception before create group-by result block

2023-06-27 Thread via GitHub
walterddr commented on PR #10984: URL: https://github.com/apache/pinot/pull/10984#issuecomment-1610380414 +1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [pinot] Jackie-Jiang commented on pull request #10984: [bugfix] check exception before create group-by result block

2023-06-27 Thread via GitHub
Jackie-Jiang commented on PR #10984: URL: https://github.com/apache/pinot/pull/10984#issuecomment-1610379488 We don't really need to merge the exceptions since we don't return the result anyway. Early termination should be more desired. See #10988 -- This is an automated message from the

[GitHub] [pinot] Jackie-Jiang opened a new pull request, #10988: Enhance early terminate for combine operator

2023-06-27 Thread via GitHub
Jackie-Jiang opened a new pull request, #10988: URL: https://github.com/apache/pinot/pull/10988 - When exception happens during query execution, stop processing the remaining segments for all threads - (bugfix) Remove the member variable override in `BaseSingleBlockCombineOperator` which

[GitHub] [pinot] swaminathanmanish commented on pull request #10987: (Still WIP) Adding segmentRowComputer interface for SegmentProcessorFramework to use

2023-06-27 Thread via GitHub
swaminathanmanish commented on PR #10987: URL: https://github.com/apache/pinot/pull/10987#issuecomment-1610354443 cc @snleee - Please take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [pinot] swaminathanmanish opened a new pull request, #10987: (Still WIP) Adding segmentRowComputer interface for SegmentProcessorFramework to use

2023-06-27 Thread via GitHub
swaminathanmanish opened a new pull request, #10987: URL: https://github.com/apache/pinot/pull/10987 **Problem**: Currently, we dont have size based segment creation for batch ingestion that uses SegmentProcessorFramework. With maxNumRecordsPerSegment, we cannot create segment of desire

[GitHub] [pinot] wirybeaver commented on issue #10981: json_extract_scalar times out when the expression contains index

2023-06-27 Thread via GitHub
wirybeaver commented on issue #10981: URL: https://github.com/apache/pinot/issues/10981#issuecomment-1610353025 All segments are in Consuming State, i.e., mutable segment -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [pinot] walterddr commented on a diff in pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-27 Thread via GitHub
walterddr commented on code in PR #10982: URL: https://github.com/apache/pinot/pull/10982#discussion_r1244468924 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/utils/TableConfigUtilsTest.java: ## @@ -1755,6 +1755,14 @@ public void testTaskConfig() { } c

[GitHub] [pinot] wirybeaver commented on issue #10981: json_extract_scalar times out when the expression contains index

2023-06-27 Thread via GitHub
wirybeaver commented on issue #10981: URL: https://github.com/apache/pinot/issues/10981#issuecomment-1610334868 ``` SELECT * FROM rta_odin_analytics_staging_audit_v6 where JSON_MATCH(direct_tags, '"$.root_workflow[0].tag_value" = ''foo''') ``` works -- This is an automated m

[GitHub] [pinot] wirybeaver commented on issue #10981: json_extract_scalar times out when the expression contains index

2023-06-27 Thread via GitHub
wirybeaver commented on issue #10981: URL: https://github.com/apache/pinot/issues/10981#issuecomment-1610332723 The pinot field spec: { "name": "direct_tags", "dataType": "STRING", "maxLength": 102400 } -- This is an automated message from the Apach

[GitHub] [pinot] shwin opened a new issue, #10986: Count discrepancy among `in`/`=` and `between` queries using timestamp index

2023-06-27 Thread via GitHub
shwin opened a new issue, #10986: URL: https://github.com/apache/pinot/issues/10986 we're seeing something strange with timestamp indices and segment pruning. ``` We have a timestamp index like this: { "name": "blockTimestamp", "encodingType": "DICTIONARY",

[GitHub] [pinot] andimiller commented on a diff in pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-27 Thread via GitHub
andimiller commented on code in PR #10982: URL: https://github.com/apache/pinot/pull/10982#discussion_r1244438998 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/utils/TableConfigUtilsTest.java: ## @@ -1755,6 +1755,14 @@ public void testTaskConfig() { }

[GitHub] [pinot] Jackie-Jiang opened a new pull request, #10985: [Clean up] Do not count DISTINCT as aggregation

2023-06-27 Thread via GitHub
Jackie-Jiang opened a new pull request, #10985: URL: https://github.com/apache/pinot/pull/10985 Context: - When introducing the first version of `DISTINCT`, we treat it as aggregation. Then we found a lot of limitations because of that. To solve the limitations we implemented a dedicate

[GitHub] [pinot] andimiller commented on a diff in pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-27 Thread via GitHub
andimiller commented on code in PR #10982: URL: https://github.com/apache/pinot/pull/10982#discussion_r124492 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/utils/TableConfigUtilsTest.java: ## @@ -1755,6 +1755,14 @@ public void testTaskConfig() { }

[GitHub] [pinot] andimiller commented on a diff in pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-27 Thread via GitHub
andimiller commented on code in PR #10982: URL: https://github.com/apache/pinot/pull/10982#discussion_r124492 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/utils/TableConfigUtilsTest.java: ## @@ -1755,6 +1755,14 @@ public void testTaskConfig() { }

[GitHub] [pinot] andimiller commented on a diff in pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-27 Thread via GitHub
andimiller commented on code in PR #10982: URL: https://github.com/apache/pinot/pull/10982#discussion_r1244438998 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/utils/TableConfigUtilsTest.java: ## @@ -1755,6 +1755,14 @@ public void testTaskConfig() { }

[GitHub] [pinot] codecov-commenter commented on pull request #10984: [bugfix] check exception before create group-by result block

2023-06-27 Thread via GitHub
codecov-commenter commented on PR #10984: URL: https://github.com/apache/pinot/pull/10984#issuecomment-1610304258 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10984?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] robertzych commented on a diff in pull request #10463: Segment compaction for upsert real-time tables

2023-06-27 Thread via GitHub
robertzych commented on code in PR #10463: URL: https://github.com/apache/pinot/pull/10463#discussion_r1244432499 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -489,14 +489,16 @@ static void setDefaultTimeValueIfInva

[GitHub] [pinot] chenboat commented on issue #10981: json_extract_scalar times out when the expression contains index

2023-06-27 Thread via GitHub
chenboat commented on issue #10981: URL: https://github.com/apache/pinot/issues/10981#issuecomment-1610280711 cc @Jackie-Jiang @mayankshriv -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [pinot] walterddr commented on a diff in pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-27 Thread via GitHub
walterddr commented on code in PR #10982: URL: https://github.com/apache/pinot/pull/10982#discussion_r1244407923 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/utils/TableConfigUtilsTest.java: ## @@ -1755,6 +1755,14 @@ public void testTaskConfig() { } c

[GitHub] [pinot] walterddr commented on a diff in pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-27 Thread via GitHub
walterddr commented on code in PR #10982: URL: https://github.com/apache/pinot/pull/10982#discussion_r1244407923 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/utils/TableConfigUtilsTest.java: ## @@ -1755,6 +1755,14 @@ public void testTaskConfig() { } c

[GitHub] [pinot] walterddr commented on a diff in pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-27 Thread via GitHub
walterddr commented on code in PR #10982: URL: https://github.com/apache/pinot/pull/10982#discussion_r1244407923 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/utils/TableConfigUtilsTest.java: ## @@ -1755,6 +1755,14 @@ public void testTaskConfig() { } c

[GitHub] [pinot] walterddr commented on a diff in pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-27 Thread via GitHub
walterddr commented on code in PR #10982: URL: https://github.com/apache/pinot/pull/10982#discussion_r1244407923 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/utils/TableConfigUtilsTest.java: ## @@ -1755,6 +1755,14 @@ public void testTaskConfig() { } c

[GitHub] [pinot] andimiller commented on a diff in pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-27 Thread via GitHub
andimiller commented on code in PR #10982: URL: https://github.com/apache/pinot/pull/10982#discussion_r1244392095 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/utils/TableConfigUtilsTest.java: ## @@ -1755,6 +1755,14 @@ public void testTaskConfig() { }

[GitHub] [pinot] walterddr opened a new pull request, #10984: [bugfix] check exception before create group-by result block

2023-06-27 Thread via GitHub
walterddr opened a new pull request, #10984: URL: https://github.com/apache/pinot/pull/10984 fix NPE when indexTable is not created but exception is thrown. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [pinot] codecov-commenter commented on pull request #10983: Message batch ingestion lag fix

2023-06-27 Thread via GitHub
codecov-commenter commented on PR #10983: URL: https://github.com/apache/pinot/pull/10983#issuecomment-1610245244 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10983?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] codecov-commenter commented on pull request #10979: Skip generating rollup task with all merged segments

2023-06-27 Thread via GitHub
codecov-commenter commented on PR #10979: URL: https://github.com/apache/pinot/pull/10979#issuecomment-1610240202 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10979?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] walterddr commented on a diff in pull request #10937: New API to get tenant tags

2023-06-27 Thread via GitHub
walterddr commented on code in PR #10937: URL: https://github.com/apache/pinot/pull/10937#discussion_r1244338599 ## pinot-broker/src/main/java/org/apache/pinot/broker/api/resources/InstanceResource.java: ## @@ -0,0 +1,69 @@ +/** + * Licensed to the Apache Software Foundation (AS

[pinot] branch master updated (b001c40182 -> 238629c41c)

2023-06-27 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from b001c40182 [multistage][feature] pipeline breaker on broker reduce (#10971) add 238629c41c Support NULL for multi

[GitHub] [pinot] Jackie-Jiang merged pull request #10938: Support NULL for multi-column GROUP BY.

2023-06-27 Thread via GitHub
Jackie-Jiang merged PR #10938: URL: https://github.com/apache/pinot/pull/10938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] klsince commented on a diff in pull request #10977: allow to add custom context into query trace

2023-06-27 Thread via GitHub
klsince commented on code in PR #10977: URL: https://github.com/apache/pinot/pull/10977#discussion_r1244334712 ## pinot-core/src/main/java/org/apache/pinot/core/util/trace/BuiltInTracer.java: ## @@ -50,6 +50,15 @@ public void close() { } TraceContext.logTime(operat

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10905: Startree index build enhancement

2023-06-27 Thread via GitHub
Jackie-Jiang commented on code in PR #10905: URL: https://github.com/apache/pinot/pull/10905#discussion_r1244331242 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilder.java: ## @@ -83,7 +86,11 @@ public MultipleTreesBuilde

[GitHub] [pinot] klsince commented on a diff in pull request #10977: allow to add custom context into query trace

2023-06-27 Thread via GitHub
klsince commented on code in PR #10977: URL: https://github.com/apache/pinot/pull/10977#discussion_r1244334712 ## pinot-core/src/main/java/org/apache/pinot/core/util/trace/BuiltInTracer.java: ## @@ -50,6 +50,15 @@ public void close() { } TraceContext.logTime(operat

[GitHub] [pinot] walterddr merged pull request #10971: [multistage][feature] pipeline breaker on broker reduce

2023-06-27 Thread via GitHub
walterddr merged PR #10971: URL: https://github.com/apache/pinot/pull/10971 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

[pinot] branch master updated: use MultiStageRequestIdGenerator in v1 engine (#10966)

2023-06-27 Thread rongr
This is an automated email from the ASF dual-hosted git repository. rongr pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 3b07def331 use MultiStageRequestIdGenerator in v1

[GitHub] [pinot] klsince merged pull request #10977: allow to add custom context into query trace

2023-06-27 Thread via GitHub
klsince merged PR #10977: URL: https://github.com/apache/pinot/pull/10977 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

[pinot] branch master updated (fc9f56b07b -> b001c40182)

2023-06-27 Thread rongr
This is an automated email from the ASF dual-hosted git repository. rongr pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from fc9f56b07b allow to add custom context into query trace (#10977) add b001c40182 [multistage][feature] pipeline brea

[GitHub] [pinot] walterddr closed issue #10131: [multistage] Broker busy waiting

2023-06-27 Thread via GitHub
walterddr closed issue #10131: [multistage] Broker busy waiting URL: https://github.com/apache/pinot/issues/10131 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

[pinot] branch master updated (3b07def331 -> fc9f56b07b)

2023-06-27 Thread xbli
This is an automated email from the ASF dual-hosted git repository. xbli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 3b07def331 use MultiStageRequestIdGenerator in v1 engine (#10966) add fc9f56b07b allow to add custom context into qu

[GitHub] [pinot] walterddr merged pull request #10966: use MultiStageRequestIdGenerator in v1 engine

2023-06-27 Thread via GitHub
walterddr merged PR #10966: URL: https://github.com/apache/pinot/pull/10966 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

[GitHub] [pinot] walterddr commented on a diff in pull request #10977: allow to add custom context into query trace

2023-06-27 Thread via GitHub
walterddr commented on code in PR #10977: URL: https://github.com/apache/pinot/pull/10977#discussion_r1244329566 ## pinot-core/src/main/java/org/apache/pinot/core/util/trace/BuiltInTracer.java: ## @@ -50,6 +50,15 @@ public void close() { } TraceContext.logTime(oper

[GitHub] [pinot] walterddr commented on a diff in pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-27 Thread via GitHub
walterddr commented on code in PR #10982: URL: https://github.com/apache/pinot/pull/10982#discussion_r1244327305 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/utils/TableConfigUtilsTest.java: ## @@ -1755,6 +1755,14 @@ public void testTaskConfig() { } c

[GitHub] [pinot] walterddr commented on a diff in pull request #10983: Message batch ingestion lag fix

2023-06-27 Thread via GitHub
walterddr commented on code in PR #10983: URL: https://github.com/apache/pinot/pull/10983#discussion_r1244326440 ## pinot-spi/src/test/java/org/apache/pinot/spi/stream/StreamDataDecoderImplTest.java: ## @@ -41,7 +41,7 @@ public void testDecodeValueOnly() messageDecoder.init

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10463: Segment compaction for upsert real-time tables

2023-06-27 Thread via GitHub
Jackie-Jiang commented on code in PR #10463: URL: https://github.com/apache/pinot/pull/10463#discussion_r1244318587 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -489,14 +489,16 @@ static void setDefaultTimeValueIfIn

[GitHub] [pinot] summerhasama-stripe opened a new pull request, #10983: Message batch ingestion lag fix

2023-06-27 Thread via GitHub
summerhasama-stripe opened a new pull request, #10983: URL: https://github.com/apache/pinot/pull/10983 We saw a lot of logs for the FreshnessBasedConsumptionStatusChecker that show the servers are not aware of the freshness of incoming events Through some debugging, we determined the

[GitHub] [pinot] robertzych commented on a diff in pull request #10463: Segment compaction for upsert real-time tables

2023-06-27 Thread via GitHub
robertzych commented on code in PR #10463: URL: https://github.com/apache/pinot/pull/10463#discussion_r1244251665 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -489,14 +489,16 @@ static void setDefaultTimeValueIfInva

[GitHub] [pinot] ksnijjer commented on pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-27 Thread via GitHub
ksnijjer commented on PR #10982: URL: https://github.com/apache/pinot/pull/10982#issuecomment-1610041218 @mayankshriv FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [pinot] codecov-commenter commented on pull request #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-27 Thread via GitHub
codecov-commenter commented on PR #10982: URL: https://github.com/apache/pinot/pull/10982#issuecomment-1610021184 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10982?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] walterddr commented on a diff in pull request #10873: Use pinot-query-planner utils to extract table names in pinot-controller

2023-06-27 Thread via GitHub
walterddr commented on code in PR #10873: URL: https://github.com/apache/pinot/pull/10873#discussion_r1244103752 ## pinot-query-planner/src/main/java/org/apache/pinot/query/QueryEnvironment.java: ## @@ -208,6 +210,21 @@ public String explainQuery(String sqlQuery) { return e

[GitHub] [pinot] andimiller opened a new pull request, #10982: Allow extra aggregation types in RealtimeToOfflineSegmentsTask

2023-06-27 Thread via GitHub
andimiller opened a new pull request, #10982: URL: https://github.com/apache/pinot/pull/10982 The only thing blocking these being used was the table validator in `TableConfigUtils` so I've updated it. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [pinot] walterddr commented on a diff in pull request #10873: Use pinot-query-planner utils to extract table names in pinot-controller

2023-06-27 Thread via GitHub
walterddr commented on code in PR #10873: URL: https://github.com/apache/pinot/pull/10873#discussion_r1244098059 ## pinot-query-planner/src/main/java/org/apache/pinot/query/QueryEnvironment.java: ## @@ -208,6 +210,24 @@ public String explainQuery(String sqlQuery) { return e

[GitHub] [pinot] gortiz commented on pull request #10953: Add a PrioritizedFilterOperator

2023-06-27 Thread via GitHub
gortiz commented on PR #10953: URL: https://github.com/apache/pinot/pull/10953#issuecomment-1609272970 Can we merge this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [pinot] tsegismont commented on pull request #10326: Provide results in CompletableFuture for java clients and expose metrics

2023-06-27 Thread via GitHub
tsegismont commented on PR #10326: URL: https://github.com/apache/pinot/pull/10326#issuecomment-1609021564 Thank you @Jackie-Jiang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [pinot] jtao15 commented on a diff in pull request #10964: Allow custom segment grouping in MergeRollupTask

2023-06-27 Thread via GitHub
jtao15 commented on code in PR #10964: URL: https://github.com/apache/pinot/pull/10964#discussion_r1243289538 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/lineage/DefaultLineageManager.java: ## @@ -105,6 +107,11 @@ public void updateLineageForRetentio