Re: [I] Create DATETIMECONVERTWINDOWHOP function [pinot]

2023-10-13 Thread via GitHub
dario-liberman commented on issue #11775: URL: https://github.com/apache/pinot/issues/11775#issuecomment-1762670074 @jackjlli - I see in https://github.com/apache/pinot/issues/7213 you wanted to be able to calculate aggregations on moving time windows, such as avg COVID cases in a 7 day mov

[pinot] branch master updated: Minor improve on leaf execution stats NPE handling (#11805)

2023-10-13 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 7873b7bb90 Minor improve on leaf execution stats

Re: [PR] Minor improve on leaf execution stats NPE handling [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang merged PR #11805: URL: https://github.com/apache/pinot/pull/11805 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [I] Add partition, validDocId as the derived column [pinot]

2023-10-13 Thread via GitHub
snleee commented on issue #11580: URL: https://github.com/apache/pinot/issues/11580#issuecomment-1762607612 @Jackie-Jiang Good point. Per record partition id would be better as this is getting appended per each row. -- This is an automated message from the Apache Git Service. To respond

Re: [I] Create DATETIMECONVERTWINDOWHOP function [pinot]

2023-10-13 Thread via GitHub
dario-liberman commented on issue #11775: URL: https://github.com/apache/pinot/issues/11775#issuecomment-1762591744 > BTW i dont understand what the 2 other argument `'1:MILLISECONDS:EPOCH'`, and `'1:SECONDS:EPOCH'` means See existing support for tumbling window here: https://docs

Re: [I] double NaN as default value can massivly expand off heap mutable dictionary [pinot]

2023-10-13 Thread via GitHub
snleee commented on issue #10697: URL: https://github.com/apache/pinot/issues/10697#issuecomment-1762547917 MySQL: not allow `NaN` Postgres: allow `NaN` official documentation states, `In order to allow numeric values to be sorted and used in tree-based indexes, PostgreSQL treats NaN

Re: [PR] Add pending segments to reload job status [pinot]

2023-10-13 Thread via GitHub
walterddr commented on code in PR #11785: URL: https://github.com/apache/pinot/pull/11785#discussion_r1359064010 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java: ## @@ -700,19 +699,24 @@ public ServerReloadControllerJo

Re: [PR] Add pending segments to reload job status [pinot]

2023-10-13 Thread via GitHub
walterddr commented on code in PR #11785: URL: https://github.com/apache/pinot/pull/11785#discussion_r1359064010 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java: ## @@ -700,19 +699,24 @@ public ServerReloadControllerJo

Re: [PR] Add controller API for reload segment task status [pinot]

2023-10-13 Thread via GitHub
walterddr commented on code in PR #8828: URL: https://github.com/apache/pinot/pull/8828#discussion_r1359063941 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java: ## @@ -577,6 +599,107 @@ public SuccessResponse reloadSegm

Re: [PR] Minor improve on leaf execution stats NPE handling [pinot]

2023-10-13 Thread via GitHub
codecov-commenter commented on PR #11805: URL: https://github.com/apache/pinot/pull/11805#issuecomment-1762464755 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11805?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [I] UI does not show full stacktrace for minion tasks [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on issue #11795: URL: https://github.com/apache/pinot/issues/11795#issuecomment-1762458945 So this is not a UI issue. @snleee @swaminathanmanish Please help take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [I] Upgrade inconsistency: Confusing behaviour with noDictionaryConfig and fieldConfig for enabling compression on raw forward index. [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed issue #11745: Upgrade inconsistency: Confusing behaviour with noDictionaryConfig and fieldConfig for enabling compression on raw forward index. URL: https://github.com/apache/pinot/issues/11745 -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] Fix regression in ForwardIndexType detected in https://github.com/apache/pinot/issues/11745 [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang merged PR #11784: URL: https://github.com/apache/pinot/pull/11784 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [I] Upgrade inconsistency: Confusing behaviour with noDictionaryConfig and fieldConfig for enabling compression on raw forward index. [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on issue #11745: URL: https://github.com/apache/pinot/issues/11745#issuecomment-1762457525 Fixed with #11784 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[pinot] branch master updated: Fix regression in ForwardIndexType detected in https://github.com/apache/pinot/issues/11745 (#11784)

2023-10-13 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new d7d504b7f6 Fix regression in ForwardIndexType det

Re: [I] pinot-controller pod clash over and over again in kubernetes [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on issue #11764: URL: https://github.com/apache/pinot/issues/11764#issuecomment-1762437328 One possible explanation is that when committing a segment, controller will list all the files within the table data dir, and clean up the unnecessary ones (see `PinotLLCRealtim

[PR] Minor improve on leaf execution stats NPE handling [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang opened a new pull request, #11805: URL: https://github.com/apache/pinot/pull/11805 Better handling of NPE over #11801 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Support Upsert deletion for TTL: construct queryableDocIds when adding segments out of TTL [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on code in PR #11791: URL: https://github.com/apache/pinot/pull/11791#discussion_r1358998624 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java: ## @@ -138,15 +138,27 @@ public void addSegment

Re: [PR] [SQL Compatible] Fix literal type handling [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on code in PR #11763: URL: https://github.com/apache/pinot/pull/11763#discussion_r1358992155 ## pinot-core/src/test/java/org/apache/pinot/core/operator/transform/function/AdditionTransformFunctionTest.java: ## @@ -69,8 +69,8 @@ public void testAdditionTran

Re: [PR] Separate pr test scripts and codecov [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on PR #11804: URL: https://github.com/apache/pinot/pull/11804#issuecomment-1762420143 We used to have about 70% test coverage. Can you double check if all the coverage files are properly merged? -- This is an automated message from the Apache Git Service. To respond

Re: [PR] Add pending segments to reload job status [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on code in PR #11785: URL: https://github.com/apache/pinot/pull/11785#discussion_r1358990509 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java: ## @@ -700,19 +699,24 @@ public ServerReloadControlle

Re: [I] API to validate two tables are co-located [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on issue #11788: URL: https://github.com/apache/pinot/issues/11788#issuecomment-1762413589 @ankitsultana This is mostly for multi-stage engine colocated join, where 2 tables should have all segments for the same partition served by the same server -- This is an auto

[pinot] branch master updated: separate pr test scripts and codecov (#11804)

2023-10-13 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 1e04c8a095 separate pr test scripts and codecov

Re: [PR] Separate pr test scripts and codecov [pinot]

2023-10-13 Thread via GitHub
xiangfu0 merged PR #11804: URL: https://github.com/apache/pinot/pull/11804 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] [SQL Compatible] Fix literal type handling [pinot]

2023-10-13 Thread via GitHub
xiangfu0 commented on code in PR #11763: URL: https://github.com/apache/pinot/pull/11763#discussion_r1358951699 ## pinot-core/src/test/java/org/apache/pinot/core/operator/transform/function/AdditionTransformFunctionTest.java: ## @@ -69,8 +69,8 @@ public void testAdditionTransfor

Re: [PR] [SQL Compatible] Fix literal type handling [pinot]

2023-10-13 Thread via GitHub
xiangfu0 commented on code in PR #11763: URL: https://github.com/apache/pinot/pull/11763#discussion_r1358951699 ## pinot-core/src/test/java/org/apache/pinot/core/operator/transform/function/AdditionTransformFunctionTest.java: ## @@ -69,8 +69,8 @@ public void testAdditionTransfor

Re: [PR] [SQL Compatible] Fix literal type handling [pinot]

2023-10-13 Thread via GitHub
xiangfu0 commented on code in PR #11763: URL: https://github.com/apache/pinot/pull/11763#discussion_r1358951699 ## pinot-core/src/test/java/org/apache/pinot/core/operator/transform/function/AdditionTransformFunctionTest.java: ## @@ -69,8 +69,8 @@ public void testAdditionTransfor

Re: [PR] Separate pr test scripts and codecov [pinot]

2023-10-13 Thread via GitHub
codecov-commenter commented on PR #11804: URL: https://github.com/apache/pinot/pull/11804#issuecomment-1762377984 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11804?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [I] Dropping Out-of-order events for upserts [pinot]

2023-10-13 Thread via GitHub
tibrewalpratik17 commented on issue #11796: URL: https://github.com/apache/pinot/issues/11796#issuecomment-1762375387 Cool! Can you assign this to me? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] Dropping Out-of-order events for upserts [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on issue #11796: URL: https://github.com/apache/pinot/issues/11796#issuecomment-1762365629 Good suggestion. We may introduce a config `dropOutOfOrderRecord`, and modify `addRecord` to return whether the record should be kept in the segment. For the second request, i

Re: [PR] [SQL Compatible] Fix literal type handling [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on code in PR #11763: URL: https://github.com/apache/pinot/pull/11763#discussion_r1358926538 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/plan/server/ServerPlanRequestUtils.java: ## @@ -255,7 +255,8 @@ private static List computeInO

Re: [PR] Fix job submission time for reload and foce commit job [pinot]

2023-10-13 Thread via GitHub
codecov-commenter commented on PR #11803: URL: https://github.com/apache/pinot/pull/11803#issuecomment-1762331935 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11803?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] 11085: Added commons-configuration2 dependency. [pinot]

2023-10-13 Thread via GitHub
abhioncbr commented on PR #11792: URL: https://github.com/apache/pinot/pull/11792#issuecomment-1762325097 > > I saw an extra file added : `pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-hadoop/pinot-plugins.tar.gz`, is it required? https://user-images.githubusercontent.com/120212

Re: [PR] Java 21 pipeline [pinot]

2023-10-13 Thread via GitHub
xiangfu0 commented on code in PR #11672: URL: https://github.com/apache/pinot/pull/11672#discussion_r1358918958 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/memory/PinotDataBuffer.java: ## @@ -152,17 +153,22 @@ public static PinotBufferFactory getFactory() {

Re: [PR] [SQL Compatible] Fix literal type handling [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on code in PR #11763: URL: https://github.com/apache/pinot/pull/11763#discussion_r1358917066 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/AggregationFunctionFactory.java: ## @@ -111,8 +111,8 @@ public static AggregationFunct

[PR] Separate pr test scripts and codecov [pinot]

2023-10-13 Thread via GitHub
xiangfu0 opened a new pull request, #11804: URL: https://github.com/apache/pinot/pull/11804 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] [SQL Compatible] Fix literal type handling [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on code in PR #11763: URL: https://github.com/apache/pinot/pull/11763#discussion_r1358916763 ## pinot-core/src/main/java/org/apache/pinot/core/geospatial/transform/function/StPointFunction.java: ## @@ -65,7 +65,7 @@ public void init(List arguments, Map c

Re: [PR] [SQL Compatible] Fix literal type handling [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on code in PR #11763: URL: https://github.com/apache/pinot/pull/11763#discussion_r1358916446 ## pinot-common/src/main/java/org/apache/pinot/common/utils/request/RequestUtils.java: ## @@ -116,15 +116,22 @@ public static Expression getLiteralExpression(SqlLi

Re: [PR] [SQL Compatible] Fix literal type handling [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on code in PR #11763: URL: https://github.com/apache/pinot/pull/11763#discussion_r1358915901 ## pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java: ## @@ -493,9 +493,13 @@ private static String getErrorMessage(HttpUriRequest req

Re: [PR] [SQL Compatible] Fix literal type handling [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on code in PR #11763: URL: https://github.com/apache/pinot/pull/11763#discussion_r1358913092 ## pinot-common/src/main/java/org/apache/pinot/common/request/context/LiteralContext.java: ## @@ -18,179 +18,231 @@ */ package org.apache.pinot.common.request.c

Re: [PR] [SQL Compatible] Fix literal type handling [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on code in PR #11763: URL: https://github.com/apache/pinot/pull/11763#discussion_r1358911984 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/custom/GeoSpatialTest.java: ## @@ -304,11 +304,11 @@ public void testStPointFunctionWit

Re: [PR] [SQL Compatible] Fix literal type handling [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on code in PR #11763: URL: https://github.com/apache/pinot/pull/11763#discussion_r1358911168 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/BooleanUtils.java: ## @@ -64,6 +64,13 @@ public static int toInt(String booleanString) { return toBoolea

Re: [PR] [Issue 7519] Adds support for multiple filtered/unfiltered aggregations with GROUP BY [pinot]

2023-10-13 Thread via GitHub
egalpin commented on PR #1: URL: https://github.com/apache/pinot/pull/1#issuecomment-1762306908 Thanks! https://github.com/pinot-contrib/pinot-docs/pull/250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] segmentReloadStatus API never completes [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on issue #11603: URL: https://github.com/apache/pinot/issues/11603#issuecomment-1762301210 This is actually a bug. #11803 is the fix -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[PR] Fix job submission time for reload and foce commit job [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang opened a new pull request, #11803: URL: https://github.com/apache/pinot/pull/11803 For reload and force commit job, we should use the time when the request is received as the job submission time. Currently the submission time is the time after operation is done, which is too la

Re: [PR] Support helm chart server separate liveness and readiness probe endpoints [pinot]

2023-10-13 Thread via GitHub
xiangfu0 commented on code in PR #11800: URL: https://github.com/apache/pinot/pull/11800#discussion_r1358866439 ## helm/pinot/templates/server/statefulset.yaml: ## @@ -88,15 +88,23 @@ spec: initialDelaySeconds: {{ .Values.probes.initialDelaySeconds }} perio

Re: [PR] Reduce direct memory OOM chances on broker with a per server query response size budget [pinot]

2023-10-13 Thread via GitHub
vvivekiyer commented on PR #11710: URL: https://github.com/apache/pinot/pull/11710#issuecomment-1762247888 @Jackie-Jiang added perServer and perQuery configs. Please take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] Gapfill: Add support for lowercase datatypes [pinot]

2023-10-13 Thread via GitHub
codecov-commenter commented on PR #11722: URL: https://github.com/apache/pinot/pull/11722#issuecomment-1762246945 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11722?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [I] PinotS3 : Connection Pool Shutdown after one hour on EKS on release-1.0.0 [pinot]

2023-10-13 Thread via GitHub
swaminathanmanish commented on issue #11761: URL: https://github.com/apache/pinot/issues/11761#issuecomment-1762238755 > > > What do you think about downgrading the AWS-SDK version? (only in the environment with the problem, not in the official release of course) > > > > > > Just

Re: [I] LOOKUP not supported in v2 engine [pinot]

2023-10-13 Thread via GitHub
walterddr commented on issue #11651: URL: https://github.com/apache/pinot/issues/11651#issuecomment-1762238266 automating hints are a much wider topic (getting cost-factory to hook-in, getting table metadata to feed into cost factory etc) and i would like to keep that separate. but t

Re: [PR] 11085: Added commons-configuration2 dependency. [pinot]

2023-10-13 Thread via GitHub
abhioncbr commented on PR #11792: URL: https://github.com/apache/pinot/pull/11792#issuecomment-1762221840 > I saw an extra file added : `pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-hadoop/pinot-plugins.tar.gz`, is it required? https://user-images.githubusercontent.com/1202120/

Re: [I] CAST + alias + IS_NULL/IS_NOT_NULL filter expression does not mix [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on issue #11139: URL: https://github.com/apache/pinot/issues/11139#issuecomment-1762183802 Alias should not be applied to filter. Fixed in #11610 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] CAST + alias + IS_NULL/IS_NOT_NULL filter expression does not mix [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed issue #11139: CAST + alias + IS_NULL/IS_NOT_NULL filter expression does not mix URL: https://github.com/apache/pinot/issues/11139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] LOOKUP not supported in v2 engine [pinot]

2023-10-13 Thread via GitHub
cbalci commented on issue #11651: URL: https://github.com/apache/pinot/issues/11651#issuecomment-1762167976 Thanks for the investigation and the proposal @walterddr. 'LOOKUP' as a join strategy makes more sense to me. I also imagine we could automatically set the join_strategy to 'loo

Re: [PR] [POC] Working null solution for transform functions [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed pull request #10280: [POC] Working null solution for transform functions URL: https://github.com/apache/pinot/pull/10280 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] [bugfix] Fix to string representation of literal [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed pull request #10621: [bugfix] Fix to string representation of literal URL: https://github.com/apache/pinot/pull/10621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [feature] Parse null literal from calcite to thrift and pass down to LiteralContext [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed pull request #10257: [feature] Parse null literal from calcite to thrift and pass down to LiteralContext URL: https://github.com/apache/pinot/pull/10257 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] [multistage] Reproduce infinite loop in MailboxContentStreamObserver [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed pull request #9776: [multistage] Reproduce infinite loop in MailboxContentStreamObserver URL: https://github.com/apache/pinot/pull/9776 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [Not for review] Reproduce inequi join error [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed pull request #9743: [Not for review] Reproduce inequi join error URL: https://github.com/apache/pinot/pull/9743 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] [multistage] [experiment] Experiment manual grpc flow control [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed pull request #10036: [multistage] [experiment] Experiment manual grpc flow control URL: https://github.com/apache/pinot/pull/10036 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] 11085: Added commons-configuration2 dependency. [pinot]

2023-10-13 Thread via GitHub
xiangfu0 commented on PR #11792: URL: https://github.com/apache/pinot/pull/11792#issuecomment-1762165072 I saw an extra file added : `pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-hadoop/pinot-plugins.tar.gz`, is it required? -- This is an automated message from the Apache Gi

Re: [PR] [feature] null literal support for literal transform func [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed pull request #10082: [feature] null literal support for literal transform func URL: https://github.com/apache/pinot/pull/10082 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Add a pre-shutdown method to make deleting tables faster [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on PR #10602: URL: https://github.com/apache/pinot/pull/10602#issuecomment-1762162825 #11380 is picked instead -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Add a pre-shutdown method to make deleting tables faster [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed pull request #10602: Add a pre-shutdown method to make deleting tables faster URL: https://github.com/apache/pinot/pull/10602 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] [Feature] Null support in Pinot [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on issue #10252: URL: https://github.com/apache/pinot/issues/10252#issuecomment-1762159243 Majority of them are already supported. Use #11802 to track the remaining items -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] The race condition that TableDataManager is removed during segment download because of incorrect segment count. [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed issue #8423: The race condition that TableDataManager is removed during segment download because of incorrect segment count. URL: https://github.com/apache/pinot/issues/8423 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] [Do NOT REVIEW ]Working null solution test [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed pull request #10283: [Do NOT REVIEW ]Working null solution test URL: https://github.com/apache/pinot/pull/10283 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [I] [Feature] Null support in Pinot [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed issue #10252: [Feature] Null support in Pinot URL: https://github.com/apache/pinot/issues/10252 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

[I] Single-stage engine null support TODO list [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang opened a new issue, #11802: URL: https://github.com/apache/pinot/issues/11802 Use this issue to track the remaining items for null support -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] Add COALESCE or NULL value functions [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed issue #8251: Add COALESCE or NULL value functions URL: https://github.com/apache/pinot/issues/8251 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [I] UI Validation for upsert config errors [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed issue #8226: UI Validation for upsert config errors URL: https://github.com/apache/pinot/issues/8226 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] Add DATETIMECONVERTWINDOWHOP function [pinot]

2023-10-13 Thread via GitHub
dario-liberman commented on PR #11773: URL: https://github.com/apache/pinot/pull/11773#issuecomment-1762063968 @Jackie-Jiang - you can try it out yourself, works like a charm on single stage. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [I] Null Pointer exception When trying to add new schema [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed issue #8332: Null Pointer exception When trying to add new schema URL: https://github.com/apache/pinot/issues/8332 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [I] Improvement in NULL Handling for Datetime columns [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed issue #8254: Improvement in NULL Handling for Datetime columns URL: https://github.com/apache/pinot/issues/8254 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] parsing query to get tablename [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed pull request #9263: parsing query to get tablename URL: https://github.com/apache/pinot/pull/9263 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [I] Table selector won't work if table name is null [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on issue #8842: URL: https://github.com/apache/pinot/issues/8842#issuecomment-1762061809 Solved with #9902 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] parsing query to get tablename [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang commented on PR #9263: URL: https://github.com/apache/pinot/pull/9263#issuecomment-1762061561 #9902 of the same purpose was merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Table selector won't work if table name is null [pinot]

2023-10-13 Thread via GitHub
Jackie-Jiang closed issue #8842: Table selector won't work if table name is null URL: https://github.com/apache/pinot/issues/8842 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Create DATETIMECONVERTWINDOWHOP function [pinot]

2023-10-13 Thread via GitHub
dario-liberman commented on issue #11775: URL: https://github.com/apache/pinot/issues/11775#issuecomment-1761999763 > hmm. i am a bit confused. isn't the SQL you described above similar to https://calcite.apache.org/docs/reference.html#table-functions? > > ``` > select id, > c

Re: [I] Create DATETIMECONVERTWINDOWHOP function [pinot]

2023-10-13 Thread via GitHub
dario-liberman commented on issue #11775: URL: https://github.com/apache/pinot/issues/11775#issuecomment-1761997985 > @dario-liberman Thanks for providing the context! The docs you referenced are actually window functions, and here are some discussions around it #7213. It is not really poss

[pinot] branch master updated: Reverts interface refactor on PinotBrokerRestletResource.java

2023-10-13 Thread rongr
This is an automated email from the ASF dual-hosted git repository. rongr pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new aff30d6380 Reverts interface refactor on PinotBro

Re: [PR] This commit reverts the PR https://github.com/apache/pinot/pull/8852 [pinot]

2023-10-13 Thread via GitHub
walterddr merged PR #11797: URL: https://github.com/apache/pinot/pull/11797 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

Re: [I] Create DATETIMECONVERTWINDOWHOP function [pinot]

2023-10-13 Thread via GitHub
walterddr commented on issue #11775: URL: https://github.com/apache/pinot/issues/11775#issuecomment-1761904083 hmm. i am a bit confused. isn't the SQL you described above similar to https://calcite.apache.org/docs/reference.html#table-functions? ``` select id, created_at_times

Re: [I] Create an API for cloning a table [pinot]

2023-10-13 Thread via GitHub
walterddr commented on issue #11781: URL: https://github.com/apache/pinot/issues/11781#issuecomment-1761888254 plus one on cloning should also clone deep store. cloning (even only for zk metadata) is not atomic and thus doing it on a REALTIME table will have various corner cases. but

Re: [PR] Add more information in RequestContext class [pinot]

2023-10-13 Thread via GitHub
walterddr commented on code in PR #11708: URL: https://github.com/apache/pinot/pull/11708#discussion_r1358568153 ## pinot-spi/src/main/java/org/apache/pinot/spi/trace/DefaultRequestContext.java: ## @@ -402,6 +419,136 @@ public void setReduceTimeMillis(long reduceTimeMillis) {

Re: [PR] Add more information in RequestContext class [pinot]

2023-10-13 Thread via GitHub
tibrewalpratik17 commented on PR #11708: URL: https://github.com/apache/pinot/pull/11708#issuecomment-1761832999 hey @walterddr can you help in reviewing this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] Support helm chart server separate liveness and readiness probe endpoints [pinot]

2023-10-13 Thread via GitHub
walterddr commented on PR #11800: URL: https://github.com/apache/pinot/pull/11800#issuecomment-1761810959 > @walterddr I don't quite follow, do you mean I should change the values.yaml? Can you tell me where should I change? > > I thought I should keep values.yaml unchanged, it is the

Re: [PR] reloadStatus should only count online segments [pinot]

2023-10-13 Thread via GitHub
walterddr merged PR #11787: URL: https://github.com/apache/pinot/pull/11787 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

Re: [PR] reloadStatus should only count online segments [pinot]

2023-10-13 Thread via GitHub
walterddr commented on PR #11787: URL: https://github.com/apache/pinot/pull/11787#issuecomment-1761800686 merging this one first. we will continue discussion on #11785 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[pinot] branch master updated: reload status should only count online/consuming segments (#11787)

2023-10-13 Thread rongr
This is an automated email from the ASF dual-hosted git repository. rongr pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 16989370ce reload status should only count online/

[pinot] branch master updated: [UI] use indexes API instead of fetching all segment metadata (#11793)

2023-10-13 Thread rongr
This is an automated email from the ASF dual-hosted git repository. rongr pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 2eacd30295 [UI] use indexes API instead of fetchin

Re: [PR] [UI] use indexes API instead of fetching all segment metadata [pinot]

2023-10-13 Thread via GitHub
walterddr merged PR #11793: URL: https://github.com/apache/pinot/pull/11793 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

Re: [PR] Issue-6420 Add support for Apache Datasketches CPC sketch [pinot]

2023-10-13 Thread via GitHub
davecromberge commented on PR #11774: URL: https://github.com/apache/pinot/pull/11774#issuecomment-1761688844 I have tested the following features on this PR: - ingestion transformation function using baseball stats - segment rollup aggregation with MergeRollupTask - StarTree aggrega

Re: [I] UI does not show full stacktrace for minion tasks [pinot]

2023-10-13 Thread via GitHub
jayeshchoudhary commented on issue #11795: URL: https://github.com/apache/pinot/issues/11795#issuecomment-1761624346 we can add show more logs button and call a detailed API but I'll let @Jackie-Jiang to decide. -- This is an automated message from the Apache Git Service. To respond to

Re: [I] UI does not show full stacktrace for minion tasks [pinot]

2023-10-13 Thread via GitHub
piby180 commented on issue #11795: URL: https://github.com/apache/pinot/issues/11795#issuecomment-1761621131 Maybe we can have a button to manually fetch the logs on user action and not prefill it . Trimmed stack trace is of not much use from my perspective. -- This is an automated mess

Re: [I] UI does not show full stacktrace for minion tasks [pinot]

2023-10-13 Thread via GitHub
jayeshchoudhary commented on issue #11795: URL: https://github.com/apache/pinot/issues/11795#issuecomment-1761606689 @Jackie-Jiang UI shows whatever the API is returning as is. So I guess we are trimming data from backend. But sending entire stack trace to UI is not a good idea. the respo

Re: [I] UI does not show full stacktrace for minion tasks [pinot]

2023-10-13 Thread via GitHub
jayeshchoudhary commented on issue #11795: URL: https://github.com/apache/pinot/issues/11795#issuecomment-1761607948 UI is getting response. subtaskInfos[0].info and displaying everything present in that key -- This is an automated message from the Apache Git Service. To respond to the me

Re: [I] Allow schema to be created with JSON config in UI [pinot]

2023-10-13 Thread via GitHub
jayeshchoudhary commented on issue #11789: URL: https://github.com/apache/pinot/issues/11789#issuecomment-1761546095 Sure, nice idea. I'll add a way to directly add schema config. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[pinot] branch master updated: Upgrade spark from 3.2 to 3.5 (#11702)

2023-10-13 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new cedac51f29 Upgrade spark from 3.2 to 3.5 (#11702

Re: [PR] Upgrade spark from 3.2 to 3.5 [pinot]

2023-10-13 Thread via GitHub
xiangfu0 merged PR #11702: URL: https://github.com/apache/pinot/pull/11702 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

  1   2   >