Re: [PR] For non-aggregation group-by query, if select clause is not matching to group by clause, return original query [pinot]

2024-08-13 Thread via GitHub
tarun11Mavani commented on PR #13792: URL: https://github.com/apache/pinot/pull/13792#issuecomment-2285714401 > > In #9605, we introduced a change to not resolve non-aggregation group by queries into distinct query, if select clause is not exactly matching to group-by clause. This is a brea

Re: [PR] Improve null handling performance for nullable single input aggregation functions [pinot]

2024-08-13 Thread via GitHub
gortiz commented on PR #13791: URL: https://github.com/apache/pinot/pull/13791#issuecomment-2285732500 > Btw, to be clear I am supportive of the change and the rationale behind it. But (if time permits) it may be useful to see results from real use case or at least some kind of CPU profilin

Re: [PR] Improve logging in SegmentPreProcessor::needProcess [pinot]

2024-08-13 Thread via GitHub
yashmayya commented on code in PR #13700: URL: https://github.com/apache/pinot/pull/13700#discussion_r1714940541 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessor.java: ## @@ -180,40 +181,42 @@ public boolean needProcess

Re: [PR] For non-aggregation group-by query, if select clause is not matching to group by clause, return original query [pinot]

2024-08-13 Thread via GitHub
tibrewalpratik17 commented on PR #13792: URL: https://github.com/apache/pinot/pull/13792#issuecomment-2285854632 @tarun11Mavani there were few UTs removed in https://github.com/apache/pinot/pull/9605/files as they were not supported post that backward-incompatible change. Can you also add t

[I] How can I use Grovvy Scripts in Pinot while in Web Application [pinot]

2024-08-13 Thread via GitHub
Drun1baby opened a new issue, #13807: URL: https://github.com/apache/pinot/issues/13807 I just tried a post request ```http http://127.0.0.1:9000/schemas/validate { "schemaName": "flights", "enableColumnBasedNullHandling": false, "dateTimeFieldSpecs": [

[PR] Bump org.codehaus.mojo:exec-maven-plugin from 3.4.0 to 3.4.1 [pinot]

2024-08-13 Thread via GitHub
dependabot[bot] opened a new pull request, #13808: URL: https://github.com/apache/pinot/pull/13808 Bumps [org.codehaus.mojo:exec-maven-plugin](https://github.com/mojohaus/exec-maven-plugin) from 3.4.0 to 3.4.1. Release notes Sourced from https://github.com/mojohaus/exec-maven-plug

[PR] Bump org.apache.spark:spark-sql_2.12 from 2.4.8 to 3.5.2 [pinot]

2024-08-13 Thread via GitHub
dependabot[bot] opened a new pull request, #13809: URL: https://github.com/apache/pinot/pull/13809 Bumps org.apache.spark:spark-sql_2.12 from 2.4.8 to 3.5.2. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=org.a

[PR] Bump software.amazon.awssdk:bom from 2.27.2 to 2.27.3 [pinot]

2024-08-13 Thread via GitHub
dependabot[bot] opened a new pull request, #13810: URL: https://github.com/apache/pinot/pull/13810 Bumps software.amazon.awssdk:bom from 2.27.2 to 2.27.3. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=software

[PR] Bump com.h2database:h2 from 2.3.230 to 2.3.232 [pinot]

2024-08-13 Thread via GitHub
dependabot[bot] opened a new pull request, #13811: URL: https://github.com/apache/pinot/pull/13811 Bumps [com.h2database:h2](https://github.com/h2database/h2database) from 2.3.230 to 2.3.232. Release notes Sourced from https://github.com/h2database/h2database/releases";>com.h2datab

Re: [I] Ingest multiple streams into 1 Pinot table [pinot]

2024-08-13 Thread via GitHub
mcvsubbu commented on issue #5647: URL: https://github.com/apache/pinot/issues/5647#issuecomment-2286647197 Another duplicate of this in Issue #8806 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] Aggregation filter is being pushed down in a way it breaks SQL semantics [pinot]

2024-08-13 Thread via GitHub
bziobrowski commented on issue #12647: URL: https://github.com/apache/pinot/issues/12647#issuecomment-2286805741 Hello @gortiz . I've spent some time debugging and found that this filter pushdown happens during construction of operators (FilteredGroupByOperator to be precise). I think f

Re: [PR] Make NetUtils.getHostAddress ipv6 compatible [pinot]

2024-08-13 Thread via GitHub
real-mj-song commented on PR #13805: URL: https://github.com/apache/pinot/pull/13805#issuecomment-2286808911 Yes this change should support those mixed env scenarios (at least in terms of getting local address). Expanding what I described in PR description, these will be the behavior after

[I] Flaky compatibility regression test (against release-1.1.0) [pinot]

2024-08-13 Thread via GitHub
real-mj-song opened a new issue, #13812: URL: https://github.com/apache/pinot/issues/13812 # Issue Looks like `Pinot Compatibility Regression Testing against release-1.1.0` fails transiently. I found multiple PRs where builds are failing for the exact same issue `routing table has not be

Re: [PR] Bump org.apache.spark:spark-sql_2.12 from 2.4.8 to 3.5.2 [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang closed pull request #13809: Bump org.apache.spark:spark-sql_2.12 from 2.4.8 to 3.5.2 URL: https://github.com/apache/pinot/pull/13809 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Bump software.amazon.awssdk:bom from 2.27.2 to 2.27.3 [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang merged PR #13810: URL: https://github.com/apache/pinot/pull/13810 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Bump org.codehaus.mojo:exec-maven-plugin from 3.4.0 to 3.4.1 [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang merged PR #13808: URL: https://github.com/apache/pinot/pull/13808 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Bump org.apache.spark:spark-sql_2.12 from 2.4.8 to 3.5.2 [pinot]

2024-08-13 Thread via GitHub
dependabot[bot] commented on PR #13809: URL: https://github.com/apache/pinot/pull/13809#issuecomment-2286939332 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] Bump com.h2database:h2 from 2.3.230 to 2.3.232 [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang merged PR #13811: URL: https://github.com/apache/pinot/pull/13811 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Remove ingestion metrics when consuming segment relocates [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on code in PR #13668: URL: https://github.com/apache/pinot/pull/13668#discussion_r1715806024 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/IngestionDelayTracker.java: ## @@ -214,7 +230,7 @@ private long getPartitionOffsetLag(Inges

Re: [I] How can I use Grovvy Scripts in Pinot while in Web Application [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on issue #13807: URL: https://github.com/apache/pinot/issues/13807#issuecomment-2286965106 I think it is triggered successfully. The request is to **validate** the schema, and it does return the validated schema. -- This is an automated message from the Apache Git S

Re: [PR] #12117 Support for Server API to check for Segments reload of a table in a server [pinot]

2024-08-13 Thread via GitHub
npawar commented on code in PR #13789: URL: https://github.com/apache/pinot/pull/13789#discussion_r1715809310 ## pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java: ## @@ -952,4 +955,41 @@ public TableSegmentValidationInfo validateTableSegmentS

Re: [I] How can I use Grovvy Scripts in Pinot while in Web Application [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on issue #13807: URL: https://github.com/apache/pinot/issues/13807#issuecomment-2286966802 You may get more help if you post the question in the Slack `troubleshooting` channel -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] Make NetUtils.getHostAddress ipv6 compatible [pinot]

2024-08-13 Thread via GitHub
real-mj-song commented on code in PR #13805: URL: https://github.com/apache/pinot/pull/13805#discussion_r1715833587 ## pinot-spi/src/test/java/org/apache/pinot/spi/utils/NetUtilsTest.java: ## @@ -0,0 +1,194 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one +

[PR] more assertions in PinotDriverTest [pinot]

2024-08-13 Thread via GitHub
sullis opened a new pull request, #13813: URL: https://github.com/apache/pinot/pull/13813 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

Re: [PR] more assertions in PinotDriverTest [pinot]

2024-08-13 Thread via GitHub
codecov-commenter commented on PR #13813: URL: https://github.com/apache/pinot/pull/13813#issuecomment-2287087655 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13813?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[I] UI improvement for tagging instances [pinot]

2024-08-13 Thread via GitHub
dd-willgan opened a new issue, #13814: URL: https://github.com/apache/pinot/issues/13814 We run a multi-tenant cluster at my company. When onboarding a new table or scaling up resources for an existing table, we have to tag newly provisioned servers under the table's tenant, but currently t

Re: [PR] Add more unit test for `AbstractMetricsTest` [pinot]

2024-08-13 Thread via GitHub
dttung2905 commented on code in PR #13697: URL: https://github.com/apache/pinot/pull/13697#discussion_r1715899840 ## pinot-common/src/test/java/org/apache/pinot/common/metrics/AbstractMetricsTest.java: ## @@ -48,4 +49,63 @@ public void testAddOrUpdateGauge() { controllerMet

Re: [PR] more assertions in PinotDriverTest [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on code in PR #13813: URL: https://github.com/apache/pinot/pull/13813#discussion_r1715909765 ## pinot-clients/pinot-jdbc-client/src/test/java/org/apache/pinot/client/PinotDriverTest.java: ## @@ -66,6 +68,7 @@ public void testDriver() public void testDri

Re: [PR] #12117 Support for Server API to check for Segments reload of a table in a server [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on code in PR #13789: URL: https://github.com/apache/pinot/pull/13789#discussion_r1715921352 ## pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java: ## @@ -952,4 +956,42 @@ public TableSegmentValidationInfo validateTableSe

Re: [PR] Allows multiple requests per server per request ID [pinot]

2024-08-13 Thread via GitHub
egalpin commented on PR #13742: URL: https://github.com/apache/pinot/pull/13742#issuecomment-2287145250 @siddharthteotia @jackjlli @vvivekiyer I'd love to get some guidance on the regression suites. I believe I understand why the failures exist, because depending on the order of which comp

Re: [PR] Support madvise for MmapMemory [pinot]

2024-08-13 Thread via GitHub
jasperjiaguo commented on code in PR #13721: URL: https://github.com/apache/pinot/pull/13721#discussion_r1715927679 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/memory/unsafe/MmapMemoryConfig.java: ## @@ -0,0 +1,81 @@ +/** + * Licensed to the Apache Software F

Re: [PR] [Feature] Add the ability to compress and decompress tar archive file with ZStandard and LZ4 compressors [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on code in PR #13782: URL: https://github.com/apache/pinot/pull/13782#discussion_r1715943217 ## pinot-common/src/main/java/org/apache/pinot/common/utils/FileUploadDownloadClient.java: ## @@ -1209,7 +1210,7 @@ public int downloadFile(URI uri, File dest, Aut

[I] NPE at DistinctCountHLLAggregationFunction.extractFinalResult() [pinot]

2024-08-13 Thread via GitHub
deepix opened a new issue, #13815: URL: https://github.com/apache/pinot/issues/13815 We are seeing an NPE in Pinot. We run Pinot 1.0.0 on Kubernetes via statefulsets. This happened when we were trying to increase disk space for Pinot, we made the change and then restarted the statefu

Re: [PR] Handle null values appropriately during segment reload for new derived columns [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on code in PR #13212: URL: https://github.com/apache/pinot/pull/13212#discussion_r1715954303 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/defaultcolumn/BaseDefaultColumnHandler.java: ## @@ -812,6 +775,274 @@ priv

Re: [PR] Handle null values appropriately during segment reload for new derived columns [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on code in PR #13212: URL: https://github.com/apache/pinot/pull/13212#discussion_r1715958604 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/defaultcolumn/BaseDefaultColumnHandler.java: ## @@ -575,44 +587,59 @@ priv

Re: [PR] Handle null values appropriately during segment reload for new derived columns [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang merged PR #13212: URL: https://github.com/apache/pinot/pull/13212 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] [null support] Fill the derived column with the default null value when transform function failed to execute [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang closed pull request #12763: [null support] Fill the derived column with the default null value when transform function failed to execute URL: https://github.com/apache/pinot/pull/12763 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] [null support] Fill the derived column with the default null value when transform function failed to execute [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on PR #12763: URL: https://github.com/apache/pinot/pull/12763#issuecomment-2287189216 Covered by #13212 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [I] [null support] Transform function not handle null value [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang closed issue #12762: [null support] Transform function not handle null value URL: https://github.com/apache/pinot/issues/12762 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Allows multiple requests per server per request ID [pinot]

2024-08-13 Thread via GitHub
jackjlli commented on code in PR #13742: URL: https://github.com/apache/pinot/pull/13742#discussion_r1715967855 ## pinot-core/src/main/java/org/apache/pinot/core/transport/AsyncQueryResponse.java: ## @@ -112,15 +139,22 @@ public Map getFinalResponses() public String getServe

Re: [PR] Optimize NullableSingleInputAggregationFunction when entire block is null [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on code in PR #13758: URL: https://github.com/apache/pinot/pull/13758#discussion_r1715974434 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/NullableSingleInputAggregationFunction.java: ## @@ -81,7 +81,9 @@ public void forEachN

Re: [PR] Enable `noRawDataForTextIndex` for the mutable segment when re-use mutable index is enabled [pinot]

2024-08-13 Thread via GitHub
itschrispeck commented on PR #13776: URL: https://github.com/apache/pinot/pull/13776#issuecomment-2287205289 > Can we not use the `noForwardIndex` feature to get this behavior ? > > When we had added the `noRawDataForTextIndex` support, `noForwardIndex` feature did not exist and so it

Re: [PR] Fix literal type handling in LiteralValueExtractor [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on code in PR #13715: URL: https://github.com/apache/pinot/pull/13715#discussion_r1715978481 ## pinot-core/src/main/java/org/apache/pinot/core/query/reduce/filter/LiteralValueExtractor.java: ## @@ -18,31 +18,34 @@ */ package org.apache.pinot.core.query.

Re: [PR] #12117 Support for Server API to check for Segments reload of a table in a server [pinot]

2024-08-13 Thread via GitHub
deepthi912 commented on code in PR #13789: URL: https://github.com/apache/pinot/pull/13789#discussion_r1715982997 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java: ## @@ -1013,6 +1013,47 @@ public boolean tryLoadExistingSegment(SegmentZKMe

Re: [PR] #12117 Support for Server API to check for Segments reload of a table in a server [pinot]

2024-08-13 Thread via GitHub
deepthi912 commented on code in PR #13789: URL: https://github.com/apache/pinot/pull/13789#discussion_r1715984483 ## pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java: ## @@ -952,4 +955,41 @@ public TableSegmentValidationInfo validateTableSegm

Re: [I] [Backward incompatible issues in 1.1] segmentAssignmentStrategy can be NULL after it is deprecated in SegmentValidationAndRetentionConfig [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on issue #13659: URL: https://github.com/apache/pinot/issues/13659#issuecomment-2287218913 Can you elaborate more on the problem? We didn't introduce backward incompatibility in the PR (I commented and the contributor reverted the change). Basically it is okay to have

Re: [PR] Allows multiple requests per server per request ID [pinot]

2024-08-13 Thread via GitHub
egalpin commented on code in PR #13742: URL: https://github.com/apache/pinot/pull/13742#discussion_r1715991070 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandler.java: ## @@ -629,44 +631,32 @@ protected BrokerResponse handleR

Re: [PR] improve exception assertions in PinotDriverTest [pinot]

2024-08-13 Thread via GitHub
sullis commented on code in PR #13813: URL: https://github.com/apache/pinot/pull/13813#discussion_r1716005620 ## pinot-clients/pinot-jdbc-client/src/test/java/org/apache/pinot/client/PinotDriverTest.java: ## @@ -66,6 +68,7 @@ public void testDriver() public void testDriverBad

Re: [PR] #12117 Support for Server API to check for Segments reload of a table in a server [pinot]

2024-08-13 Thread via GitHub
deepthi912 commented on code in PR #13789: URL: https://github.com/apache/pinot/pull/13789#discussion_r1716075330 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java: ## @@ -1013,6 +1013,47 @@ public boolean tryLoadExistingSegment(SegmentZKMe

Re: [PR] #12117 Support for Server API to check for Segments reload of a table in a server [pinot]

2024-08-13 Thread via GitHub
deepthi912 commented on code in PR #13789: URL: https://github.com/apache/pinot/pull/13789#discussion_r1716075639 ## pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java: ## @@ -952,4 +956,42 @@ public TableSegmentValidationInfo validateTableSegm

Re: [PR] #12117 Support for Server API to check for Segments reload of a table in a server [pinot]

2024-08-13 Thread via GitHub
deepthi912 commented on code in PR #13789: URL: https://github.com/apache/pinot/pull/13789#discussion_r1716075852 ## pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java: ## @@ -952,4 +956,42 @@ public TableSegmentValidationInfo validateTableSegm

Re: [PR] Remove ingestion metrics when consuming segment relocates [pinot]

2024-08-13 Thread via GitHub
sajjad-moradi commented on code in PR #13668: URL: https://github.com/apache/pinot/pull/13668#discussion_r1716070749 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/IngestionDelayTracker.java: ## @@ -214,7 +230,7 @@ private long getPartitionOffsetLag(Inge

Re: [PR] Allows multiple requests per server per request ID [pinot]

2024-08-13 Thread via GitHub
egalpin commented on code in PR #13742: URL: https://github.com/apache/pinot/pull/13742#discussion_r1716084206 ## pinot-core/src/main/java/org/apache/pinot/core/query/reduce/ExecutionStatsAggregator.java: ## @@ -70,10 +71,13 @@ public ExecutionStatsAggregator(boolean enableTrace

Re: [PR] Allows multiple requests per server per request ID [pinot]

2024-08-13 Thread via GitHub
egalpin commented on code in PR #13742: URL: https://github.com/apache/pinot/pull/13742#discussion_r1716084599 ## pinot-core/src/main/java/org/apache/pinot/core/transport/ServerRoutingInstance.java: ## @@ -23,43 +23,36 @@ import java.util.Map; import java.util.concurrent.Concu

Re: [I] Multiple upstream source ingestion support on Pinot [pinot]

2024-08-13 Thread via GitHub
sajjad-moradi commented on issue #13780: URL: https://github.com/apache/pinot/issues/13780#issuecomment-2287375803 A few years ago, I had a working POC for multi-topic consumption. I also wrote a design doc for it, but we ended up not using it in linkedin, so I did not publish it at all. Th

Re: [PR] Allows multiple requests per server per request ID [pinot]

2024-08-13 Thread via GitHub
egalpin commented on code in PR #13742: URL: https://github.com/apache/pinot/pull/13742#discussion_r1716089990 ## pinot-core/src/main/java/org/apache/pinot/core/transport/ServerRoutingInstance.java: ## @@ -103,16 +93,16 @@ public boolean equals(Object o) { return false;

Re: [PR] Allows multiple requests per server per request ID [pinot]

2024-08-13 Thread via GitHub
egalpin commented on code in PR #13742: URL: https://github.com/apache/pinot/pull/13742#discussion_r1716092758 ## pinot-core/src/main/java/org/apache/pinot/core/transport/ServerRoutingInstance.java: ## @@ -103,16 +93,16 @@ public boolean equals(Object o) { return false;

Re: [PR] Allows multiple requests per server per request ID [pinot]

2024-08-13 Thread via GitHub
egalpin commented on code in PR #13742: URL: https://github.com/apache/pinot/pull/13742#discussion_r1716093986 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandler.java: ## @@ -771,15 +756,14 @@ protected BrokerResponse handleR

Re: [PR] Remove ingestion metrics when consuming segment relocates [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang merged PR #13668: URL: https://github.com/apache/pinot/pull/13668 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [I] Realtime servers reporting lag for partitions they don't own after rebalancing w/o including consuming segments [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang closed issue #11448: Realtime servers reporting lag for partitions they don't own after rebalancing w/o including consuming segments URL: https://github.com/apache/pinot/issues/11448 -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] UI Load time Improvement API Fixes #13278 [pinot]

2024-08-13 Thread via GitHub
deepthi912 commented on code in PR #13296: URL: https://github.com/apache/pinot/pull/13296#discussion_r1716106932 ## pinot-controller/src/main/resources/app/utils/PinotMethodUtils.ts: ## @@ -436,7 +434,7 @@ const getAllTableDetails = (tablesList) => { if (tablesList.length) {

Re: [PR] Add TablePauseStatus to track the pause details [pinot]

2024-08-13 Thread via GitHub
sajjad-moradi commented on code in PR #13803: URL: https://github.com/apache/pinot/pull/13803#discussion_r1716109524 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PauseStatus.java: ## @@ -22,20 +22,27 @@ import com.fasterxml.jackson.annotation.Json

Re: [PR] #12117 Support for Server API to check for Segments reload of a table in a server [pinot]

2024-08-13 Thread via GitHub
jackjlli commented on code in PR #13789: URL: https://github.com/apache/pinot/pull/13789#discussion_r1716111730 ## pinot-common/src/main/java/org/apache/pinot/common/restlet/resources/SegmentsReloadCheckResponse.java: ## @@ -0,0 +1,48 @@ +/** + * Licensed to the Apache Software

Re: [PR] #12117 Support for Server API to check for Segments reload of a table in a server [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on code in PR #13789: URL: https://github.com/apache/pinot/pull/13789#discussion_r1716159772 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/data/manager/TableDataManager.java: ## @@ -122,6 +122,19 @@ void addNewOnlineSegment(SegmentZKM

Re: [PR] #12117 Support for Server API to check for Segments reload of a table in a server [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on code in PR #13789: URL: https://github.com/apache/pinot/pull/13789#discussion_r1716160748 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java: ## @@ -1013,8 +1013,30 @@ public boolean tryLoadExistingSegment(SegmentZK

Re: [PR] Support polymorphic scalar comparison functions in the multi-stage query engine [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on code in PR #13711: URL: https://github.com/apache/pinot/pull/13711#discussion_r1716063997 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/MultiStageEngineIntegrationTest.java: ## @@ -704,6 +704,97 @@ public void testVariadicF

Re: [I] NPE at DistinctCountHLLAggregationFunction.extractFinalResult() [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on issue #13815: URL: https://github.com/apache/pinot/issues/13815#issuecomment-2287585884 Since `1.0` we have made a lot of improvements to multi-stage engine, and it is very hard to track a bug in `1.0`. Can you try to reproduce this with latest code and see if it g

Re: [PR] Fixing requestId generator bug in PinotBrokerDebug Resource [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on PR #13716: URL: https://github.com/apache/pinot/pull/13716#issuecomment-2287597396 Are we going to create a new class for each http request? That is surprising -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [Feature] Support configurable Lucene analyzer with args and configurable query parser [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on PR #13003: URL: https://github.com/apache/pinot/pull/13003#issuecomment-2287602181 Thanks for the contribution! Can you help also update the pinot doc about this new feature? -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] Flaky compatibility regression test (against release-1.1.0) [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on issue #13812: URL: https://github.com/apache/pinot/issues/13812#issuecomment-2287637534 This could be related to #9921 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] #12117 Support for Server API to check for Segments reload of a table in a server [pinot]

2024-08-13 Thread via GitHub
deepthi912 commented on code in PR #13789: URL: https://github.com/apache/pinot/pull/13789#discussion_r1715982997 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java: ## @@ -1013,6 +1013,47 @@ public boolean tryLoadExistingSegment(SegmentZKMe

Re: [PR] #12117 Support for Server API to check for Segments reload of a table in a server [pinot]

2024-08-13 Thread via GitHub
deepthi912 commented on code in PR #13789: URL: https://github.com/apache/pinot/pull/13789#discussion_r1716192276 ## pinot-common/src/main/java/org/apache/pinot/common/restlet/resources/SegmentsReloadCheckResponse.java: ## @@ -0,0 +1,48 @@ +/** + * Licensed to the Apache Softwar

Re: [I] How can I use Grovvy Scripts in Pinot while in Web Application [pinot]

2024-08-13 Thread via GitHub
Drun1baby commented on issue #13807: URL: https://github.com/apache/pinot/issues/13807#issuecomment-2287712941 But actually that Grovvy scripts not triggered, and as a result, i can not get any info from logs -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] Add OOM Protection Support for Multi-Stage Queries [pinot]

2024-08-13 Thread via GitHub
vrajat commented on code in PR #13598: URL: https://github.com/apache/pinot/pull/13598#discussion_r1716278467 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/HashJoinOperator.java: ## @@ -260,6 +260,7 @@ private void buildBroadcastHashTable()

Re: [PR] Add OOM Protection Support for Multi-Stage Queries [pinot]

2024-08-13 Thread via GitHub
vrajat commented on code in PR #13598: URL: https://github.com/apache/pinot/pull/13598#discussion_r1716278736 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/operator/MultiStageAccountingTest.java: ## @@ -0,0 +1,246 @@ +/** Review Comment: Since the trhea

Re: [PR] Optimize NullableSingleInputAggregationFunction when entire block is null [pinot]

2024-08-13 Thread via GitHub
yashmayya commented on code in PR #13758: URL: https://github.com/apache/pinot/pull/13758#discussion_r1716280555 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/NullableSingleInputAggregationFunction.java: ## @@ -81,7 +81,9 @@ public void forEachNotN

Re: [PR] Optimize NullableSingleInputAggregationFunction when entire block is null [pinot]

2024-08-13 Thread via GitHub
yashmayya commented on code in PR #13758: URL: https://github.com/apache/pinot/pull/13758#discussion_r1716284699 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/NullableSingleInputAggregationFunction.java: ## @@ -81,7 +81,9 @@ public void forEachNotN

Re: [PR] Add TablePauseStatus to track the pause details [pinot]

2024-08-13 Thread via GitHub
shounakmk219 commented on code in PR #13803: URL: https://github.com/apache/pinot/pull/13803#discussion_r1716288662 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PauseStatus.java: ## @@ -22,20 +22,27 @@ import com.fasterxml.jackson.annotation.JsonI

Re: [PR] Add OOM Protection Support for Multi-Stage Queries [pinot]

2024-08-13 Thread via GitHub
vrajat commented on code in PR #13598: URL: https://github.com/apache/pinot/pull/13598#discussion_r1716292146 ## pinot-core/src/main/java/org/apache/pinot/core/accounting/PerQueryCPUMemAccountantFactory.java: ## @@ -286,17 +286,18 @@ public void setThreadResourceUsageProvider(T

Re: [I] How can I use Groovy Scripts in Pinot while in Web Application [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on issue #13807: URL: https://github.com/apache/pinot/issues/13807#issuecomment-2287841548 Hmm.. It doesn't work this way, and actually you are trying to attack the system because the system shouldn't allow accessing local files through table creation or query. --

Re: [PR] Fix literal type handling in LiteralValueExtractor [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang merged PR #13715: URL: https://github.com/apache/pinot/pull/13715 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] improve exception assertions in PinotDriverTest [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang merged PR #13813: URL: https://github.com/apache/pinot/pull/13813 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Optimize NullableSingleInputAggregationFunction when entire block is null [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on code in PR #13758: URL: https://github.com/apache/pinot/pull/13758#discussion_r1716305239 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/NullableSingleInputAggregationFunction.java: ## @@ -119,6 +122,12 @@ public A foldNot

Re: [PR] add metadata ttl field to DedupConfig [pinot]

2024-08-13 Thread via GitHub
klsince commented on code in PR #13636: URL: https://github.com/apache/pinot/pull/13636#discussion_r1716301593 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/dedup/BaseTableDedupMetadataManager.java: ## @@ -19,39 +19,60 @@ package org.apache.pinot.segment.l

Re: [PR] Upgrade Helix dependency to 1.4.0 [pinot]

2024-08-13 Thread via GitHub
Jackie-Jiang commented on PR #13486: URL: https://github.com/apache/pinot/pull/13486#issuecomment-2287854296 @jackjlli But this doesn't explain why the tests are not failing before the upgrade. For `testListConfigs()`, why do we need to wait for EV to appear? Table list API should only chec

Re: [PR] Move BrokerSelector and implementations to pinot-common [pinot]

2024-08-13 Thread via GitHub
vrajat closed pull request #13361: Move BrokerSelector and implementations to pinot-common URL: https://github.com/apache/pinot/pull/13361 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[PR] Fix bugs in DynamicBrokerSelector [pinot]

2024-08-13 Thread via GitHub
vrajat opened a new pull request, #13816: URL: https://github.com/apache/pinot/pull/13816 DynamicBrokerSelector.getBrokers returned a list of zkServers instead of brokers. BrokerSelectorUtils.getTablesCommonBrokers made inplace updates to lists in brokerData. Make a copy of the lists

Re: [PR] Fix bugs in DynamicBrokerSelector [pinot]

2024-08-13 Thread via GitHub
codecov-commenter commented on PR #13816: URL: https://github.com/apache/pinot/pull/13816#issuecomment-2287955875 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13816?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Fix bugs in DynamicBrokerSelector [pinot]

2024-08-13 Thread via GitHub
gortiz commented on code in PR #13816: URL: https://github.com/apache/pinot/pull/13816#discussion_r1716386801 ## pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/utils/BrokerSelectorUtils.java: ## @@ -57,11 +59,13 @@ public static List getTablesCommonBroker

Re: [PR] Fix bugs in DynamicBrokerSelector [pinot]

2024-08-14 Thread via GitHub
vrajat commented on code in PR #13816: URL: https://github.com/apache/pinot/pull/13816#discussion_r1716420580 ## pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/utils/BrokerSelectorUtils.java: ## @@ -57,11 +59,13 @@ public static List getTablesCommonBroker

Re: [PR] Optimize NullableSingleInputAggregationFunction when entire block is null [pinot]

2024-08-14 Thread via GitHub
Jackie-Jiang merged PR #13758: URL: https://github.com/apache/pinot/pull/13758 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[PR] Add javadoc and improve usability of FluentQueryTest [pinot]

2024-08-14 Thread via GitHub
gortiz opened a new pull request, #13817: URL: https://github.com/apache/pinot/pull/13817 This PR adds some improvements on `FluentQueryTest`. Specifically javadoc has been added to (almost) all methods and two new ways to use the framework have been added: 1. `FluentQueryTest.open()

Re: [PR] Fix bugs in DynamicBrokerSelector [pinot]

2024-08-14 Thread via GitHub
yashmayya commented on PR #13816: URL: https://github.com/apache/pinot/pull/13816#issuecomment-2288358604 @vrajat looks like there's an NPE in `org.apache.pinot.client.ConnectionFactoryTest` with these changes: ``` WARN [ExternalViewReader] [main] Exception while reading External v

Re: [PR] Add javadoc and improve usability of FluentQueryTest [pinot]

2024-08-14 Thread via GitHub
yashmayya commented on code in PR #13817: URL: https://github.com/apache/pinot/pull/13817#discussion_r1716695667 ## pinot-core/src/test/java/org/apache/pinot/queries/FluentQueryTest.java: ## @@ -101,11 +182,22 @@ public static class DeclaringTable { _extraQueryOptions = e

Re: [PR] Add javadoc and improve usability of FluentQueryTest [pinot]

2024-08-14 Thread via GitHub
codecov-commenter commented on PR #13817: URL: https://github.com/apache/pinot/pull/13817#issuecomment-2288425553 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13817?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Improve error messages when creating star-tree indexes [pinot]

2024-08-14 Thread via GitHub
gortiz opened a new pull request, #13818: URL: https://github.com/apache/pinot/pull/13818 This PR improves the error messages when star-tree or forward index creation end up failing due to integer overflow produced when chunks are too large. -- This is an automated message from the Apache

Re: [PR] Improve error messages when creating star-tree indexes [pinot]

2024-08-14 Thread via GitHub
gortiz commented on code in PR #13818: URL: https://github.com/apache/pinot/pull/13818#discussion_r1716728448 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/BaseSingleTreeBuilder.java: ## @@ -520,11 +520,22 @@ private void createForwardIn

[PR] Bump kerby.version from 2.0.3 to 2.1.0 [pinot]

2024-08-14 Thread via GitHub
dependabot[bot] opened a new pull request, #13819: URL: https://github.com/apache/pinot/pull/13819 Bumps `kerby.version` from 2.0.3 to 2.1.0. Updates `org.apache.kerby:kerb-core` from 2.0.3 to 2.1.0 Updates `org.apache.kerby:kerb-simplekdc` from 2.0.3 to 2.1.0 Dependabot

[PR] Bump software.amazon.awssdk:bom from 2.27.3 to 2.27.4 [pinot]

2024-08-14 Thread via GitHub
dependabot[bot] opened a new pull request, #13820: URL: https://github.com/apache/pinot/pull/13820 Bumps software.amazon.awssdk:bom from 2.27.3 to 2.27.4. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=software

Re: [PR] Improve error messages when creating star-tree indexes [pinot]

2024-08-14 Thread via GitHub
gortiz commented on code in PR #13818: URL: https://github.com/apache/pinot/pull/13818#discussion_r1716727378 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/writer/impl/BaseChunkForwardIndexWriter.java: ## @@ -186,7 +186,8 @@ protected void writeChunk() {

  1   2   3   4   5   6   7   8   9   10   >