[GitHub] [pinot] codecov-commenter commented on pull request #11571: Always copy schema when creating new table

2023-09-12 Thread via GitHub
codecov-commenter commented on PR #11571: URL: https://github.com/apache/pinot/pull/11571#issuecomment-1715120676 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11571?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] codecov-commenter commented on pull request #11568: Add integration test for rebalance in upsert tables

2023-09-12 Thread via GitHub
codecov-commenter commented on PR #11568: URL: https://github.com/apache/pinot/pull/11568#issuecomment-1715122869 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] codecov-commenter commented on pull request #11572: Allow override/force options when add schema

2023-09-12 Thread via GitHub
codecov-commenter commented on PR #11572: URL: https://github.com/apache/pinot/pull/11572#issuecomment-1715127510 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11572?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[pinot] annotated tag release-1.0.0-rc3 deleted (was a17ca7705d)

2023-09-12 Thread saurabhd336
This is an automated email from the ASF dual-hosted git repository. saurabhd336 pushed a change to annotated tag release-1.0.0-rc3 in repository https://gitbox.apache.org/repos/asf/pinot.git *** WARNING: tag release-1.0.0-rc3 was deleted! *** tag was a17ca7705d The revisions that were on t

[GitHub] [pinot] siddharthteotia commented on pull request #11564: Use builder method for initalizing immutable map in integration test

2023-09-12 Thread via GitHub
siddharthteotia commented on PR #11564: URL: https://github.com/apache/pinot/pull/11564#issuecomment-1715162473 Seems like a harmless change to make it use builder. However, what exactly is the issue ? -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [pinot] saurabhd336 commented on issue #11563: Pinot build in local is getting into error

2023-09-12 Thread via GitHub
saurabhd336 commented on issue #11563: URL: https://github.com/apache/pinot/issues/11563#issuecomment-1715165327 The same command passes for me. Can you check the guava version in your dependency tree? -- This is an automated message from the Apache Git Service. To respond to the message,

[pinot] branch release-1.0.0-rc updated (432afadef0 -> b297245f1c)

2023-09-12 Thread saurabhd336
This is an automated email from the ASF dual-hosted git repository. saurabhd336 pushed a change to branch release-1.0.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git discard 432afadef0 [maven-release-plugin] prepare for next development iteration discard eaf6db90d9 [maven-rel

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11558: Handle interrupt gracefully for Lucene mutable index

2023-09-12 Thread via GitHub
siddharthteotia commented on code in PR #11558: URL: https://github.com/apache/pinot/pull/11558#discussion_r1322615606 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneTextIndexSearcherPool.java: ## @@ -0,0 +1,56 @@ +/

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11558: Handle interrupt gracefully for Lucene mutable index

2023-09-12 Thread via GitHub
siddharthteotia commented on code in PR #11558: URL: https://github.com/apache/pinot/pull/11558#discussion_r1322617905 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneTextIndex.java: ## @@ -109,27 +112,43 @@ public Im

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11558: Handle interrupt gracefully for Lucene mutable index

2023-09-12 Thread via GitHub
siddharthteotia commented on code in PR #11558: URL: https://github.com/apache/pinot/pull/11558#discussion_r1322620614 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneDocIdCollector.java: ## @@ -60,9 +61,16 @@ public

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11558: Handle interrupt gracefully for Lucene mutable index

2023-09-12 Thread via GitHub
siddharthteotia commented on code in PR #11558: URL: https://github.com/apache/pinot/pull/11558#discussion_r1322620614 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneDocIdCollector.java: ## @@ -60,9 +61,16 @@ public

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11558: Handle interrupt gracefully for Lucene mutable index

2023-09-12 Thread via GitHub
siddharthteotia commented on code in PR #11558: URL: https://github.com/apache/pinot/pull/11558#discussion_r1322626318 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneTextIndex.java: ## @@ -109,27 +112,43 @@ public Im

[pinot] branch release-1.0.0-rc updated: [maven-release-plugin] prepare for next development iteration

2023-09-12 Thread saurabhd336
This is an automated email from the ASF dual-hosted git repository. saurabhd336 pushed a commit to branch release-1.0.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-1.0.0-rc by this push: new ef71cec9e7 [maven-releas

[pinot] annotated tag release-1.0.0-rc3 updated (b6bdf6c968 -> 10b489bc4f)

2023-09-12 Thread saurabhd336
This is an automated email from the ASF dual-hosted git repository. saurabhd336 pushed a change to annotated tag release-1.0.0-rc3 in repository https://gitbox.apache.org/repos/asf/pinot.git *** WARNING: tag release-1.0.0-rc3 was modified! *** from b6bdf6c968 (commit) to 10b489bc4f (t

[pinot] branch release-1.0.0-rc updated: [maven-release-plugin] prepare release release-1.0.0-rc3

2023-09-12 Thread saurabhd336
This is an automated email from the ASF dual-hosted git repository. saurabhd336 pushed a commit to branch release-1.0.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-1.0.0-rc by this push: new b6bdf6c968 [maven-releas

[GitHub] [pinot] xiangfu0 opened a new pull request, #11574: Adding a controller task to update schema name

2023-09-12 Thread via GitHub
xiangfu0 opened a new pull request, #11574: URL: https://github.com/apache/pinot/pull/11574 - Adding a controller periodical task to validate and clean up schema name -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [pinot] tibrewalpratik17 commented on a diff in pull request #11437: Add broker query event listener

2023-09-12 Thread via GitHub
tibrewalpratik17 commented on code in PR #11437: URL: https://github.com/apache/pinot/pull/11437#discussion_r1322755425 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java: ## @@ -248,13 +252,15 @@ public BrokerResponse handleReques

[GitHub] [pinot] tibrewalpratik17 commented on a diff in pull request #11437: Add broker query event listener

2023-09-12 Thread via GitHub
tibrewalpratik17 commented on code in PR #11437: URL: https://github.com/apache/pinot/pull/11437#discussion_r1322755823 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java: ## @@ -248,13 +252,15 @@ public BrokerResponse handleReques

[GitHub] [pinot] tibrewalpratik17 commented on a diff in pull request #11437: Add broker query event listener

2023-09-12 Thread via GitHub
tibrewalpratik17 commented on code in PR #11437: URL: https://github.com/apache/pinot/pull/11437#discussion_r1322757245 ## pinot-spi/src/main/java/org/apache/pinot/spi/queryeventlistener/BrokerQueryEventInfo.java: ## @@ -0,0 +1,462 @@ +/** + * Licensed to the Apache Software Fou

svn commit: r63938 - /dev/pinot/apache-pinot-1.0.0-rc3/

2023-09-12 Thread saurabhd336
Author: saurabhd336 Date: Tue Sep 12 09:46:51 2023 New Revision: 63938 Log: Update apache-pinot-1.0.0-rc3 Added: dev/pinot/apache-pinot-1.0.0-rc3/ dev/pinot/apache-pinot-1.0.0-rc3/apache-pinot-1.0.0-bin.tar.gz (with props) dev/pinot/apache-pinot-1.0.0-rc3/apache-pinot-1.0.0-bin.tar

[GitHub] [pinot] codecov-commenter commented on pull request #11574: Adding a controller task to update schema name

2023-09-12 Thread via GitHub
codecov-commenter commented on PR #11574: URL: https://github.com/apache/pinot/pull/11574#issuecomment-1715450200 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11574?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] gortiz opened a new pull request, #11575: Add metrics to export netty direct memory used and max

2023-09-12 Thread via GitHub
gortiz opened a new pull request, #11575: URL: https://github.com/apache/pinot/pull/11575 Lately we have been seeing some OOM exceptions related to direct memory in Netty channels. Right now we are mostly focused on dealing with the error (see https://github.com/apache/pinot/pull/11496), bu

[GitHub] [pinot] gortiz commented on pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
gortiz commented on PR #11496: URL: https://github.com/apache/pinot/pull/11496#issuecomment-1715520421 Related to this PR, I've added some netty buffer metrics in: https://github.com/apache/pinot/pull/11575 -- This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [pinot] saurabhd336 opened a new pull request, #11576: Add table indexes API

2023-09-12 Thread via GitHub
saurabhd336 opened a new pull request, #11576: URL: https://github.com/apache/pinot/pull/11576 This API is primarily for the controller UI -> tables -> -> reload status -> STATUS ui element. Right now it uses the `/segments/{tableName}/metadata?type={type}&columns=*` API to display the co

[GitHub] [pinot] saurabhd336 commented on pull request #11576: Add table indexes API

2023-09-12 Thread via GitHub
saurabhd336 commented on PR #11576: URL: https://github.com/apache/pinot/pull/11576#issuecomment-1715540806 https://github.com/apache/pinot/assets/7672118/9e17c699-fe41-43ce-a057-81c9c158fe84";> API calls involved when viewing this modal. For a large enough table, this UI element take

[GitHub] [pinot] codecov-commenter commented on pull request #11576: Add table indexes API

2023-09-12 Thread via GitHub
codecov-commenter commented on PR #11576: URL: https://github.com/apache/pinot/pull/11576#issuecomment-1715599000 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11576?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] codecov-commenter commented on pull request #11575: Add metrics to export netty direct memory used and max

2023-09-12 Thread via GitHub
codecov-commenter commented on PR #11575: URL: https://github.com/apache/pinot/pull/11575#issuecomment-1715599971 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11575?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] tibrewalpratik17 commented on a diff in pull request #11330: [multistage] Fixing pinot client resolve table name

2023-09-12 Thread via GitHub
tibrewalpratik17 commented on code in PR #11330: URL: https://github.com/apache/pinot/pull/11330#discussion_r1322951203 ## pinot-common/src/main/java/org/apache/pinot/common/utils/request/RequestUtils.java: ## @@ -290,4 +302,52 @@ public static Map getOptionsFromJson(JsonNode r

[GitHub] [pinot] ankitsultana commented on a diff in pull request #11558: Handle interrupt gracefully for Lucene mutable index

2023-09-12 Thread via GitHub
ankitsultana commented on code in PR #11558: URL: https://github.com/apache/pinot/pull/11558#discussion_r1323163212 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneTextIndex.java: ## @@ -109,27 +112,46 @@ public Immut

[GitHub] [pinot] ankitsultana commented on a diff in pull request #11437: Add broker query event listener

2023-09-12 Thread via GitHub
ankitsultana commented on code in PR #11437: URL: https://github.com/apache/pinot/pull/11437#discussion_r1323167542 ## pinot-spi/src/main/java/org/apache/pinot/spi/eventlistener/query/PinotBrokerQueryEventListenerUtils.java: ## @@ -0,0 +1,105 @@ +/** + * Licensed to the Apache S

[GitHub] [pinot] ankitsultana commented on a diff in pull request #11561: Decouple metrics Key , client ID in stream consumer and add identifiers for kafka steam clientID to prevent mBean already exis

2023-09-12 Thread via GitHub
ankitsultana commented on code in PR #11561: URL: https://github.com/apache/pinot/pull/11561#discussion_r1323170830 ## pinot-spi/src/main/java/org/apache/pinot/spi/stream/PartitionGroupMetadataFetcher.java: ## @@ -63,8 +63,9 @@ public Exception getException() { @Override p

[GitHub] [pinot] ankitsultana commented on a diff in pull request #11561: Decouple metrics Key , client ID in stream consumer and add identifiers for kafka steam clientID to prevent mBean already exis

2023-09-12 Thread via GitHub
ankitsultana commented on code in PR #11561: URL: https://github.com/apache/pinot/pull/11561#discussion_r1323172892 ## pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/src/main/java/org/apache/pinot/plugin/stream/kafka20/KafkaConsumerFactory.java: ## @@ -25,18 +25,22 @@ p

[GitHub] [pinot] ege-st commented on a diff in pull request #11576: Add table indexes API

2023-09-12 Thread via GitHub
ege-st commented on code in PR #11576: URL: https://github.com/apache/pinot/pull/11576#discussion_r1323257805 ## pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java: ## @@ -285,15 +289,60 @@ public String getSegmentMetadata( return ResourceU

[GitHub] [pinot] soumitra-st commented on issue #11483: HTTP 1.1 authentication challenge

2023-09-12 Thread via GitHub
soumitra-st commented on issue #11483: URL: https://github.com/apache/pinot/issues/11483#issuecomment-1716030378 @jvenant , Great find! What is the AccessControlFactory set to in the test environment? [Broker request filter](https://github.com/apache/pinot/blob/61dcea6b71a99746805bc7f322dae

[GitHub] [pinot] vvivekiyer commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
vvivekiyer commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1323279081 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] [pinot] soumitra-st commented on a diff in pull request #11501: Change the Forbidden error to Unauthorized

2023-09-12 Thread via GitHub
soumitra-st commented on code in PR #11501: URL: https://github.com/apache/pinot/pull/11501#discussion_r1323288750 ## pinot-broker/src/main/java/org/apache/pinot/broker/api/resources/PinotBrokerDebug.java: ## @@ -158,7 +158,7 @@ public Map> getRoutingTableForQuery( if (!

[GitHub] [pinot] soumitra-st commented on a diff in pull request #11501: Change the Forbidden error to Unauthorized

2023-09-12 Thread via GitHub
soumitra-st commented on code in PR #11501: URL: https://github.com/apache/pinot/pull/11501#discussion_r1323288750 ## pinot-broker/src/main/java/org/apache/pinot/broker/api/resources/PinotBrokerDebug.java: ## @@ -158,7 +158,7 @@ public Map> getRoutingTableForQuery( if (!

[GitHub] [pinot] soumitra-st commented on a diff in pull request #11501: Change the Forbidden error to Unauthorized

2023-09-12 Thread via GitHub
soumitra-st commented on code in PR #11501: URL: https://github.com/apache/pinot/pull/11501#discussion_r1323288750 ## pinot-broker/src/main/java/org/apache/pinot/broker/api/resources/PinotBrokerDebug.java: ## @@ -158,7 +158,7 @@ public Map> getRoutingTableForQuery( if (!

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1323308080 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1323309261 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1323314926 ## pinot-core/src/main/java/org/apache/pinot/core/transport/InstanceRequestHandler.java: ## @@ -321,6 +322,10 @@ private void sendResponse(ChannelHandlerContext

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1323315675 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1323316058 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[GitHub] [pinot] tibrewalpratik17 commented on a diff in pull request #11437: Add broker query event listener

2023-09-12 Thread via GitHub
tibrewalpratik17 commented on code in PR #11437: URL: https://github.com/apache/pinot/pull/11437#discussion_r1323318749 ## pinot-spi/src/main/java/org/apache/pinot/spi/eventlistener/query/PinotBrokerQueryEventListenerUtils.java: ## @@ -0,0 +1,105 @@ +/** + * Licensed to the Apac

[GitHub] [pinot] itschrispeck commented on a diff in pull request #11558: Handle interrupt gracefully for Lucene mutable index

2023-09-12 Thread via GitHub
itschrispeck commented on code in PR #11558: URL: https://github.com/apache/pinot/pull/11558#discussion_r1323331770 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneDocIdCollector.java: ## @@ -60,9 +61,16 @@ public voi

[GitHub] [pinot] itschrispeck commented on a diff in pull request #11558: Handle interrupt gracefully for Lucene mutable index

2023-09-12 Thread via GitHub
itschrispeck commented on code in PR #11558: URL: https://github.com/apache/pinot/pull/11558#discussion_r1323332979 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneTextIndex.java: ## @@ -109,27 +112,43 @@ public Immut

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11575: Add metrics to export netty direct memory used and max

2023-09-12 Thread via GitHub
siddharthteotia commented on code in PR #11575: URL: https://github.com/apache/pinot/pull/11575#discussion_r1323334725 ## pinot-core/src/main/java/org/apache/pinot/core/transport/ServerChannels.java: ## @@ -151,7 +153,24 @@ private class ServerChannel { ServerChannel(Serv

[GitHub] [pinot] itschrispeck commented on a diff in pull request #11558: Handle interrupt gracefully for Lucene mutable index

2023-09-12 Thread via GitHub
itschrispeck commented on code in PR #11558: URL: https://github.com/apache/pinot/pull/11558#discussion_r1323331770 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneDocIdCollector.java: ## @@ -60,9 +61,16 @@ public voi

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
siddharthteotia commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1323340895 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,95 @@ +/** + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11575: Add metrics to export netty direct memory used and max

2023-09-12 Thread via GitHub
siddharthteotia commented on code in PR #11575: URL: https://github.com/apache/pinot/pull/11575#discussion_r1323343206 ## pinot-common/src/main/java/org/apache/pinot/common/metrics/ServerGauge.java: ## @@ -46,6 +47,26 @@ public enum ServerGauge implements AbstractMetrics.Gauge {

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11575: Add metrics to export netty direct memory used and max

2023-09-12 Thread via GitHub
siddharthteotia commented on code in PR #11575: URL: https://github.com/apache/pinot/pull/11575#discussion_r1323344023 ## pinot-common/src/main/java/org/apache/pinot/common/metrics/ServerGauge.java: ## @@ -46,6 +47,26 @@ public enum ServerGauge implements AbstractMetrics.Gauge {

[GitHub] [pinot] itschrispeck commented on a diff in pull request #11558: Handle interrupt gracefully for Lucene mutable index

2023-09-12 Thread via GitHub
itschrispeck commented on code in PR #11558: URL: https://github.com/apache/pinot/pull/11558#discussion_r1323331770 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneDocIdCollector.java: ## @@ -60,9 +61,16 @@ public voi

[GitHub] [pinot] Jackie-Jiang commented on issue #11519: Log when creating all index types

2023-09-12 Thread via GitHub
Jackie-Jiang commented on issue #11519: URL: https://github.com/apache/pinot/issues/11519#issuecomment-1716139742 @shwin Good suggestion. Would you like to contribute this? We can go over all the index creators and log some useful stats when an index is created -- This is an automated mes

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #11556: Add argument check for aggregation functions with single argument

2023-09-12 Thread via GitHub
siddharthteotia commented on code in PR #11556: URL: https://github.com/apache/pinot/pull/11556#discussion_r1323359088 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/BooleanOrAggregationFunction.java: ## @@ -19,18 +19,21 @@ package org.apache.pin

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
jasperjiaguo commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1323362776 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,95 @@ +/** + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [pinot] Jackie-Jiang commented on issue #11520: Pinot minions not building NATIVE indices

2023-09-12 Thread via GitHub
Jackie-Jiang commented on issue #11520: URL: https://github.com/apache/pinot/issues/11520#issuecomment-1716165848 @gortiz Can you help take a look? I took a quick look and seems `TextIndexType.getFileExtension()` always returns lucene index extension. Not sure if that is related -- This

[GitHub] [pinot] soumitra-st commented on pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
soumitra-st commented on PR #11496: URL: https://github.com/apache/pinot/pull/11496#issuecomment-1716166853 Sorry for starting a new thread here, but while we are trying to "handle" OOM by resetting the netty channels, queries may still fail (I think). So, System.exit can be called instead,

[GitHub] [pinot] Jackie-Jiang commented on issue #11529: Reloading segments does not update LUCENE vs NATIVE text indices

2023-09-12 Thread via GitHub
Jackie-Jiang commented on issue #11529: URL: https://github.com/apache/pinot/issues/11529#issuecomment-1716167677 Might be related to #11520 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [pinot] Jackie-Jiang commented on issue #11531: flaky SimpleMinionClusterIntegrationTest

2023-09-12 Thread via GitHub
Jackie-Jiang commented on issue #11531: URL: https://github.com/apache/pinot/issues/11531#issuecomment-1716173056 This is caused by a Helix bug. Reverted Helix version in #11560 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [pinot] Jackie-Jiang closed issue #11531: flaky SimpleMinionClusterIntegrationTest

2023-09-12 Thread via GitHub
Jackie-Jiang closed issue #11531: flaky SimpleMinionClusterIntegrationTest URL: https://github.com/apache/pinot/issues/11531 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [pinot] rajagopr commented on a diff in pull request #11540: Keep header parsing logic the same in default CSVRecordReader and the line iterator

2023-09-12 Thread via GitHub
rajagopr commented on code in PR #11540: URL: https://github.com/apache/pinot/pull/11540#discussion_r1323399278 ## pinot-plugins/pinot-input-format/pinot-csv/src/main/java/org/apache/pinot/plugin/inputformat/csv/CSVRecordReader.java: ## @@ -104,50 +103,42 @@ public void init(Fil

[GitHub] [pinot] somandal opened a new pull request, #11577: [multistage] Add ROWS support for aggregation window functions with ORDER BY within OVER

2023-09-12 Thread via GitHub
somandal opened a new pull request, #11577: URL: https://github.com/apache/pinot/pull/11577 This PR adds support for `ROWS` type window frames for aggregation window functions with an `ORDER BY` clause within the `OVER()`. The default frame type for `ORDER BY` within `OVER()` is `RANGE BETW

[GitHub] [pinot] codecov-commenter commented on pull request #11577: [multistage] Add ROWS support for aggregation window functions with ORDER BY within OVER

2023-09-12 Thread via GitHub
codecov-commenter commented on PR #11577: URL: https://github.com/apache/pinot/pull/11577#issuecomment-1716291672 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11577?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[GitHub] [pinot] Jackie-Jiang commented on pull request #11566: Support up to 15 boolean expressions for casewhen scalar function

2023-09-12 Thread via GitHub
Jackie-Jiang commented on PR #11566: URL: https://github.com/apache/pinot/pull/11566#issuecomment-1716293076 @snleee When else is not configured, we should just return `null` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [pinot] shwin commented on issue #11519: Log when creating all index types

2023-09-12 Thread via GitHub
shwin commented on issue #11519: URL: https://github.com/apache/pinot/issues/11519#issuecomment-1716295505 @Jackie-Jiang yeah I'd be happy to contribute to this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
jasperjiaguo commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1323497068 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,95 @@ +/** + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [pinot] ankitsultana commented on a diff in pull request #11437: Add broker query event listener

2023-09-12 Thread via GitHub
ankitsultana commented on code in PR #11437: URL: https://github.com/apache/pinot/pull/11437#discussion_r1323500172 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java: ## @@ -242,19 +246,23 @@ public BrokerResponse handleRequest(Js

[GitHub] [pinot] ankitsultana commented on a diff in pull request #11437: Add broker query event listener

2023-09-12 Thread via GitHub
ankitsultana commented on code in PR #11437: URL: https://github.com/apache/pinot/pull/11437#discussion_r1323500172 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java: ## @@ -242,19 +246,23 @@ public BrokerResponse handleRequest(Js

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
jasperjiaguo commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1323501260 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [pinot] ege-st commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
ege-st commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1323502750 ## pinot-core/src/main/java/org/apache/pinot/core/transport/InstanceRequestHandler.java: ## @@ -321,6 +322,10 @@ private void sendResponse(ChannelHandlerContext ctx, St

[GitHub] [pinot] jasperjiaguo commented on pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
jasperjiaguo commented on PR #11496: URL: https://github.com/apache/pinot/pull/11496#issuecomment-1716347496 > Sorry for starting a new thread here, but while we are trying to "handle" OOM by resetting the netty channels, queries may still fail (I think). So, System.exit can be called inste

[GitHub] [pinot] ege-st commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
ege-st commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1323504740 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under o

[GitHub] [pinot] soumitra-st commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
soumitra-st commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1323505609 ## pinot-core/src/main/java/org/apache/pinot/core/transport/InstanceRequestHandler.java: ## @@ -321,6 +322,10 @@ private void sendResponse(ChannelHandlerContext ctx

[GitHub] [pinot] ege-st commented on pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
ege-st commented on PR #11496: URL: https://github.com/apache/pinot/pull/11496#issuecomment-1716354398 > > has less unknowns > > Could you eleborate your concern here? I think the tests/heap dump/graphs show that we recover deterministically and the direct buffers are deallocated.

[GitHub] [pinot] soumitra-st commented on pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
soumitra-st commented on PR #11496: URL: https://github.com/apache/pinot/pull/11496#issuecomment-1716370562 Thanks @jasperjiaguo for you comments! > Could you eleborate your concern here? I think the tests/heap dump/graphs show that we recover deterministically and the direct buffers

[GitHub] [pinot] snleee merged pull request #11540: Keep header parsing logic the same in default CSVRecordReader and the line iterator

2023-09-12 Thread via GitHub
snleee merged PR #11540: URL: https://github.com/apache/pinot/pull/11540 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

[pinot] branch master updated (bc0d9fb659 -> c254e393ef)

2023-09-12 Thread snlee
This is an automated email from the ASF dual-hosted git repository. snlee pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from bc0d9fb659 Fail reload if derived columns can't be created (#11559) add c254e393ef Keep header parsing logic the sa

[GitHub] [pinot] siddharthteotia commented on pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
siddharthteotia commented on PR #11496: URL: https://github.com/apache/pinot/pull/11496#issuecomment-1716388980 > Thanks @jasperjiaguo for you comments! > > > Could you eleborate your concern here? I think the tests/heap dump/graphs show that we recover deterministically and the direc

[GitHub] [pinot] deemoliu commented on pull request #11346: Add hyperLogLogPlus aggregation function for distinct count

2023-09-12 Thread via GitHub
deemoliu commented on PR #11346: URL: https://github.com/apache/pinot/pull/11346#issuecomment-1716413409 Test Query ``` SELECT DISTINCTCOUNTHLLPLUS(some_id, 12) FROM test_pinot_table where time_ms >= 169438000 time_ms <= 169438200 and filter1 > 0

[GitHub] [pinot] ege-st commented on a diff in pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
ege-st commented on code in PR #11496: URL: https://github.com/apache/pinot/pull/11496#discussion_r1323558463 ## pinot-core/src/main/java/org/apache/pinot/core/transport/DirectOOMHandler.java: ## @@ -0,0 +1,100 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under o

[GitHub] [pinot] siddharthteotia commented on pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
siddharthteotia commented on PR #11496: URL: https://github.com/apache/pinot/pull/11496#issuecomment-1716427968 > My concern is that we are trying to prove that the fix is working using tests/heap dump, etc. vs the restart will just work. Let me elaborate a bit on the nature of probl

[GitHub] [pinot] Jackie-Jiang merged pull request #11534: do not emit lag once tabledatamanager shutdown

2023-09-12 Thread via GitHub
Jackie-Jiang merged PR #11534: URL: https://github.com/apache/pinot/pull/11534 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[pinot] branch master updated: do not emit lag once tabledatamanager shutdown (#11534)

2023-09-12 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 30e29abe60 do not emit lag once tabledatamanager

[GitHub] [pinot] Jackie-Jiang closed issue #11541: Pulsar Stream Metadata not being written to Pinot

2023-09-12 Thread via GitHub
Jackie-Jiang closed issue #11541: Pulsar Stream Metadata not being written to Pinot URL: https://github.com/apache/pinot/issues/11541 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [pinot] Jackie-Jiang commented on issue #11543: Queries that involve string timestamps do not work properly unless the string is singled quoted

2023-09-12 Thread via GitHub
Jackie-Jiang commented on issue #11543: URL: https://github.com/apache/pinot/issues/11543#issuecomment-1716448923 This is incorrect. `2023-08-31-23` will be parsed as `2023 - 8 - 31 - 23` (minus) ==> `1961`. The query itself parses but is semantically wrong. String literal must be quoted

[GitHub] [pinot] Jackie-Jiang closed issue #11543: Queries that involve string timestamps do not work properly unless the string is singled quoted

2023-09-12 Thread via GitHub
Jackie-Jiang closed issue #11543: Queries that involve string timestamps do not work properly unless the string is singled quoted URL: https://github.com/apache/pinot/issues/11543 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [pinot] itschrispeck commented on a diff in pull request #11558: Handle interrupt gracefully for Lucene mutable index

2023-09-12 Thread via GitHub
itschrispeck commented on code in PR #11558: URL: https://github.com/apache/pinot/pull/11558#discussion_r1323585652 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneTextIndexSearcherPool.java: ## @@ -0,0 +1,56 @@ +/**

[GitHub] [pinot] itschrispeck commented on a diff in pull request #11558: Handle interrupt gracefully for Lucene mutable index

2023-09-12 Thread via GitHub
itschrispeck commented on code in PR #11558: URL: https://github.com/apache/pinot/pull/11558#discussion_r1323586218 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneTextIndex.java: ## @@ -109,27 +112,43 @@ public Immut

[GitHub] [pinot] soumitra-st commented on pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
soumitra-st commented on PR #11496: URL: https://github.com/apache/pinot/pull/11496#issuecomment-1716461492 > @soumitra-st @ege-st - I hope this gives some insight into where we are coming from. We can also chat offline and align if need be I agree with the operational toil and thanks

[GitHub] [pinot] ege-st commented on pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
ege-st commented on PR #11496: URL: https://github.com/apache/pinot/pull/11496#issuecomment-1716467737 > > My concern is that we are trying to prove that the fix is working using tests/heap dump, etc. vs the restart will just work. > > Let me elaborate a bit on the nature of problem w

[GitHub] [pinot] siddharthteotia merged pull request #11496: Fix direct memory OOM on broker

2023-09-12 Thread via GitHub
siddharthteotia merged PR #11496: URL: https://github.com/apache/pinot/pull/11496 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pi

[pinot] branch master updated: Fix direct memory OOM on broker (#11496)

2023-09-12 Thread siddteotia
This is an automated email from the ASF dual-hosted git repository. siddteotia pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new e62a271221 Fix direct memory OOM on broker (

[GitHub] [pinot] tibrewalpratik17 commented on a diff in pull request #11437: Add broker query event listener

2023-09-12 Thread via GitHub
tibrewalpratik17 commented on code in PR #11437: URL: https://github.com/apache/pinot/pull/11437#discussion_r1323605056 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java: ## @@ -242,19 +246,23 @@ public BrokerResponse handleReques

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11545: [multistage] (improve #11518) disable calcite implicit float to double cast

2023-09-12 Thread via GitHub
Jackie-Jiang commented on code in PR #11545: URL: https://github.com/apache/pinot/pull/11545#discussion_r1323618552 ## pinot-query-planner/src/main/java/org/apache/calcite/rex/PinotRexBuilder.java: ## @@ -0,0 +1,64 @@ +/** + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] [pinot] jasperjiaguo opened a new pull request, #11578: pre-configuration based assignment

2023-09-12 Thread via GitHub
jasperjiaguo opened a new pull request, #11578: URL: https://github.com/apache/pinot/pull/11578 Assign instances based on a pre-configured instance assignment map -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [pinot] zhtaoxiang closed pull request #11545: [multistage] (improve #11518) disable calcite implicit float to double cast

2023-09-12 Thread via GitHub
zhtaoxiang closed pull request #11545: [multistage] (improve #11518) disable calcite implicit float to double cast URL: https://github.com/apache/pinot/pull/11545 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [pinot] walterddr opened a new pull request, #11579: [multistage] default enable metadata manager for colocated join quickstart

2023-09-12 Thread via GitHub
walterddr opened a new pull request, #11579: URL: https://github.com/apache/pinot/pull/11579 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

[GitHub] [pinot] ankitsultana commented on a diff in pull request #11558: Handle interrupt gracefully for Lucene mutable index

2023-09-12 Thread via GitHub
ankitsultana commented on code in PR #11558: URL: https://github.com/apache/pinot/pull/11558#discussion_r1323686919 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneTextIndexSearcherPool.java: ## @@ -0,0 +1,56 @@ +/**

[GitHub] [pinot] Jackie-Jiang commented on issue #11549: Does Pinot hybrid table allowed to be created with different schema

2023-09-12 Thread via GitHub
Jackie-Jiang commented on issue #11549: URL: https://github.com/apache/pinot/issues/11549#issuecomment-1716631325 No. I think we didn't enforce that before, but no longer allow such configuration going forward -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [pinot] codecov-commenter commented on pull request #11578: pre-configuration based assignment

2023-09-12 Thread via GitHub
codecov-commenter commented on PR #11578: URL: https://github.com/apache/pinot/pull/11578#issuecomment-1716641667 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11578?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

  1   2   >