Re: [I] [feature-request] Add Support for SQL Formatting in Controller Query Editor [pinot]

2024-12-06 Thread via GitHub
ankitsultana closed issue #6: [feature-request] Add Support for SQL Formatting in Controller Query Editor URL: https://github.com/apache/pinot/issues/6 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[PR] Allow configurable initial capacity for IndexedTable [pinot]

2024-12-06 Thread via GitHub
Jackie-Jiang opened a new pull request, #14620: URL: https://github.com/apache/pinot/pull/14620 On top of #14618 One of the bottleneck for combining group-by results is to grow the `IndexedTable` as we are adding groups. To reduce the re-hashing, we can initialize it with a larger ca

Re: [PR] Add multi stream ingestion support [pinot]

2024-12-06 Thread via GitHub
chenboat commented on code in PR #13790: URL: https://github.com/apache/pinot/pull/13790#discussion_r1874178792 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java: ## @@ -791,15 +780,48 @@ Set getPartitionIds(Str

Re: [PR] use dictionary for BinaryOperatorTransformFunction when possible [pinot]

2024-12-06 Thread via GitHub
codecov-commenter commented on PR #14619: URL: https://github.com/apache/pinot/pull/14619#issuecomment-2524657618 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14619?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Add multi stream ingestion support [pinot]

2024-12-06 Thread via GitHub
chenboat commented on code in PR #13790: URL: https://github.com/apache/pinot/pull/13790#discussion_r1874150534 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/IngestionConfigUtils.java: ## @@ -46,15 +48,100 @@ private IngestionConfigUtils() { private static final int D

Re: [PR] Add multi stream ingestion support [pinot]

2024-12-06 Thread via GitHub
chenboat commented on code in PR #13790: URL: https://github.com/apache/pinot/pull/13790#discussion_r1874146072 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/utils/TableConfigUtilsTest.java: ## @@ -684,12 +684,11 @@ public void ingestionStreamConfigsTest()

Re: [PR] Add multi stream ingestion support [pinot]

2024-12-06 Thread via GitHub
chenboat commented on code in PR #13790: URL: https://github.com/apache/pinot/pull/13790#discussion_r1874141964 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java: ## @@ -284,6 +284,7 @@ public void deleteSegmentFile() {

Re: [PR] Fix query option validation for group-by queries [pinot]

2024-12-06 Thread via GitHub
codecov-commenter commented on PR #14618: URL: https://github.com/apache/pinot/pull/14618#issuecomment-2524631063 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14618?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] use dictionary for BinaryOperatorTransformFunction when possible [pinot]

2024-12-06 Thread via GitHub
xiangfu0 opened a new pull request, #14619: URL: https://github.com/apache/pinot/pull/14619 For BinaryOperatorTransformFunction, we don't need to read the values then compare but use dictionary. Further optimization could be using predicate evaluator. -- This is an automated messa

[PR] Fix query option validation for group-by queries [pinot]

2024-12-06 Thread via GitHub
Jackie-Jiang opened a new pull request, #14618: URL: https://github.com/apache/pinot/pull/14618 #14158 introduced the query option validation, but some of the validations are too strict and some are too loose. This PR changes the validations of the following query options: - NUM_R

Re: [PR] #13049 Refactored RecordTransformer & merged RecordEnricher [pinot]

2024-12-06 Thread via GitHub
Jackie-Jiang closed pull request #13086: #13049 Refactored RecordTransformer & merged RecordEnricher URL: https://github.com/apache/pinot/pull/13086 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

(pinot) branch master updated (a506df38a5 -> 694546b3b2)

2024-12-06 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from a506df38a5 Safeguard AbstractBuilder copy constructor from empty TextIndexConfig (#14616) add 694546b3b2 Make Rec

Re: [I] Merge `RecordEnricher` into `RecordTransformer` [pinot]

2024-12-06 Thread via GitHub
Jackie-Jiang closed issue #13049: Merge `RecordEnricher` into `RecordTransformer` URL: https://github.com/apache/pinot/issues/13049 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Make RecordEnricher extend RecordTransformer [pinot]

2024-12-06 Thread via GitHub
Jackie-Jiang merged PR #14601: URL: https://github.com/apache/pinot/pull/14601 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Merge RecordEnricher into RecordTransformer [pinot]

2024-12-06 Thread via GitHub
Jackie-Jiang commented on PR #13704: URL: https://github.com/apache/pinot/pull/13704#issuecomment-2524228043 Merge #14601 instead -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Merge RecordEnricher into RecordTransformer [pinot]

2024-12-06 Thread via GitHub
Jackie-Jiang closed pull request #13704: Merge RecordEnricher into RecordTransformer URL: https://github.com/apache/pinot/pull/13704 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] #13049 Refactored RecordTransformer & merged RecordEnricher [pinot]

2024-12-06 Thread via GitHub
Jackie-Jiang commented on PR #13086: URL: https://github.com/apache/pinot/pull/13086#issuecomment-2524227482 Merge #14601 instead -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Make RecordEnricher extend RecordTransformer [pinot]

2024-12-06 Thread via GitHub
Jackie-Jiang commented on code in PR #14601: URL: https://github.com/apache/pinot/pull/14601#discussion_r1874024636 ## pinot-plugins/pinot-segment-writer/pinot-segment-writer-file-based/src/main/java/org/apache/pinot/plugin/segmentwriter/filebased/FileBasedSegmentWriter.java: ##

(pinot) branch master updated: Safeguard AbstractBuilder copy constructor from empty TextIndexConfig (#14616)

2024-12-06 Thread manishswaminathan
This is an automated email from the ASF dual-hosted git repository. manishswaminathan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new a506df38a5 Safeguard AbstractBuilder c

Re: [PR] Safeguard AbstractBuilder copy constructor from empty TextIndexConfig [pinot]

2024-12-06 Thread via GitHub
swaminathanmanish merged PR #14616: URL: https://github.com/apache/pinot/pull/14616 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@

Re: [PR] Safeguard AbstractBuilder copy constructor from empty TextIndexConfig [pinot]

2024-12-06 Thread via GitHub
shounakmk219 commented on code in PR #14616: URL: https://github.com/apache/pinot/pull/14616#discussion_r1873530036 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/store/TextIndexUtilsTest.java: ## @@ -45,7 +45,8 @@ public void testRoundTripProperties

Re: [PR] Safeguard AbstractBuilder copy constructor from empty TextIndexConfig [pinot]

2024-12-06 Thread via GitHub
swaminathanmanish commented on code in PR #14616: URL: https://github.com/apache/pinot/pull/14616#discussion_r1873503014 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/store/TextIndexUtilsTest.java: ## @@ -45,7 +45,8 @@ public void testRoundTripPrope

Re: [PR] Safeguard AbstractBuilder copy constructor from empty TextIndexConfig [pinot]

2024-12-06 Thread via GitHub
codecov-commenter commented on PR #14616: URL: https://github.com/apache/pinot/pull/14616#issuecomment-2523404804 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14616?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Improve performance of multi-stage queries with large IN clauses [pinot]

2024-12-06 Thread via GitHub
codecov-commenter commented on PR #14615: URL: https://github.com/apache/pinot/pull/14615#issuecomment-2523344529 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14615?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Store index metadata file for Lucene text indexes [pinot]

2024-12-06 Thread via GitHub
shounakmk219 commented on PR #13948: URL: https://github.com/apache/pinot/pull/13948#issuecomment-2523327764 Raised a fix https://github.com/apache/pinot/pull/14616 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[PR] Safeguard AbstractBuilder copy constructor from empty TextIndexConfig [pinot]

2024-12-06 Thread via GitHub
shounakmk219 opened a new pull request, #14616: URL: https://github.com/apache/pinot/pull/14616 Refer https://github.com/apache/pinot/pull/13948#issuecomment-2523157822 The PR also fixes the default value of `useANDForMultiTermQueries` which was being set to `true` via `AbstractBuilde

[PR] Improve performance of multi-stage queries with large IN clauses [pinot]

2024-12-06 Thread via GitHub
yashmayya opened a new pull request, #14615: URL: https://github.com/apache/pinot/pull/14615 - 2x performance improvement by eliminating optimization overhead in `CoreRules.FILTER_REDUCE_EXPRESSIONS` from huge number of filter predicates created by `PinotFilterExpandSearchRule` (`IN` -> `OR

Re: [PR] Store index metadata file for Lucene text indexes [pinot]

2024-12-06 Thread via GitHub
swaminathanmanish commented on PR #13948: URL: https://github.com/apache/pinot/pull/13948#issuecomment-2523157822 @chenboat , @itschrispeck - We ran into an NPE with this change. When we don't have all fields populated in the TextIndexConfig, we will run into NPE, via code path that calls.

Re: [PR] Fix flaky quota integration tests [pinot]

2024-12-06 Thread via GitHub
bziobrowski commented on code in PR #14602: URL: https://github.com/apache/pinot/pull/14602#discussion_r1873043118 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/QueryQuotaClusterIntegrationTest.java: ## @@ -234,30 +235,44 @@ public void testApplica

Re: [PR] Override return type for NOW to TIMESTAMP in the v2 engine [pinot]

2024-12-06 Thread via GitHub
codecov-commenter commented on PR #14614: URL: https://github.com/apache/pinot/pull/14614#issuecomment-2522650820 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14614?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

[PR] Override return type for NOW to TIMESTAMP in the v2 engine [pinot]

2024-12-06 Thread via GitHub
yashmayya opened a new pull request, #14614: URL: https://github.com/apache/pinot/pull/14614 - The `NOW` scalar function returns a value of `LONG` type representing milliseconds since epoch. It was added before the `TIMESTAMP` data type was introduced to Pinot. - The function can't curre