Re: [I] commons-configuration2 performance issue during segment metadata load [pinot]

2024-02-15 Thread via GitHub
xiangfu0 commented on issue #12433: URL: https://github.com/apache/pinot/issues/12433#issuecomment-1947857684 I think this is already fixed in https://github.com/apache/pinot/pull/12405, can you please check? -- This is an automated message from the Apache Git Service. To respond to the m

Re: [I] Not able to set dimension table quota size beyond 200MB [pinot]

2024-02-15 Thread via GitHub
xiangfu0 commented on issue #12434: URL: https://github.com/apache/pinot/issues/12434#issuecomment-1947855778 You need to set this config in controller `controller.dimTable.maxSize= 1G` to lift the table quota constraint, default is `200M`. Code ref: https://github.com/apache/pinot/b

Re: [PR] UI: add preferred node version, ignore prettier formatting [pinot]

2024-02-15 Thread via GitHub
jadami10 commented on code in PR #12409: URL: https://github.com/apache/pinot/pull/12409#discussion_r1492031199 ## pinot-controller/src/main/resources/.prettierignore: ## @@ -0,0 +1 @@ +* Review Comment: what's the reasoning here? If anything I was hoping we'd start enforcin

Re: [PR] Readme - How to setup Pinot UI for development [pinot]

2024-02-15 Thread via GitHub
jadami10 commented on code in PR #12408: URL: https://github.com/apache/pinot/pull/12408#discussion_r1492030188 ## pinot-controller/src/main/resources/Readme.md: ## @@ -0,0 +1,16 @@ +# How to setup Pinot UI for development + +1. Make sure pinot backend is running on port 9000.

Re: [PR] Fix Bug in Handling Equal Comparison Column Values in Upsert [pinot]

2024-02-15 Thread via GitHub
tibrewalpratik17 commented on code in PR #12395: URL: https://github.com/apache/pinot/pull/12395#discussion_r1492024212 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java: ## @@ -167,6 +172,45 @@ protected

[I] Not able to set dimension table quota size beyond 200MB [pinot]

2024-02-15 Thread via GitHub
madhusudhannsn opened a new issue, #12434: URL: https://github.com/apache/pinot/issues/12434 Hi, I tried creating a dimension table with below quota size `"quota": {"storage": "800M"}` But I am getting the 500 error `2024/02/16 09:42:10.150 INFO [AddTableCommand] [main] {"code":500,

Re: [I] commons-configuration2 performance issue during segment metadata load [pinot]

2024-02-15 Thread via GitHub
abhioncbr commented on issue #12433: URL: https://github.com/apache/pinot/issues/12433#issuecomment-1947688248 I am planning to work on this. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] Add Prefix, Suffix and Ngram UDFs [pinot]

2024-02-15 Thread via GitHub
deemoliu commented on code in PR #12392: URL: https://github.com/apache/pinot/pull/12392#discussion_r1487311632 ## pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java: ## @@ -570,6 +572,81 @@ public static String[] split(String input, String

Re: [PR] Upgrade google library versions [pinot]

2024-02-15 Thread via GitHub
codecov-commenter commented on PR #12432: URL: https://github.com/apache/pinot/pull/12432#issuecomment-1947524724 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12432?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] Interning for OnHeapByteDictionary [pinot]

2024-02-15 Thread via GitHub
codecov-commenter commented on PR #12342: URL: https://github.com/apache/pinot/pull/12342#issuecomment-1947515678 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12342?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[PR] Upgrade google library versions [pinot]

2024-02-15 Thread via GitHub
Jackie-Jiang opened a new pull request, #12432: URL: https://github.com/apache/pinot/pull/12432 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Add upsert-snapshot timer metric [pinot]

2024-02-15 Thread via GitHub
ankitsultana commented on code in PR #12383: URL: https://github.com/apache/pinot/pull/12383#discussion_r1491724125 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java: ## @@ -660,8 +666,9 @@ protected void doTakeSna

Re: [PR] Fix bug in logging in UpsertCompaction task [pinot]

2024-02-15 Thread via GitHub
ankitsultana commented on code in PR #12419: URL: https://github.com/apache/pinot/pull/12419#discussion_r1491707054 ## pinot-controller/src/main/java/org/apache/pinot/controller/util/ServerSegmentMetadataReader.java: ## @@ -269,12 +269,18 @@ public List getValidDocIdsMetadataFr

Re: [PR] [draft] Mode null benchmark [pinot]

2024-02-15 Thread via GitHub
Jackie-Jiang commented on PR #12354: URL: https://github.com/apache/pinot/pull/12354#issuecomment-1947365697 For benchmark purpose, can we try the same idea on a cheaper aggregation such as `SUM`? This way we can amplify the performance impact -- This is an automated message from the Apac

Re: [PR] Fix bug to return validDocIDsMetadata from all servers [pinot]

2024-02-15 Thread via GitHub
codecov-commenter commented on PR #12431: URL: https://github.com/apache/pinot/pull/12431#issuecomment-1947173608 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12431?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

(pinot) branch full-auto-poc updated (3c1ecd9495 -> 37dc958fbc)

2024-02-15 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch full-auto-poc in repository https://gitbox.apache.org/repos/asf/pinot.git discard 3c1ecd9495 Initial POC code for hybrid table new 37dc958fbc Initial POC code for hybrid table This update ad

(pinot) 01/01: Initial POC code for hybrid table

2024-02-15 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch full-auto-poc in repository https://gitbox.apache.org/repos/asf/pinot.git commit 37dc958fbc8cc33ab500167e43e78d764f239297 Author: jlli_LinkedIn AuthorDate: Mon Feb 12 23:36:47 2024 -0800 Initi

Re: [PR] [multistage][feature] FunctionRegistry unification [pinot]

2024-02-15 Thread via GitHub
walterddr commented on code in PR #12302: URL: https://github.com/apache/pinot/pull/12302#discussion_r1491512462 ## pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java: ## @@ -94,83 +155,196 @@ private FunctionRegistry() { public static void init

Re: [PR] [multistage][feature] FunctionRegistry unification [pinot]

2024-02-15 Thread via GitHub
walterddr commented on code in PR #12302: URL: https://github.com/apache/pinot/pull/12302#discussion_r1491512462 ## pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java: ## @@ -94,83 +155,196 @@ private FunctionRegistry() { public static void init

[PR] Fix bug to return validDocIDsMetadata from all servers [pinot]

2024-02-15 Thread via GitHub
tibrewalpratik17 opened a new pull request, #12431: URL: https://github.com/apache/pinot/pull/12431 label: `bugfix` Fixes #12423 We were sending `serverToEndpoints` mapping instead of `endpointsToServer` which is expected by CompletionService. That's why CompletionService

(pinot) 01/01: Initial POC code for hybrid table

2024-02-15 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch full-auto-poc in repository https://gitbox.apache.org/repos/asf/pinot.git commit 3c1ecd9495d9cb479b8ab026e7e5c7a77246a2d9 Author: jlli_LinkedIn AuthorDate: Mon Feb 12 23:36:47 2024 -0800 Initi

(pinot) branch full-auto-poc updated (e4acc6e70c -> 3c1ecd9495)

2024-02-15 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch full-auto-poc in repository https://gitbox.apache.org/repos/asf/pinot.git discard e4acc6e70c Initial POC code for hybrid table new 3c1ecd9495 Initial POC code for hybrid table This update ad

Re: [I] Rename "Deleted Segments" feature to "Archived Segments" [pinot]

2024-02-15 Thread via GitHub
cbalci commented on issue #12424: URL: https://github.com/apache/pinot/issues/12424#issuecomment-1946957794 > regarding "Archival format": > is archiving a one way thing or should there also be a "restore" option which then needs to be able to deal with the different formats (and all thes

Re: [PR] UI: add preferred node version, ignore prettier formatting [pinot]

2024-02-15 Thread via GitHub
jayeshchoudhary commented on PR #12409: URL: https://github.com/apache/pinot/pull/12409#issuecomment-1946874346 @saurabhd336 @jadami10 please take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Readme - How to setup Pinot UI for development [pinot]

2024-02-15 Thread via GitHub
jayeshchoudhary commented on PR #12408: URL: https://github.com/apache/pinot/pull/12408#issuecomment-1946873934 does it make sense to link this doc in the home readme? here? - https://github.com/apache/pinot?tab=readme-ov-file#building-pinot -- This is an automated message from the Apac

Re: [PR] Readme - How to setup Pinot UI for development [pinot]

2024-02-15 Thread via GitHub
jayeshchoudhary commented on PR #12408: URL: https://github.com/apache/pinot/pull/12408#issuecomment-1946872675 @saurabhd336 @jadami10 please take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] UI: add preferred node version, ignore prettier formatting [pinot]

2024-02-15 Thread via GitHub
jayeshchoudhary commented on PR #12409: URL: https://github.com/apache/pinot/pull/12409#issuecomment-1946864418 does it make sense to link this doc in the home readme? here? - https://github.com/apache/pinot?tab=readme-ov-file#building-pinot -- This is an automated message from the Apac

Re: [PR] UI: add preferred node version, ignore prettier formatting [pinot]

2024-02-15 Thread via GitHub
jayeshchoudhary commented on PR #12409: URL: https://github.com/apache/pinot/pull/12409#issuecomment-1946862402 @saurabhd336 @jadami10 please take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [I] 1.0.1 release timeline [pinot]

2024-02-15 Thread via GitHub
hpvd commented on issue #12428: URL: https://github.com/apache/pinot/issues/12428#issuecomment-1946652528 of course it would be good to look into/solve the known/reported open vulnerabilities before: https://github.com/apache/pinot/issues?q=is%3Aissue+label%3Avulnerability+is%3Aopen+ -

Re: [PR] Ensure FieldConfig.getEncodingType() is never null [pinot]

2024-02-15 Thread via GitHub
codecov-commenter commented on PR #12430: URL: https://github.com/apache/pinot/pull/12430#issuecomment-1946615702 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12430?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [I] 1.0.1 release timeline [pinot]

2024-02-15 Thread via GitHub
mosabua commented on issue #12428: URL: https://github.com/apache/pinot/issues/12428#issuecomment-1946475653 We would really need this for Trino to avoid a regression from Pinot 1.0.0 - https://github.com/apache/pinot/issues/11676 - which was already merged https://github.com/apache/pinot/p

Re: [PR] [multistage][feature] FunctionRegistry unification [pinot]

2024-02-15 Thread via GitHub
gortiz commented on code in PR #12302: URL: https://github.com/apache/pinot/pull/12302#discussion_r1491271325 ## pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java: ## @@ -94,83 +155,196 @@ private FunctionRegistry() { public static void init()

Re: [PR] [multistage][feature] FunctionRegistry unification [pinot]

2024-02-15 Thread via GitHub
gortiz commented on code in PR #12302: URL: https://github.com/apache/pinot/pull/12302#discussion_r1491269838 ## pinot-common/src/main/java/org/apache/pinot/common/function/FunctionRegistry.java: ## @@ -94,83 +155,196 @@ private FunctionRegistry() { public static void init()

Re: [I] 1.0.1 release timeline [pinot]

2024-02-15 Thread via GitHub
hpvd commented on issue #12428: URL: https://github.com/apache/pinot/issues/12428#issuecomment-1946294646 btw the milestone section seems to be orphaned... https://github.com/apache/pinot/milestones -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] 1.0.1 release timeline [pinot]

2024-02-15 Thread via GitHub
hpvd commented on issue #12428: URL: https://github.com/apache/pinot/issues/12428#issuecomment-1946279794 +1 maybe worth it since there were already >450 merged pr since last release: https://github.com/apache/pinot/pulls?q=is%3Apr+merged%3A%3E%3D2023-09-12+ -- This is an automated m

[I] DISTINCTCOUNTMV returns incorrect value when field is included in select/group by [pinot]

2024-02-15 Thread via GitHub
aw381246 opened a new issue, #12429: URL: https://github.com/apache/pinot/issues/12429 If the field passed into the distinctcountmv function is included in the list of select / group by fields, it will return incorrect counts. In the case below, the result for each row should be a dis

Re: [I] Rename "Deleted Segments" feature to "Archived Segments" [pinot]

2024-02-15 Thread via GitHub
hpvd commented on issue #12424: URL: https://github.com/apache/pinot/issues/12424#issuecomment-1946208585 from functional pov: is there an intersection between "archive" and the mass export topic: https://github.com/apache/pinot/issues/12315 e.g. is "archive" an "export with dele

Re: [I] Rename "Deleted Segments" feature to "Archived Segments" [pinot]

2024-02-15 Thread via GitHub
hpvd commented on issue #12424: URL: https://github.com/apache/pinot/issues/12424#issuecomment-1946196006 regarding "Archival format": is archiving a one way thing or should there also be a "restore" option which then needs to be able to deal with the different formats (and all these con

Re: [I] [Multi-stage] Stage stats are very misleading [pinot]

2024-02-15 Thread via GitHub
KKcorps commented on issue #12388: URL: https://github.com/apache/pinot/issues/12388#issuecomment-1946184142 Ohhh, iirc, I did that to ensure that all the existing stats for V1 engine are also available for each table. I must missed the tracing duplication part. Assigning this to myself

Re: [PR] Record enricher [pinot]

2024-02-15 Thread via GitHub
gortiz commented on PR #12243: URL: https://github.com/apache/pinot/pull/12243#issuecomment-1945966525 Just as a reminder, we need to document this feature! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Record enricher [pinot]

2024-02-15 Thread via GitHub
gortiz commented on code in PR #12243: URL: https://github.com/apache/pinot/pull/12243#discussion_r1490909390 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/ingestion/EnrichmentConfig.java: ## @@ -0,0 +1,49 @@ +/** + * Licensed to the Apache Software Foundation (AS

Re: [PR] Record enricher [pinot]

2024-02-15 Thread via GitHub
gortiz commented on code in PR #12243: URL: https://github.com/apache/pinot/pull/12243#discussion_r1490905367 ## pinot-spi/src/main/java/org/apache/pinot/spi/recordenricher/RecordEnricherFactoryInterface.java: ## @@ -0,0 +1,29 @@ +/** + * Licensed to the Apache Software Foundati

Re: [PR] Record enricher [pinot]

2024-02-15 Thread via GitHub
gortiz commented on code in PR #12243: URL: https://github.com/apache/pinot/pull/12243#discussion_r1490903529 ## pinot-spi/src/main/java/org/apache/pinot/spi/recordenricher/RecordEnricher.java: ## @@ -0,0 +1,28 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] Record enricher [pinot]

2024-02-15 Thread via GitHub
gortiz commented on code in PR #12243: URL: https://github.com/apache/pinot/pull/12243#discussion_r1490897426 ## pinot-spi/src/main/java/org/apache/pinot/spi/annotations/RecordEnricherFactory.java: ## @@ -0,0 +1,30 @@ +/** + * Licensed to the Apache Software Foundation (ASF) und

Re: [I] Overview for text search topic [pinot]

2024-02-15 Thread via GitHub
hpvd commented on issue #12427: URL: https://github.com/apache/pinot/issues/12427#issuecomment-1945825119 maybe these named items of a great ecommerce search could be a possibility to group an overview of the text search topic... -- This is an automated message from the Apache Git Service

Re: [I] Overview for text search topic [pinot]

2024-02-15 Thread via GitHub
hpvd commented on issue #12427: URL: https://github.com/apache/pinot/issues/12427#issuecomment-1945782201 Why not having in a stunning OLAP-system like Pinot the same search possibilities and same result quality like a great Ecommerce store has?? with - fast results - well ranke

[I] Overview for text search topic [pinot]

2024-02-15 Thread via GitHub
hpvd opened a new issue, #12427: URL: https://github.com/apache/pinot/issues/12427 At this time, it's a little hard to build a complete picture of the field of "text search" topic in Pinot with - available capabilities, - ongoing efforts, - ideas and - plans. Would be good