Re: [PR] Revert not null requirement on FieldConfig.getEncodingType [pinot]

2024-02-14 Thread via GitHub
gortiz closed pull request #12407: Revert not null requirement on FieldConfig.getEncodingType URL: https://github.com/apache/pinot/pull/12407 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

(pinot) branch full-auto-poc updated (35e96d1eb5 -> e4acc6e70c)

2024-02-14 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch full-auto-poc in repository https://gitbox.apache.org/repos/asf/pinot.git discard 35e96d1eb5 Initial POC code for hybrid table new e4acc6e70c Initial POC code for hybrid table This update ad

(pinot) 01/01: Initial POC code for hybrid table

2024-02-14 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch full-auto-poc in repository https://gitbox.apache.org/repos/asf/pinot.git commit e4acc6e70c2be0125fbdd54dd3c2fe99db0d705d Author: jlli_LinkedIn AuthorDate: Mon Feb 12 23:36:47 2024 -0800 Initi

Re: [I] InterruptedException when acquiring partitonConsumerSemaphore [pinot]

2024-02-14 Thread via GitHub
ankitsultana commented on issue #12390: URL: https://github.com/apache/pinot/issues/12390#issuecomment-1945473584 Took a deep-dive today and found the root-cause. Here's the sequence of events which triggers this: * We have a consuming segment on a server. On time of commit, it receiv

Re: [PR] Record enricher [pinot]

2024-02-14 Thread via GitHub
saurabhd336 commented on PR #12243: URL: https://github.com/apache/pinot/pull/12243#issuecomment-1945463953 @gortiz @ege-st Thanks for review. Updated the PR as per the suggestion to move to a more concrete config specification. Please take a look again. -- This is an automated message fr

Re: [PR] Record enricher [pinot]

2024-02-14 Thread via GitHub
saurabhd336 commented on code in PR #12243: URL: https://github.com/apache/pinot/pull/12243#discussion_r1490488156 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/ingestion/EnrichmentConfig.java: ## @@ -0,0 +1,51 @@ +/** + * Licensed to the Apache Software Foundatio

Re: [PR] Record enricher [pinot]

2024-02-14 Thread via GitHub
saurabhd336 commented on code in PR #12243: URL: https://github.com/apache/pinot/pull/12243#discussion_r1490487956 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordenricher/clp/CLPEncodingEnricher.java: ## @@ -0,0 +1,102 @@ +/** + * Licensed to the Apach

Re: [PR] Record enricher [pinot]

2024-02-14 Thread via GitHub
saurabhd336 commented on code in PR #12243: URL: https://github.com/apache/pinot/pull/12243#discussion_r1490487201 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java: ## @@ -157,17 +159,20 @@ public Record

Re: [PR] Record enricher [pinot]

2024-02-14 Thread via GitHub
saurabhd336 commented on code in PR #12243: URL: https://github.com/apache/pinot/pull/12243#discussion_r1490486639 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordenricher/clp/CLPEncodingEnricher.java: ## @@ -0,0 +1,102 @@ +/** + * Licensed to the Apach

Re: [PR] Record enricher [pinot]

2024-02-14 Thread via GitHub
saurabhd336 commented on code in PR #12243: URL: https://github.com/apache/pinot/pull/12243#discussion_r1490486161 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordenricher/clp/CLPEncodingEnricher.java: ## @@ -0,0 +1,102 @@ +/** + * Licensed to the Apach

Re: [PR] Record enricher [pinot]

2024-02-14 Thread via GitHub
saurabhd336 commented on code in PR #12243: URL: https://github.com/apache/pinot/pull/12243#discussion_r1490484922 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordenricher/RecordEnricherPipeline.java: ## @@ -0,0 +1,76 @@ +/** + * Licensed to the Apache

Re: [I] [Multi-stage] OpChain stats handling improvement [pinot]

2024-02-14 Thread via GitHub
Jackie-Jiang commented on issue #12426: URL: https://github.com/apache/pinot/issues/12426#issuecomment-1945211294 cc @gortiz @KKcorps -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[I] [Multi-stage] OpChain stats handling improvement [pinot]

2024-02-14 Thread via GitHub
Jackie-Jiang opened a new issue, #12426: URL: https://github.com/apache/pinot/issues/12426 - OpChain stats is a mapping from worker id to serialized map, then serialized as JSON bytes. It is very inefficient. Since we know the metadata key upfront, we can encode the metadata much more effic

Re: [I] [Multi-stage] Stage stats are very misleading [pinot]

2024-02-14 Thread via GitHub
Jackie-Jiang commented on issue #12388: URL: https://github.com/apache/pinot/issues/12388#issuecomment-1945206587 By default, we collect stats for leaf operator and send operator. The docs gathered for each stage is the aggregated value. IMO we should only count the rows sent by the send op

Re: [PR] Adding support to insecure TLS when creating SSLFactory [pinot]

2024-02-14 Thread via GitHub
codecov-commenter commented on PR #12425: URL: https://github.com/apache/pinot/pull/12425#issuecomment-1945012769 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12425?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[PR] Adding support to insecure TLS when creating SSLFactory [pinot]

2024-02-14 Thread via GitHub
zhtaoxiang opened a new pull request, #12425: URL: https://github.com/apache/pinot/pull/12425 What is insecure mode - [curl -k](https://curl.se/docs/manpage.html#-k) The insecure mode flag is introduced recently in https://github.com/apache/pinot/pull/12416 Insecure mode is dis

[I] Rename "Deleted Segments" feature to "Archived Segments" [pinot]

2024-02-14 Thread via GitHub
cbalci opened a new issue, #12424: URL: https://github.com/apache/pinot/issues/12424 Proposing a somewhat cosmetic change and some related small features. Pinot already has a mechanism for keeping 'deleted' segments around for a configurable period under `/Deleted_segments` directory.

Re: [I] MultiStage Engine Fails to Parse Table Names with `-` (Dash) [pinot]

2024-02-14 Thread via GitHub
Ferix9288 commented on issue #11639: URL: https://github.com/apache/pinot/issues/11639#issuecomment-1944530686 No worries and appreciate you guys for looking into it! Understood that us quoting the table in V1 engine as a workaround for non-standard SQL table won't work for multistage.

Re: [I] MultiStage Engine Fails to Parse Table Names with `-` (Dash) [pinot]

2024-02-14 Thread via GitHub
walterddr commented on issue #11639: URL: https://github.com/apache/pinot/issues/11639#issuecomment-1944513230 going to close this as a non-issue. feel free to discuss further in separate thread if thats not the case or other issues related -- This is an automated message from the Apache

Re: [I] MultiStage Engine Fails to Parse Table Names with `-` (Dash) [pinot]

2024-02-14 Thread via GitHub
walterddr closed issue #11639: MultiStage Engine Fails to Parse Table Names with `-` (Dash) URL: https://github.com/apache/pinot/issues/11639 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [I] [partial-upsert] Partial Upsert Table Data Inconsistency Issues [pinot]

2024-02-14 Thread via GitHub
deemoliu commented on issue #12396: URL: https://github.com/apache/pinot/issues/12396#issuecomment-1944497834 +1 as @Jackie-Jiang mentioned current expected behavior is updating the record location when there is a tie to keep the newer record. However realistically, message orders are not g

(pinot) branch master updated: Adding support of insecure TLS (#12416)

2024-02-14 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 38d86b0a64 Adding support of insecure TLS (#12416

Re: [PR] Adding support of insecure TLS [pinot]

2024-02-14 Thread via GitHub
Jackie-Jiang merged PR #12416: URL: https://github.com/apache/pinot/pull/12416 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [I] Aggregation Function with '=' Operator Fails for Multivalued Columns in Apache Pinot [pinot]

2024-02-14 Thread via GitHub
abhioncbr commented on issue #12374: URL: https://github.com/apache/pinot/issues/12374#issuecomment-191513 Please assign it to me. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [I] Invalid transform function JSONPATHDOUBLE [pinot]

2024-02-14 Thread via GitHub
abhioncbr commented on issue #12422: URL: https://github.com/apache/pinot/issues/12422#issuecomment-190642 Seems like a transform function looking for the `cpu_util ` value in the JSON, however as per the transformConfigs it's `machine_util `. Is this correct? -- This is an automate

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-02-14 Thread via GitHub
Jackie-Jiang commented on PR #12417: URL: https://github.com/apache/pinot/pull/12417#issuecomment-1944413043 cc @zhtaoxiang to also take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Fix bug in logging in UpsertCompaction task [pinot]

2024-02-14 Thread via GitHub
tibrewalpratik17 closed pull request #12419: Fix bug in logging in UpsertCompaction task URL: https://github.com/apache/pinot/pull/12419 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[I] CompletionServiceHelper's doMultiPostRequest returns whenever 1 server responds instead of waiting for all [pinot]

2024-02-14 Thread via GitHub
tibrewalpratik17 opened a new issue, #12423: URL: https://github.com/apache/pinot/issues/12423 labels: bug This seems to be associated with this change #12275 When using UpsertCompaction, we make a `doMultiPostRequest` to all the servers for a table with the list of validDo

Re: [I] Support uptime API in all pinot components [pinot]

2024-02-14 Thread via GitHub
suyashpatel98 commented on issue #12298: URL: https://github.com/apache/pinot/issues/12298#issuecomment-1944246524 @xiangfu0 Please review when you find time. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[I] Invalid transform function JSONPATHDOUBLE [pinot]

2024-02-14 Thread via GitHub
vineethvp opened a new issue, #12422: URL: https://github.com/apache/pinot/issues/12422 I'm working on a pinot cluster with version 1.1.0. I have created a table schema with one field as JSON element. I have defined a JSON field and DOUBLE field in my schema. { "name" : "dat

Re: [PR] Attempt to apply timestamp expr override in V2 [pinot]

2024-02-14 Thread via GitHub
codecov-commenter commented on PR #12421: URL: https://github.com/apache/pinot/pull/12421#issuecomment-1944107924 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12421?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[PR] Attempt to apply timestamp expr override in V2 [pinot]

2024-02-14 Thread via GitHub
gortiz opened a new pull request, #12421: URL: https://github.com/apache/pinot/pull/12421 This is a draft -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

[PR] WIP: Async Kafka Consumer [POC] [pinot]

2024-02-14 Thread via GitHub
KKcorps opened a new pull request, #12420: URL: https://github.com/apache/pinot/pull/12420 Instructions: 1. The PR has to be tagged with at least one of the following labels (*): 1. `feature` 2. `bugfix` 3. `performance` 4. `ui` 5. `backward-incompat` 6.

Re: [PR] Fix bug in logging in UpsertCompaction task [pinot]

2024-02-14 Thread via GitHub
codecov-commenter commented on PR #12419: URL: https://github.com/apache/pinot/pull/12419#issuecomment-1943807503 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12419?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[PR] Fix bug in logging in UpsertCompaction task [pinot]

2024-02-14 Thread via GitHub
tibrewalpratik17 opened a new pull request, #12419: URL: https://github.com/apache/pinot/pull/12419 label: `bugfix` At present, the log message comes like: ``` Unable to get validDocIdsMetadata from all servers. Expected: 0, Actual: 1 ``` This is wrong due to fo

Re: [I] Expand Full Text Search Capabilities [pinot]

2024-02-14 Thread via GitHub
hpvd commented on issue #7218: URL: https://github.com/apache/pinot/issues/7218#issuecomment-1943669533 maybe this is fyi @kkrugler -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [I] Expand Full Text Search Capabilities [pinot]

2024-02-14 Thread via GitHub
hpvd commented on issue #7218: URL: https://github.com/apache/pinot/issues/7218#issuecomment-1943661027 this is an relatively old issue, and some things are already implemented. Coming from the ecommerce field where good text search is an absolute must-have, the question is why sho

Re: [PR] Bump com.google.api.grpc:proto-google-iam-v1 from 1.0.0 to 1.29.0 [pinot]

2024-02-14 Thread via GitHub
codecov-commenter commented on PR #12418: URL: https://github.com/apache/pinot/pull/12418#issuecomment-1943638964 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12418?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

(pinot) branch dependabot/maven/com.google.api.grpc-proto-google-iam-v1-1.29.0 created (now 880cb591a6)

2024-02-14 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.google.api.grpc-proto-google-iam-v1-1.29.0 in repository https://gitbox.apache.org/repos/asf/pinot.git at 880cb591a6 Bump com.google.api.grpc:proto-google-iam-v1 f

Re: [PR] Bump com.google.api.grpc:proto-google-iam-v1 from 1.0.0 to 1.28.0 [pinot]

2024-02-14 Thread via GitHub
dependabot[bot] commented on PR #12348: URL: https://github.com/apache/pinot/pull/12348#issuecomment-1943581601 Superseded by #12418. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[PR] Bump com.google.api.grpc:proto-google-iam-v1 from 1.0.0 to 1.29.0 [pinot]

2024-02-14 Thread via GitHub
dependabot[bot] opened a new pull request, #12418: URL: https://github.com/apache/pinot/pull/12418 Bumps [com.google.api.grpc:proto-google-iam-v1](https://github.com/googleapis/sdk-platform-java) from 1.0.0 to 1.29.0. Release notes Sourced from https://github.com/googleapis/sdk-pl

(pinot) branch dependabot/maven/com.google.api.grpc-proto-google-iam-v1-1.28.0 deleted (was 2e9a103cfa)

2024-02-14 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.google.api.grpc-proto-google-iam-v1-1.28.0 in repository https://gitbox.apache.org/repos/asf/pinot.git was 2e9a103cfa Bump com.google.api.grpc:proto-google-iam-v1 f

Re: [PR] Bump com.google.api.grpc:proto-google-iam-v1 from 1.0.0 to 1.28.0 [pinot]

2024-02-14 Thread via GitHub
dependabot[bot] closed pull request #12348: Bump com.google.api.grpc:proto-google-iam-v1 from 1.0.0 to 1.28.0 URL: https://github.com/apache/pinot/pull/12348 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] Record enricher [pinot]

2024-02-14 Thread via GitHub
gortiz commented on PR #12243: URL: https://github.com/apache/pinot/pull/12243#issuecomment-1943432997 Could we use a different configuration model? We have historically abuse of `properties` fields. They are easy to implement (just map them to `Map` with Jackson!), but they are a ve

Re: [PR] Record enricher [pinot]

2024-02-14 Thread via GitHub
gortiz commented on PR #12243: URL: https://github.com/apache/pinot/pull/12243#issuecomment-1943416791 ``` ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [I] Proposal for a Kubernetes Operator [pinot]

2024-02-14 Thread via GitHub
hpvd commented on issue #6464: URL: https://github.com/apache/pinot/issues/6464#issuecomment-1943390507 see also: https://github.com/datainfrahq/pinot-control-plane-k8s -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [I] Feature Request: Support for hooking different tokenizer or configuring existing for enhance text search capabilities [pinot]

2024-02-14 Thread via GitHub
hpvd commented on issue #10374: URL: https://github.com/apache/pinot/issues/10374#issuecomment-1943314168 @rohityadav1993 since this is pretty interesting, is there any news on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

(pinot) branch master updated: [Multi-stage] Fix SortedMailboxReceiveOperator to not pull 2 EOS blocks (#12406)

2024-02-14 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new a5ae4d738a [Multi-stage] Fix SortedMailboxReceive

Re: [PR] [Multi-stage] Fix SortedMailboxReceiveOperator to not pull 2 EOS blocks [pinot]

2024-02-14 Thread via GitHub
Jackie-Jiang merged PR #12406: URL: https://github.com/apache/pinot/pull/12406 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot