(pinot) branch release-1.1.0-rc updated: [maven-release-plugin] prepare release release-1.1.0-rc0

2024-03-01 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a commit to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-1.1.0-rc by this push: new 49ee95a75b [maven-release

(pinot) branch release-1.1.0-rc updated: [maven-release-plugin] prepare for next development iteration

2024-03-01 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a commit to branch release-1.1.0-rc in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/release-1.1.0-rc by this push: new 64f9d792f1 [maven-release

(pinot) annotated tag release-1.1.0-rc0 updated (49ee95a75b -> 37bd7be690)

2024-03-01 Thread vvivekiyer
This is an automated email from the ASF dual-hosted git repository. vvivekiyer pushed a change to annotated tag release-1.1.0-rc0 in repository https://gitbox.apache.org/repos/asf/pinot.git *** WARNING: tag release-1.1.0-rc0 was modified! *** from 49ee95a75b (commit) to 37bd7be690 (ta

Re: [I] Configurable heap size for query and realtime segment persistence. [pinot]

2024-03-01 Thread via GitHub
Lvnszn commented on issue #12503: URL: https://github.com/apache/pinot/issues/12503#issuecomment-1972735574 > Is the problem resolved automatically? We'll need more context in order to debug it Add a server and rebalance table to a new server then the problem was resolved -- This

Re: [I] String to bytes autocasting [pinot]

2024-03-01 Thread via GitHub
gortiz commented on issue #12457: URL: https://github.com/apache/pinot/issues/12457#issuecomment-1972834050 As a partial solution, @Jackie-Jiang and I are planning to merge https://github.com/apache/pinot/pull/12475, which forbids implicit or explicit casts from anything to Bytes/BINARY onl

Re: [PR] Add a post-validator visitor that verifies there are no cast to bytes [pinot]

2024-03-01 Thread via GitHub
gortiz commented on code in PR #12475: URL: https://github.com/apache/pinot/pull/12475#discussion_r1508720811 ## pinot-query-planner/src/main/java/org/apache/pinot/query/validate/BytesCastVisitor.java: ## @@ -0,0 +1,73 @@ +/** + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-03-01 Thread via GitHub
shounakmk219 commented on code in PR #12417: URL: https://github.com/apache/pinot/pull/12417#discussion_r1508835134 ## pinot-common/src/main/java/org/apache/pinot/common/utils/DatabaseUtils.java: ## @@ -0,0 +1,91 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-03-01 Thread via GitHub
shounakmk219 commented on code in PR #12417: URL: https://github.com/apache/pinot/pull/12417#discussion_r1508834724 ## pinot-broker/src/main/java/org/apache/pinot/broker/api/resources/PinotBrokerRouting.java: ## @@ -61,8 +69,23 @@ public class PinotBrokerRouting { @ApiRes

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-03-01 Thread via GitHub
shounakmk219 commented on code in PR #12417: URL: https://github.com/apache/pinot/pull/12417#discussion_r1508836250 ## pinot-common/src/main/java/org/apache/pinot/common/utils/DatabaseUtils.java: ## @@ -0,0 +1,91 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

Re: [I] 1.0.1 release timeline [pinot]

2024-03-01 Thread via GitHub
hpvd commented on issue #12428: URL: https://github.com/apache/pinot/issues/12428#issuecomment-1973139915 has moved to 1.1.0-rc https://github.com/apache/pinot/tree/release-1.1.0-rc -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] 1.0.1 release timeline [pinot]

2024-03-01 Thread via GitHub
hpvd commented on issue #12428: URL: https://github.com/apache/pinot/issues/12428#issuecomment-1973142317 1.1.0-rc was released: https://github.com/apache/pinot/releases/tag/release-1.1.0-rc0 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] 1.0.1 release timeline [pinot]

2024-03-01 Thread via GitHub
hpvd commented on issue #12428: URL: https://github.com/apache/pinot/issues/12428#issuecomment-1973159953 so maybe this issue can already be closed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] [feature] use async methods to read from Pulsar topic [pinot]

2024-03-01 Thread via GitHub
enzo-dechaene commented on PR #12061: URL: https://github.com/apache/pinot/pull/12061#issuecomment-1973221062 Hii @KKcorps, can we merge this PR in the 1.1.0 release ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Throw exception when schema name doesn't match table name during table creation [pinot]

2024-03-01 Thread via GitHub
estebanz01 commented on PR #11591: URL: https://github.com/apache/pinot/pull/11591#issuecomment-1973253789 Hola 👋 ! sorry to comment on a merged PR, but I don't see in the release docs for 1.1.0-rc a reference to this change, which is a breaking change for our infrastructure. Is it going to

[PR] Json extract index mv [pinot]

2024-03-01 Thread via GitHub
saurabhd336 opened a new pull request, #12532: URL: https://github.com/apache/pinot/pull/12532 inspired by JSONEXTRACTINDEX, allows extracting contexual json array elements matching a given json match predicate, using the json index eg: For a dataset like ``` {"jsonDoc": [{"n

Re: [PR] Allow passing custom record reader to be inited/closed in SegmentProcessorFramework [pinot]

2024-03-01 Thread via GitHub
swaminathanmanish commented on PR #12529: URL: https://github.com/apache/pinot/pull/12529#issuecomment-1973361126 > LGTM > > Is the agreement is to not use the context? Yes it looks like the purpose of a RecordTransformer is to transform a GenericRecord. independent of file/rec

Re: [PR] Json extract index mv [pinot]

2024-03-01 Thread via GitHub
codecov-commenter commented on PR #12532: URL: https://github.com/apache/pinot/pull/12532#issuecomment-1973403465 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12532?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[PR] use Temurin instead of AdoptOpenJdk [pinot]

2024-03-01 Thread via GitHub
sullis opened a new pull request, #12533: URL: https://github.com/apache/pinot/pull/12533 - AdoptOpenJDK is obsolete - use Temurin JDK -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Add the support to extract json array elements from json index for the transform function jsonExtractIndex [pinot]

2024-03-01 Thread via GitHub
wirybeaver commented on code in PR #12466: URL: https://github.com/apache/pinot/pull/12466#discussion_r1509270121 ## pinot-core/src/test/java/org/apache/pinot/core/operator/transform/function/BaseTransformFunctionTest.java: ## @@ -158,6 +159,11 @@ public void setUp()

Re: [I] [Flaky test] CaseTransformFunctionTest. testCaseTransformFunctionWithIntResults() [pinot]

2024-03-01 Thread via GitHub
aditya0811 commented on issue #12367: URL: https://github.com/apache/pinot/issues/12367#issuecomment-1973584832 Hey Jackie, when you say expected result is wrong, you mean the comparison in the case block in L721 in CaseTransformFunctionTest.java? ``` case FLOAT_SV_COLUMN:

Re: [PR] use Temurin instead of AdoptOpenJdk [pinot]

2024-03-01 Thread via GitHub
codecov-commenter commented on PR #12533: URL: https://github.com/apache/pinot/pull/12533#issuecomment-1973621247 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12533?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] Custom configuration property reader for segment metadata files [pinot]

2024-03-01 Thread via GitHub
abhioncbr commented on code in PR #12440: URL: https://github.com/apache/pinot/pull/12440#discussion_r1509441264 ## pinot-spi/src/main/java/org/apache/pinot/spi/env/CommonsConfigurationUtils.java: ## @@ -287,4 +321,35 @@ private static PropertiesConfiguration createPropertiesCo

Re: [I] Consuming segment flush rows significantly reduced after force commit [pinot]

2024-03-01 Thread via GitHub
sajjad-moradi commented on issue #12509: URL: https://github.com/apache/pinot/issues/12509#issuecomment-1973795958 I believe as part of this [PR](https://github.com/apache/pinot/pull/12188/files), this issue was fixed. Look here: https://github.com/apache/pinot/blob/9cfce82385dfcd11ee

svn commit: r67657 - /dev/pinot/apache-pinot-1.1.0-rc0/

2024-03-01 Thread vvivekiyer
Author: vvivekiyer Date: Fri Mar 1 20:43:54 2024 New Revision: 67657 Log: Update apache-pinot-1.1.0-rc0 Added: dev/pinot/apache-pinot-1.1.0-rc0/ dev/pinot/apache-pinot-1.1.0-rc0/apache-pinot-1.1.0-bin.tar.gz (with props) dev/pinot/apache-pinot-1.1.0-rc0/apache-pinot-1.1.0-bin.tar.

Re: [PR] Enter segment preloading at partition level [pinot]

2024-03-01 Thread via GitHub
klsince commented on code in PR #12451: URL: https://github.com/apache/pinot/pull/12451#discussion_r1509628827 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java: ## @@ -359,6 +357,12 @@ public boolean isPartialUpsertEnabled() {

Re: [PR] Metrics for Realtime Rows Fetched and Stream Consumer Create Exceptions [pinot]

2024-03-01 Thread via GitHub
zhtaoxiang commented on code in PR #12522: URL: https://github.com/apache/pinot/pull/12522#discussion_r1509633595 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java: ## @@ -1661,22 +1666,35 @@ private void makeStreamConsumer(S

[I] `VarByteChunkForwardIndexReaderV4` fails to decompress some chunks [pinot]

2024-03-01 Thread via GitHub
itschrispeck opened a new issue, #12534: URL: https://github.com/apache/pinot/issues/12534 Within a segment, certain chunks cannot be decompressed. I can time bound a query within a single segment, and the query will succeed. e.g. for a segment with data [chunk1][chunk2][chunk3]:

Re: [I] `VarByteChunkForwardIndexReaderV4` fails to decompress some chunks [pinot]

2024-03-01 Thread via GitHub
itschrispeck commented on issue #12534: URL: https://github.com/apache/pinot/issues/12534#issuecomment-1974066176 @richardstartin for your thoughts -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] maven no transfer progress [pinot]

2024-03-01 Thread via GitHub
Jackie-Jiang merged PR #12528: URL: https://github.com/apache/pinot/pull/12528 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: maven: no transfer progress (#12528)

2024-03-01 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 1bfc6ac4dd maven: no transfer progress (#12528) 1

Re: [PR] Egalpin/skip indexes minor changes [pinot]

2024-03-01 Thread via GitHub
Jackie-Jiang merged PR #12514: URL: https://github.com/apache/pinot/pull/12514 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: Egalpin/skip indexes minor changes (#12514)

2024-03-01 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new f51c34fcbd Egalpin/skip indexes minor changes (#1

Re: [PR] Egalpin/skip indexes minor changes [pinot]

2024-03-01 Thread via GitHub
Jackie-Jiang commented on PR #12514: URL: https://github.com/apache/pinot/pull/12514#issuecomment-1974098078 Thank you for addressing the comments! Can you help contribute this new query option to the pinot doc? -- This is an automated message from the Apache Git Service. To respond to th

Re: [I] Consuming segment flush rows significantly reduced after force commit [pinot]

2024-03-01 Thread via GitHub
Jackie-Jiang closed issue #12509: Consuming segment flush rows significantly reduced after force commit URL: https://github.com/apache/pinot/issues/12509 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] use Temurin instead of AdoptOpenJdk [pinot]

2024-03-01 Thread via GitHub
Jackie-Jiang merged PR #12533: URL: https://github.com/apache/pinot/pull/12533 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: use Temurin instead of AdoptOpenJdk (#12533)

2024-03-01 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 954f5d7e51 use Temurin instead of AdoptOpenJdk (#

Re: [I] [Flaky test] CaseTransformFunctionTest. testCaseTransformFunctionWithIntResults() [pinot]

2024-03-01 Thread via GitHub
Jackie-Jiang commented on issue #12367: URL: https://github.com/apache/pinot/issues/12367#issuecomment-1974115082 Yes. When we use float as literal (e.g. `WHERE col = 0.1`), it is actually parsed as double. If the `col` is of type `FLOAT`, it might not match this filter because `(double) 0.

Re: [PR] reformat ClusterTest.java [pinot]

2024-03-01 Thread via GitHub
Jackie-Jiang merged PR #12531: URL: https://github.com/apache/pinot/pull/12531 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: reformat ClusterTest.java (#12531)

2024-03-01 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 7b71babb73 reformat ClusterTest.java (#12531) 7b7

Re: [PR] Prepare for next development iteration [pinot]

2024-03-01 Thread via GitHub
Jackie-Jiang merged PR #12530: URL: https://github.com/apache/pinot/pull/12530 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

(pinot) branch master updated: Prepare for next development iteration (#12530)

2024-03-01 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 60d23ca97b Prepare for next development iteration

[I] Segment Compaction tasks fail when snapshot file doesn't exist on lagging server [pinot]

2024-03-01 Thread via GitHub
robertzych opened a new issue, #12535: URL: https://github.com/apache/pinot/issues/12535 When the compaction is configured with a `bufferTimePeriod` value `< (2 * segmentCommitInterval)` the tasks fail due to the following `NotFoundException`: ![image](https://github.com/apache/pinot

Re: [PR] adding support of mega_bytes when configuring broker response size [pinot]

2024-03-01 Thread via GitHub
Jackie-Jiang commented on code in PR #12510: URL: https://github.com/apache/pinot/pull/12510#discussion_r1509748576 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java: ## @@ -1773,15 +1774,17 @@ private void setMaxServerResponseSiz

Re: [PR] Allow passing custom record reader to be inited/closed in SegmentProcessorFramework [pinot]

2024-03-01 Thread via GitHub
snleee merged PR #12529: URL: https://github.com/apache/pinot/pull/12529 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

(pinot) branch master updated: Allow passing custom record reader to be inited/closed in SegmentProcessorFramework (#12529)

2024-03-01 Thread snlee
This is an automated email from the ASF dual-hosted git repository. snlee pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 9a8fa79199 Allow passing custom record reader to b

[PR] mockito 5.11.0 [pinot]

2024-03-01 Thread via GitHub
sullis opened a new pull request, #12536: URL: https://github.com/apache/pinot/pull/12536 mockito 5.11.0 is the latest version of mockito. ## mockito-inline mockito-inline no longer exists see: https://github.com/mockito/mockito/issues/2877 -- This is an automated

Re: [PR] mockito 5.11.0 [pinot]

2024-03-01 Thread via GitHub
codecov-commenter commented on PR #12536: URL: https://github.com/apache/pinot/pull/12536#issuecomment-1974202097 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12536?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[PR] DRAFT: Adding SHOW TABLES/DESCRIBE table support [pinot]

2024-03-01 Thread via GitHub
aadilkhalifa opened a new pull request, #12537: URL: https://github.com/apache/pinot/pull/12537 PR to add support for "SHOW TABLES" and "DESCRIBE table" queries. Current status: - [x] SHOW TABLES - [ ] DESCRIBE table -- This is an automated message from the Apache Git Service.

[PR] [experiment] Add Deterministic Hash Function for Upserts for Encoding Type-4 UUIDs [pinot]

2024-03-01 Thread via GitHub
ankitsultana opened a new pull request, #12538: URL: https://github.com/apache/pinot/pull/12538 This can help reduce memory usage for scenarios where all columns of the Primary Key are type-4 UUIDs. This can also handle scenarios where some values are not valid type-4 UUIDs, by simply switc

Re: [PR] [experiment] Deterministic Hash Function for Upsert Type-4 UUID Primary Keys [pinot]

2024-03-01 Thread via GitHub
ankitsultana commented on code in PR #12538: URL: https://github.com/apache/pinot/pull/12538#discussion_r1509906166 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/HashUtils.java: ## @@ -36,6 +40,35 @@ public static byte[] hashMD5(byte[] bytes) { r

Re: [PR] [experiment] Deterministic Hash Function for Upsert Type-4 UUID Primary Keys [pinot]

2024-03-01 Thread via GitHub
ankitsultana commented on code in PR #12538: URL: https://github.com/apache/pinot/pull/12538#discussion_r1509906862 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/HashUtils.java: ## @@ -44,6 +77,8 @@ public static Object hashPrimaryKey(PrimaryKey prima