[GitHub] [pinot] richardstartin commented on issue #8607: Explain plan can be misleading

2022-05-03 Thread GitBox
richardstartin commented on issue #8607: URL: https://github.com/apache/pinot/issues/8607#issuecomment-1115856106 > One approach could be: > > * Send query to all segments and all servers > * On the broker, pick the response with deepest tree and avoid merging across segments >

[GitHub] [pinot] atris commented on issue #8585: Support balanced segment assignment for consuming segments

2022-05-03 Thread GitBox
atris commented on issue #8585: URL: https://github.com/apache/pinot/issues/8585#issuecomment-1115996286 @sajjad-moradi Are you actively working on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [pinot] suddendust commented on issue #8617: Add CSV Decoder for realtime

2022-05-03 Thread GitBox
suddendust commented on issue #8617: URL: https://github.com/apache/pinot/issues/8617#issuecomment-1116064297 @npawar Can take this up, working on a very similar [feature](https://github.com/apache/pinot/issues/8399). -- This is an automated message from the Apache Git Service. To respond

[GitHub] [pinot] richardstartin opened a new pull request, #8628: Broker tracing

2022-05-03 Thread GitBox
richardstartin opened a new pull request, #8628: URL: https://github.com/apache/pinot/pull/8628 This repurposes `RequestStatistics` so it can be overridden by a third party tracer so that e.g. JFR events can be emitted for queries which breach a latency threshold. -- This is an automate

[GitHub] [pinot] nizarhejazi commented on a diff in pull request #8622: Support BigDecimal raw value forward index; Support BigDecimal in many transforms and operators

2022-05-03 Thread GitBox
nizarhejazi commented on code in PR #8622: URL: https://github.com/apache/pinot/pull/8622#discussion_r863836030 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java: ## @@ -374,6 +374,7 @@ public static void

[GitHub] [pinot] nizarhejazi commented on a diff in pull request #8622: Support BigDecimal raw value forward index; Support BigDecimal in many transforms and operators

2022-05-03 Thread GitBox
nizarhejazi commented on code in PR #8622: URL: https://github.com/apache/pinot/pull/8622#discussion_r863836591 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/fwd/SingleValueFixedByteRawIndexCreator.java: ## @@ -30,7 +31,7 @@ /** *

[GitHub] [pinot] richardstartin opened a new pull request, #8629: Sampling driven tracing

2022-05-03 Thread GitBox
richardstartin opened a new pull request, #8629: URL: https://github.com/apache/pinot/pull/8629 This builds on #8628 to allow the tracer implementation to make a head sampling decision which can be used to enabled tracing automatically. By default this is disabled and does not affect Pinot

[GitHub] [pinot] npawar commented on issue #8617: Add CSV Decoder for realtime

2022-05-03 Thread GitBox
npawar commented on issue #8617: URL: https://github.com/apache/pinot/issues/8617#issuecomment-1116211204 Thanks Prashant! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [pinot] yupeng9 commented on a diff in pull request #8620: Handle nested conversion of geo column

2022-05-03 Thread GitBox
yupeng9 commented on code in PR #8620: URL: https://github.com/apache/pinot/pull/8620#discussion_r863891845 ## pinot-core/src/main/java/org/apache/pinot/core/util/GeoColumnUtils.java: ## @@ -0,0 +1,43 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or

[GitHub] [pinot] yupeng9 commented on pull request #8620: Handle nested conversion of geo column

2022-05-03 Thread GitBox
yupeng9 commented on PR #8620: URL: https://github.com/apache/pinot/pull/8620#issuecomment-1116220449 plz take a look at the tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #8628: Broker tracing

2022-05-03 Thread GitBox
siddharthteotia commented on code in PR #8628: URL: https://github.com/apache/pinot/pull/8628#discussion_r863942853 ## pinot-spi/src/main/java/org/apache/pinot/spi/trace/RequestStatistics.java: ## @@ -0,0 +1,159 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [pinot] snleee commented on issue #8585: Support balanced segment assignment for consuming segments

2022-05-03 Thread GitBox
snleee commented on issue #8585: URL: https://github.com/apache/pinot/issues/8585#issuecomment-1116282958 @atris Thank you for the interest. @GSharayu is working on this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #8628: Broker tracing

2022-05-03 Thread GitBox
siddharthteotia commented on code in PR #8628: URL: https://github.com/apache/pinot/pull/8628#discussion_r863948002 ## pinot-spi/src/main/java/org/apache/pinot/spi/trace/Tracer.java: ## @@ -43,6 +43,14 @@ public interface Tracer { */ InvocationScope createScope(Class

[GitHub] [pinot] siddharthteotia commented on pull request #8628: Broker tracing

2022-05-03 Thread GitBox
siddharthteotia commented on PR #8628: URL: https://github.com/apache/pinot/pull/8628#issuecomment-1116284905 Overall looks neat!! Just want to clarify the changes needed on the client (user of RequestStatistics) side per my above comment. -- This is an automated message from the A

[GitHub] [pinot] atris commented on issue #8585: Support balanced segment assignment for consuming segments

2022-05-03 Thread GitBox
atris commented on issue #8585: URL: https://github.com/apache/pinot/issues/8585#issuecomment-1116285557 Cool, thanks. I had somebody I wanted to introduce to Pinot so thought this might be a good issue :D -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [pinot] richardstartin commented on a diff in pull request #8628: Broker tracing

2022-05-03 Thread GitBox
richardstartin commented on code in PR #8628: URL: https://github.com/apache/pinot/pull/8628#discussion_r863951473 ## pinot-spi/src/main/java/org/apache/pinot/spi/trace/Tracer.java: ## @@ -43,6 +43,14 @@ public interface Tracer { */ InvocationScope createScope(Class c

[GitHub] [pinot] kkrugler commented on pull request #6703: Remove slf4j-impl (#6364)

2022-05-03 Thread GitBox
kkrugler commented on PR #6703: URL: https://github.com/apache/pinot/pull/6703#issuecomment-1116325269 @KKcorps - yes, exactly. Plus other (lower-level) Pinot jars that include logging implementations. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [pinot] richardstartin commented on a diff in pull request #8628: Broker tracing

2022-05-03 Thread GitBox
richardstartin commented on code in PR #8628: URL: https://github.com/apache/pinot/pull/8628#discussion_r863951473 ## pinot-spi/src/main/java/org/apache/pinot/spi/trace/Tracer.java: ## @@ -43,6 +43,14 @@ public interface Tracer { */ InvocationScope createScope(Class c

[GitHub] [pinot] siddharthteotia merged pull request #8628: Broker tracing

2022-05-03 Thread GitBox
siddharthteotia merged PR #8628: URL: https://github.com/apache/pinot/pull/8628 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pino

[pinot] branch master updated (4e14101f57 -> 72e1844559)

2022-05-03 Thread siddteotia
This is an automated email from the ASF dual-hosted git repository. siddteotia pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 4e14101f57 Add unrecognizedProperties to schema and tableConfigs APIs (#8606) add 72e1844559 Broker tracing (

[GitHub] [pinot] richardstartin commented on pull request #8629: Sampling driven tracing

2022-05-03 Thread GitBox
richardstartin commented on PR #8629: URL: https://github.com/apache/pinot/pull/8629#issuecomment-1116389141 @siddharthteotia what was `RequestStatistics` changes to `RequestContext` and has an `isSampled()` which the tracer implementation can control. Code which previously used `RequestSta

[GitHub] [pinot] Jackie-Jiang commented on issue #8627: Support filtering for literals

2022-05-03 Thread GitBox
Jackie-Jiang commented on issue #8627: URL: https://github.com/apache/pinot/issues/8627#issuecomment-1116421356 Let's list down the standard SQL semantics for using different data type as boolean. These rules should be applied to all expressions in addition to literals. -- This is an aut

[GitHub] [pinot] Jackie-Jiang merged pull request #8622: Support BigDecimal raw value forward index; Support BigDecimal in many transforms and operators

2022-05-03 Thread GitBox
Jackie-Jiang merged PR #8622: URL: https://github.com/apache/pinot/pull/8622 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8589: Implement builtin-purge task

2022-05-03 Thread GitBox
Jackie-Jiang commented on code in PR #8589: URL: https://github.com/apache/pinot/pull/8589#discussion_r864087024 ## pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/purge/PurgeTaskGenerator.java: ## @@ -0,0 +1,134 @@

[GitHub] [pinot] jackjlli commented on pull request #8602: Change shading prefix so as to minimize conflicts with other libaries

2022-05-03 Thread GitBox
jackjlli commented on PR #8602: URL: https://github.com/apache/pinot/pull/8602#issuecomment-1116445461 > Will this remain backward compatible? A recent discussion in slack made me think about this one: https://apache-pinot.slack.com/archives/CDRCA57FC/p1651496807383319 @npawar After

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8582: Support conjugates for scalar functions, add more scalar functions

2022-05-03 Thread GitBox
Jackie-Jiang commented on code in PR #8582: URL: https://github.com/apache/pinot/pull/8582#discussion_r864096070 ## pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ComparisonFunctions.java: ## @@ -0,0 +1,65 @@ +/** + * Licensed to the Apache Software Foundatio

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8621: Update default helm image to latest

2022-05-03 Thread GitBox
Jackie-Jiang commented on code in PR #8621: URL: https://github.com/apache/pinot/pull/8621#discussion_r864098268 ## kubernetes/helm/pinot/values.yaml: ## @@ -21,8 +21,8 @@ image: repository: apachepinot/pinot - tag: latest-jdk11 # release-0.7.1 - pullPolicy: IfNotPresent

[GitHub] [pinot] Jackie-Jiang merged pull request #8467: pinot-java-client controller based broker selector

2022-05-03 Thread GitBox
Jackie-Jiang merged PR #8467: URL: https://github.com/apache/pinot/pull/8467 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

[pinot] branch master updated: pinot-java-client controller based broker selector (#8467)

2022-05-03 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 01dda9ddfa pinot-java-client controller based bro

[GitHub] [pinot] Jackie-Jiang closed issue #7256: Removing ZKClient dependency from Pinot Client

2022-05-03 Thread GitBox
Jackie-Jiang closed issue #7256: Removing ZKClient dependency from Pinot Client URL: https://github.com/apache/pinot/issues/7256 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #8621: Update default helm image to latest

2022-05-03 Thread GitBox
xiangfu0 commented on code in PR #8621: URL: https://github.com/apache/pinot/pull/8621#discussion_r864120079 ## kubernetes/helm/pinot/values.yaml: ## @@ -21,8 +21,8 @@ image: repository: apachepinot/pinot - tag: latest-jdk11 # release-0.7.1 - pullPolicy: IfNotPresent +

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #8621: Update default helm image to latest

2022-05-03 Thread GitBox
xiangfu0 commented on code in PR #8621: URL: https://github.com/apache/pinot/pull/8621#discussion_r864120304 ## kubernetes/helm/pinot/values.yaml: ## @@ -21,8 +21,8 @@ image: repository: apachepinot/pinot - tag: latest-jdk11 # release-0.7.1 - pullPolicy: IfNotPresent +

[pinot] branch master updated (01dda9ddfa -> f693bd3edc)

2022-05-03 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 01dda9ddfa pinot-java-client controller based broker selector (#8467) add f693bd3edc Update default helm image to

[GitHub] [pinot] xiangfu0 merged pull request #8621: Update default helm image to latest

2022-05-03 Thread GitBox
xiangfu0 merged PR #8621: URL: https://github.com/apache/pinot/pull/8621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

[GitHub] [pinot] ddcprg commented on a diff in pull request #8625: Add support to select timestamp arrays

2022-05-03 Thread GitBox
ddcprg commented on code in PR #8625: URL: https://github.com/apache/pinot/pull/8625#discussion_r863186931 ## pinot-core/src/test/java/org/apache/pinot/core/query/selection/SelectionOperatorServiceTest.java: ## @@ -235,11 +238,11 @@ public void testCompatibleRowsRenderSelection

[GitHub] [pinot] siddharthteotia opened a new pull request, #8630: Upgrade jackson to 2.13.1

2022-05-03 Thread GitBox
siddharthteotia opened a new pull request, #8630: URL: https://github.com/apache/pinot/pull/8630 cc @vvivekiyer @jasperjiaguo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[GitHub] [pinot] deemoliu commented on issue #8579: Data loss in offline table after tenant migration and rebalancing (no server restart)

2022-05-03 Thread GitBox
deemoliu commented on issue #8579: URL: https://github.com/apache/pinot/issues/8579#issuecomment-1116546213 Thanks @Jackie-Jiang for taking a look. We have successfully reproduce the issue yesterday by rebalancing the table. We find that something updated the hdfs url for this segment

[GitHub] [pinot] navina opened a new pull request, #8631: [Bugfix] Fix incorrect time string format in KinesisDataProducer

2022-05-03 Thread GitBox
navina opened a new pull request, #8631: URL: https://github.com/apache/pinot/pull/8631 Incorrect string representation of retry delay `1L` results in `NumberFormatException`. Replaced with `1` in `KinesisDataProducer` -- This is an automated message from the Apache Git Service.

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8601: Add support for indexes during offline segment creation

2022-05-03 Thread GitBox
Jackie-Jiang commented on code in PR #8601: URL: https://github.com/apache/pinot/pull/8601#discussion_r864267277 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/IndexingConfig.java: ## @@ -64,6 +64,7 @@ public class IndexingConfig extends BaseJsonConfig { // TODO

[GitHub] [pinot] codecov-commenter commented on pull request #8631: [Bugfix] Fix incorrect time string format in KinesisDataProducer

2022-05-03 Thread GitBox
codecov-commenter commented on PR #8631: URL: https://github.com/apache/pinot/pull/8631#issuecomment-111246 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8631?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8625: Add support to select timestamp arrays

2022-05-03 Thread GitBox
Jackie-Jiang commented on code in PR #8625: URL: https://github.com/apache/pinot/pull/8625#discussion_r864272152 ## pinot-core/src/main/java/org/apache/pinot/core/query/selection/SelectionOperatorUtils.java: ## @@ -597,6 +599,14 @@ public static Serializable getFormattedValue(Ob

[GitHub] [pinot] Jackie-Jiang commented on issue #8623: [Feature] can i use apache pinot to query data already stored in a Nosql DB deployed on cloud.

2022-05-03 Thread GitBox
Jackie-Jiang commented on issue #8623: URL: https://github.com/apache/pinot/issues/8623#issuecomment-1116682415 Currently there is no way to directly pull data from an external DB. You will need to first dump the data from the DB into a supported file format, then use the Pinot ingestion fl

[GitHub] [pinot] Jackie-Jiang commented on issue #8399: Persist raw data from kafka topic as it is

2022-05-03 Thread GitBox
Jackie-Jiang commented on issue #8399: URL: https://github.com/apache/pinot/issues/8399#issuecomment-1116686955 > @Jackie-Jiang If this property is set to true, then we should just ignore the filter and transform configs? Or should be provide a way to store the record as a JSON AFTER applyi

[GitHub] [pinot] npawar commented on issue #8581: DatetimeConvert does not support timezone when output format is epoch

2022-05-03 Thread GitBox
npawar commented on issue #8581: URL: https://github.com/apache/pinot/issues/8581#issuecomment-1116701575 @Jackie-Jiang is this something we can pick up? Is it a beginner issue? Or would you prefer just getting to the DateTimeFieldSpec refactoring that you were suggesting? If so, mind add

[GitHub] [pinot] npawar commented on issue #8579: Data loss in offline table after tenant migration and rebalancing (no server restart)

2022-05-03 Thread GitBox
npawar commented on issue #8579: URL: https://github.com/apache/pinot/issues/8579#issuecomment-1116703184 Hey @deemoliu , what were your further findings? Were the urls updated by some operation on your end, or did it happen because of some Pinot operation that needs to be debugged further?

[GitHub] [pinot] npawar commented on issue #8547: ST_Contain function can use H3 native different polyfill mode when they release

2022-05-03 Thread GitBox
npawar commented on issue #8547: URL: https://github.com/apache/pinot/issues/8547#issuecomment-1116705284 @WangCHX does your PR address and fix this? Can this be closed or does it need further work? -- This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [pinot] npawar commented on issue #8492: Need ways to check whether segment reload/refresh is done on server side

2022-05-03 Thread GitBox
npawar commented on issue #8492: URL: https://github.com/apache/pinot/issues/8492#issuecomment-1116709729 FYI @saurabhd336 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [pinot] npawar commented on issue #8476: pinot-controller uses multiple NPMs that have security issues

2022-05-03 Thread GitBox
npawar commented on issue #8476: URL: https://github.com/apache/pinot/issues/8476#issuecomment-1116710686 @pjfanning i saw some PRs from you. Are these all addressed? If so, would you please close this issue? -- This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8584: Enable uploading segments to realtime tables

2022-05-03 Thread GitBox
Jackie-Jiang commented on code in PR #8584: URL: https://github.com/apache/pinot/pull/8584#discussion_r864295648 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/RealtimeSegmentAssignment.java: ## @@ -104,18 +104,15 @@ public void init(

[GitHub] [pinot] Jackie-Jiang commented on issue #8537: Pulsar test instability

2022-05-03 Thread GitBox
Jackie-Jiang commented on issue #8537: URL: https://github.com/apache/pinot/issues/8537#issuecomment-1116713496 I'm still seeing the exact same failure. @KKcorps Please take another look -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [pinot] npawar commented on issue #8423: The race condition that TableDataManager is removed during segment download because of incorrect segment count.

2022-05-03 Thread GitBox
npawar commented on issue #8423: URL: https://github.com/apache/pinot/issues/8423#issuecomment-1116715448 has this been resolved? can the issue be closed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [pinot] npawar closed issue #8418: Add support for BigDecimal as a Pinot data type

2022-05-03 Thread GitBox
npawar closed issue #8418: Add support for BigDecimal as a Pinot data type URL: https://github.com/apache/pinot/issues/8418 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8355: add a experiment API for upsert heap memory estimation

2022-05-03 Thread GitBox
Jackie-Jiang commented on code in PR #8355: URL: https://github.com/apache/pinot/pull/8355#discussion_r864302884 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotUpsertRestletResource.java: ## @@ -0,0 +1,153 @@ +/** + * Licensed to the Apache Soft

[GitHub] [pinot] npawar commented on issue #8406: Pinot returns empty result set when there is an invalid column

2022-05-03 Thread GitBox
npawar commented on issue #8406: URL: https://github.com/apache/pinot/issues/8406#issuecomment-1116717755 @elonazoulay can you confirm regarding the schema and if it is an issue even though you have a schema? -- This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [pinot] Jackie-Jiang closed issue #8318: Indicate what fields are getting ignored when adding configs

2022-05-03 Thread GitBox
Jackie-Jiang closed issue #8318: Indicate what fields are getting ignored when adding configs URL: https://github.com/apache/pinot/issues/8318 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [pinot] npawar commented on a diff in pull request #8582: Support conjugates for scalar functions, add more scalar functions

2022-05-03 Thread GitBox
npawar commented on code in PR #8582: URL: https://github.com/apache/pinot/pull/8582#discussion_r864309112 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/recordtransformer/RecordTransformerTest.java: ## @@ -178,6 +179,150 @@ public void testSanitationTransfo

[pinot] branch master updated: bugfix on timeout string format in KinesisDataProducer (#8631)

2022-05-03 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 2e0f74d208 bugfix on timeout string format in Kin

[GitHub] [pinot] Jackie-Jiang merged pull request #8631: [Bugfix] Fix incorrect time string format in KinesisDataProducer

2022-05-03 Thread GitBox
Jackie-Jiang merged PR #8631: URL: https://github.com/apache/pinot/pull/8631 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

[GitHub] [pinot] Jackie-Jiang closed issue #8547: ST_Contain function can use H3 native different polyfill mode when they release

2022-05-03 Thread GitBox
Jackie-Jiang closed issue #8547: ST_Contain function can use H3 native different polyfill mode when they release URL: https://github.com/apache/pinot/issues/8547 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [pinot] Jackie-Jiang commented on issue #8581: DatetimeConvert does not support timezone when output format is epoch

2022-05-03 Thread GitBox
Jackie-Jiang commented on issue #8581: URL: https://github.com/apache/pinot/issues/8581#issuecomment-1116730010 @npawar They are orthogonal. I can create a separate issue for the new time format @cyrilou242 Do you want to help contribute this enhancement? -- This is an automated me

[GitHub] [pinot] amrishlal commented on issue #8607: Explain plan can be misleading

2022-05-03 Thread GitBox
amrishlal commented on issue #8607: URL: https://github.com/apache/pinot/issues/8607#issuecomment-1116735978 If we want to be 100% accurate, then probably the only option is to move towards full evaluation of EXPLAIN PLAN over all segments on all servers. To start with maybe we could evalu

[GitHub] [pinot] WangCHX commented on pull request #8620: Handle nested conversion of geo column

2022-05-03 Thread GitBox
WangCHX commented on PR #8620: URL: https://github.com/apache/pinot/pull/8620#issuecomment-1116742625 > This is not the correct fix. The filter operator relies on the H3 index on the column. If the column is within an expression, the result will be wrong. E.g. `ST_DISTANCE(geoCol, 'point')`

[GitHub] [pinot] Jackie-Jiang opened a new issue, #8632: Re-design the date time format

2022-05-03 Thread GitBox
Jackie-Jiang opened a new issue, #8632: URL: https://github.com/apache/pinot/issues/8632 Currently the date time format is of the format: - `::(:( tz())` E.g. - `1:SECONDS:EPOCH` - `1:DAYS:SIMPLE_DATE_FORMAT:-MM-dd tz(America/Chicago)` - `1:MILLISECONDS:TIMESTAMP`

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8582: Support conjugates for scalar functions, add more scalar functions

2022-05-03 Thread GitBox
Jackie-Jiang commented on code in PR #8582: URL: https://github.com/apache/pinot/pull/8582#discussion_r864325793 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/recordtransformer/RecordTransformerTest.java: ## @@ -178,6 +179,150 @@ public void testSanitationT

[GitHub] [pinot] Jackie-Jiang commented on pull request #8620: Handle nested conversion of geo column

2022-05-03 Thread GitBox
Jackie-Jiang commented on PR #8620: URL: https://github.com/apache/pinot/pull/8620#issuecomment-1116753483 > > This is not the correct fix. The filter operator relies on the H3 index on the column. If the column is within an expression, the result will be wrong. E.g. `ST_DISTANCE(geoCol, 'p

[GitHub] [pinot] Jackie-Jiang merged pull request #8612: Store batch message ids vs calculating them

2022-05-03 Thread GitBox
Jackie-Jiang merged PR #8612: URL: https://github.com/apache/pinot/pull/8612 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

[pinot] branch master updated (2e0f74d208 -> cced818177)

2022-05-03 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 2e0f74d208 bugfix on timeout string format in KinesisDataProducer (#8631) add cced818177 Store batch message ids

[GitHub] [pinot] Jackie-Jiang commented on pull request #8612: Store batch message ids vs calculating them

2022-05-03 Thread GitBox
Jackie-Jiang commented on PR #8612: URL: https://github.com/apache/pinot/pull/8612#issuecomment-1116756998 Related to #8537 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [pinot] Jackie-Jiang closed issue #8537: Pulsar test instability

2022-05-03 Thread GitBox
Jackie-Jiang closed issue #8537: Pulsar test instability URL: https://github.com/apache/pinot/issues/8537 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail

[GitHub] [pinot] WangCHX commented on pull request #8620: Handle nested conversion of geo column

2022-05-03 Thread GitBox
WangCHX commented on PR #8620: URL: https://github.com/apache/pinot/pull/8620#issuecomment-1116762409 > > > This is not the correct fix. The filter operator relies on the H3 index on the column. If the column is within an expression, the result will be wrong. E.g. `ST_DISTANCE(geoCol, 'poin

[GitHub] [pinot] deemoliu commented on issue #8579: Data loss in offline table after tenant migration and rebalancing (no server restart)

2022-05-03 Thread GitBox
deemoliu commented on issue #8579: URL: https://github.com/apache/pinot/issues/8579#issuecomment-1116774756 hi @npawar thanks for looking into the issue. I think the URL is updated by some user operation. The user has three hive to Pinot jobs that used the same input hive table and write to

[GitHub] [pinot] Jackie-Jiang opened a new pull request, #8633: Fix the potential NPE for timestamp index rewrite

2022-05-03 Thread GitBox
Jackie-Jiang opened a new pull request, #8633: URL: https://github.com/apache/pinot/pull/8633 Fix the potential NPE for the timestamp index rewrite when the ingestion config or the transform configs are not configured in the table config -- This is an automated message from the Apache Git

[GitHub] [pinot] Jackie-Jiang merged pull request #8384: Implement Native Text Operator

2022-05-03 Thread GitBox
Jackie-Jiang merged PR #8384: URL: https://github.com/apache/pinot/pull/8384 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

[pinot] branch master updated (cced818177 -> 907b0239b1)

2022-05-03 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from cced818177 Store batch message ids vs calculating them (#8612) add 907b0239b1 Implement Native Text Operator (#838

[GitHub] [pinot] Jackie-Jiang commented on pull request #8620: Handle nested conversion of geo column

2022-05-03 Thread GitBox
Jackie-Jiang commented on PR #8620: URL: https://github.com/apache/pinot/pull/8620#issuecomment-1116824711 @yupeng9 Can we apply `ST_Contains` to the geography objects? I think it should work on both geometry and geography -- This is an automated message from the Apache Git Service. To re

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #8633: Fix the potential NPE for timestamp index rewrite

2022-05-03 Thread GitBox
xiangfu0 commented on code in PR #8633: URL: https://github.com/apache/pinot/pull/8633#discussion_r864378489 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/TimestampIndexGranularity.java: ## @@ -104,6 +104,6 @@ public static FieldSpec getFieldSpecForTimestampColum

[GitHub] [pinot] Jackie-Jiang closed pull request #7252: support ST_Contains with H3 optimization

2022-05-03 Thread GitBox
Jackie-Jiang closed pull request #7252: support ST_Contains with H3 optimization URL: https://github.com/apache/pinot/pull/7252 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [pinot] deemoliu commented on pull request #8355: add a experiment API for upsert heap memory estimation

2022-05-03 Thread GitBox
deemoliu commented on PR #8355: URL: https://github.com/apache/pinot/pull/8355#issuecomment-1116840737 > Please update the release note of the PR description for the new added rest endpoint Thanks @Jackie-Jiang , done. -- This is an automated message from the Apache Git Service. To

[GitHub] [pinot] codecov-commenter commented on pull request #8633: Fix the potential NPE for timestamp index rewrite

2022-05-03 Thread GitBox
codecov-commenter commented on PR #8633: URL: https://github.com/apache/pinot/pull/8633#issuecomment-1116847601 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8633?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[GitHub] [pinot] Jackie-Jiang merged pull request #8626: [PQL Cleanup] Remove all PQL related code

2022-05-03 Thread GitBox
Jackie-Jiang merged PR #8626: URL: https://github.com/apache/pinot/pull/8626 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

[GitHub] [pinot] Jackie-Jiang closed issue #7430: Remove PQL query endpoint and response format after 0.9.0 release

2022-05-03 Thread GitBox
Jackie-Jiang closed issue #7430: Remove PQL query endpoint and response format after 0.9.0 release URL: https://github.com/apache/pinot/issues/7430 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [pinot] Jackie-Jiang merged pull request #8633: Fix the potential NPE for timestamp index rewrite

2022-05-03 Thread GitBox
Jackie-Jiang merged PR #8633: URL: https://github.com/apache/pinot/pull/8633 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.a

[pinot] branch master updated (1b2c278dae -> 7edad89d61)

2022-05-03 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 1b2c278dae [PQL Cleanup] Remove all PQL related code (#8626) add 7edad89d61 Fix the potential NPE for timestamp in

[GitHub] [pinot] xiangfu0 commented on issue #7430: Remove PQL query endpoint and response format after 0.9.0 release

2022-05-03 Thread GitBox
xiangfu0 commented on issue #7430: URL: https://github.com/apache/pinot/issues/7430#issuecomment-1116905295 This is huge! Big congrats! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [pinot] saurabhd336 commented on a diff in pull request #8582: Support conjugates for scalar functions, add more scalar functions

2022-05-03 Thread GitBox
saurabhd336 commented on code in PR #8582: URL: https://github.com/apache/pinot/pull/8582#discussion_r864452023 ## pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ComparisonFunctions.java: ## @@ -0,0 +1,65 @@ +/** + * Licensed to the Apache Software Foundation

[GitHub] [pinot] saurabhd336 commented on a diff in pull request #8582: Support conjugates for scalar functions, add more scalar functions

2022-05-03 Thread GitBox
saurabhd336 commented on code in PR #8582: URL: https://github.com/apache/pinot/pull/8582#discussion_r864452097 ## pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ObjectFunctions.java: ## @@ -0,0 +1,37 @@ +/** + * Licensed to the Apache Software Foundation (AS

[GitHub] [pinot] saurabhd336 commented on a diff in pull request #8582: Support conjugates for scalar functions, add more scalar functions

2022-05-03 Thread GitBox
saurabhd336 commented on code in PR #8582: URL: https://github.com/apache/pinot/pull/8582#discussion_r864453910 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/function/InbuiltFunctionEvaluator.java: ## @@ -68,17 +69,27 @@ private ExecutableNode planExecution

[GitHub] [pinot] saurabhd336 commented on a diff in pull request #8582: Support conjugates for scalar functions, add more scalar functions

2022-05-03 Thread GitBox
saurabhd336 commented on code in PR #8582: URL: https://github.com/apache/pinot/pull/8582#discussion_r864454190 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/function/InbuiltFunctionEvaluator.java: ## @@ -68,17 +69,27 @@ private ExecutableNode planExecution

[GitHub] [pinot] siddharthteotia commented on pull request #8629: Sampling driven tracing

2022-05-03 Thread GitBox
siddharthteotia commented on PR #8629: URL: https://github.com/apache/pinot/pull/8629#issuecomment-1116942079 LGTM @richardstartin can you please rebase ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [pinot] KKcorps commented on pull request #8602: Change shading prefix so as to minimize conflicts with other libaries

2022-05-03 Thread GitBox
KKcorps commented on PR #8602: URL: https://github.com/apache/pinot/pull/8602#issuecomment-1116958049 @jackjlli @npawar Yep, it will be backward-incompatible which is why I haven't merged it yet. -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [pinot] pjfanning commented on issue #8476: pinot-controller uses multiple NPMs that have security issues

2022-05-03 Thread GitBox
pjfanning commented on issue #8476: URL: https://github.com/apache/pinot/issues/8476#issuecomment-1116959243 @npawar I didn't submit any NPM changes. I am not a UI expert and do not feel I would be a good person to test any changes. -- This is an automated message from the Apache Git Serv

[GitHub] [pinot] KKcorps commented on pull request #8630: Upgrade jackson to 2.13.1

2022-05-03 Thread GitBox
KKcorps commented on PR #8630: URL: https://github.com/apache/pinot/pull/8630#issuecomment-1116962154 Upgrading/Downgrading Jackson breaks a lot of stuff such as Spark ingestion and other plugins. Not sure if we need this. -- This is an automated message from the Apache Git Service. To re

[GitHub] [pinot] KKcorps closed issue #5837: Apache Pulsar Connector integration

2022-05-03 Thread GitBox
KKcorps closed issue #5837: Apache Pulsar Connector integration URL: https://github.com/apache/pinot/issues/5837 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [pinot] KKcorps closed issue #6000: Fixing fetch broker information API

2022-05-03 Thread GitBox
KKcorps closed issue #6000: Fixing fetch broker information API URL: https://github.com/apache/pinot/issues/6000 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [pinot] KKcorps closed issue #6519: Update document for JDBC client

2022-05-03 Thread GitBox
KKcorps closed issue #6519: Update document for JDBC client URL: https://github.com/apache/pinot/issues/6519 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

[GitHub] [pinot] KKcorps commented on issue #6992: End-to-end integration test for Kinesis connector

2022-05-03 Thread GitBox
KKcorps commented on issue #6992: URL: https://github.com/apache/pinot/issues/6992#issuecomment-1116963890 Is this test still flaky? I see it is currently disabled. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [pinot] siddharthteotia commented on pull request #8630: Upgrade jackson to 2.13.1

2022-05-03 Thread GitBox
siddharthteotia commented on PR #8630: URL: https://github.com/apache/pinot/pull/8630#issuecomment-1116970957 > Details I am still working on this. There is a dependency conflict I am trying to address as a short-term fix. Haven't settled on the fix yet. -- This is an automated me

[GitHub] [pinot] KKcorps closed issue #5703: add support for Range Index support for non-dictionary columns

2022-05-03 Thread GitBox
KKcorps closed issue #5703: add support for Range Index support for non-dictionary columns URL: https://github.com/apache/pinot/issues/5703 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [pinot] KKcorps commented on issue #8632: Re-design the date time format

2022-05-03 Thread GitBox
KKcorps commented on issue #8632: URL: https://github.com/apache/pinot/issues/8632#issuecomment-1116978789 I feel both if these formats cause a lot of confusion for new users. What each value represents is not implicit and requires going through the doc to change every time. I propose a

[GitHub] [pinot] codecov-commenter commented on pull request #8629: Sampling driven tracing

2022-05-03 Thread GitBox
codecov-commenter commented on PR #8629: URL: https://github.com/apache/pinot/pull/8629#issuecomment-1116983389 # [Codecov](https://codecov.io/gh/apache/pinot/pull/8629?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

  1   2   >