[GitHub] [incubator-pinot] codecov-io commented on pull request #6246: Adding validation for json_extract_key and json_extract_scalar functions

2020-11-05 Thread GitBox
codecov-io commented on pull request #6246: URL: https://github.com/apache/incubator-pinot/pull/6246#issuecomment-722928327 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6246?src=pr&el=h1) Report > Merging [#6246](https://codecov.io/gh/apache/incubator-pinot/pull/6246?s

[incubator-pinot] branch validate_json_path_function_in_sql_compile_time created (now 1ba42e8)

2020-11-05 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch validate_json_path_function_in_sql_compile_time in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at 1ba42e8 Adding validation for json_extract_key and json_extract

[incubator-pinot] branch validate_json_path_function_in_sql_compile_time created (now 1ba42e8)

2020-11-05 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch validate_json_path_function_in_sql_compile_time in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at 1ba42e8 Adding validation for json_extract_key and json_extract

[GitHub] [incubator-pinot] fx19880617 opened a new pull request #6246: Adding validation for json_extract_key and json_extract_scalar functions

2020-11-05 Thread GitBox
fx19880617 opened a new pull request #6246: URL: https://github.com/apache/incubator-pinot/pull/6246 ## Description Adding validation for json_extract_key and json_extract_scalar functions during sql compilation phase to avoid empty response If user mistakenly put double quote on

[GitHub] [incubator-pinot] zhangloo333 opened a new pull request #6245: [TE](feat): add formatting pipeline for js, hbs code

2020-11-05 Thread GitBox
zhangloo333 opened a new pull request #6245: URL: https://github.com/apache/incubator-pinot/pull/6245 ## Description [TE](feat): add formatting pipeline for js, hbs code - add eslint rules - build pipeline to formatting -

[GitHub] [incubator-pinot] yupeng9 commented on issue #6236: Reset the consumer offsets for a table actively ingesting

2020-11-05 Thread GitBox
yupeng9 commented on issue #6236: URL: https://github.com/apache/incubator-pinot/issues/6236#issuecomment-722771778 Why do this in Pinot? This can be done via kafka tools to update the offsets for both high level and low level consumers Also agree with Subbu that this sounds like a

svn commit: r44368 - /dev/incubator/pinot/apache-pinot-incubating-0.6.0-rc0/

2020-11-05 Thread jlli
Author: jlli Date: Fri Nov 6 00:50:19 2020 New Revision: 44368 Log: Removing release-0.6.0-rc0 Removed: dev/incubator/pinot/apache-pinot-incubating-0.6.0-rc0/ - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org

[GitHub] [incubator-pinot] mcvsubbu commented on issue #6240: Epoch time format does not work with size other than 1

2020-11-05 Thread GitBox
mcvsubbu commented on issue #6240: URL: https://github.com/apache/incubator-pinot/issues/6240#issuecomment-722723112 OK, my bad. We don't have a use case like this. But, for those who do, what is your migration path?

svn commit: r44367 - /dev/incubator/pinot/apache-pinot-incubating-0.6.0-rc1/

2020-11-05 Thread jlli
Author: jlli Date: Thu Nov 5 23:38:53 2020 New Revision: 44367 Log: Update apache-pinot-incubating-0.6.0-rc1 Added: dev/incubator/pinot/apache-pinot-incubating-0.6.0-rc1/ dev/incubator/pinot/apache-pinot-incubating-0.6.0-rc1/apache-pinot-incubating-0.6.0-bin.tar.gz (with props)

[GitHub] [incubator-pinot] jihaozh merged pull request #6238: [TE] Remove maven-shade-plugin and switch from org.reflections to io.classgraph

2020-11-05 Thread GitBox
jihaozh merged pull request #6238: URL: https://github.com/apache/incubator-pinot/pull/6238 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[incubator-pinot] branch master updated (ca8545b -> 2d446d2)

2020-11-05 Thread jihao
This is an automated email from the ASF dual-hosted git repository. jihao pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from ca8545b [TE] frontend - abachuk/alert-details-ui-tweaks - formatting alert header (#6210) add 2d446d2

[incubator-pinot] branch release-0.6.0-rc-1 created (now bb646ba)

2020-11-05 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch release-0.6.0-rc-1 in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at bb646ba [maven-release-plugin] prepare release release-0.6.0-rc1 No new revisions were added by

[incubator-pinot] branch release-0.6.0-rc-1 updated: [maven-release-plugin] prepare for next development iteration

2020-11-05 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch release-0.6.0-rc-1 in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/release-0.6.0-rc-1 by this push: new 9525e93 [maven-

[incubator-pinot] annotated tag release-0.6.0-rc1 updated (bb646ba -> df7d40f)

2020-11-05 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to annotated tag release-0.6.0-rc1 in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. *** WARNING: tag release-0.6.0-rc1 was modified! *** from bb646ba (commit) to df7d40f (t

[incubator-pinot] branch master updated: [TE] frontend - abachuk/alert-details-ui-tweaks - formatting alert header (#6210)

2020-11-05 Thread jihao
This is an automated email from the ASF dual-hosted git repository. jihao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new ca8545b [TE] frontend - abachuk/alert-d

[GitHub] [incubator-pinot] jihaozh merged pull request #6210: [TE] frontend - abachuk/alert-details-ui-tweaks - formatting alert header

2020-11-05 Thread GitBox
jihaozh merged pull request #6210: URL: https://github.com/apache/incubator-pinot/pull/6210 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [incubator-pinot] npawar commented on pull request #6211: add api for cluster manager to get table state

2020-11-05 Thread GitBox
npawar commented on pull request #6211: URL: https://github.com/apache/incubator-pinot/pull/6211#issuecomment-722649835 > > @galalen is there anything pending in this PR? > > @kishoreg, @npawar Yes, I'm working on unit and the integration test. > I want to make sure reading the Ide

[GitHub] [incubator-pinot] vincentchenjl commented on pull request #6238: [TE] Remove maven-shade-plugin and switch from org.reflections to io.classgraph

2020-11-05 Thread GitBox
vincentchenjl commented on pull request #6238: URL: https://github.com/apache/incubator-pinot/pull/6238#issuecomment-722636179 > Quick Question: Does this new library work with higher versions of Java. org.reflections had issues with Java 9 ad above. Yes, the new library works with Java

[GitHub] [incubator-pinot] venkatvghub closed pull request #6244: Fix timezone

2020-11-05 Thread GitBox
venkatvghub closed pull request #6244: URL: https://github.com/apache/incubator-pinot/pull/6244 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [incubator-pinot] venkatvghub opened a new pull request #6244: Fix timezone

2020-11-05 Thread GitBox
venkatvghub opened a new pull request #6244: URL: https://github.com/apache/incubator-pinot/pull/6244 ## Description This PR tries to fix the timezone of the detection from PDT to IST. Additionally also removes the image push to the internal harbor registry A good description should i

[GitHub] [incubator-pinot] fx19880617 commented on issue #6243: Support more than 1 condition in CASE WHEN statement

2020-11-05 Thread GitBox
fx19880617 commented on issue #6243: URL: https://github.com/apache/incubator-pinot/issues/6243#issuecomment-722591145 This query can be written as: ``` SELECT CASE WHEN 'timestamp' >= 1587077057690 THEN 0 WHEN 'timestamp' >= 1587073457690 THEN 1

[GitHub] [incubator-pinot] yupeng9 commented on issue #5919: Add endpoint for external to get information about UDFs

2020-11-05 Thread GitBox
yupeng9 commented on issue #5919: URL: https://github.com/apache/incubator-pinot/issues/5919#issuecomment-722585604 > > How do you plan to maintain this endpoint? > > Not sure if this can help with query push down as the behavior must be identical on presto side and pinot side.

[GitHub] [incubator-pinot] snleee commented on a change in pull request #6094: Implement the segment merge task generator

2020-11-05 Thread GitBox
snleee commented on a change in pull request #6094: URL: https://github.com/apache/incubator-pinot/pull/6094#discussion_r518297224 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/generator/SegmentMergeRollupTaskGenerator.java ## @@ -0

[GitHub] [incubator-pinot] snleee commented on a change in pull request #6094: Implement the segment merge task generator

2020-11-05 Thread GitBox
snleee commented on a change in pull request #6094: URL: https://github.com/apache/incubator-pinot/pull/6094#discussion_r518296337 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/generator/TaskGeneratorUtils.java ## @@ -68,4 +73,30 @@

[GitHub] [incubator-pinot] jackjlli opened a new issue #6243: Support more than 1 condition in CASE WHEN statement

2020-11-05 Thread GitBox
jackjlli opened a new issue #6243: URL: https://github.com/apache/incubator-pinot/issues/6243 With this PR (https://github.com/apache/incubator-pinot/pull/5461), we now can support CASE WHEN statement, while this PR only assumes that there is only 1 condition after CASE WHEN. Queries conta

[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6234: error when recreating a deleted table with the same name

2020-11-05 Thread GitBox
Jackie-Jiang commented on issue #6234: URL: https://github.com/apache/incubator-pinot/issues/6234#issuecomment-722567415 @flykent1990 From the exception log, seems like you don't have the column `timestamp` in your schema but defined it as the time column in the table config. If that is n

[GitHub] [incubator-pinot] codecov-io commented on pull request #6241: Sort columns by dependency before groovy transformation

2020-11-05 Thread GitBox
codecov-io commented on pull request #6241: URL: https://github.com/apache/incubator-pinot/pull/6241#issuecomment-722564814 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6241?src=pr&el=h1) Report > Merging [#6241](https://codecov.io/gh/apache/incubator-pinot/pull/6241?s

[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6240: Epoch time format does not work with size other than 1

2020-11-05 Thread GitBox
Jackie-Jiang commented on issue #6240: URL: https://github.com/apache/incubator-pinot/issues/6240#issuecomment-722563255 @mcvsubbu Then we can use that to verify the behavior. Please check the segment metadata / ZK metadata and see if the start/end time is converted to epoch in minutes in

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6239: Always read start/end time in millis from the segment ZK metadata

2020-11-05 Thread GitBox
Jackie-Jiang commented on a change in pull request #6239: URL: https://github.com/apache/incubator-pinot/pull/6239#discussion_r518275105 ## File path: pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentZKMetadata.java ## @@ -105,54 +97,40 @@ public void

[GitHub] [incubator-pinot] npawar opened a new issue #6242: Add streamConfigs and batchConfigs into the ingestionConfig

2020-11-05 Thread GitBox
npawar opened a new issue #6242: URL: https://github.com/apache/incubator-pinot/issues/6242 Currently when running a batch ingestion job, all the configs are provided via the ingestion-job-spec yaml file to the job. This will not be possible if we want to move to a pull-based ingestion mod

[GitHub] [incubator-pinot] hotienvu opened a new pull request #6241: Sort columns by dependency before groovy transformation

2020-11-05 Thread GitBox
hotienvu opened a new pull request #6241: URL: https://github.com/apache/incubator-pinot/pull/6241 This PR aims to solve the problem with transformation dependencies https://github.com/apache/incubator-pinot/issues/5351 We try to build the dependency graph between columns and sort them

[GitHub] [incubator-pinot] mcvsubbu commented on pull request #6233: Make sure DatetimeFormatter always uses UTC

2020-11-05 Thread GitBox
mcvsubbu commented on pull request #6233: URL: https://github.com/apache/incubator-pinot/pull/6233#issuecomment-722517538 Seems like better to change the test, since there may be backward incompatibility introduced. This is

[GitHub] [incubator-pinot] flykent1990 commented on issue #6234: error when recreating a deleted table with the same name

2020-11-05 Thread GitBox
flykent1990 commented on issue #6234: URL: https://github.com/apache/incubator-pinot/issues/6234#issuecomment-722359281 everything has been deleted ... even in zookeeper PROPERTYSTORE is not available but cannot create realtime table --

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6211: add api for cluster manager to get table state

2020-11-05 Thread GitBox
codecov-io edited a comment on pull request #6211: URL: https://github.com/apache/incubator-pinot/pull/6211#issuecomment-718863847 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6211?src=pr&el=h1) Report > Merging [#6211](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6211: add api for cluster manager to get table state

2020-11-05 Thread GitBox
codecov-io edited a comment on pull request #6211: URL: https://github.com/apache/incubator-pinot/pull/6211#issuecomment-718863847 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6211?src=pr&el=h1) Report > Merging [#6211](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] galalen commented on a change in pull request #6211: add api for cluster manager to get table state

2020-11-05 Thread GitBox
galalen commented on a change in pull request #6211: URL: https://github.com/apache/incubator-pinot/pull/6211#discussion_r517961369 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java ## @@ -558,4 +558,54 @@ pub

[GitHub] [incubator-pinot] hotienvu commented on pull request #6233: Make sure DatetimeFormatter always uses UTC

2020-11-05 Thread GitBox
hotienvu commented on pull request #6233: URL: https://github.com/apache/incubator-pinot/pull/6233#issuecomment-722248030 > Seems this change only impact segment name generation, does it also impact the query results? @fx19880617 Please see example above, the output of `dateTimeFormatt

[GitHub] [incubator-pinot] hotienvu commented on pull request #6233: Make sure DatetimeFormatter always uses UTC

2020-11-05 Thread GitBox
hotienvu commented on pull request #6233: URL: https://github.com/apache/incubator-pinot/pull/6233#issuecomment-722239601 Since the output we want is millis since epoch, this would still work regardless of whether user specify a time zone or not. ``` DateTimeFormat.forPattern("MM

[GitHub] [incubator-pinot] hotienvu commented on pull request #6233: Make sure DatetimeFormatter always uses UTC

2020-11-05 Thread GitBox
hotienvu commented on pull request #6233: URL: https://github.com/apache/incubator-pinot/pull/6233#issuecomment-722234395 @Jackie-Jiang Actually when I tried to run the test in my local machine, this is what it gives: (my local timezone is UTC+8) ``` java.lang.IllegalStateException:

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6094: Implement the segment merge task generator

2020-11-05 Thread GitBox
codecov-io edited a comment on pull request #6094: URL: https://github.com/apache/incubator-pinot/pull/6094#issuecomment-709807920 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6094?src=pr&el=h1) Report > Merging [#6094](https://codecov.io/gh/apache/incubator-pinot/pull

[GitHub] [incubator-pinot] snleee commented on a change in pull request #6094: Implement the segment merge task generator

2020-11-05 Thread GitBox
snleee commented on a change in pull request #6094: URL: https://github.com/apache/incubator-pinot/pull/6094#discussion_r517861506 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/FileUploadDownloadClient.java ## @@ -148,6 +153,23 @@ public static URI ge

[GitHub] [incubator-pinot] snleee commented on a change in pull request #6094: Implement the segment merge task generator

2020-11-05 Thread GitBox
snleee commented on a change in pull request #6094: URL: https://github.com/apache/incubator-pinot/pull/6094#discussion_r517861403 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/FileUploadDownloadClient.java ## @@ -148,6 +153,23 @@ public static URI ge