[GitHub] [incubator-doris] HangyuanLiu opened a new issue #2962: Query fail because "version already has been merged"

2020-02-20 Thread GitBox
HangyuanLiu opened a new issue #2962: Query fail because "version already has been merged" URL: https://github.com/apache/incubator-doris/issues/2962 **Describe the bug** Query fail because "version already has been merged **To Reproduce** sometimes https://user-images.g

[incubator-doris] branch master updated (70d2ccf -> 8291a45)

2020-02-20 Thread lingbin
This is an automated email from the ASF dual-hosted git repository. lingbin pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 70d2ccf Add spark load design (#2856) add 8291a45 Assign each status type an constant explicitly (#2

[GitHub] [incubator-doris] lingbin merged pull request #2960: Assign each status type an constant explicitly

2020-02-20 Thread GitBox
lingbin merged pull request #2960: Assign each status type an constant explicitly URL: https://github.com/apache/incubator-doris/pull/2960 This is an automated message from the Apache Git Service. To respond to the message,

[incubator-doris] branch master updated (35b09ec -> 70d2ccf)

2020-02-20 Thread zhaoc
This is an automated email from the ASF dual-hosted git repository. zhaoc pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 35b09ec [JDK] Support OpenJDK (#2804) add 70d2ccf Add spark load design (#2856) No new revisions were

[GitHub] [incubator-doris] imay merged pull request #2856: add spark load design

2020-02-20 Thread GitBox
imay merged pull request #2856: add spark load design URL: https://github.com/apache/incubator-doris/pull/2856 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [incubator-doris] imay commented on a change in pull request #2961: [ut] fix the test case of SegmentReaderWriterTest::TestBitmapPredicate

2020-02-20 Thread GitBox
imay commented on a change in pull request #2961: [ut] fix the test case of SegmentReaderWriterTest::TestBitmapPredicate URL: https://github.com/apache/incubator-doris/pull/2961#discussion_r382416704 ## File path: be/test/olap/tablet_schema_helper.h ## @@ -31,6 +31,7 @@ T

[GitHub] [incubator-doris] trueeyu opened a new pull request #2961: [ut] fix the test case of SegmentReaderWriterTest::TestBitmapPredicate

2020-02-20 Thread GitBox
trueeyu opened a new pull request #2961: [ut] fix the test case of SegmentReaderWriterTest::TestBitmapPredicate URL: https://github.com/apache/incubator-doris/pull/2961 function create_int_key() will create a TableColumn instance with data memger: _aggregation=(random value) if _ag

[GitHub] [incubator-doris] worker24h commented on a change in pull request #2958: Auto Resume RoutineLoadJob

2020-02-20 Thread GitBox
worker24h commented on a change in pull request #2958: Auto Resume RoutineLoadJob URL: https://github.com/apache/incubator-doris/pull/2958#discussion_r382403361 ## File path: fe/src/main/java/org/apache/doris/common/InternalErrorCode.java ## @@ -0,0 +1,72 @@ +// Licensed t

[GitHub] [incubator-doris] worker24h commented on a change in pull request #2958: Auto Resume RoutineLoadJob

2020-02-20 Thread GitBox
worker24h commented on a change in pull request #2958: Auto Resume RoutineLoadJob URL: https://github.com/apache/incubator-doris/pull/2958#discussion_r382403312 ## File path: fe/src/main/java/org/apache/doris/load/routineload/KafkaRoutineLoadJob.java ## @@ -299,7 +306,11

[GitHub] [incubator-doris] worker24h commented on a change in pull request #2958: Auto Resume RoutineLoadJob

2020-02-20 Thread GitBox
worker24h commented on a change in pull request #2958: Auto Resume RoutineLoadJob URL: https://github.com/apache/incubator-doris/pull/2958#discussion_r382403269 ## File path: fe/src/main/java/org/apache/doris/load/routineload/ScheduleRule.java ## @@ -0,0 +1,78 @@ +// Licen

[GitHub] [incubator-doris] worker24h commented on a change in pull request #2958: Auto Resume RoutineLoadJob

2020-02-20 Thread GitBox
worker24h commented on a change in pull request #2958: Auto Resume RoutineLoadJob URL: https://github.com/apache/incubator-doris/pull/2958#discussion_r382403138 ## File path: fe/src/main/java/org/apache/doris/common/InternalErrorCode.java ## @@ -0,0 +1,72 @@ +// Licensed t

[GitHub] [incubator-doris] worker24h commented on a change in pull request #2958: Auto Resume RoutineLoadJob

2020-02-20 Thread GitBox
worker24h commented on a change in pull request #2958: Auto Resume RoutineLoadJob URL: https://github.com/apache/incubator-doris/pull/2958#discussion_r382403211 ## File path: fe/src/main/java/org/apache/doris/load/routineload/ScheduleRule.java ## @@ -0,0 +1,78 @@ +// Licen

[GitHub] [incubator-doris] worker24h commented on a change in pull request #2958: Auto Resume RoutineLoadJob

2020-02-20 Thread GitBox
worker24h commented on a change in pull request #2958: Auto Resume RoutineLoadJob URL: https://github.com/apache/incubator-doris/pull/2958#discussion_r382403102 ## File path: fe/src/main/java/org/apache/doris/load/routineload/ScheduleRule.java ## @@ -0,0 +1,78 @@ +// Licen

[GitHub] [incubator-doris] lingbin opened a new pull request #2960: Assign each status type an constant explicitly

2020-02-20 Thread GitBox
lingbin opened a new pull request #2960: Assign each status type an constant explicitly URL: https://github.com/apache/incubator-doris/pull/2960 The `TStatusCode` struct is used in all FEs and BEs. In order to be able to avoid errors when identifying status_codes in RPC when upgrading Do

[GitHub] [incubator-doris] imay commented on issue #2916: doris on es report 500 Internal Privoxy Error

2020-02-20 Thread GitBox
imay commented on issue #2916: doris on es report 500 Internal Privoxy Error URL: https://github.com/apache/incubator-doris/issues/2916#issuecomment-589489360 > @imay @morningman Please help me Could we talk with weixin? My id is ApacheZhaoc -

[GitHub] [incubator-doris] hexian55 commented on issue #2916: doris on es report 500 Internal Privoxy Error

2020-02-20 Thread GitBox
hexian55 commented on issue #2916: doris on es report 500 Internal Privoxy Error URL: https://github.com/apache/incubator-doris/issues/2916#issuecomment-589483637 @imay @morningman Please help me This is an automated mess

[GitHub] [incubator-doris] gaodayue commented on issue #2953: [segment_v2] Switch to Unified and Extensible Page Format

2020-02-20 Thread GitBox
gaodayue commented on issue #2953: [segment_v2] Switch to Unified and Extensible Page Format URL: https://github.com/apache/incubator-doris/pull/2953#issuecomment-589467268 The failed UT `GenericPoolTest` is un-related to this PR

[GitHub] [incubator-doris] kangpinghuang commented on a change in pull request #2939: spark etl design

2020-02-20 Thread GitBox
kangpinghuang commented on a change in pull request #2939: spark etl design URL: https://github.com/apache/incubator-doris/pull/2939#discussion_r382360310 ## File path: docs/documentation/cn/internal/spark_etl.md ## @@ -0,0 +1,342 @@ + + +# Doris spark导入ETL逻辑设计 + +## 背景 + +

[GitHub] [incubator-doris] kangpinghuang commented on a change in pull request #2939: spark etl design

2020-02-20 Thread GitBox
kangpinghuang commented on a change in pull request #2939: spark etl design URL: https://github.com/apache/incubator-doris/pull/2939#discussion_r382360503 ## File path: docs/documentation/cn/internal/spark_etl.md ## @@ -0,0 +1,342 @@ + + +# Doris spark导入ETL逻辑设计 + +## 背景 + +

[GitHub] [incubator-doris] kangpinghuang commented on a change in pull request #2939: spark etl design

2020-02-20 Thread GitBox
kangpinghuang commented on a change in pull request #2939: spark etl design URL: https://github.com/apache/incubator-doris/pull/2939#discussion_r382359959 ## File path: docs/documentation/cn/internal/spark_etl.md ## @@ -0,0 +1,323 @@ +# Doris spark导入ETL逻辑设计 + +## 背景 + +Dori

[GitHub] [incubator-doris] kangpinghuang commented on a change in pull request #2939: spark etl design

2020-02-20 Thread GitBox
kangpinghuang commented on a change in pull request #2939: spark etl design URL: https://github.com/apache/incubator-doris/pull/2939#discussion_r382359984 ## File path: docs/documentation/cn/internal/spark_etl.md ## @@ -0,0 +1,323 @@ +# Doris spark导入ETL逻辑设计 + +## 背景 + +Dori

[GitHub] [incubator-doris] morningman opened a new pull request #2959: Routine load submit

2020-02-20 Thread GitBox
morningman opened a new pull request #2959: Routine load submit URL: https://github.com/apache/incubator-doris/pull/2959 NOT READY! This is an automated message from the Apache Git Service. To respond to the message, please lo

[incubator-doris] branch master updated (ccc3412 -> 35b09ec)

2020-02-20 Thread morningman
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from ccc3412 Fix bug: Error of exporting double type data to hdfs (#2924) (#2925) add 35b09ec [JDK] S

[GitHub] [incubator-doris] morningman merged pull request #2804: [JDK] Support OpenJDK

2020-02-20 Thread GitBox
morningman merged pull request #2804: [JDK] Support OpenJDK URL: https://github.com/apache/incubator-doris/pull/2804 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [incubator-doris] morningman commented on a change in pull request #2958: Auto Resume RoutineLoadJob

2020-02-20 Thread GitBox
morningman commented on a change in pull request #2958: Auto Resume RoutineLoadJob URL: https://github.com/apache/incubator-doris/pull/2958#discussion_r382068262 ## File path: fe/src/main/java/org/apache/doris/common/InternalErrorCode.java ## @@ -0,0 +1,72 @@ +// Licensed

[GitHub] [incubator-doris] morningman commented on a change in pull request #2958: Auto Resume RoutineLoadJob

2020-02-20 Thread GitBox
morningman commented on a change in pull request #2958: Auto Resume RoutineLoadJob URL: https://github.com/apache/incubator-doris/pull/2958#discussion_r382060571 ## File path: fe/src/main/java/org/apache/doris/load/routineload/ScheduleRule.java ## @@ -0,0 +1,78 @@ +// Lice

[GitHub] [incubator-doris] morningman commented on a change in pull request #2958: Auto Resume RoutineLoadJob

2020-02-20 Thread GitBox
morningman commented on a change in pull request #2958: Auto Resume RoutineLoadJob URL: https://github.com/apache/incubator-doris/pull/2958#discussion_r382045606 ## File path: fe/src/main/java/org/apache/doris/load/routineload/ScheduleRule.java ## @@ -0,0 +1,78 @@ +// Lice

[GitHub] [incubator-doris] morningman commented on a change in pull request #2958: Auto Resume RoutineLoadJob

2020-02-20 Thread GitBox
morningman commented on a change in pull request #2958: Auto Resume RoutineLoadJob URL: https://github.com/apache/incubator-doris/pull/2958#discussion_r382063037 ## File path: fe/src/main/java/org/apache/doris/load/routineload/KafkaRoutineLoadJob.java ## @@ -299,7 +306,11

[GitHub] [incubator-doris] morningman commented on a change in pull request #2958: Auto Resume RoutineLoadJob

2020-02-20 Thread GitBox
morningman commented on a change in pull request #2958: Auto Resume RoutineLoadJob URL: https://github.com/apache/incubator-doris/pull/2958#discussion_r382059620 ## File path: fe/src/main/java/org/apache/doris/load/routineload/ScheduleRule.java ## @@ -0,0 +1,78 @@ +// Lice

[GitHub] [incubator-doris] morningman commented on a change in pull request #2958: Auto Resume RoutineLoadJob

2020-02-20 Thread GitBox
morningman commented on a change in pull request #2958: Auto Resume RoutineLoadJob URL: https://github.com/apache/incubator-doris/pull/2958#discussion_r382052529 ## File path: fe/src/main/java/org/apache/doris/common/InternalErrorCode.java ## @@ -0,0 +1,72 @@ +// Licensed

[GitHub] [incubator-doris] morningman commented on a change in pull request #2958: Auto Resume RoutineLoadJob

2020-02-20 Thread GitBox
morningman commented on a change in pull request #2958: Auto Resume RoutineLoadJob URL: https://github.com/apache/incubator-doris/pull/2958#discussion_r382067831 ## File path: fe/src/main/java/org/apache/doris/common/InternalErrorCode.java ## @@ -0,0 +1,72 @@ +// Licensed

[GitHub] [incubator-doris] morningman commented on a change in pull request #2956: Add multi-thread ver olaptablesink for LOAD

2020-02-20 Thread GitBox
morningman commented on a change in pull request #2956: Add multi-thread ver olaptablesink for LOAD URL: https://github.com/apache/incubator-doris/pull/2956#discussion_r382032981 ## File path: fe/src/main/java/org/apache/doris/load/loadv2/LoadJob.java ## @@ -17,6 +17,20 @@

[GitHub] [incubator-doris] morningman commented on a change in pull request #2956: Add multi-thread ver olaptablesink for LOAD

2020-02-20 Thread GitBox
morningman commented on a change in pull request #2956: Add multi-thread ver olaptablesink for LOAD URL: https://github.com/apache/incubator-doris/pull/2956#discussion_r382035039 ## File path: gensrc/thrift/DataSinks.thrift ## @@ -105,6 +105,9 @@ struct TOlapTableSink {

[GitHub] [incubator-doris] morningman commented on a change in pull request #2956: Add multi-thread ver olaptablesink for LOAD

2020-02-20 Thread GitBox
morningman commented on a change in pull request #2956: Add multi-thread ver olaptablesink for LOAD URL: https://github.com/apache/incubator-doris/pull/2956#discussion_r382034126 ## File path: fe/src/main/java/org/apache/doris/planner/OlapTableSink.java ## @@ -88,12 +92,29

[GitHub] [incubator-doris] morningman commented on a change in pull request #2956: Add multi-thread ver olaptablesink for LOAD

2020-02-20 Thread GitBox
morningman commented on a change in pull request #2956: Add multi-thread ver olaptablesink for LOAD URL: https://github.com/apache/incubator-doris/pull/2956#discussion_r382036639 ## File path: be/src/exec/tablet_sink.h ## @@ -141,26 +191,75 @@ class IndexChannel { std

[incubator-doris] branch master updated: Fix bug: Error of exporting double type data to hdfs (#2924) (#2925)

2020-02-20 Thread zhaoc
This is an automated email from the ASF dual-hosted git repository. zhaoc pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new ccc3412 Fix bug: Error of exporting dou

[GitHub] [incubator-doris] imay merged pull request #2925: fix bug: Error of exporting double type data to hdfs (#2924)

2020-02-20 Thread GitBox
imay merged pull request #2925: fix bug: Error of exporting double type data to hdfs (#2924) URL: https://github.com/apache/incubator-doris/pull/2925 This is an automated message from the Apache Git Service. To respond to th

[GitHub] [incubator-doris] worker24h opened a new pull request #2958: Auto Resume RoutineLoadJob

2020-02-20 Thread GitBox
worker24h opened a new pull request #2958: Auto Resume RoutineLoadJob URL: https://github.com/apache/incubator-doris/pull/2958 When all backends restart, the routineload job can't RUNNING. This is an automated message from the

[GitHub] [incubator-doris] imay merged pull request #2952: Fix some function DATE type priority

2020-02-20 Thread GitBox
imay merged pull request #2952: Fix some function DATE type priority URL: https://github.com/apache/incubator-doris/pull/2952 This is an automated message from the Apache Git Service. To respond to the message, please log on

[incubator-doris] branch master updated (839ec45 -> ece8740)

2020-02-20 Thread zhaoc
This is an automated email from the ASF dual-hosted git repository. zhaoc pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 839ec45 Remove llvm relative code from be/src/exec (#2955) add ece8740 Fix some function DATE type pri

[incubator-doris] branch master updated (da945c8 -> 839ec45)

2020-02-20 Thread zhaoc
This is an automated email from the ASF dual-hosted git repository. zhaoc pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from da945c8 Add log to track problem in small_file_mgr_test (#2951) add 839ec45 Remove llvm relative code

[GitHub] [incubator-doris] imay merged pull request #2955: Remove unused LLVM related codes of directory:be/src/exec (#2910)

2020-02-20 Thread GitBox
imay merged pull request #2955: Remove unused LLVM related codes of directory:be/src/exec (#2910) URL: https://github.com/apache/incubator-doris/pull/2955 This is an automated message from the Apache Git Service. To respond

[GitHub] [incubator-doris] kangkaisen commented on a change in pull request #2952: Fix some function DATE type priority

2020-02-20 Thread GitBox
kangkaisen commented on a change in pull request #2952: Fix some function DATE type priority URL: https://github.com/apache/incubator-doris/pull/2952#discussion_r381953273 ## File path: fe/src/main/java/org/apache/doris/planner/SingleNodePlanner.java ## @@ -802,7 +802,6 @@

[GitHub] [incubator-doris] vagetablechicken edited a comment on issue #2780: OlapTableSink::send is low efficient?

2020-02-20 Thread GitBox
vagetablechicken edited a comment on issue #2780: OlapTableSink::send is low efficient? URL: https://github.com/apache/incubator-doris/issues/2780#issuecomment-588156273 # OlapTableSink Multithreading Solution Design ## single-thread model (original model) ![image](https://user-

[GitHub] [incubator-doris] vagetablechicken edited a comment on issue #2780: OlapTableSink::send is low efficient?

2020-02-20 Thread GitBox
vagetablechicken edited a comment on issue #2780: OlapTableSink::send is low efficient? URL: https://github.com/apache/incubator-doris/issues/2780#issuecomment-588156273 # OlapTableSink Multithreading Solution Design ## single-thread model (original model) ![image](https://user-

[GitHub] [incubator-doris] yangzhg opened a new pull request #2957: implement the planner for set operation

2020-02-20 Thread GitBox
yangzhg opened a new pull request #2957: implement the planner for set operation URL: https://github.com/apache/incubator-doris/pull/2957 implemnet intersect and except planner, examples: ``` mysql> explain select k1,k2 from t where k1='a' intersect select k1,k2 from t where k1='b'

[GitHub] [incubator-doris] vagetablechicken opened a new pull request #2956: Add multi-ver olaptablesink for LOAD

2020-02-20 Thread GitBox
vagetablechicken opened a new pull request #2956: Add multi-ver olaptablesink for LOAD URL: https://github.com/apache/incubator-doris/pull/2956 Ref #2780 So we can use ``` LOAD LABEL ... PROPERTIES ( "buffer_num"="5", "mem_limit_

[GitHub] [incubator-doris] EmmyMiao87 opened a new pull request #2821: The new materialized view selector

2020-02-20 Thread GitBox
EmmyMiao87 opened a new pull request #2821: The new materialized view selector URL: https://github.com/apache/incubator-doris/pull/2821 This commit mainly implements the new materialized view selector which supports SPJ<->SPJG. Two parameters are currently used to regulate this function.

[GitHub] [incubator-doris] trueeyu opened a new pull request #2955: Remove unused LLVM related codes of directory:be/src/exec (#2910)

2020-02-20 Thread GitBox
trueeyu opened a new pull request #2955: Remove unused LLVM related codes of directory:be/src/exec (#2910) URL: https://github.com/apache/incubator-doris/pull/2955 Remove unused LLVM related codes of directory:be/src/exec (#2910) there are many LLVM related codes in code base, but th

[GitHub] [incubator-doris] wangbo commented on issue #2940: spark etl build global dict doc

2020-02-20 Thread GitBox
wangbo commented on issue #2940: spark etl build global dict doc URL: https://github.com/apache/incubator-doris/pull/2940#issuecomment-588817101 @morningman 懂你的意思了。 有两个问题需要解决 第一个是输入信息的方式,这个具体可以等元信息详细接口出来之后再定,如果通过kv的方式描述,放在dml确实很复杂的话,也可以通过http提交。建议是dml和http上传同时支持,原因是根据我经历的用户使用场景,其实连接信息

[GitHub] [incubator-doris] WingsGo opened a new issue #2954: [Proposal]Support config default storage_medium and default storage_cooldown_time

2020-02-20 Thread GitBox
WingsGo opened a new issue #2954: [Proposal]Support config default storage_medium and default storage_cooldown_time URL: https://github.com/apache/incubator-doris/issues/2954 **Is your feature request related to a problem? Please describe.** Doris support choose medium when create table,

[GitHub] [incubator-doris] gaodayue opened a new pull request #2953: [segment_v2] Switch to Unified and Extensible Page Format

2020-02-20 Thread GitBox
gaodayue opened a new pull request #2953: [segment_v2] Switch to Unified and Extensible Page Format URL: https://github.com/apache/incubator-doris/pull/2953 Fixes #2892 IMPORTANT NOTICE: this CL makes incompatible changes to V2 storage format, developers need to create new tables f

[GitHub] [incubator-doris] morningman edited a comment on issue #2940: spark etl build global dict doc

2020-02-20 Thread GitBox
morningman edited a comment on issue #2940: spark etl build global dict doc URL: https://github.com/apache/incubator-doris/pull/2940#issuecomment-588769716 > 目前这个job的sql版我已经写完了,并跑通基本测试了。接下来主要是测试,这部分花时间应该会比较长,也是最重要的一部分,测试内容包括资源用量以及性能,至少要和kylin现有的能力持平。所以sql表达的问题不是重点。 不是,你理解错了,我说的不是构建字典

[GitHub] [incubator-doris] morningman commented on issue #2940: spark etl build global dict doc

2020-02-20 Thread GitBox
morningman commented on issue #2940: spark etl build global dict doc URL: https://github.com/apache/incubator-doris/pull/2940#issuecomment-588769716 > 目前这个job的sql版我已经写完了,并跑通基本测试了。接下来主要是测试,这部分花时间应该会比较长,也是最重要的一部分,测试内容包括资源用量以及性能,至少要和kylin现有的能力持平。所以sql表达的问题不是重点。 不是,你理解错误,我说的不是构建字典使用的spar

[GitHub] [incubator-doris] lingbin merged pull request #2951: [ut] Add log to track problem in small_file_mgr_test

2020-02-20 Thread GitBox
lingbin merged pull request #2951: [ut] Add log to track problem in small_file_mgr_test URL: https://github.com/apache/incubator-doris/pull/2951 This is an automated message from the Apache Git Service. To respond to the mes

[incubator-doris] branch master updated (180bf02 -> da945c8)

2020-02-20 Thread lingbin
This is an automated email from the ASF dual-hosted git repository. lingbin pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 180bf02 [Bug] Missing `in memory` property for restore meta info (#2950) add da945c8 Add log to tra