[GitHub] [incubator-doris] HangyuanLiu opened a new issue #3779: [BUG] Fix _hdfs_location misuse in AggFnEvaluator

2020-06-04 Thread GitBox
HangyuanLiu opened a new issue #3779: URL: https://github.com/apache/incubator-doris/issues/3779 The current location uses the wrong variable in AggFnEvaluator, the variable _hdfs_location in _fn should be used This is an au

[GitHub] [incubator-doris] kangkaisen commented on issue #3778: [Catalog] Should not use `getInstance()` of Catalog anymore

2020-06-04 Thread GitBox
kangkaisen commented on issue #3778: URL: https://github.com/apache/incubator-doris/issues/3778#issuecomment-639276716 +1. This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-doris] vagetablechicken edited a comment on issue #3469: [Memory Engine] Write Pipeline: internal data processing and storage

2020-06-04 Thread GitBox
vagetablechicken edited a comment on issue #3469: URL: https://github.com/apache/incubator-doris/issues/3469#issuecomment-631289944 ### WAL file format - need to rename WAL file will save multiple partial rows. One write TX's WAL may be split by size => generates multiple {txn_id}_{segm

[GitHub] [incubator-doris] morningman opened a new issue #3778: [Catalog] Should not use `getInstance()` of Catalog anymore

2020-06-04 Thread GitBox
morningman opened a new issue #3778: URL: https://github.com/apache/incubator-doris/issues/3778 Catalog is currently a singleton class. In the Master FE, this singleton class actually has two instances: One is the Catalog used for formal services(Called `Formal Catalog`), and the ot

[GitHub] [incubator-doris] imay commented on a change in pull request #3768: Disable Bitmap or Hll type in keys or in values with incorrect agg-type

2020-06-04 Thread GitBox
imay commented on a change in pull request #3768: URL: https://github.com/apache/incubator-doris/pull/3768#discussion_r435656472 ## File path: fe/src/main/java/org/apache/doris/analysis/ColumnDef.java ## @@ -131,6 +131,16 @@ public void analyze(boolean isOlap) throws AnalysisE

[GitHub] [incubator-doris] chaoyli closed issue #3746: Ignore broken disks in the storage_data_path

2020-06-04 Thread GitBox
chaoyli closed issue #3746: URL: https://github.com/apache/incubator-doris/issues/3746 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [incubator-doris] chaoyli merged pull request #3741: Ignore broken disks when BE starts up

2020-06-04 Thread GitBox
chaoyli merged pull request #3741: URL: https://github.com/apache/incubator-doris/pull/3741 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[incubator-doris] branch master updated (73719f2 -> ed9022a)

2020-06-04 Thread lichaoyong
This is an automated email from the ASF dual-hosted git repository. lichaoyong pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 73719f2 Fix document (#3773) add ed9022a Ignore broken disk when BE starts up (#3741) No new rev

[incubator-doris] branch master updated: Fix document (#3773)

2020-06-04 Thread zhaoc
This is an automated email from the ASF dual-hosted git repository. zhaoc pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 73719f2 Fix document (#3773) 73719f2 is

[GitHub] [incubator-doris] imay merged pull request #3773: fix some flaw in format-code tutorial

2020-06-04 Thread GitBox
imay merged pull request #3773: URL: https://github.com/apache/incubator-doris/pull/3773 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[incubator-doris] branch master updated: Add some log to make it easier to find out bug (#3770)

2020-06-04 Thread zhaoc
This is an automated email from the ASF dual-hosted git repository. zhaoc pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new cdd1733 Add some log to make it easier

[GitHub] [incubator-doris] imay merged pull request #3770: Add some log to make it easier to find out bug

2020-06-04 Thread GitBox
imay merged pull request #3770: URL: https://github.com/apache/incubator-doris/pull/3770 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [incubator-doris] imay merged pull request #3777: [Website] Fix CREATE RESOURCE sidebar text and link not right bug

2020-06-04 Thread GitBox
imay merged pull request #3777: URL: https://github.com/apache/incubator-doris/pull/3777 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[incubator-doris] branch master updated: [Website] Fix CREATE RESOURCE sidebar text and link not right bug (#3777)

2020-06-04 Thread zhaoc
This is an automated email from the ASF dual-hosted git repository. zhaoc pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new fdf3415 [Website] Fix CREATE RESOURCE s

[GitHub] [incubator-doris] wyb edited a comment on issue #3433: [Spark load] Doris support Spark load

2020-06-04 Thread GitBox
wyb edited a comment on issue #3433: URL: https://github.com/apache/incubator-doris/issues/3433#issuecomment-635180109 **Resource manager** #3418 [Spark load] Add resource manager (Merged) **Fe schedule job execution** #3712 [Spark load][Fe 1/5] Add spark etl job config (Merged

[GitHub] [incubator-doris] wuyunfeng closed issue #3752: [Doris On ES] Incorrect result when `enable_docvalue_scan`

2020-06-04 Thread GitBox
wuyunfeng closed issue #3752: URL: https://github.com/apache/incubator-doris/issues/3752 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [incubator-doris] acelyc111 commented on a change in pull request #3750: [Bug] Fix a bug that tablet may be modified to BETA_ROWSET after cloned

2020-06-04 Thread GitBox
acelyc111 commented on a change in pull request #3750: URL: https://github.com/apache/incubator-doris/pull/3750#discussion_r435397028 ## File path: be/src/olap/tablet_schema.h ## @@ -107,22 +113,30 @@ class TabletSchema { inline void set_is_in_memory (bool is_in_memory) {

[GitHub] [incubator-doris] sduzh commented on issue #3688: Query with a cast in where clause returned wrong result

2020-06-04 Thread GitBox
sduzh commented on issue #3688: URL: https://github.com/apache/incubator-doris/issues/3688#issuecomment-638926751 > why Expected behavior has value 0 It's a mistake, should not including 0. This is an automated messag

[GitHub] [incubator-doris] imay commented on a change in pull request #3750: [Bug] Fix a bug that tablet may be modified to BETA_ROWSET after cloned

2020-06-04 Thread GitBox
imay commented on a change in pull request #3750: URL: https://github.com/apache/incubator-doris/pull/3750#discussion_r435340647 ## File path: be/src/olap/tablet_schema.h ## @@ -107,22 +113,30 @@ class TabletSchema { inline void set_is_in_memory (bool is_in_memory) {

[GitHub] [incubator-doris] imay commented on a change in pull request #3761: [StorageEngine] Make StorageEngine::open return more detailed info

2020-06-04 Thread GitBox
imay commented on a change in pull request #3761: URL: https://github.com/apache/incubator-doris/pull/3761#discussion_r43570 ## File path: be/src/olap/storage_engine.cpp ## @@ -170,22 +167,28 @@ OLAPStatus StorageEngine::_open() { _parse_default_rowset_type(); -

[GitHub] [incubator-doris] acelyc111 commented on a change in pull request #3761: [StorageEngine] Make StorageEngine::open return more detailed info

2020-06-04 Thread GitBox
acelyc111 commented on a change in pull request #3761: URL: https://github.com/apache/incubator-doris/pull/3761#discussion_r435302914 ## File path: be/src/olap/data_dir.cpp ## @@ -267,12 +281,13 @@ Status DataDir::_init_meta() { _meta = new (std::nothrow) OlapMeta(_path);

[GitHub] [incubator-doris] wyb commented on a change in pull request #3729: (#3728) Using SparkDpp to complete some calculation in Spark Load

2020-06-04 Thread GitBox
wyb commented on a change in pull request #3729: URL: https://github.com/apache/incubator-doris/pull/3729#discussion_r435246339 ## File path: fe/src/main/java/org/apache/doris/load/loadv2/dpp/ColumnParser.java ## @@ -0,0 +1,175 @@ +// Licensed to the Apache Software Foundation

[GitHub] [incubator-doris] wyb commented on a change in pull request #3742: [Spark load][Be 1/1] Be handle push task

2020-06-04 Thread GitBox
wyb commented on a change in pull request #3742: URL: https://github.com/apache/incubator-doris/pull/3742#discussion_r435205288 ## File path: be/src/olap/push_handler.cpp ## @@ -248,6 +259,149 @@ void PushHandler::_get_tablet_infos(const vector& tablet_vars, } } +OLAPSta

[GitHub] [incubator-doris] wyb opened a new pull request #3777: [Website] Fix CREATE RESOURCE sidebar text and link not right bug

2020-06-04 Thread GitBox
wyb opened a new pull request #3777: URL: https://github.com/apache/incubator-doris/pull/3777 Add title and language header This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-doris] wyb opened a new issue #3776: [Website] CREATE RESOURCE sidebar text and link not right

2020-06-04 Thread GitBox
wyb opened a new issue #3776: URL: https://github.com/apache/incubator-doris/issues/3776 **Describe the bug** CREATE RESOURCE sidebar text and link not right ![image](https://user-images.githubusercontent.com/972887/83757918-cbf60700-a6a3-11ea-9bd5-fd483caf3d4e.png)

[GitHub] [incubator-doris] caiconghui edited a comment on pull request #3775: Support read and write lock in table level to reduce lock competition

2020-06-04 Thread GitBox
caiconghui edited a comment on pull request #3775: URL: https://github.com/apache/incubator-doris/pull/3775#issuecomment-638814025 ref #3743 This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [incubator-doris] caiconghui commented on pull request #3775: Support read and write lock in table level to reduce lock competition

2020-06-04 Thread GitBox
caiconghui commented on pull request #3775: URL: https://github.com/apache/incubator-doris/pull/3775#issuecomment-638814025 ref #3775 This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [incubator-doris] caiconghui opened a new pull request #3775: Support read and write lock in table level to reduce lock competition

2020-06-04 Thread GitBox
caiconghui opened a new pull request #3775: URL: https://github.com/apache/incubator-doris/pull/3775 This PR is to reduce lock competition by supporting read and write lock in table level. When we modify or read table's meta, we don't need to get db lock, just get table write or read lock.

[GitHub] [incubator-doris] wuyunfeng edited a comment on issue #3759: [Doris On ES] [WIP] Add Debug mode for ES table

2020-06-04 Thread GitBox
wuyunfeng edited a comment on issue #3759: URL: https://github.com/apache/incubator-doris/issues/3759#issuecomment-638799164 https://github.com/apache/incubator-doris/pull/3774 add basic ability class BoolQueryBuilder to generate query DSL and full UT test

[GitHub] [incubator-doris] wuyunfeng commented on issue #3759: [Doris On ES] [WIP] Add Debug mode for ES table

2020-06-04 Thread GitBox
wuyunfeng commented on issue #3759: URL: https://github.com/apache/incubator-doris/issues/3759#issuecomment-638799164 https://github.com/apache/incubator-doris/pull/3774 This is an automated message from the Apache Git Servi

[GitHub] [incubator-doris] wuyunfeng opened a new pull request #3774: [Doris On ES] Add ES QueryBuilders for debug mode

2020-06-04 Thread GitBox
wuyunfeng opened a new pull request #3774: URL: https://github.com/apache/incubator-doris/pull/3774 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [incubator-doris] morningman commented on a change in pull request #3738: [Bug]fix the crash of checksum task #3735

2020-06-04 Thread GitBox
morningman commented on a change in pull request #3738: URL: https://github.com/apache/incubator-doris/pull/3738#discussion_r435191500 ## File path: be/src/olap/task/engine_checksum_task.cpp ## @@ -118,8 +113,10 @@ OLAPStatus EngineChecksumTask::_compute_checksum() {

[GitHub] [incubator-doris] morningman commented on pull request #3741: Ignore broken disks when BE starts up

2020-06-04 Thread GitBox
morningman commented on pull request #3741: URL: https://github.com/apache/incubator-doris/pull/3741#issuecomment-638796007 Hi @chaoyli, Any more comment? This is an automated message from the Apache Git Service. To respond

[incubator-doris] branch master updated: Fix the error selectedIndexId when keysType of table is UNIQUE (#3772)

2020-06-04 Thread lingmiao
This is an automated email from the ASF dual-hosted git repository. lingmiao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 0a74866 Fix the error selectedIndexI

[GitHub] [incubator-doris] EmmyMiao87 closed issue #3771: Error selectedIndexId when KeysType of table is UNIQUE

2020-06-04 Thread GitBox
EmmyMiao87 closed issue #3771: URL: https://github.com/apache/incubator-doris/issues/3771 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [incubator-doris] EmmyMiao87 merged pull request #3772: Fix the error selectedIndexId when keysType of table is UNIQUE

2020-06-04 Thread GitBox
EmmyMiao87 merged pull request #3772: URL: https://github.com/apache/incubator-doris/pull/3772 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-doris] yangzhg commented on issue #3688: Query with a cast in where clause returned wrong result

2020-06-04 Thread GitBox
yangzhg commented on issue #3688: URL: https://github.com/apache/incubator-doris/issues/3688#issuecomment-638786240 why Expected behavior has value 0 This is an automated message from the Apache Git Service. To respond to th

[GitHub] [incubator-doris] morningman commented on a change in pull request #3750: [Bug] Fix a bug that tablet may be modified to BETA_ROWSET after cloned

2020-06-04 Thread GitBox
morningman commented on a change in pull request #3750: URL: https://github.com/apache/incubator-doris/pull/3750#discussion_r434942618 ## File path: be/src/olap/tablet_schema.cpp ## @@ -337,14 +337,12 @@ void TabletColumn::to_schema_pb(ColumnPB* column) { } } -TabletSch

[GitHub] [incubator-doris] morningman commented on a change in pull request #3760: Support external users to contribute udf

2020-06-04 Thread GitBox
morningman commented on a change in pull request #3760: URL: https://github.com/apache/incubator-doris/pull/3760#discussion_r434952908 ## File path: custom_udf/CMakeLists.txt ## @@ -0,0 +1,58 @@ +cmake_minimum_required(VERSION 2.8.10) + +# set CMAKE_C_COMPILER, this must set b

[GitHub] [incubator-doris] xy720 opened a new pull request #3773: fix some flaw in formate-code tutorial

2020-06-04 Thread GitBox
xy720 opened a new pull request #3773: URL: https://github.com/apache/incubator-doris/pull/3773 Fix some problems found when using MAC to refer to formate-code tutorial. This commit adds some details to the document in formate-code tutorial. -

[GitHub] [incubator-doris] chaoyli commented on issue #3767: Txn is not cleared when load been cancelled.

2020-06-04 Thread GitBox
chaoyli commented on issue #3767: URL: https://github.com/apache/incubator-doris/issues/3767#issuecomment-638761800 ![image](https://user-images.githubusercontent.com/44767902/83745693-824ff100-a690-11ea-8c8a-b37d82531817.png) -

[GitHub] [incubator-doris] chaoyli closed issue #3767: Txn is not cleared when load been cancelled.

2020-06-04 Thread GitBox
chaoyli closed issue #3767: URL: https://github.com/apache/incubator-doris/issues/3767 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[incubator-doris] branch master updated: [Bug] Clear Txn when load been cancelled (#3766)

2020-06-04 Thread lichaoyong
This is an automated email from the ASF dual-hosted git repository. lichaoyong pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 9b2cf1c [Bug] Clear Txn when load

[GitHub] [incubator-doris] chaoyli merged pull request #3766: [Bug] Clear Txn when load been cancelled

2020-06-04 Thread GitBox
chaoyli merged pull request #3766: URL: https://github.com/apache/incubator-doris/pull/3766 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [incubator-doris] spaces-X commented on a change in pull request #3768: Disable Bitmap or Hll type in keys or in values with incorrect agg-type

2020-06-04 Thread GitBox
spaces-X commented on a change in pull request #3768: URL: https://github.com/apache/incubator-doris/pull/3768#discussion_r435135824 ## File path: fe/src/main/java/org/apache/doris/catalog/AggregateType.java ## @@ -85,20 +85,22 @@ compatibilityMap.put(MAX, EnumSet.copy

[GitHub] [incubator-doris] morningman commented on a change in pull request #3761: [StorageEngine] Make StorageEngine::open return more detailed info

2020-06-04 Thread GitBox
morningman commented on a change in pull request #3761: URL: https://github.com/apache/incubator-doris/pull/3761#discussion_r435128372 ## File path: be/src/olap/data_dir.cpp ## @@ -267,12 +281,13 @@ Status DataDir::_init_meta() { _meta = new (std::nothrow) OlapMeta(_path);

[GitHub] [incubator-doris] morningman commented on a change in pull request #3768: Disable Bitmap or Hll type in keys or in values with incorrect agg-type

2020-06-04 Thread GitBox
morningman commented on a change in pull request #3768: URL: https://github.com/apache/incubator-doris/pull/3768#discussion_r435125457 ## File path: fe/src/main/java/org/apache/doris/catalog/AggregateType.java ## @@ -85,20 +85,22 @@ compatibilityMap.put(MAX, EnumSet.co

[GitHub] [incubator-doris] morningman commented on a change in pull request #3770: Add some log to make it easier to find out bug

2020-06-04 Thread GitBox
morningman commented on a change in pull request #3770: URL: https://github.com/apache/incubator-doris/pull/3770#discussion_r435121454 ## File path: fe/src/main/java/org/apache/doris/qe/Coordinator.java ## @@ -425,6 +428,7 @@ public void exec() throws Exception { L

[GitHub] [incubator-doris] imay merged pull request #3751: [Doris On ES] Bug-Fix for incorrect result for docvalue scan mode

2020-06-04 Thread GitBox
imay merged pull request #3751: URL: https://github.com/apache/incubator-doris/pull/3751 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[incubator-doris] branch master updated: [Doirs On ES] fix bug for sparse docvalue context and remove the mistake usage of total (#3751)

2020-06-04 Thread zhaoc
This is an automated email from the ASF dual-hosted git repository. zhaoc pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 484e7de [Doirs On ES] fix bug for spars

[GitHub] [incubator-doris] xy720 commented on a change in pull request #3742: [Spark load][Be 1/1] Be handle push task

2020-06-04 Thread GitBox
xy720 commented on a change in pull request #3742: URL: https://github.com/apache/incubator-doris/pull/3742#discussion_r435064627 ## File path: be/src/olap/push_handler.cpp ## @@ -761,6 +915,126 @@ OLAPStatus LzoBinaryReader::_next_block() { return res; } +OLAPStatus Push

[GitHub] [incubator-doris] EmmyMiao87 opened a new pull request #3772: Fix the error selectedIndexId when keysType of table is UNIQUE

2020-06-04 Thread GitBox
EmmyMiao87 opened a new pull request #3772: URL: https://github.com/apache/incubator-doris/pull/3772 The unique table also should be compensated candidate index. The reason is the same as the agg table type. Fixed #3771. Change-Id: Ic04b0360a0b178cb0b6ee635e56f48852092ec09 ---

[GitHub] [incubator-doris] EmmyMiao87 opened a new issue #3771: Error selectedIndexId when KeysType of table is UNIQUE

2020-06-04 Thread GitBox
EmmyMiao87 opened a new issue #3771: URL: https://github.com/apache/incubator-doris/issues/3771 **Describe the bug** When user query the UNIQUE table, the new mv selector could not select correct indexId. The warning log like ``` 2020-06-04 14:45:58,361 WARN 101 [OlapScanN

[GitHub] [incubator-doris] spaces-X commented on pull request #3741: Ignore broken disks when BE starts up

2020-06-04 Thread GitBox
spaces-X commented on pull request #3741: URL: https://github.com/apache/incubator-doris/pull/3741#issuecomment-638667370 > Compilation failed. Did you miss `util/auto_deallocator.h`? Hi, i have removed the redundant `include` line, and rebuild it successfully in my machine. How

[GitHub] [incubator-doris] xy720 commented on a change in pull request #3742: [Spark load][Be 1/1] Be handle push task

2020-06-04 Thread GitBox
xy720 commented on a change in pull request #3742: URL: https://github.com/apache/incubator-doris/pull/3742#discussion_r435053583 ## File path: be/src/olap/push_handler.h ## @@ -209,6 +216,44 @@ class LzoBinaryReader: public IBinaryReader { size_t _next_row_start; }; +c

[GitHub] [incubator-doris] xy720 commented on a change in pull request #3742: [Spark load][Be 1/1] Be handle push task

2020-06-04 Thread GitBox
xy720 commented on a change in pull request #3742: URL: https://github.com/apache/incubator-doris/pull/3742#discussion_r435050694 ## File path: be/src/olap/push_handler.cpp ## @@ -761,6 +915,126 @@ OLAPStatus LzoBinaryReader::_next_block() { return res; } +OLAPStatus Push

[GitHub] [incubator-doris] xy720 commented on a change in pull request #3742: [Spark load][Be 1/1] Be handle push task

2020-06-04 Thread GitBox
xy720 commented on a change in pull request #3742: URL: https://github.com/apache/incubator-doris/pull/3742#discussion_r435048880 ## File path: be/src/olap/push_handler.cpp ## @@ -761,6 +915,126 @@ OLAPStatus LzoBinaryReader::_next_block() { return res; } +OLAPStatus Push

[GitHub] [incubator-doris] spaces-X commented on pull request #3768: Disable Bitmap or Hll type in keys or in values with incorrect agg-type

2020-06-04 Thread GitBox
spaces-X commented on pull request #3768: URL: https://github.com/apache/incubator-doris/pull/3768#issuecomment-638660609 > Please add UT for this case. > You can reference to `CreateTableStmtTest.java` Done. This i

[GitHub] [incubator-doris] spaces-X commented on pull request #3768: Disable Bitmap or Hll type in keys or in values with incorrect agg-type

2020-06-04 Thread GitBox
spaces-X commented on pull request #3768: URL: https://github.com/apache/incubator-doris/pull/3768#issuecomment-638659009 [Fix #3769](https://github.com/apache/incubator-doris/issues/3769) This is an automated message from th