[GitHub] [incubator-doris] caiconghui commented on a change in pull request #4006: [Feature] Support InPredicate in delete statement

caiconghui commented on a change in pull request #4006: URL: https://github.com/apache/incubator-doris/pull/4006#discussion_r450646519 ## File path: fe/src/main/java/org/apache/doris/load/DeleteHandler.java ## @@ -569,6 +587,19 @@ private void checkDeleteV2(OlapTable table, Par

[GitHub] [incubator-doris] chaoyli commented on pull request #4009: [Bug]Fix some schema change not work right

chaoyli commented on pull request #4009: URL: https://github.com/apache/incubator-doris/pull/4009#issuecomment-654633518 > > It's better add more discrete message in commit info like you issue. > > Ok, done, and I tested the schema change supported in doc, but I found that convert La

[GitHub] [incubator-doris] caiconghui commented on a change in pull request #4006: [Feature] Support InPredicate in delete statement

caiconghui commented on a change in pull request #4006: URL: https://github.com/apache/incubator-doris/pull/4006#discussion_r450638700 ## File path: fe/src/main/java/org/apache/doris/load/DeleteHandler.java ## @@ -450,7 +451,22 @@ public DeleteJob getDeleteJob(long transactionI

[GitHub] [incubator-doris] caiconghui commented on a change in pull request #4006: [Feature] Support InPredicate in delete statement

caiconghui commented on a change in pull request #4006: URL: https://github.com/apache/incubator-doris/pull/4006#discussion_r450637016 ## File path: fe/src/main/java/org/apache/doris/load/DeleteHandler.java ## @@ -507,7 +516,19 @@ private void checkDeleteV2(OlapTable table, Par

[GitHub] [incubator-doris] wutiangan commented on a change in pull request #4006: [Feature] Support InPredicate in delete statement

wutiangan commented on a change in pull request #4006: URL: https://github.com/apache/incubator-doris/pull/4006#discussion_r450632141 ## File path: fe/src/main/java/org/apache/doris/load/DeleteHandler.java ## @@ -507,7 +516,19 @@ private void checkDeleteV2(OlapTable table, Part

[GitHub] [incubator-doris] morningman commented on a change in pull request #4034: [Bug][Broker] Fix bug that the error exception is not caught when closing the broker writer

morningman commented on a change in pull request #4034: URL: https://github.com/apache/incubator-doris/pull/4034#discussion_r450629168 ## File path: fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/FileSystemManager.java ## @@ -586,6 +586,8 @@ public vo

[GitHub] [incubator-doris] xy720 commented on pull request #4034: [Bug][Broker] Fix bug that the error exception is not caught when closing the broker writer

xy720 commented on pull request #4034: URL: https://github.com/apache/incubator-doris/pull/4034#issuecomment-654621955 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-doris] wutiangan commented on a change in pull request #4006: [Feature] Support InPredicate in delete statement

wutiangan commented on a change in pull request #4006: URL: https://github.com/apache/incubator-doris/pull/4006#discussion_r450628421 ## File path: fe/src/main/java/org/apache/doris/load/DeleteHandler.java ## @@ -450,7 +451,22 @@ public DeleteJob getDeleteJob(long transactionId

[GitHub] [incubator-doris] wutiangan commented on a change in pull request #4006: [Feature] Support InPredicate in delete statement

wutiangan commented on a change in pull request #4006: URL: https://github.com/apache/incubator-doris/pull/4006#discussion_r450627407 ## File path: fe/src/main/java/org/apache/doris/load/DeleteHandler.java ## @@ -569,6 +587,19 @@ private void checkDeleteV2(OlapTable table, Part

[GitHub] [incubator-doris] xy720 commented on a change in pull request #4034: [Bug][Broker] Fix bug that the error exception is not caught when closing the broker writer

xy720 commented on a change in pull request #4034: URL: https://github.com/apache/incubator-doris/pull/4034#discussion_r450627361 ## File path: fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/FileSystemManager.java ## @@ -586,6 +586,8 @@ public void cl

[GitHub] [incubator-doris] xy720 removed a comment on pull request #4034: [Bug][Broker] Fix bug that the error exception is not caught when closing the broker writer

xy720 removed a comment on pull request #4034: URL: https://github.com/apache/incubator-doris/pull/4034#issuecomment-654615792 The md5sum was generated before the local file upload to remote, which means the local file is a empty file. Is it normal we have a empty local file? -

[GitHub] [incubator-doris] xy720 commented on pull request #4034: [Bug][Broker] Fix bug that the error exception is not caught when closing the broker writer

xy720 commented on pull request #4034: URL: https://github.com/apache/incubator-doris/pull/4034#issuecomment-654615792 The md5sum was generated before the local file upload to remote, which means the local file is a empty file. Is it normal we have a empty local file? -

[GitHub] [incubator-doris] chaoyli merged pull request #4024: Correct typos for zh-CN doc

chaoyli merged pull request #4024: URL: https://github.com/apache/incubator-doris/pull/4024 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[incubator-doris] branch master updated: Correct typos (#4024)

This is an automated email from the ASF dual-hosted git repository. lichaoyong pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new d396408 Correct typos (#4024) d396

[GitHub] [incubator-doris] chaoyli commented on a change in pull request #4011: Remove rs when version conflict when insert to a tablet

chaoyli commented on a change in pull request #4011: URL: https://github.com/apache/incubator-doris/pull/4011#discussion_r450619615 ## File path: be/src/olap/tablet.cpp ## @@ -826,6 +832,33 @@ void Tablet::_print_missed_versions(const std::vector& missed_versions) LOG(WAR

[GitHub] [incubator-doris] wuyunfeng opened a new pull request #4035: [Doris On ES] Add UT test for all search phase

wuyunfeng opened a new pull request #4035: URL: https://github.com/apache/incubator-doris/pull/4035 I forget push some UT test in this PR https://github.com/apache/incubator-doris/pull/4012, and remove `_cluster/state` resource because DOE does not rely the full ES cluster state meta. -

[GitHub] [incubator-doris] morningman opened a new pull request #4034: [Bug][Broker] Fix bug that the error exception is not caught when closing the broker writer

morningman opened a new pull request #4034: URL: https://github.com/apache/incubator-doris/pull/4034 Broker should return the error status. Fix #4033 This is an automated message from the Apache Git Service. To respond to

[GitHub] [incubator-doris] morningman opened a new issue #4033: [Restore]

morningman opened a new issue #4033: URL: https://github.com/apache/incubator-doris/issues/4033 **Describe the bug** Restore failed because of following errors shows in `SHOW RESTORE` result. ``` [19685: invalid md5 of downloaded file: /path/to/palobe/data/snapshot/202006301129

[GitHub] [incubator-doris] wangbo commented on issue #4030: Be coredump when mem_pool->allocate returns null in ColumnReader

wangbo commented on issue #4030: URL: https://github.com/apache/incubator-doris/issues/4030#issuecomment-654591856 > Solution2 could be better. But this will involve a lot of code. > How about adding a new method mentioned in solution 2, and call it where needed? It seems that ad

[GitHub] [incubator-doris] wangbo commented on issue #4027: DeltaWriter's FlushStatistic needs a metric

wangbo commented on issue #4027: URL: https://github.com/apache/incubator-doris/issues/4027#issuecomment-654591438 I see. I am curious about detail of ```some reason the flush time or count in one BE is always longer/higher than others```. Under what circumstances will this problem

[GitHub] [incubator-doris] wuyunfeng closed issue #3960: [Doris On ES] All DOE table sync meta-data failure after upgrading FE

wuyunfeng closed issue #3960: URL: https://github.com/apache/incubator-doris/issues/3960 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [incubator-doris] fanlongmengjd opened a new issue #4032: colocate is true on Master but is false : reason group is not stable

fanlongmengjd opened a new issue #4032: URL: https://github.com/apache/incubator-doris/issues/4032 **Describe the bug** same sql was sent to master and other followers. the one to master could use colocate, the another to other follower could not use colocate **To Reproduc

[GitHub] [incubator-doris] morningman commented on a change in pull request #4020: [Load][Json] Refactor json load logic to make it more reasonable

morningman commented on a change in pull request #4020: URL: https://github.com/apache/incubator-doris/pull/4020#discussion_r450582767 ## File path: be/src/exec/json_scanner.cpp ## @@ -406,54 +459,60 @@ Status JsonReader::handle_simple_json(Tuple* tuple, const std::vector& slo

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #4014: Add framework of mv selector

EmmyMiao87 commented on a change in pull request #4014: URL: https://github.com/apache/incubator-doris/pull/4014#discussion_r450581152 ## File path: fe/src/main/java/org/apache/doris/rewrite/mvrewrite/SlotRefEqualRule.java ## @@ -0,0 +1,39 @@ +// Licensed to the Apache Softwar

[GitHub] [incubator-doris] vagetablechicken commented on issue #4027: DeltaWriter's FlushStatistic needs a metric

vagetablechicken commented on issue #4027: URL: https://github.com/apache/incubator-doris/issues/4027#issuecomment-654562912 > I think the main pain points here is that the flush log isn't associated with a `transaction id` or `query id`, so it's hard to know how much time cost on `DeltaW

[GitHub] [incubator-doris] morningman merged pull request #4012: [Doris On ES][Refactor] refactor and enchanment ES sync meta logic

morningman merged pull request #4012: URL: https://github.com/apache/incubator-doris/pull/4012 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[incubator-doris] branch master updated: [Doris On ES][Refactor] refactor and enchanment ES sync meta logic (#4012)

This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 3ba38e3 [Doris On ES][Refactor] re

[GitHub] [incubator-doris] caiconghui edited a comment on pull request #4029: Support check commited txns before catalog drop meta, like db, table, partition etc

caiconghui edited a comment on pull request #4029: URL: https://github.com/apache/incubator-doris/pull/4029#issuecomment-654309078 > If I use `DROPP Table` to forcibly drop a table with COMMITTED txns. > Can those COMMITTED txns be cleaned automatically? @morningman in fe, the txn

[GitHub] [incubator-doris] caiconghui commented on a change in pull request #4029: Support check commited txns before catalog drop meta, like db, table, partition etc

caiconghui commented on a change in pull request #4029: URL: https://github.com/apache/incubator-doris/pull/4029#discussion_r450338931 ## File path: fe/src/main/java/org/apache/doris/catalog/Catalog.java ## @@ -3308,6 +3315,20 @@ public void dropPartition(Database db, OlapTable

[GitHub] [incubator-doris] morningman commented on issue #4030: Be coredump when mem_pool->allocate returns null in ColumnReader

morningman commented on issue #4030: URL: https://github.com/apache/incubator-doris/issues/4030#issuecomment-654324046 Solution2 could be better. But this will involve a lot of code. How about adding a new method mentioned in solution 2, and call it where needed? -

[GitHub] [incubator-doris] morningman opened a new pull request #4031: [Config] Enable some features by default

morningman opened a new pull request #4031: URL: https://github.com/apache/incubator-doris/pull/4031 1. Enable FE plugins by setting `plugin_enable=true` 2. Enable dynamic partition by setting `dynamic_partition_enable=true` Also modify installation doc, add download link of MySQL

[GitHub] [incubator-doris] caiconghui commented on pull request #4029: Support check commited txns before catalog drop meta, like db, table, partition etc

caiconghui commented on pull request #4029: URL: https://github.com/apache/incubator-doris/pull/4029#issuecomment-654309078 > If I use `DROPP Table` to forcibly drop a table with COMMITTED txns. > Can those COMMITTED txns be cleaned automatically? @morningman in fe, the txn will b

[GitHub] [incubator-doris] morningman commented on a change in pull request #4006: [Feature] Support InPredicate in delete statement

morningman commented on a change in pull request #4006: URL: https://github.com/apache/incubator-doris/pull/4006#discussion_r450280629 ## File path: fe/src/main/java/org/apache/doris/load/DeleteHandler.java ## @@ -516,25 +537,22 @@ private void checkDeleteV2(OlapTable table, Pa

[GitHub] [incubator-doris] morningman commented on pull request #4029: Support check commited txns before catalog drop meta, like db, table, partition etc

morningman commented on pull request #4029: URL: https://github.com/apache/incubator-doris/pull/4029#issuecomment-654287503 If I use `DROPP Table` to forcibly drop a table with COMMITTED txns. Can those COMMITTED txns be cleaned automatically? ---

[GitHub] [incubator-doris] morningman commented on a change in pull request #4029: Support check commited txns before catalog drop meta, like db, table, partition etc

morningman commented on a change in pull request #4029: URL: https://github.com/apache/incubator-doris/pull/4029#discussion_r450272944 ## File path: fe/src/main/java/org/apache/doris/alter/MaterializedViewHandler.java ## @@ -716,7 +716,6 @@ public void processBatchDropRollup(Li

[GitHub] [incubator-doris] wangbo opened a new issue #4030: Be coredump when mem_pool->allocate returns null in ColumnReader

wangbo opened a new issue #4030: URL: https://github.com/apache/incubator-doris/issues/4030 **be core dump when mem_pool allocate memory failed** ``` #0 0x7fe8daf3dfdb in __memcpy_ssse3_back () from /lib64/libc.so.6 #1 0x00f586c2 in get (length=, dst_size=, dst=, this=

[GitHub] [incubator-doris] morningman commented on a change in pull request #4014: Add framework of mv selector

morningman commented on a change in pull request #4014: URL: https://github.com/apache/incubator-doris/pull/4014#discussion_r450094583 ## File path: fe/src/main/java/org/apache/doris/rewrite/mvrewrite/HLLHashToSlotRefRule.java ## @@ -0,0 +1,114 @@ +// Licensed to the Apache So

[GitHub] [incubator-doris] wangbo commented on issue #4027: DeltaWriter's FlushStatistic needs a metric

wangbo commented on issue #4027: URL: https://github.com/apache/incubator-doris/issues/4027#issuecomment-654257097 I think the main pain points here is that the flush log isn't associated with a ```transaction id``` or ```query id```, so it's hard to know how much time cost on ```DeltaWri

[GitHub] [incubator-doris] caiconghui commented on a change in pull request #4006: [Feature] Support InPredicate in delete statement

caiconghui commented on a change in pull request #4006: URL: https://github.com/apache/incubator-doris/pull/4006#discussion_r450222851 ## File path: fe/src/main/java/org/apache/doris/load/DeleteHandler.java ## @@ -516,25 +537,22 @@ private void checkDeleteV2(OlapTable table, Pa

[GitHub] [incubator-doris] caiconghui closed issue #3949: Many stream load requests are rpc timeout cause number of threads in fe skyrocket and finally cause fe core

caiconghui closed issue #3949: URL: https://github.com/apache/incubator-doris/issues/3949 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [incubator-doris] caiconghui commented on pull request #4029: Support check commited txns before catalog drop meta, like db, table, partition etc

caiconghui commented on pull request #4029: URL: https://github.com/apache/incubator-doris/pull/4029#issuecomment-654177442 for #3988 This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [incubator-doris] caiconghui opened a new pull request #4029: Support check commited txns before catalog drop meta, like db, table, partition etc

caiconghui opened a new pull request #4029: URL: https://github.com/apache/incubator-doris/pull/4029 This PR is to ensure that dropped db , table or partition can be with normal state after recovered by user. Commited txns can not be aborted, because the partitions's commited versions hav

[GitHub] [incubator-doris] acelyc111 commented on pull request #4028: [shell] Fix BE unit test directory not match bug

acelyc111 commented on pull request #4028: URL: https://github.com/apache/incubator-doris/pull/4028#issuecomment-654150920 I introduced this bug by #3990 :( This is an automated message from the Apache Git Service. To respon

[GitHub] [incubator-doris] acelyc111 opened a new pull request #4028: [shell] Fix BE unit test directory not match bug

acelyc111 opened a new pull request #4028: URL: https://github.com/apache/incubator-doris/pull/4028 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [incubator-doris] vagetablechicken opened a new issue #4027: DeltaWriter's FlushStatistic needs a metric

vagetablechicken opened a new issue #4027: URL: https://github.com/apache/incubator-doris/issues/4027 More user-friendly way to debug, instead of grep logs. DeltaWriter can flush many times, and there are too much writers opened, so we can get the max flush time in 15s. Or avg? ---

[GitHub] [incubator-doris] liutang123 commented on pull request #4026: Fix mini load NPE

liutang123 commented on pull request #4026: URL: https://github.com/apache/incubator-doris/pull/4026#issuecomment-654132393 @morningman This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [incubator-doris] liutang123 opened a new pull request #4026: Fix mini load NPE

liutang123 opened a new pull request #4026: URL: https://github.com/apache/incubator-doris/pull/4026 for #4025 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [incubator-doris] liutang123 opened a new issue #4025: NPE when use MiniLoad

liutang123 opened a new issue #4025: URL: https://github.com/apache/incubator-doris/issues/4025 In our cluster, we use `mini load` instead of `stream load` although `mini load` ia deprecated. But, when update to 0.12, `mini load` will throw NPE.

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #4014: Add framework of mv selector

EmmyMiao87 commented on a change in pull request #4014: URL: https://github.com/apache/incubator-doris/pull/4014#discussion_r450099097 ## File path: fe/src/main/java/org/apache/doris/qe/StmtExecutor.java ## @@ -440,58 +439,10 @@ public void analyze(TQueryOptions tQueryOptions)

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #4014: Add framework of mv selector

EmmyMiao87 commented on a change in pull request #4014: URL: https://github.com/apache/incubator-doris/pull/4014#discussion_r450093057 ## File path: fe/src/main/java/org/apache/doris/analysis/QueryStmt.java ## @@ -113,6 +113,40 @@ // represent the "INTO OUTFILE" clause

[GitHub] [incubator-doris] morningman commented on a change in pull request #4014: Add framework of mv selector

morningman commented on a change in pull request #4014: URL: https://github.com/apache/incubator-doris/pull/4014#discussion_r450045369 ## File path: fe/src/main/java/org/apache/doris/planner/SingleNodePlanner.java ## @@ -773,19 +776,26 @@ public void selectMaterializedView(Quer

[GitHub] [incubator-doris] HangyuanLiu commented on a change in pull request #4014: Add framework of mv selector

HangyuanLiu commented on a change in pull request #4014: URL: https://github.com/apache/incubator-doris/pull/4014#discussion_r450031609 ## File path: fe/src/main/java/org/apache/doris/analysis/MVColumnHLLUnionPatten.java ## @@ -0,0 +1,55 @@ +// Licensed to the Apache Software