[GitHub] [incubator-doris] HappenLee opened a new issue #4137: [Bug] MemExceed in SortNode when enable spilling and use multi spill operator

2020-07-21 Thread GitBox
HappenLee opened a new issue #4137: URL: https://github.com/apache/incubator-doris/issues/4137 **Describe the bug** MemExceed in SortNode when enable spilling and use multi spill operator, when we use PartitionAggregateNode or PartitionHash join node also need to spill to disk. *

[GitHub] [incubator-doris] worker24h commented on a change in pull request #4136: Fixbug: json load

2020-07-21 Thread GitBox
worker24h commented on a change in pull request #4136: URL: https://github.com/apache/incubator-doris/pull/4136#discussion_r458559252 ## File path: be/src/exec/json_scanner.h ## @@ -144,9 +141,9 @@ class JsonReader { RuntimeProfile::Counter* _bytes_read_counter; Runti

[GitHub] [incubator-doris] worker24h commented on a change in pull request #4136: Fixbug: json load

2020-07-21 Thread GitBox
worker24h commented on a change in pull request #4136: URL: https://github.com/apache/incubator-doris/pull/4136#discussion_r458558896 ## File path: be/src/exec/json_scanner.cpp ## @@ -197,30 +201,52 @@ JsonReader::~JsonReader() { _close(); } -Status JsonReader::init() {

[GitHub] [incubator-doris] worker24h commented on a change in pull request #4136: Fixbug: json load

2020-07-21 Thread GitBox
worker24h commented on a change in pull request #4136: URL: https://github.com/apache/incubator-doris/pull/4136#discussion_r458558969 ## File path: be/src/exec/json_scanner.h ## @@ -106,33 +105,31 @@ struct JsonPath; class JsonReader { public: JsonReader(RuntimeState* st

[GitHub] [incubator-doris] WingsGo commented on pull request #4110: [Feature][ThreadPool]Add Web Page to display thread's stats

2020-07-21 Thread GitBox
WingsGo commented on pull request #4110: URL: https://github.com/apache/incubator-doris/pull/4110#issuecomment-662262529 > Hi @WingsGo Could you please add a document for this new feature? > I suggest `docs/zh-CN/administrator-guide/ui/be-ui.md` OK, I will add it later. -

[GitHub] [incubator-doris] morningman commented on a change in pull request #4136: Fixbug: json load

2020-07-21 Thread GitBox
morningman commented on a change in pull request #4136: URL: https://github.com/apache/incubator-doris/pull/4136#discussion_r458546950 ## File path: be/src/exec/json_scanner.h ## @@ -106,33 +105,31 @@ struct JsonPath; class JsonReader { public: JsonReader(RuntimeState* s

[GitHub] [incubator-doris] liutang123 removed a comment on pull request #4105: Delete bad replicas when no BE can be used to create new replica. for…

2020-07-21 Thread GitBox
liutang123 removed a comment on pull request #4105: URL: https://github.com/apache/incubator-doris/pull/4105#issuecomment-662202086 @morningman hi,mingyu, how to make ut run again? This is an automated message from the Apache

[GitHub] [incubator-doris] liutang123 commented on pull request #4105: Delete bad replicas when no BE can be used to create new replica. for…

2020-07-21 Thread GitBox
liutang123 commented on pull request #4105: URL: https://github.com/apache/incubator-doris/pull/4105#issuecomment-662202086 @morningman hi,mingyu, how to make ut run again? This is an automated message from the Apache Git Ser

[GitHub] [incubator-doris] liutang123 closed pull request #4105: Delete bad replicas when no BE can be used to create new replica. for…

2020-07-21 Thread GitBox
liutang123 closed pull request #4105: URL: https://github.com/apache/incubator-doris/pull/4105 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[incubator-doris] branch master updated: [Bug] fix use-after-poison bug in ut schema_change_test (#4118)

2020-07-21 Thread lichaoyong
This is an automated email from the ASF dual-hosted git repository. lichaoyong pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 46c8c25 [Bug] fix use-after-poison

[GitHub] [incubator-doris] chaoyli merged pull request #4118: [Bug] fix use-after-poison bug in ut schema_change_test

2020-07-21 Thread GitBox
chaoyli merged pull request #4118: URL: https://github.com/apache/incubator-doris/pull/4118 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [incubator-doris] morningman commented on a change in pull request #4125: [Bug][Load][Json] #4124 Load json format with stream load failed

2020-07-21 Thread GitBox
morningman commented on a change in pull request #4125: URL: https://github.com/apache/incubator-doris/pull/4125#discussion_r458207883 ## File path: be/src/exec/json_scanner.cpp ## @@ -243,6 +243,7 @@ Status JsonReader::_parse_json_doc(bool* eof) { uint8_t* json_str = null

[GitHub] [incubator-doris] liutang123 closed pull request #4105: Delete bad replicas when no BE can be used to create new replica. for…

2020-07-21 Thread GitBox
liutang123 closed pull request #4105: URL: https://github.com/apache/incubator-doris/pull/4105 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-doris] acelyc111 commented on issue #3529: [Bug] BE crash in brpc

2020-07-21 Thread GitBox
acelyc111 commented on issue #3529: URL: https://github.com/apache/incubator-doris/issues/3529#issuecomment-661848746 I saw this stack again: ``` Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `/home/work/app/doris/c3prc-bigbi/be/package/be/lib/p

[GitHub] [incubator-doris] yangzhg edited a comment on issue #4051: [Proposal] Doris supports batch delete

2020-07-21 Thread GitBox
yangzhg edited a comment on issue #4051: URL: https://github.com/apache/incubator-doris/issues/4051#issuecomment-655317768 # Doris 支持批量删除设计文档 ## 背景 ​目前Doris 支持broker load, routine load, stream load 等多种导入方式,对于数据的删除目前只能通过delete 语句进行删除,使用delete 语句的方式删除时,每执行一次delete

[GitHub] [incubator-doris] worker24h opened a new pull request #4136: Fixbug: json load

2020-07-21 Thread GitBox
worker24h opened a new pull request #4136: URL: https://github.com/apache/incubator-doris/pull/4136 1) FixBug: load a json object is failed in routine load for example json-data: {"category":"11","title":"SayingsoftheCentury","price":895,"timestamp":1589191587} 2) add `json_r

[GitHub] [incubator-doris] acelyc111 closed issue #3911: [Proposal] Make BE web site more pretty

2020-07-21 Thread GitBox
acelyc111 closed issue #3911: URL: https://github.com/apache/incubator-doris/issues/3911 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [incubator-doris] vagetablechicken opened a new pull request #4135: [MemTracker] make all MemTrackers shared

2020-07-21 Thread GitBox
vagetablechicken opened a new pull request #4135: URL: https://github.com/apache/incubator-doris/pull/4135 Ref https://github.com/apache/incubator-doris/issues/3714 We make all MemTrackers shared. As follows: 1. all MemTracker raw ptr -> shared_ptr 2. Use CreateTracker() to creat

[GitHub] [incubator-doris] ZhangYu0123 opened a new issue #4134: Add compaction point changing log

2020-07-21 Thread GitBox
ZhangYu0123 opened a new issue #4134: URL: https://github.com/apache/incubator-doris/issues/4134 Add log to record the compaction point changing. When OLAP_ERR_BE_SEGMENTS_OVERLAPPING happens, it will be used to track the bugs. -

[GitHub] [incubator-doris] morningman commented on a change in pull request #4105: Delete bad replicas when no BE can be used to create new replica. for…

2020-07-21 Thread GitBox
morningman commented on a change in pull request #4105: URL: https://github.com/apache/incubator-doris/pull/4105#discussion_r457433461 ## File path: fe/src/main/java/org/apache/doris/catalog/Tablet.java ## @@ -17,6 +17,11 @@ package org.apache.doris.catalog; +import com.go

[GitHub] [incubator-doris] yangzhg opened a new pull request #4133: generate jave files using maven

2020-07-21 Thread GitBox
yangzhg opened a new pull request #4133: URL: https://github.com/apache/incubator-doris/pull/4133 ## Proposed changes generate generated-java files using maven instead of by build.sh ## Types of changes What types of changes does your code introduce to Doris? _Put an

[GitHub] [incubator-doris] caiconghui commented on pull request #4131: Use LongAdder or volatile long to replace AtomicLong in some scenarios

2020-07-21 Thread GitBox
caiconghui commented on pull request #4131: URL: https://github.com/apache/incubator-doris/pull/4131#issuecomment-661718793 for #4132 This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [incubator-doris] caiconghui opened a new issue #4132: Use LongAdder or volatile long to replace AtomicLong in some scenarios

2020-07-21 Thread GitBox
caiconghui opened a new issue #4132: URL: https://github.com/apache/incubator-doris/issues/4132 In the statistical summation scenario, LongAdder(introduced by jdk1.8) has better performance than AtomicLong in high concurrency update scenario. And if we just want to keep get and set operati

[GitHub] [incubator-doris] caiconghui opened a new pull request #4131: Use LongAdder or volatile long to replace AtomicLong in some scenarios

2020-07-21 Thread GitBox
caiconghui opened a new pull request #4131: URL: https://github.com/apache/incubator-doris/pull/4131 ## Proposed changes This PR is to use LongAdder or volatile long to replace AtomicLong in some scenarios. In the statistical summation scenario, LongAdder(introduced by jdk1.8) has be

[GitHub] [incubator-doris] yuanfeng0905 opened a new issue #4130: [BUG] from_base64

2020-07-21 Thread GitBox
yuanfeng0905 opened a new issue #4130: URL: https://github.com/apache/incubator-doris/issues/4130 example sql: ```sql select a,from_base64(a) from ( select "aGVsbG8=" a union all select null a ) t ``` The expect result is: | a | from_ba

[GitHub] [incubator-doris] worker24h commented on a change in pull request #4125: [Bug][Load][Json] #4124 Load json format with stream load failed

2020-07-21 Thread GitBox
worker24h commented on a change in pull request #4125: URL: https://github.com/apache/incubator-doris/pull/4125#discussion_r457912482 ## File path: be/src/exec/json_scanner.cpp ## @@ -243,6 +243,7 @@ Status JsonReader::_parse_json_doc(bool* eof) { uint8_t* json_str = nullp

[GitHub] [incubator-doris] worker24h commented on a change in pull request #4125: [Bug][Load][Json] #4124 Load json format with stream load failed

2020-07-21 Thread GitBox
worker24h commented on a change in pull request #4125: URL: https://github.com/apache/incubator-doris/pull/4125#discussion_r457912482 ## File path: be/src/exec/json_scanner.cpp ## @@ -243,6 +243,7 @@ Status JsonReader::_parse_json_doc(bool* eof) { uint8_t* json_str = nullp

[GitHub] [incubator-doris] stalary commented on issue #4129: Compaction error

2020-07-21 Thread GitBox
stalary commented on issue #4129: URL: https://github.com/apache/incubator-doris/issues/4129#issuecomment-661701900 ![截屏2020-07-21 下午4 01 42](https://user-images.githubusercontent.com/24592373/88028687-90d46800-cb6b-11ea-9520-7f7f05a4361f.png)

[GitHub] [incubator-doris] stalary opened a new issue #4129: Compaction error

2020-07-21 Thread GitBox
stalary opened a new issue #4129: URL: https://github.com/apache/incubator-doris/issues/4129 **Describe the bug** compaction appear "[0-6323] 1 DATA NONOVERLAPPING" **Version** master: https://github.com/apache/incubator-doris/commit/725ebafd999e9e027ac4f5c10da4d8b4636474fb -

[GitHub] [incubator-doris] ZhangYu0123 opened a new pull request #4128: Add compaction point changing log

2020-07-21 Thread GitBox
ZhangYu0123 opened a new pull request #4128: URL: https://github.com/apache/incubator-doris/pull/4128 Add log to record the compaction point changing. When OLAP_ERR_BE_SEGMENTS_OVERLAPPING happens, it will be used to track the bugs. ---

[GitHub] [incubator-doris] WingsGo commented on pull request #4127: [Bug][DynamicPartition]Fix bug that Modify a dynamic partition property in a non-dynamic partition table will throw a Exception

2020-07-21 Thread GitBox
WingsGo commented on pull request #4127: URL: https://github.com/apache/incubator-doris/pull/4127#issuecomment-661694471 Ref #4126 This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [incubator-doris] WingsGo opened a new pull request #4127: [Bug][DynamicPartition]Fix bug that Modify a dynamic partition property in a non-dynamic partition table will throw a Exception

2020-07-21 Thread GitBox
WingsGo opened a new pull request #4127: URL: https://github.com/apache/incubator-doris/pull/4127 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [incubator-doris] wutiangan commented on a change in pull request #4125: [Bug][Load][Json] #4124 Load json format with stream load failed

2020-07-21 Thread GitBox
wutiangan commented on a change in pull request #4125: URL: https://github.com/apache/incubator-doris/pull/4125#discussion_r457892910 ## File path: be/src/runtime/stream_load/stream_load_pipe.h ## @@ -85,30 +87,14 @@ class StreamLoadPipe : public MessageBodySink, public FileRe

[GitHub] [incubator-doris] wutiangan commented on a change in pull request #4125: [Bug][Load][Json] #4124 Load json format with stream load failed

2020-07-21 Thread GitBox
wutiangan commented on a change in pull request #4125: URL: https://github.com/apache/incubator-doris/pull/4125#discussion_r457892910 ## File path: be/src/runtime/stream_load/stream_load_pipe.h ## @@ -85,30 +87,14 @@ class StreamLoadPipe : public MessageBodySink, public FileRe

[GitHub] [incubator-doris] wutiangan commented on a change in pull request #4125: [Bug][Load][Json] #4124 Load json format with stream load failed

2020-07-21 Thread GitBox
wutiangan commented on a change in pull request #4125: URL: https://github.com/apache/incubator-doris/pull/4125#discussion_r457892464 ## File path: be/src/exec/json_scanner.cpp ## @@ -243,6 +243,7 @@ Status JsonReader::_parse_json_doc(bool* eof) { uint8_t* json_str = nullp

[GitHub] [incubator-doris] wutiangan commented on a change in pull request #4125: [Bug][Load][Json] #4124 Load json format with stream load failed

2020-07-21 Thread GitBox
wutiangan commented on a change in pull request #4125: URL: https://github.com/apache/incubator-doris/pull/4125#discussion_r457892231 ## File path: be/src/http/action/stream_load.cpp ## @@ -312,7 +312,7 @@ Status StreamLoadAction::_process_put(HttpRequest* http_req, StreamLoad

[GitHub] [incubator-doris] morningman commented on a change in pull request #4109: [Spark Load] add job granularity global dict lock

2020-07-21 Thread GitBox
morningman commented on a change in pull request #4109: URL: https://github.com/apache/incubator-doris/pull/4109#discussion_r457882817 ## File path: fe/src/main/java/org/apache/doris/load/loadv2/LoadJobScheduler.java ## @@ -47,6 +51,10 @@ private LinkedBlockingQueue need

[GitHub] [incubator-doris] WingsGo opened a new issue #4126: [Bug]Modify a dynamic partition property in a non-dynamic partition table will throw a Exception

2020-07-21 Thread GitBox
WingsGo opened a new issue #4126: URL: https://github.com/apache/incubator-doris/issues/4126 **Describe the bug** When user use cmd like `alter table xxx set ("dynamic_partition.enable"="true");` , if the table is not a dynamic partition table, It will throw a Exception like following