[GitHub] [incubator-doris] decster commented on a change in pull request #3668: [Memory Engine] Add TabletType to PartitionInfo and TabletMeta

decster commented on a change in pull request #3668: URL: https://github.com/apache/incubator-doris/pull/3668#discussion_r432253780 ## File path: be/src/olap/tablet_meta.cpp ## @@ -101,6 +89,8 @@ TabletMeta::TabletMeta(int64_t table_id, int64_t partition_id, tablet_meta_p

[GitHub] [incubator-doris] decster commented on a change in pull request #3668: [Memory Engine] Add TabletType to PartitionInfo and TabletMeta

decster commented on a change in pull request #3668: URL: https://github.com/apache/incubator-doris/pull/3668#discussion_r432253819 ## File path: be/src/olap/base_tablet.h ## @@ -84,6 +85,10 @@ inline const TabletMetaSharedPtr BaseTablet::tablet_meta() { return _tablet_met

[GitHub] [incubator-doris] EmmyMiao87 closed pull request #3705: Serialize origin stmt in Rollup Job and MV Meta

EmmyMiao87 closed pull request #3705: URL: https://github.com/apache/incubator-doris/pull/3705 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-doris] HappenLee opened a new pull request #3722: [Bug] Fix bug that runningprofile show time problem in FE web page and add the runingprofile doc

HappenLee opened a new pull request #3722: URL: https://github.com/apache/incubator-doris/pull/3722 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [incubator-doris] chaoyli opened a new issue #3721: fix mixed used of counter

chaoyli opened a new issue #3721: URL: https://github.com/apache/incubator-doris/issues/3721 MysqlResultWriter _sent_rows_counter and _result_send_timer are mixed used. It will results core dump when calling this function in RuntimeProfille ``` DCHECK(counter->type() == TUnit::TIME_

[GitHub] [incubator-doris] chaoyli opened a new pull request #3720: [Bug] fix mixed used of counter

chaoyli opened a new pull request #3720: URL: https://github.com/apache/incubator-doris/pull/3720 MysqlResultWriter _sent_rows_counter and _result_send_timer are mixed used. It will results core dump when checking counter->type().

[GitHub] [incubator-doris] chaoyli merged pull request #3710: [Bug] Set the 404 error for QueryProfile HttpRequest when query_id not set

chaoyli merged pull request #3710: URL: https://github.com/apache/incubator-doris/pull/3710 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [incubator-doris] chaoyli closed issue #3711: The 404 error for QueryProfile HttpRequest is not set

chaoyli closed issue #3711: URL: https://github.com/apache/incubator-doris/issues/3711 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[incubator-doris] branch master updated: [Bug] Set the HttpResponseStatus for QueryProfile when query_id been not set (#3710)

This is an automated email from the ASF dual-hosted git repository. lichaoyong pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 5f1d25a [Bug] Set the HttpResponse

[GitHub] [incubator-doris] imay commented on a change in pull request #3716: [Spark load] Fe submit spark etl job

imay commented on a change in pull request #3716: URL: https://github.com/apache/incubator-doris/pull/3716#discussion_r432214780 ## File path: fe/src/main/java/org/apache/doris/common/util/BrokerUtil.java ## @@ -139,4 +142,326 @@ public static String printBroker(String brokerNa

[GitHub] [incubator-doris] imay commented on a change in pull request #3718: [Spark load] Update push task thrift interface

imay commented on a change in pull request #3718: URL: https://github.com/apache/incubator-doris/pull/3718#discussion_r432210442 ## File path: gensrc/thrift/Types.thrift ## @@ -138,7 +138,8 @@ enum TAggregationType { enum TPushType { LOAD, DELETE, -LOAD_DELETE +

[GitHub] [incubator-doris] lcldragon opened a new issue #3719: tcmalloc 使用错误

lcldragon opened a new issue #3719: URL: https://github.com/apache/incubator-doris/issues/3719 doris_main.cpp 这个文件中的 #if !defined(ADDRESS_SANITIZER) && !defined(LEAK_SANITIZER) && !defined(THREAD_SANITIZER) MallocExtension::instance()->SetNumericProperty("tcmalloc.aggressive_mem

[GitHub] [incubator-doris] imay commented on a change in pull request #3715: [Spark load] Fe create job

imay commented on a change in pull request #3715: URL: https://github.com/apache/incubator-doris/pull/3715#discussion_r432209858 ## File path: fe/src/main/java/org/apache/doris/analysis/BrokerDesc.java ## @@ -17,61 +17,36 @@ package org.apache.doris.analysis; -import org.a

[GitHub] [incubator-doris] imay commented on a change in pull request #3712: [Spark load] Add spark etl job config

imay commented on a change in pull request #3712: URL: https://github.com/apache/incubator-doris/pull/3712#discussion_r432205716 ## File path: fe/src/main/java/org/apache/doris/load/loadv2/etl/EtlJobConfig.java ## @@ -0,0 +1,487 @@ +// Licensed to the Apache Software Foundation

[GitHub] [incubator-doris] wyb edited a comment on issue #3433: [Spark load] Doris support Spark load

wyb edited a comment on issue #3433: URL: https://github.com/apache/incubator-doris/issues/3433#issuecomment-635180109 [Spark load] Add spark etl job config #3712 [Spark load] Update push task thrift interface #3718 This

[GitHub] [incubator-doris] wyb opened a new pull request #3718: [Spark load] Update push task thrift interface

wyb opened a new pull request #3718: URL: https://github.com/apache/incubator-doris/pull/3718 1. Add TBrokerScanRange and TDescriptorTable used by ParquetScanner 2. Add new TPushType LOAD_V2 for spark load #3433 Th

[GitHub] [incubator-doris] wyb commented on issue #3433: [Spark load] Doris support Spark load

wyb commented on issue #3433: URL: https://github.com/apache/incubator-doris/issues/3433#issuecomment-635482036 [Spark load] Fe process etl and loading state job #3717 This is an automated message from the Apache Git Service

[GitHub] [incubator-doris] wyb opened a new pull request #3717: [Spark load] Fe process etl and loading state job

wyb opened a new pull request #3717: URL: https://github.com/apache/incubator-doris/pull/3717 1. Fe checks the status of etl job regularly 1.1 If status is RUNNING, update etl job progress 1.2 If status is CANCELLED, cancel load job 1.3 If status is FINISHED, get the etl output fil

[GitHub] [incubator-doris] wyb commented on issue #3433: [Spark load] Doris support Spark load

wyb commented on issue #3433: URL: https://github.com/apache/incubator-doris/issues/3433#issuecomment-635469749 [Spark load] Fe submit spark etl job #3716 This is an automated message from the Apache Git Service. To respond

[GitHub] [incubator-doris] wyb opened a new pull request #3716: [Spark load] Fe submit spark etl job

wyb opened a new pull request #3716: URL: https://github.com/apache/incubator-doris/pull/3716 After user creates a spark load job which status is PENDING, Fe will schedule and submit the spark etl job. 1. Begin transaction 2. Create a SparkLoadPendingTask for submitting etl job 2.1

[GitHub] [incubator-doris] wangbo commented on pull request #3604: (#3464) fix Query failed when fact table has no data in join case

wangbo commented on pull request #3604: URL: https://github.com/apache/incubator-doris/pull/3604#issuecomment-635401808 > > @morningman > > The key point is that it's currently impossible to accurately estimate the size of the memory footprint at runtime according to table size.Otherwise

[incubator-doris] branch master updated (e76f712 -> 9c85d05)

This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from e76f712 [Bug] Load data is error in json load add 9c85d05 [Bug] RuntimeState should be destructed

[GitHub] [incubator-doris] morningman merged pull request #3709: [Bug] RuntimeState should be destructed after DataSink

morningman merged pull request #3709: URL: https://github.com/apache/incubator-doris/pull/3709 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-doris] morningman closed issue #3706: BE cores because RuntimeState is deallocated before DataSink when fragment is cancelled

morningman closed issue #3706: URL: https://github.com/apache/incubator-doris/issues/3706 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[incubator-doris] branch master updated: [Bug] Load data is error in json load

This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new e76f712 [Bug] Load data is error i

[GitHub] [incubator-doris] morningman merged pull request #3702: FixBug: Load data Is error in json load

morningman merged pull request #3702: URL: https://github.com/apache/incubator-doris/pull/3702 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-doris] WingsGo commented on a change in pull request #3713: Use BackendStatus to show BE's infomation in `show backends;`

WingsGo commented on a change in pull request #3713: URL: https://github.com/apache/incubator-doris/pull/3713#discussion_r431702383 ## File path: fe/src/main/java/org/apache/doris/master/ReportHandler.java ## @@ -247,6 +249,13 @@ private static void tabletReport(long backendId,

[GitHub] [incubator-doris] caiconghui edited a comment on pull request #3703: [Load] Modify Some config in be to make doris get better performance both in stream load and query

caiconghui edited a comment on pull request #3703: URL: https://github.com/apache/incubator-doris/pull/3703#issuecomment-635199557 As a conclusion, the http server is based on libevent , which use reactor model to process http request, some http requests are processed by one thread. it a

[GitHub] [incubator-doris] caiconghui edited a comment on pull request #3703: [Load] Modify Some config in be to make doris get better performance both in stream load and query

caiconghui edited a comment on pull request #3703: URL: https://github.com/apache/incubator-doris/pull/3703#issuecomment-635199557 As a conclusion, the http server is based on libevent , which use reactor model to process http request, some http requests are processed by one thread.

[GitHub] [incubator-doris] caiconghui edited a comment on pull request #3703: [Load] Modify Some config in be to make doris get better performance both in stream load and query

caiconghui edited a comment on pull request #3703: URL: https://github.com/apache/incubator-doris/pull/3703#issuecomment-635199557 As a conclusion, the http server is based on libevent , which use reactor model to process http request, some http requests are processed by one thread. it as

[GitHub] [incubator-doris] morningman commented on a change in pull request #3713: Use BackendStatus to show BE's infomation in `show backends;`

morningman commented on a change in pull request #3713: URL: https://github.com/apache/incubator-doris/pull/3713#discussion_r431672560 ## File path: fe/src/main/java/org/apache/doris/system/Backend.java ## @@ -94,6 +94,9 @@ // this field is set by tablet report, and just f

[GitHub] [incubator-doris] caiconghui commented on pull request #3703: [Load] Modify Some config in be to make doris get better performance both in stream load and query

caiconghui commented on pull request #3703: URL: https://github.com/apache/incubator-doris/pull/3703#issuecomment-635199557 As a conclusion, the http server is based on libevent , which use reactor model to process http request, some http requests are processed by one thread. it assumes t

[GitHub] [incubator-doris] caiconghui closed pull request #3703: [Load] Modify Some config in be to make doris get better performance both in stream load and query

caiconghui closed pull request #3703: URL: https://github.com/apache/incubator-doris/pull/3703 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-doris] decster commented on a change in pull request #3668: [Memory Engine] Add TabletType to PartitionInfo and TabletMeta

decster commented on a change in pull request #3668: URL: https://github.com/apache/incubator-doris/pull/3668#discussion_r431665339 ## File path: fe/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java ## @@ -233,6 +236,23 @@ public static TStorageType analyzeStorag

[GitHub] [incubator-doris] morningman commented on a change in pull request #3712: Add spark etl job config

morningman commented on a change in pull request #3712: URL: https://github.com/apache/incubator-doris/pull/3712#discussion_r431659000 ## File path: fe/src/main/java/org/apache/doris/load/loadv2/etl/EtlJobConfig.java ## @@ -0,0 +1,487 @@ +// Licensed to the Apache Software Foun

[GitHub] [incubator-doris] WingsGo commented on pull request #3713: Use BackendStatus to show BE's infomation in `show backends;`

WingsGo commented on pull request #3713: URL: https://github.com/apache/incubator-doris/pull/3713#issuecomment-635186222 For #3698 This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [incubator-doris] WingsGo opened a new pull request #3713: Use BackendStatus to show BE's infomation in `show backends;`

WingsGo opened a new pull request #3713: URL: https://github.com/apache/incubator-doris/pull/3713 The infomation is displayed in JSON format.For example: ` {"lastTabletReportTime":"N/A"} ` This is an automated message from

[GitHub] [incubator-doris] acelyc111 commented on a change in pull request #3690: [compaction] Update cumulative point calculate algorithm

acelyc111 commented on a change in pull request #3690: URL: https://github.com/apache/incubator-doris/pull/3690#discussion_r431654202 ## File path: be/src/olap/tablet.cpp ## @@ -68,7 +68,7 @@ Tablet::Tablet(TabletMetaSharedPtr tablet_meta, DataDir* data_dir) : _last_b

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #3705: Serialize origin stmt in Rollup Job and MV Meta

EmmyMiao87 commented on a change in pull request #3705: URL: https://github.com/apache/incubator-doris/pull/3705#discussion_r431651862 ## File path: fe/src/main/java/org/apache/doris/catalog/MaterializedIndexMeta.java ## @@ -46,9 +55,11 @@ private TStorageType storageType;

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #3705: Serialize origin stmt in Rollup Job and MV Meta

EmmyMiao87 commented on a change in pull request #3705: URL: https://github.com/apache/incubator-doris/pull/3705#discussion_r431651256 ## File path: fe/src/main/java/org/apache/doris/alter/AlterJobV2.java ## @@ -220,16 +233,20 @@ protected boolean checkTableStable(Database db)

[GitHub] [incubator-doris] wyb commented on issue #3433: [Spark load] Doris support Spark load

wyb commented on issue #3433: URL: https://github.com/apache/incubator-doris/issues/3433#issuecomment-635181748 Add resource manager #3418 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [incubator-doris] wyb commented on issue #3433: [Spark load] Doris support Spark load

wyb commented on issue #3433: URL: https://github.com/apache/incubator-doris/issues/3433#issuecomment-635180109 Add spark etl job config #3712 This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [incubator-doris] wyb opened a new pull request #3712: Add spark etl job config

wyb opened a new pull request #3712: URL: https://github.com/apache/incubator-doris/pull/3712 Add spark etl job config, includes: 1. Schema of the load tables, including columns, partitions and rollups 2. Infos of the source file, including split rules, corresponding columns, and conv

[GitHub] [incubator-doris] wyb commented on issue #3010: Spark load interface

wyb commented on issue #3010: URL: https://github.com/apache/incubator-doris/issues/3010#issuecomment-635174675 update new etl job config ``` { "tables": { 10014: { "indexes": [{ "index_id": 10014,

[GitHub] [incubator-doris] caiconghui commented on a change in pull request #3703: [Load] Modify Some config in be to make doris get better performance both in stream load and query

caiconghui commented on a change in pull request #3703: URL: https://github.com/apache/incubator-doris/pull/3703#discussion_r431633255 ## File path: be/src/common/config.h ## @@ -137,7 +140,7 @@ namespace config { // the maximum number of bytes to display on the debug webs

[GitHub] [incubator-doris] chaoyli commented on issue #3711: The 404 error for QueryProfile HttpRequest is not set

chaoyli commented on issue #3711: URL: https://github.com/apache/incubator-doris/issues/3711#issuecomment-635159021  -

[GitHub] [incubator-doris] morningman commented on a change in pull request #3705: Serialize origin stmt in Rollup Job and MV Meta

morningman commented on a change in pull request #3705: URL: https://github.com/apache/incubator-doris/pull/3705#discussion_r431628375 ## File path: fe/src/main/java/org/apache/doris/analysis/CreateMaterializedViewStmt.java ## @@ -100,9 +102,8 @@ public KeysType getMVKeysType(

[GitHub] [incubator-doris] chaoyli opened a new issue #3711: The 404 error for QueryProfile HttpRequest is not set

chaoyli opened a new issue #3711: URL: https://github.com/apache/incubator-doris/issues/3711 Doris can get query profile by HttpRequest ``` http://fe_host:web_port/query_profile?query_id=123456 ``` Now, if query_id is not found, the 404 error is not set in HttpHeader. -

[GitHub] [incubator-doris] chaoyli opened a new pull request #3710: [Bug] Set the HttpResponseStatus for QueryProfile when query_id been not set

chaoyli opened a new pull request #3710: URL: https://github.com/apache/incubator-doris/pull/3710 Doris can get query profile by HttpRequest ``` http://fe_host:web_port/query_profile?query_id=123456 ``` Now, if query_id is not found, the return value is not set error in HttpHea