[GitHub] [incubator-doris] KevinLinGit opened a new pull request #1259: modify udf docs

2019-06-05 Thread GitBox
KevinLinGit opened a new pull request #1259: modify udf docs URL: https://github.com/apache/incubator-doris/pull/1259 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [incubator-doris] qdore edited a comment on issue #1256: failed to initialize storage reader. tablet=322844.1094272791, res=-214, backend=10.90.239.23

2019-06-05 Thread GitBox
qdore edited a comment on issue #1256: failed to initialize storage reader. tablet=322844.1094272791, res=-214, backend=10.90.239.23 URL: https://github.com/apache/incubator-doris/issues/1256#issuecomment-499337855 This is the warning log of be when I triggered doris return `initialize st

[GitHub] [incubator-doris] qdore edited a comment on issue #1256: failed to initialize storage reader. tablet=322844.1094272791, res=-214, backend=10.90.239.23

2019-06-05 Thread GitBox
qdore edited a comment on issue #1256: failed to initialize storage reader. tablet=322844.1094272791, res=-214, backend=10.90.239.23 URL: https://github.com/apache/incubator-doris/issues/1256#issuecomment-499337855 initialize storage reader failed: be warning log: ``` W0606 14:33:

[GitHub] [incubator-doris] morningman commented on issue #1257: Routine load problem

2019-06-05 Thread GitBox
morningman commented on issue #1257: Routine load problem URL: https://github.com/apache/incubator-doris/issues/1257#issuecomment-499364270 That's weird. Could you please send a be.INFO and fe.log file to my mail box: morning...@163.com

[GitHub] [incubator-doris] EmmyMiao87 closed pull request #1255: Change strategy of incorrect data

2019-06-05 Thread GitBox
EmmyMiao87 closed pull request #1255: Change strategy of incorrect data URL: https://github.com/apache/incubator-doris/pull/1255 This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [incubator-doris] EmmyMiao87 opened a new pull request #1255: Change strategy of incorrect data

2019-06-05 Thread GitBox
EmmyMiao87 opened a new pull request #1255: Change strategy of incorrect data URL: https://github.com/apache/incubator-doris/pull/1255 * Change strategy of incorrect data This change adds a load property named strict_mode which is used to prohibit the incorrect data. When it is set to

[GitHub] [incubator-doris] luoyafei commented on issue #1257: Routine load problem

2019-06-05 Thread GitBox
luoyafei commented on issue #1257: Routine load problem URL: https://github.com/apache/incubator-doris/issues/1257#issuecomment-499344954 concurrent task num set only 1 but i can see BE log, always submit new tasks my doris version is 0.10.0-rc01 ![image](https://user-images.githu

[GitHub] [incubator-doris] qdore commented on issue #1256: failed to initialize storage reader. tablet=322844.1094272791, res=-214, backend=10.90.239.23

2019-06-05 Thread GitBox
qdore commented on issue #1256: failed to initialize storage reader. tablet=322844.1094272791, res=-214, backend=10.90.239.23 URL: https://github.com/apache/incubator-doris/issues/1256#issuecomment-499337855 initialize storage reader failed: be warning log: ``` W0606 10:21:45.1832

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #1235: Support SSL authentication with Kafka in routine load job

2019-06-05 Thread GitBox
EmmyMiao87 commented on a change in pull request #1235: Support SSL authentication with Kafka in routine load job URL: https://github.com/apache/incubator-doris/pull/1235#discussion_r290564228 ## File path: fe/src/main/java/org/apache/doris/load/Load.java ## @@ -2307,7 +23

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #1235: Support SSL authentication with Kafka in routine load job

2019-06-05 Thread GitBox
EmmyMiao87 commented on a change in pull request #1235: Support SSL authentication with Kafka in routine load job URL: https://github.com/apache/incubator-doris/pull/1235#discussion_r291004804 ## File path: fe/src/main/java/org/apache/doris/load/routineload/KafkaRoutineLoadJob.java

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #1235: Support SSL authentication with Kafka in routine load job

2019-06-05 Thread GitBox
EmmyMiao87 commented on a change in pull request #1235: Support SSL authentication with Kafka in routine load job URL: https://github.com/apache/incubator-doris/pull/1235#discussion_r290564252 ## File path: fe/src/main/java/org/apache/doris/load/Load.java ## @@ -2348,7 +23

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #1235: Support SSL authentication with Kafka in routine load job

2019-06-05 Thread GitBox
EmmyMiao87 commented on a change in pull request #1235: Support SSL authentication with Kafka in routine load job URL: https://github.com/apache/incubator-doris/pull/1235#discussion_r290571088 ## File path: docs/help/Contents/Administration/small_files.md ## @@ -0,0 +1,103

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #1235: Support SSL authentication with Kafka in routine load job

2019-06-05 Thread GitBox
EmmyMiao87 commented on a change in pull request #1235: Support SSL authentication with Kafka in routine load job URL: https://github.com/apache/incubator-doris/pull/1235#discussion_r291008767 ## File path: fe/src/main/java/org/apache/doris/load/routineload/KafkaRoutineLoadJob.java

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #1235: Support SSL authentication with Kafka in routine load job

2019-06-05 Thread GitBox
EmmyMiao87 commented on a change in pull request #1235: Support SSL authentication with Kafka in routine load job URL: https://github.com/apache/incubator-doris/pull/1235#discussion_r291007447 ## File path: fe/src/main/java/org/apache/doris/common/util/SmallFileMgr.java ##

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #1235: Support SSL authentication with Kafka in routine load job

2019-06-05 Thread GitBox
EmmyMiao87 commented on a change in pull request #1235: Support SSL authentication with Kafka in routine load job URL: https://github.com/apache/incubator-doris/pull/1235#discussion_r291005971 ## File path: fe/src/main/java/org/apache/doris/load/routineload/KafkaRoutineLoadJob.java

[GitHub] [incubator-doris] EmmyMiao87 opened a new pull request #1255: Change strategy of incorrect data

2019-06-05 Thread GitBox
EmmyMiao87 opened a new pull request #1255: Change strategy of incorrect data URL: https://github.com/apache/incubator-doris/pull/1255 * Change strategy of incorrect data This change adds a load property named strict_mode which is used to prohibit the incorrect data. When it is set to

[GitHub] [incubator-doris] EmmyMiao87 closed pull request #1255: Change strategy of incorrect data

2019-06-05 Thread GitBox
EmmyMiao87 closed pull request #1255: Change strategy of incorrect data URL: https://github.com/apache/incubator-doris/pull/1255 This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [incubator-doris] luoyafei commented on issue #1257: Routine load problem

2019-06-05 Thread GitBox
luoyafei commented on issue #1257: Routine load problem URL: https://github.com/apache/incubator-doris/issues/1257#issuecomment-499320986 this my routine load config; ```sql mysql> show routine load; +---+--+-+---+-+-

[GitHub] [incubator-doris] morningman commented on issue #1257: Routine load problem

2019-06-05 Thread GitBox
morningman commented on issue #1257: Routine load problem URL: https://github.com/apache/incubator-doris/issues/1257#issuecomment-499308826 @luoyafei How many routine load job have you been created in FE? And what is their concurrent task number? --

[GitHub] [incubator-doris] imay commented on issue #1258: routine load submit task stall

2019-06-05 Thread GitBox
imay commented on issue #1258: routine load submit task stall URL: https://github.com/apache/incubator-doris/issues/1258#issuecomment-499294089 #1257 This is an automated message from the Apache Git Service. To respond to t

[GitHub] [incubator-doris] imay opened a new issue #1258: routine load submit task stall

2019-06-05 Thread GitBox
imay opened a new issue #1258: routine load submit task stall URL: https://github.com/apache/incubator-doris/issues/1258 **Describe the bug** When there are too many concurrent routine load task, all `submit_task` will wait for a lock when the thread pool is full. following is the st

[GitHub] [incubator-doris] imay commented on issue #1257: Routine load problem

2019-06-05 Thread GitBox
imay commented on issue #1257: Routine load problem URL: https://github.com/apache/incubator-doris/issues/1257#issuecomment-499293113 @luoyafei Thanks for your help. I think the problem is that you have too many concurrent routine load job. I'm not sure your case. In this stack tra

[GitHub] [incubator-doris] luoyafei commented on issue #1257: Routine load problem

2019-06-05 Thread GitBox
luoyafei commented on issue #1257: Routine load problem URL: https://github.com/apache/incubator-doris/issues/1257#issuecomment-499153773 all most threads seem to be waiting for lock? [be_stack.log](https://github.com/apache/incubator-doris/files/3258036/be_stack.log) -

[GitHub] [incubator-doris] imay commented on issue #1257: Routine load problem

2019-06-05 Thread GitBox
imay commented on issue #1257: Routine load problem URL: https://github.com/apache/incubator-doris/issues/1257#issuecomment-499132342 you should look backendId 10002's LOG, and you can pstack doris_be process to see what are the threads doing. -

[GitHub] [incubator-doris] luoyafei commented on issue #1257: Routine load problem

2019-06-05 Thread GitBox
luoyafei commented on issue #1257: Routine load problem URL: https://github.com/apache/incubator-doris/issues/1257#issuecomment-499129996 like this? ![image](https://user-images.githubusercontent.com/18156435/58968393-95fc1980-87e8-11e9-900e-8febfb9fda5b.png) --

[GitHub] [incubator-doris] imay commented on issue #1257: Routine load problem

2019-06-05 Thread GitBox
imay commented on issue #1257: Routine load problem URL: https://github.com/apache/incubator-doris/issues/1257#issuecomment-499126580 It seems that FE is failed to submit load task to BE. You should look BE's INFO log to see if there is some problem? --

[GitHub] [incubator-doris] luoyafei commented on issue #1257: Routine load problem

2019-06-05 Thread GitBox
luoyafei commented on issue #1257: Routine load problem URL: https://github.com/apache/incubator-doris/issues/1257#issuecomment-499124300 There are no changes after the revision 0.10.2 What causes this exception? ![image](https://user-images.githubusercontent.com/18156435/5896733

[GitHub] [incubator-doris] imay commented on a change in pull request #1255: Change strategy of incorrect data

2019-06-05 Thread GitBox
imay commented on a change in pull request #1255: Change strategy of incorrect data URL: https://github.com/apache/incubator-doris/pull/1255#discussion_r290767524 ## File path: be/src/exec/broker_scanner.cpp ## @@ -585,24 +601,35 @@ bool BrokerScanner::fill_dest_tuple(cons

[GitHub] [incubator-doris] imay commented on a change in pull request #1255: Change strategy of incorrect data

2019-06-05 Thread GitBox
imay commented on a change in pull request #1255: Change strategy of incorrect data URL: https://github.com/apache/incubator-doris/pull/1255#discussion_r290766080 ## File path: fe/src/main/java/org/apache/doris/planner/StreamLoadScanNode.java ## @@ -240,6 +242,7 @@ public

[GitHub] [incubator-doris] imay commented on a change in pull request #1255: Change strategy of incorrect data

2019-06-05 Thread GitBox
imay commented on a change in pull request #1255: Change strategy of incorrect data URL: https://github.com/apache/incubator-doris/pull/1255#discussion_r290766431 ## File path: fe/src/main/java/org/apache/doris/planner/StreamLoadScanNode.java ## @@ -249,6 +252,7 @@ private

[GitHub] [incubator-doris] EmmyMiao87 opened a new pull request #1255: Change strategy of incorrect data

2019-06-05 Thread GitBox
EmmyMiao87 opened a new pull request #1255: Change strategy of incorrect data URL: https://github.com/apache/incubator-doris/pull/1255 * Change strategy of incorrect data This change adds a load property named strict_mode which is used to prohibit the incorrect data. When it is set to

[GitHub] [incubator-doris] imay commented on issue #1257: Routine load problem

2019-06-05 Thread GitBox
imay commented on issue #1257: Routine load problem URL: https://github.com/apache/incubator-doris/issues/1257#issuecomment-499084177 Please use 0.10.2 This is an automated message from the Apache Git Service. To respond to t

[GitHub] [incubator-doris] luoyafei opened a new issue #1257: Routine load problem

2019-06-05 Thread GitBox
luoyafei opened a new issue #1257: Routine load problem URL: https://github.com/apache/incubator-doris/issues/1257 Q: create routine load, but table no data to load. resource: >Only 1 FE, 1 BE, 1 KFK, 1 ZK >`FE`, `BE`, `KFK`,` ZK` all on the same machine kafka ver

[GitHub] [incubator-doris] EmmyMiao87 closed pull request #1255: Change strategy of incorrect data

2019-06-05 Thread GitBox
EmmyMiao87 closed pull request #1255: Change strategy of incorrect data URL: https://github.com/apache/incubator-doris/pull/1255 This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [incubator-doris] imay commented on issue #1256: failed to initialize storage reader. tablet=322844.1094272791, res=-214, backend=10.90.239.23

2019-06-05 Thread GitBox
imay commented on issue #1256: failed to initialize storage reader. tablet=322844.1094272791, res=-214, backend=10.90.239.23 URL: https://github.com/apache/incubator-doris/issues/1256#issuecomment-499062276 No, this is not a fd leak. we have a cache for all open files. When we delete a fi

[GitHub] [incubator-doris] qdore commented on issue #1256: failed to initialize storage reader. tablet=322844.1094272791, res=-214, backend=10.90.239.23

2019-06-05 Thread GitBox
qdore commented on issue #1256: failed to initialize storage reader. tablet=322844.1094272791, res=-214, backend=10.90.239.23 URL: https://github.com/apache/incubator-doris/issues/1256#issuecomment-499014209 And why palo open many deleted file fd? Does it has fd leak? ![image](https://

[GitHub] [incubator-doris] qdore opened a new issue #1256: failed to initialize storage reader. tablet=322844.1094272791, res=-214, backend=10.90.239.23

2019-06-05 Thread GitBox
qdore opened a new issue #1256: failed to initialize storage reader. tablet=322844.1094272791, res=-214, backend=10.90.239.23 URL: https://github.com/apache/incubator-doris/issues/1256 A normal sql trigger doris return this: ``` ERROR 1064 (HY000): failed to initialize storage rea

[GitHub] [incubator-doris] imay commented on a change in pull request #1255: Change strategy of incorrect data

2019-06-05 Thread GitBox
imay commented on a change in pull request #1255: Change strategy of incorrect data URL: https://github.com/apache/incubator-doris/pull/1255#discussion_r290650231 ## File path: gensrc/thrift/PlanNodes.thrift ## @@ -136,6 +136,13 @@ struct TBrokerScanRangeParams { //

[GitHub] [incubator-doris] imay commented on a change in pull request #1255: Change strategy of incorrect data

2019-06-05 Thread GitBox
imay commented on a change in pull request #1255: Change strategy of incorrect data URL: https://github.com/apache/incubator-doris/pull/1255#discussion_r290651357 ## File path: be/src/exec/broker_scanner.cpp ## @@ -585,21 +594,55 @@ bool BrokerScanner::fill_dest_tuple(cons

[GitHub] [incubator-doris] qdore closed issue #1254: Query cann't hit rollup

2019-06-05 Thread GitBox
qdore closed issue #1254: Query cann't hit rollup URL: https://github.com/apache/incubator-doris/issues/1254 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [incubator-doris] qdore commented on issue #1254: Query cann't hit rollup

2019-06-05 Thread GitBox
qdore commented on issue #1254: Query cann't hit rollup URL: https://github.com/apache/incubator-doris/issues/1254#issuecomment-499007320 I will have a test, thx This is an automated message from the Apache Git Service. To r

[GitHub] [incubator-doris] EmmyMiao87 opened a new pull request #1255: Change strategy of incorrect data

2019-06-05 Thread GitBox
EmmyMiao87 opened a new pull request #1255: Change strategy of incorrect data URL: https://github.com/apache/incubator-doris/pull/1255 * Change strategy of incorrect data This change adds a load property named strict_mode which is used to prohibit the incorrect data. When it is set to

[GitHub] [incubator-doris] imay commented on issue #207: load支持parquet 文件数据导入

2019-06-05 Thread GitBox
imay commented on issue #207: load支持parquet 文件数据导入 URL: https://github.com/apache/incubator-doris/issues/207#issuecomment-498976706 @yanchaoguo parquet格式已经在支持中了。 #1173 这个patch已经处于基本可用状态,着急的可以先试试 This is an autom

[GitHub] [incubator-doris] yanchaoguo commented on issue #207: load支持parquet 文件数据导入

2019-06-05 Thread GitBox
yanchaoguo commented on issue #207: load支持parquet 文件数据导入 URL: https://github.com/apache/incubator-doris/issues/207#issuecomment-498970493 > hive 表中存储数据为parquet 格式,建议load支持parquet 文件数据导入数据 +1 请兼容hadoop生态 This is an autom

[GitHub] [incubator-doris] imay commented on issue #1254: Query cann't hit rollup

2019-06-05 Thread GitBox
imay commented on issue #1254: Query cann't hit rollup URL: https://github.com/apache/incubator-doris/issues/1254#issuecomment-498965119 we have improve our rollup selecting in https://github.com/apache/incubator-doris/pull/947, you can check if this patch is in your Doris -