Re: Remove non-vectorization code

2022-12-06 Thread T jp 
+1

在 2022/12/6 13:38，“vin jake” 写入:

+1
It's enough to keep non-vectorization code just in 1.1-lts
It's time to remove non-vectorization code now in master

On Tue, Dec 6, 2022 at 12:23 PM GuoLei Yi  wrote:

> Hi All Developers,
>
> Vectorization is introduced into Doris in 1.0 which  is released
> earlier this year. Many companies is using 1.1.x and it is very very
> Fast and Stable.  I think it's time to remove non-vectorization code
> now. Following is my considerations:
>
> - Lots of company is using doris 1.1.x and it is stable.
> - 1.1-lts is a LTS branch, we will maintain it and fix bugs for it, If
> you need non-vectorization code, I think 1.1 will solve your problem.
> - 1.2 is now released, many features like array, JsonB, nest loop
> join, decimalv3, datev2 and etc. are not implemented on
> non-vectorization engine. So if a user is using 1.2 and using some new
> features then it could not rollback to non-vectorization engine any
> more.
> - We need to refactor some code but we need to maintain these
> non-vectorization code, it is very hard and it is very likely to
> import BUGs and there are not many developers could add feature or fix
> bugs for non-vectorization code.
>
> So that, I think it's time to remove non-vectorization code now to
> move Fast and Stable!!!
>
> --
> Best Regards,
> Yi Guo Lei
> SelectDB
> The Best Cloud Native Platform for Apache Doris
> Tel:134-3991-0228
> Email:yiguo...@gmail.com
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
> For additional commands, e-mail: dev-h...@doris.apache.org
>
>

Re:回复： Remove non-vectorization code

2022-12-06 Thread Mingyu Chen 
Agreed.
The non-vec code can be maintained on branch-1.1-lts, and branch-1.2-lts. 




--

此致！Best Regards
陈明雨 Mingyu Chen

Email:
morning...@apache.org





在 2022-12-06 16:24:00，"BiteThet"  写道：
>+1, it will be helpful
>
>
>
>
>BiteThet
>pxl...@qq.com
>
>
>
> 
>
>
>
>
>-- 原始邮件 --
>发件人:   
> "dev" 
>   发送时间: 2022年12月6日(星期二) 下午4:22
>收件人: "dev@doris.apache.org"
>主题: Re: Remove non-vectorization code
>
>
>
>+1
>
>在 2022/12/6 13:38，“vin jake”
>    +1
>    It's enough to keep non-vectorization code just in 1.1-lts
>    It's time to remove non-vectorization code now in master
>
>    On Tue, Dec 6, 2022 at 12:23 PM GuoLei Yi 
>
>    > Hi All Developers,
>    >
>    > Vectorization is introduced into Doris in 1.0 
>which  is released
>    > earlier this year. Many companies is using 1.1.x and 
>it is very very
>    > Fast and Stable.  I think it's time to remove 
>non-vectorization code
>    > now. Following is my considerations:
>    >
>    > - Lots of company is using doris 1.1.x and it is 
>stable.
>    > - 1.1-lts is a LTS branch, we will maintain it and fix 
>bugs for it, If
>    > you need non-vectorization code, I think 1.1 will 
>solve your problem.
>    > - 1.2 is now released, many features like array, 
>JsonB, nest loop
>    > join, decimalv3, datev2 and etc. are not implemented on
>    > non-vectorization engine. So if a user is using 1.2 
>and using some new
>    > features then it could not rollback to 
>non-vectorization engine any
>    > more.
>    > - We need to refactor some code but we need to 
>maintain these
>    > non-vectorization code, it is very hard and it is very 
>likely to
>    > import BUGs and there are not many developers could 
>add feature or fix
>    > bugs for non-vectorization code.
>    >
>    > So that, I think it's time to remove non-vectorization 
>code now to
>    > move Fast and Stable!!!
>    >
>    > --
>    > Best Regards,
>    > Yi Guo Lei
>    > SelectDB
>    > The Best Cloud Native Platform for Apache Doris
>    > Tel:134-3991-0228
>    > Email:yiguo...@gmail.com
>    >
>    > 
>-
>    > To unsubscribe, e-mail: 
>dev-unsubscr...@doris.apache.org
>    > For additional commands, e-mail: 
>dev-h...@doris.apache.org
>    >
>    >
>
>
>-
>To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
>For additional commands, e-mail: dev-h...@doris.apache.org

[Enhancement] make libhdfs3 compatible with hdfs2 server (PR #15)

2022-12-06 Thread junling du 
Hi all,

I have merge a pr to libhdfs3 for make libhdfs3 compatible with hdfs2
server.
And want to put a tag libhdfs3-v2.3.4 on the current the branch libhdfs3.

It is just a tag on libhdfs3 instead of release.


https://github.com/apache/doris-thirdparty/releases/tag/libhdfs3-v2.3.4




dujunling
dujlm...@gmail.com

Re:[Enhancement] make libhdfs3 compatible with hdfs2 server (PR #15)

2022-12-06 Thread Mingyu Chen 
Done,
And all doc has been updated.




--

此致！Best Regards
陈明雨 Mingyu Chen

Email:
morning...@apache.org





At 2022-12-07 10:33:00, "junling du"  wrote:
>Hi all,
>
>I have merge a pr to libhdfs3 for make libhdfs3 compatible with hdfs2
>server.
>And want to put a tag libhdfs3-v2.3.4 on the current the branch libhdfs3.
>
>It is just a tag on libhdfs3 instead of release.
>
>
>https://github.com/apache/doris-thirdparty/releases/tag/libhdfs3-v2.3.4
>
>
>
>
>dujunling
>dujlm...@gmail.com

关于咨询flink cdc任务自动关闭问题

2022-12-06 Thread 应聘者张龙飞 
新建了flink cdc任务 
测试doris的实时读写（读写都是doris），source和sink的配置都是按照官网配置的，目前现象是任务正常启动，数据同步成功，但是第一次同步成功后flink
 cdc任务就开始关闭，日志显示正常流程关闭的，也不报错2022-12-06 23:06:20,408 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: jobmanager.rpc.address, master
2022-12-06 23:06:20,414 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: jobmanager.rpc.port, 6123
2022-12-06 23:06:20,414 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: jobmanager.memory.process.size, 2048m
2022-12-06 23:06:20,414 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: taskmanager.memory.process.size, 4096m
2022-12-06 23:06:20,415 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: taskmanager.numberOfTaskSlots, 4
2022-12-06 23:06:20,415 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: parallelism.default, 2
2022-12-06 23:06:20,415 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: jobmanager.execution.failover-strategy, region
2022-12-06 23:06:20,415 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: rest.port, 8082
2022-12-06 23:06:20,416 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: rest.address, master
2022-12-06 23:06:20,416 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: rest.bind-address, 0.0.0.0
2022-12-06 23:06:20,416 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: web.submit.enable, true
2022-12-06 23:06:20,416 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: io.tmp.dirs, /data/universe/tmp
2022-12-06 23:06:20,416 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: env.pid.dir, /data/universe/apache/flink/current/pids
2022-12-06 23:06:20,417 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: jobmanager.archive.fs.dir, hdfs:///flink/completed-jobs
2022-12-06 23:06:20,417 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: historyserver.web.address, master
2022-12-06 23:06:20,417 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: historyserver.web.port, 8083
2022-12-06 23:06:20,417 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: historyserver.archive.fs.dir, 
hdfs:///flink/completed-jobs
2022-12-06 23:06:20,417 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: historyserver.archive.fs.refresh-interval, 1
2022-12-06 23:06:20,417 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: state.backend, filesystem
2022-12-06 23:06:20,417 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: state.backend.fs.checkpointdir, 
hdfs:///flink/flink-checkpoints
2022-12-06 23:06:20,418 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: high-availability, zookeeper
2022-12-06 23:06:20,418 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: high-availability.storageDir, hdfs:///flink/ha/
2022-12-06 23:06:20,418 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: high-availability.zookeeper.quorum, 
worker1:2181,worker2:2181,worker3:2181
2022-12-06 23:06:20,418 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: classloader.check-leaked-classloader, false
2022-12-06 23:06:20,418 INFO  
org.apache.flink.configuration.GlobalConfiguration   [] - Loading 
configuration property: classloader.resolve-order, parent-first
2022-12-06 23:06:20,584 INFO  
org.apache.flink.table.client.gateway.context.DefaultContext [] - Executor 
config: {execution.savepoint.ignore-unclaimed-state=false, 
execution.attached=true, execution.savepoint-restore-mode=NO_CLAIM, 
execution.shutdown-on-attached-exit=false, 
pipeline.jars=[file:/data/universe/apache/flink/flink-1.15.2/opt/flink-sql-client-1.15.2.jar,
 
file:/data/universe/apache/flink/flink-1.15.2/lib/flink-python_2.12-1.15.2.jar],
 pipeline.classpaths=[], execution.target=remote}
2022-12-06 23:06:20,774 WARN  org.apache.flink.configuration.Configuration  
   [] - Config uses deprecated configuration key 
'state.ba

[Result][Vote] Release Apache Doris 1.2.0-rc04

2022-12-06 Thread Mingyu Chen 
Hi all,
the vote is passed and closed[1].
There are 3 +1(binding) votes:


Mingyu Chen
Jiafeng Zhang
De Li


And 5 +1 (non binding) votes:


Minghong Zhou
Junling Du
Rongqian Li
William King
Guolei Yi


Thank you for your review.


[1] https://lists.apache.org/thread/878r29ln6f3r2rvw7qfd5qt2s9x9cmb9



--

此致！Best Regards
陈明雨 Mingyu Chen

Email:
morning...@apache.org