By the way, if you need a mentor, you can ping me. Regards JB
On Tue, Feb 22, 2022 at 8:08 AM Jean-Baptiste Onofré <[email protected]> wrote: > > Hi, > > It looks interesting. Quick questions about the current state of the > project/roadmap: how the users implement the ETL pipeline ? Is it > using a kind of SDK/DSL or do you plan an UI ? > Is it very coupled to Flink or the runtime is "pluggable" ? Do you > plan to have some kind of optional storage, similar to Apache Druid, > ... ? > > Regards > JB > > On Tue, Feb 22, 2022 at 7:41 AM Apache <[email protected]> wrote: > > > > > > Dear Apache Incubator Community, > > We propose to contribute Chunjun as an Apache Incubator project. > > We are still looking for possible Champion and Mentors if anyone would like > > to volunteer. Thanks a lot. > > Best Regards, > > Real-time computing engine team of DTStack. > > > > #Chunjun Proposal > > > > ##Abstract > > Chunjun is a distributed ETL tool and distributed data integration tool. > > Currently Chunjun is based on Apache Flink. It was initially known as > > FlinkX and renamed Chunjun on February 22, 2022. > > - Chunjun codebase: https://github.com/DTStack/chunjun > > > > ##Proposal > > We propose to contribute the Chunjun codebase to the Apache Software > > Foundation with the intent of forming a productive, meritocratic and open > > community around Chunjun’s continued development, according to the 'Apache > > Way'. The Chunjun's source code is already under the Apache License Version > > 2.0. > > > > ##Background > > We developed Chunjun in DTStack company in 2017, when we needed a low-code > > development and high-performance data integration tool. It has been an > > open-source project on GitHub since April 2018. Chunjun is running in > > DTStack production environment all the time. Chunjun has also been widely > > used by companies in China, including DTStack (https://www.dtstack.com/), > > Qihu360(https://www.360.cn/), Iflytek (https://www.iflytek.com/), XPeng > > Motors (https://en.xiaopeng.com/), WeBank (https://www.webank.com/), > > Asiainfo(https://asiainfo.com/), Guazi(https://www.guazi.com/), Hello Inc > > (https://www.hello-inc.com/), etc. Nowadays, Chunjun has a strong > > community in China. > > > > ##Rationale > > High-performance of Chunjun is based on Apache Flink, and Chunjun can > > integrate data from different data source. Users only need to configure a > > JSON file to complete the data reading, transformer, and writing. Users can > > implement new reader/writer plugins to meet their requirements. Chunjun > > have implemented plugins that can capture data change for MySQL to restore > > data for Apache Doris. > > Chunjun has the following feature: > > real-time and offline integrate data from different data sources. > > change data capture(CDC) to merge restore data. > > resume from broken-point. > > capture and collect dirty data. > > limit data transferring rate. > > thoughput metrics. > > capture and restore schema evolution. (TODO) > > > > ##Current Status###Meritocracy > > Since Chunjun was open-sourced, many enterprises have adopted Chunjun to > > build up their data integration system. In return, we have received many > > issue reports or enhancements from them simultaneously. The codebase is now > > mainly managed by the development team inside DTStack who's responsible for > > building internal data integration system too.###Community > > Chunjun has been building a community around contributors and users to this > > framework for the last five years. We organized one meetup in 2020. > > Currently, we communicate in Github issues and in chinese DingTalk group. > > There are about 3000 people in this group. And we believe that we can get a > > lot of help from the Apache Flink community too. We will organize a meetup > > again in 2022.###Core Developers > > (In alphabetical order) > > Chao Xu (https://github.com/zoudaokoulife) > > Gongjiang Tang, (https://github.com/kyo-tom) > > Huai Yang, (https://github.com/yanghuaiGit) > > Jiangbo Li, (https://github.com/lijiangbo) > > Luning Wong, (https://github.com/deadwind4) > > Luo Li, (https://github.com/kanata163) > > Sishu Yang, (https://github.com/yangsishu) > > Tianzhu Wen, (https://github.com/WTZ468071157) > > Weiliang Hao, (https://github.com/xiuzhu9527) > > Wenqiang Liu, (https://github.com/meng1222) > > Xing Liu, (https://github.com/simenliuxing) > > Yang Lan, (https://github.com/HiLany) > > Yanquan Lv, (https://github.com/lvyanquan) > > Yifan Hu, (https://github.com/demotto) > > Zaiyue Yu, (https://github.com/tonybobam) > > Zhangwan Zhao, (https://github.com/jiemotongxue) > > Zhiqiang Li, (https://github.com/ChestnutQiang) > > They are almost working in real-time computing engine team of DTStack. Only > > Yifan Hu working for CaoCao Tech. Most of them are Apache Flink contributor. > > > > ##Known Risks###Project Name > > The name of the project is Chunjun. Chunjun comes from mandarin chinese > > Pinyin "Chun Jun", and it is one of the top ten famous swords in > > China.###Orphaned products > > More than 20 contributors and thousands of forks and star further show that > > Chunjun is actively supported, and we seek to further prosper the community > > with the aid of Apache. As a consequence, Chunjun is unlikely to be reduced > > to an orphaned project.###Inexperience with Open Source > > Many of the Chunjun committers have experience working on open source > > projects. They are also active contributors to other Apache projects. > > ###Homogenous Developers > > The most of core developers are from DTStack, and Chunjun received some bug > > fixes and enhancements from other developers not working at DTStack. > > ###Reliance on Salaried Developers > > Currently, most of core developers are paid to work on Chunjun project by > > DTStack. We look forward to attracting more people outside DTStack to join > > this project.###Relationships with Other Apache Products > > We have integrated with Apache Flink, Apache Hadoop, Apache Common and > > Apache HttpComponents, Log4J and Maven. > > Usage of Apache projects related to Chunjun plugin > > Apache Hive > > Apache Solr > > Apache Doris > > Apache HBase > > Apache Kudu > > Apache Kafka > > Apache Pulsar (TODO)###An Excessive Fascination with the Apache Brand > > We acknowledge the value and reputation that the Apache brand would bring > > to Chunjun. However, our primary interest is in the excellent community > > provided by Apache Software Foundation, in which all the projects could > > gain stability for long-term development. > > > > ##Documentation > > A complete set of documents is provided on GitHub, including English and > > Simplified Chinese versions. > > English: https://github.com/DTStack/chunjun/blob/master/README.md > > Chinese: https://github.com/DTStack/chunjun/blob/master/README_CH.md > > > > ##Initial Code > > https://github.com/DTStack/chunjun > > > > ##Initial Source and Intellectual Property Submission Plan > > The codebase is already licensed under the Apache License 2.0 and the > > copyright is assigned to DTStack. If the project enters incubator, DTStack > > will transfer the source code & trademark ownership to ASF via a Software > > Grant Agreement. Our initial committers will submit iCLA(s), SGA, and > > CCLA(s). ##External DependenciesApache-2.0 licenses > > Apache Avro > > Apache Commons > > Apache Curator > > Apache Flink > > Apache Hadoop > > Apache HttpComponents > > Apache Log4j > > Gson > > Guava > > Jackson > > Powermock > > PrometheusEclipse Distribution License > > JUnitEPL licenses > > LogbackMIT licenses > > Mockito > > SLF4J > > > > ##Required Resources ###Git Repositories > > https://github.com/apache/incubator-chunjun###Issue Tracking > > The community would like to continue using GitHub Issues.###Mailing List > > [email protected] > > [email protected] > > [email protected]###Continuous Integration tool > > GitHub Action > > > > ##Initial Committers > > (In alphabetical order) > > Chao Xu (https://github.com/zoudaokoulife, xuchao at dtstack dot com) > > Luning Wong (https://github.com/deadwind4, gfeng48 at gmail dot com) > > Sishu Yang (https://github.com/yangsishu, sishu at dtstack dot com) > > Yang Huai (https://github.com/yanghuaiGit, dujie at dtstack dot com) > > Zhiqiang Li (https://github.com/ChestnutQiang, wujuan at dtstack dot com) > > > > > > ##Affiliations > > The initial committers are employees of DTStack. The nominated mentors and > > champion are employees of TODO. > > > > ##Sponsors > > > > ###Champion > > TODO ###Nominated Mentors > > TODO --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
