Re: Re: [Discuss][DSIP] Support Merge-On-Write implementation for UNIQUE KEY data model

2022-06-29 Thread Chen Zhang
Updated the scheduling https://cwiki.apache.org/confluence/display/DORIS/DSIP-018%3A+Support+Merge-On-Write+implementation+for+UNIQUE+KEY+data+model Best Chen Zhang 在 2022年6月27日 +0800 11:59,Chen Zhang ,写道: > Hi Devs, I've update the DISP last weekend, if you are interest on this > feature, welcom

Re: Re: [Discuss][DSIP] Support Merge-On-Write implementation for UNIQUE KEY data model

2022-06-26 Thread Chen Zhang
Hi Devs, I've update the DISP last weekend, if you are interest on this feature, welcome to review and comment, thanks https://cwiki.apache.org/confluence/display/DORIS/DSIP-018%3A+Support+Merge-On-Write+implementation+for+UNIQUE+KEY+data+model Best Chen Zhang 在 2022年6月24日 +0800 10:13,zhg yang ,写

Re: Re: [Discuss][DSIP] Support Merge-On-Write implementation for UNIQUE KEY data model

2022-06-23 Thread zhg yang
@ Chen Zhang For the more important features, it is best to send a DISP first to let everyone discuss the design Thanks Yang Zhengguo Chen Zhang 于2022年6月23日周四 22:30写道: > @Minghong We'll use a multi-version delete bitmap, only save delta for > each version. > For example, we have a rowset with

Re:Re:Re:Re: [Discuss][DSIP] Support Merge-On-Write implementation for UNIQUE KEY data model

2022-06-23 Thread 陈明雨
Done! -- 此致!Best Regards 陈明雨 Mingyu Chen Email: morning...@apache.org 在 2022-06-24 08:48:29,"Chen Zhang" 写道: >@Mingyu, my username: zhannngchen. Thanks~ > >Best >Chen Zhang >On Jun 24, 2022, 00:56 +0800, 陈明雨 , wrote: >> Hi Zhang Chen: >> I have created a DSIP-018 for this[1]. But you ne

Re:Re:Re: [Discuss][DSIP] Support Merge-On-Write implementation for UNIQUE KEY data model

2022-06-23 Thread Chen Zhang
@Mingyu, my username: zhannngchen. Thanks~ Best Chen Zhang On Jun 24, 2022, 00:56 +0800, 陈明雨 , wrote: > Hi Zhang Chen: > I have created a DSIP-018 for this[1]. But you need to create an account and > tell me your username. > > > [1] > https://cwiki.apache.org/confluence/display/DORIS/DSIP-018%3A

Re:Re:Re: [Discuss][DSIP] Support Merge-On-Write implementation for UNIQUE KEY data model

2022-06-23 Thread 陈明雨
Hi Zhang Chen: I have created a DSIP-018 for this[1]. But you need to create an account and tell me your username. [1] https://cwiki.apache.org/confluence/display/DORIS/DSIP-018%3A+Support+Merge-On-Write+implementation+for+UNIQUE+KEY+data+model -- 此致!Best Regards 陈明雨 Mingyu Chen Email: mo

Re:Re: [Discuss][DSIP] Support Merge-On-Write implementation for UNIQUE KEY data model

2022-06-23 Thread Chen Zhang
@Minghong We'll use a multi-version delete bitmap, only save delta for each version. For example, we have a rowset with version [0-98], transaction 99 updated some row in that rowset, and so does transaction 100 and 101, there would be 3 delete bitmaps on that rowset, corresponding to rows updat

Re:Re: [Discuss][DSIP] Support Merge-On-Write implementation for UNIQUE KEY data model

2022-06-23 Thread Zhou Minghong
Hi Chen Zhang one question about "and a delete bitmap (marks some rowid as deleted)”: how to handle transaction information by a bitmap? for example, transaction_100 delete a row, but this still visible to transaction_99, but not visible to trasanction_101. How to handle this case? Br/Minghong

Re: [Discuss][DSIP] Support Merge-On-Write implementation for UNIQUE KEY data model

2022-06-23 Thread Zhu,Xiaoli
Hi Chen Zhang, I am very interested in this topic, and want to participate in the development. 在 2022/6/23 下午2:44,“Chen Zhang” 写入: Hi devs, Unique-Key data model is widely used in scenarios like Flink-CDC, user profile(用户画像), E-commerce orders, but the query performance for current

[Discuss][DSIP] Support Merge-On-Write implementation for UNIQUE KEY data model

2022-06-22 Thread Chen Zhang
Hi devs, Unique-Key data model is widely used in scenarios like Flink-CDC, user profile(用户画像), E-commerce orders, but the query performance for current Merge-On-Read implementation is not good, due to the following reasons: 1. Doris can't determine whether one row in a segment file is latest o