[GitHub] [doris] nextdreamblue commented on a diff in pull request #15018: [fix](merge-on-write) calc delete bitmap need all segments which _do_flush in one memtable

Wed, 14 Dec 2022 19:46:47 -0800 


nextdreamblue commented on code in PR #15018:
URL: https://github.com/apache/doris/pull/15018#discussion_r1049187325


##########
be/src/olap/memtable.h:
##########
@@ -224,6 +224,8 @@ class MemTable {
     DeleteBitmapPtr _delete_bitmap;
     RowsetIdUnorderedSet _rowset_ids;
     int64_t _cur_max_version;
+    // seg ids in one _do_flush for calc delete bitmap
+    std::vector<int64_t> _seg_ids;

Review Comment:
   @zhannngchen 
   貌似通过segment num来判断不大行，如果打开segment compaction，segment个数其实是一直上下浮动的。
   flush_single_memtable(block)这个地方一开始就会判断是否可以进行segment 
compaction，如果可以就会触发一组segment 
compaction，会导致num_segments()不准，比如可能10个合并成1个，num_segements()返回1，继续新插入的segement的id是继续递增的，11/12/13...，必须使用这个真实的序号去获取segment的文件去load_segment才能读到正确的data文件，否则可能会读到已经被合并的不存在的文件，或者重复读取老segment导致仍有遗漏。
   所以这个地方必须精确拿到每次执行时新增的，未被compaction的segement的真实id才行



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Reply via email to