[GitHub] [doris] BiteTheDDDDt opened a new pull request, #12316: [Enhancement](compaction) reduce VMergeIterator copy block

[GitBox]

BiteTheDDDDt opened a new pull request, #12316:
URL: https://github.com/apache/doris/pull/12316

   # Proposed changes
   
   This pr change make VMergeIterator support return row reference to instead 
copy a full block.
   
   before:
   ```
   output_version=[2-11], 
   current_max_version=11, 
   segments=50, 
   input_row_num=30000000, 
   output_row_num=30000000. 
   elapsed time=53.7632s. 
   cumulative_compaction_policy=SIZE_BASED, 
   compact_row_per_second=558002
   ```
   <img width="1411" alt="图片" 
src="https://user-images.githubusercontent.com/7939630/188264198-792e2d8d-0f96-41b3-974f-563fc0338ccc.png";>
   
   after:
   ```
   output_version=[2-11], 
   current_max_version=11, 
   segments=50, 
   input_row_num=30000000, 
   output_row_num=30000000. 
   elapsed time=47.3122s. 
   cumulative_compaction_policy=SIZE_BASED, 
   compact_row_per_second=634086
   ```
   <img width="1439" alt="图片" 
src="https://user-images.githubusercontent.com/7939630/188264245-dd02ea25-7063-45f1-b891-acfef1f0f511.png";>
   
   
   test case:
   ```sql
   CREATE TABLE IF NOT EXISTS `customer` (
     `c_custkey` int(11) NOT NULL COMMENT "",
     `c_name` varchar(26) NOT NULL COMMENT "",
     `c_address` varchar(41) NOT NULL COMMENT "",
     `c_city` varchar(11) NOT NULL COMMENT "",
     `c_nation` varchar(16) NOT NULL COMMENT "",
     `c_region` varchar(13) NOT NULL COMMENT "",
     `c_phone` varchar(16) NOT NULL COMMENT "",
     `c_mktsegment` varchar(11) NOT NULL COMMENT ""
   ) ENGINE=OLAP
   DUPLICATE KEY(`c_custkey`)
   COMMENT "OLAP"
   DISTRIBUTED BY HASH(`c_custkey`) BUCKETS 1
   PROPERTIES (
   "replication_num" = "1",
   "colocate_with" = "groupa2",
   "in_memory" = "false",
   "storage_format" = "DEFAULT"
   );
   
   insert into customer select * from ssb.customer;
   insert into customer select * from ssb.customer;
   insert into customer select * from ssb.customer;
   insert into customer select * from ssb.customer;
   insert into customer select * from ssb.customer;
   insert into customer select * from ssb.customer;
   insert into customer select * from ssb.customer;
   insert into customer select * from ssb.customer;
   insert into customer select * from ssb.customer;
   insert into customer select * from ssb.customer;
   ```
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
       - [ ] Yes
       - [ ] No
       - [ ] I don't know
   2. Has unit tests been added:
       - [ ] Yes
       - [ ] No
       - [ ] No Need
   3. Has document been added or modified:
       - [ ] Yes
       - [ ] No
       - [ ] No Need
   4. Does it need to update dependencies:
       - [ ] Yes
       - [ ] No
   5. Are there any changes that cannot be rolled back:
       - [ ] Yes (If Yes, please explain WHY)
       - [ ] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org