vagetablechicken commented on issue #3469:
URL:
https://github.com/apache/incubator-doris/issues/3469#issuecomment-639358916
### Prepare Phase Detail Design
#### Sender BE:
```
Get batch from json_scanner
For partialrow in batch:
// may missing key cols(partition cols or distributed cols), should
fillin default value.
// If we want to support default values for each write txn, those
default values need to
// be pass to receiver BE(e.g. PTabletWriterOpenRequest adds a field).
And no need to
// modify rows here, coz we only need key cols here. Filling all key &
value cols
// could exec in commit phase.
calc_partition_and_tablet(partialrow, &channel, &tablet_id)
channel.add_row(partialrow, tablet_id)
```
#### Receiver BE:
```
Get batch from sender BE
For partialrow, tablet_id of batch:
get delta_writer by tablet_id
delta_writer.write(partialrow)
```
##### DeltaWriter(DeltaWriter has two mode, one for disk engine, one for
memory engine. Sets up the mode upon initialization)
```
// change arg Tuple* to void*
// disk engine MemTable will convert it to Tuple
// memory engine MemTable will convert it to PartialRow
DeltaWriter::insert(void* row):
mem_table.insert(row)
if memory_usage > config:
_flush_memtable_async()
_reset_mem_table()
```
##### WTDMemTable(WriteTxnData MemTable, flush WriteTxnData files)
Member var: _buffer, ...
```
WTDMemTable::insert(void* row):
append(_buffer, row, sizeof(uint64_t)+*(uint64_t*)row)
num_rows++;
WTDMemTable::flush():
append(_buffer, footer)
write_to_disk(_buffer)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
