vagetablechicken edited a comment on issue #3469:
URL:
https://github.com/apache/incubator-doris/issues/3469#issuecomment-626553115
### Data Flow
1. User data(json format) -> PartialRow
1. PartialRow(Type,...), Type: INSERT/UPSERT/DELETE/UPDATE
1. In POC: PartialRow(cells[0],...), cells[0].isset == 1 means DELETE.
Don't distinguish between INSERT and UPDATE.
1. PartialRowBatch ---transport---> ... --> DeltaWriter::write(PartialRow
row)
We focus on DeltaWriter.
#### DeltaWriter
About DeltaWriter, needs a new class, effects similar to MemTable,
temporarily named WALMemTable. Flush WAL when DeltaWriter closed.
1. DeltaWriter prepare works
1. DeltaWriter::write(PartialRow row)
1. DeltaWriter::close()
1. WALMemTable::flush()
1. WAL file format(the simplest): one line is a partial row, saved in
{data_dir}/wal/{shard_id}/{tablet_id}/{schema_hash}/, filename: {txn_id}.wal
1. DeltaWriter::close_wait()
#### Question
DeltaWriter needs to check the partial row's key.
But there are several specific problems to be dealt with:
1. update a non-existed row: we don't distinguish INSERT & UPDATE, so it's
fine.
1. the row which is missing some key columns: see below.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
