imay commented on a change in pull request #1798: Optimize the load performance
for large file
URL: https://github.com/apache/incubator-doris/pull/1798#discussion_r323822695
##########
File path: be/src/olap/delta_writer.h
##########
@@ -75,10 +90,16 @@ class DeltaWriter {
RowsetSharedPtr _new_rowset;
TabletSharedPtr _new_tablet;
std::unique_ptr<RowsetWriter> _rowset_writer;
- MemTable* _mem_table;
+ std::shared_ptr<MemTable> _mem_table;
Schema* _schema;
const TabletSchema* _tablet_schema;
bool _delta_written_success;
+ std::atomic<OLAPStatus> _flush_status;
+ int64_t _flush_time_ns;
+ int64_t _flush_count;
+
+ // queue for saving immable mem tables
+ BlockingQueue<std::shared_ptr<MemTable>>* _flush_queue;
Review comment:
I think it's better to have an executor, which can be system wide or a load
wide. When DeltaWriter submit a flush request to the executor, the executor
will return a future to it. If you do that way, you need not to modify
TabletChannel, and TabletChannel don't call MemTable flush. MemTable flush
operation is StorageEngine implementation related, it is a little weird to put
it out of StorageEngine.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
