dutyu commented on code in PR #27666:
URL: https://github.com/apache/doris/pull/27666#discussion_r1446170740
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/hive/event/MetastoreEventFactory.java:
##########
@@ -77,23 +79,67 @@ List<MetastoreEvent>
getMetastoreEvents(List<NotificationEvent> events, HMSExter
for (NotificationEvent event : events) {
metastoreEvents.addAll(transferNotificationEventToMetastoreEvents(event,
catalogName));
}
- return createBatchEvents(catalogName, metastoreEvents);
+ List<MetastoreEvent> mergedEvents = mergeEvents(catalogName,
metastoreEvents);
+ if (Env.getCurrentEnv().isMaster()) {
+ logMetaIdMappings(hmsExternalCatalog.getId(),
events.get(events.size() - 1).getEventId(), mergedEvents);
+ }
+ return mergedEvents;
+ }
+
+ private void logMetaIdMappings(long catalogId, long lastSyncedEventId,
List<MetastoreEvent> mergedEvents) {
+ MetaIdMappingsLog log = new MetaIdMappingsLog();
+ log.setCatalogId(catalogId);
+ log.setFromHmsEvent(true);
+ log.setLastSyncedEventId(lastSyncedEventId);
+ for (MetastoreEvent event : mergedEvents) {
+ switch (event.getEventType()) {
+ case CREATE_DATABASE:
Review Comment:
When there are some new `create db/create table/create partition` events
being produced, master `FE` assigns new ids for this database/table/partition
and send it to slave `FE` nodes through `MetaIdMappingsLog`, so master and
slave `FE` nodes can keep consistent meta ids (dbId/tblId).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
