Aravind-Suresh commented on code in PR #11183:
URL: https://github.com/apache/pinot/pull/11183#discussion_r1300782094
##########
pinot-server/src/main/java/org/apache/pinot/server/starter/helix/HelixInstanceDataManager.java:
##########
@@ -509,6 +544,44 @@ public void addOrReplaceSegment(String tableNameWithType,
String segmentName)
}
}
+ @Override
+ public void reloadTable(String tableNameWithType, boolean force,
SegmentRefreshSemaphore segmentRefreshSemaphore) {
+ TableConfig tableConfig =
ZKMetadataProvider.getTableConfig(_propertyStore, tableNameWithType);
+ AtomicReference<TableDataManager> toBeShutdown = new AtomicReference<>();
+ _tableDataManagerMap.computeIfPresent(tableNameWithType, (k, tdm) -> {
+ // create only if a table data manager doesn't exist
+ // or the current configuration and the new configuration are different
+ if (!force &&
tdm.getTableDataManagerConfig().getTableConfig().equals(tableConfig)) {
+ LOGGER.info("Not reloading table as the table data manager config is
not changed");
+ return tdm;
+ }
+ // this will load all the segments and start a new consuming segment
+ // for the time being we will be consuming 2X the memory here
+ // once all the segments are loaded we swap the new table data manager
with the old one
+ LOGGER.info("Recreating table data manager for table: {} as we received
a reload table request",
+ tableNameWithType);
+ TableDataManager newTdm = createTableDataManager(k, tableConfig);
+ List<SegmentDataManager> segments = tdm.acquireAllSegments();
+ try {
+ snapshotAndAddSegments(newTdm, tableNameWithType);
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ for (SegmentDataManager sdm : segments) {
+ tdm.releaseSegment(sdm);
+ }
+ LOGGER.info("All segments loaded for table: {}", tableNameWithType);
+ toBeShutdown.set(tdm);
+ return newTdm;
+ });
+ // shutting the old table data manager
+ TableDataManager oldTdm = toBeShutdown.get();
Review Comment:
True - added a shutdown with a delay for now. We can accurately track and do
an auto-shutdown based on reference counting later (in a follow-up PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
