xiangfu0 commented on code in PR #11574:
URL: https://github.com/apache/pinot/pull/11574#discussion_r1350784024
##########
pinot-controller/src/main/java/org/apache/pinot/controller/BaseControllerStarter.java:
##########
@@ -549,6 +556,83 @@ protected void configure() {
_serviceStatusCallbackList.add(generateServiceStatusCallback(_helixParticipantManager));
}
+ /**
+ * This method is used to fix table/schema names.
+ * TODO: in the next release, maybe 2.0.0, we can remove this method.
Meanwhile we can delete the orphan schemas
+ * that has been existed longer than a certain time period.
+ *
+ */
+ @VisibleForTesting
+ public void fixSchemaNameInTableConfig() {
+ AtomicInteger fixedTableCount = new AtomicInteger();
+ AtomicInteger failedToFixTableCount = new AtomicInteger();
+ AtomicInteger misConfiguredTableCount = new AtomicInteger();
+ List<String> allTables = _helixResourceManager.getAllTables();
+
+ allTables.forEach(table -> {
+ TableConfig tableConfig = _helixResourceManager.getTableConfig(table);
+ if ((tableConfig == null) || (tableConfig.getValidationConfig() ==
null)) {
+ LOGGER.warn("Failed to find table config for table: {}", table);
+ failedToFixTableCount.getAndIncrement();
Review Comment:
removed.
##########
pinot-controller/src/main/java/org/apache/pinot/controller/BaseControllerStarter.java:
##########
@@ -549,6 +556,83 @@ protected void configure() {
_serviceStatusCallbackList.add(generateServiceStatusCallback(_helixParticipantManager));
}
+ /**
+ * This method is used to fix table/schema names.
+ * TODO: in the next release, maybe 2.0.0, we can remove this method.
Meanwhile we can delete the orphan schemas
+ * that has been existed longer than a certain time period.
+ *
+ */
+ @VisibleForTesting
+ public void fixSchemaNameInTableConfig() {
+ AtomicInteger fixedTableCount = new AtomicInteger();
+ AtomicInteger failedToFixTableCount = new AtomicInteger();
+ AtomicInteger misConfiguredTableCount = new AtomicInteger();
+ List<String> allTables = _helixResourceManager.getAllTables();
+
+ allTables.forEach(table -> {
+ TableConfig tableConfig = _helixResourceManager.getTableConfig(table);
+ if ((tableConfig == null) || (tableConfig.getValidationConfig() ==
null)) {
+ LOGGER.warn("Failed to find table config for table: {}", table);
+ failedToFixTableCount.getAndIncrement();
+ return;
+ }
+ String rawTableName =
TableNameBuilder.extractRawTableName(tableConfig.getTableName());
+ String existSchemaName =
tableConfig.getValidationConfig().getSchemaName();
+ if (existSchemaName == null) {
+ // Although the table config is valid, we still need to ensure the
schema exists
+ if (_helixResourceManager.getSchema(rawTableName) == null) {
+ LOGGER.warn("Failed to find schema for table: {}", rawTableName);
+ failedToFixTableCount.getAndIncrement();
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
