Jackie-Jiang commented on a change in pull request #8078:
URL: https://github.com/apache/pinot/pull/8078#discussion_r795947645
##########
File path:
pinot-spi/src/main/java/org/apache/pinot/spi/config/table/SegmentsValidationAndRetentionConfig.java
##########
@@ -29,6 +29,7 @@
public class SegmentsValidationAndRetentionConfig extends BaseJsonConfig {
private String _retentionTimeUnit;
private String _retentionTimeValue;
+ private String _deletedSegmentRetentionInDays;
Review comment:
Let's make it `_deletedSegmentRetentionPeriod` (e.g. `1d`, `12h`) to be
more flexible
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java
##########
@@ -703,7 +703,14 @@ public synchronized PinotResourceManagerResponse
deleteSegments(String tableName
Preconditions.checkArgument(TableNameBuilder.isTableResource(tableNameWithType),
"Table name: %s is not a valid table name with type suffix",
tableNameWithType);
HelixHelper.removeSegmentsFromIdealState(_helixZkManager,
tableNameWithType, segmentNames);
- _segmentDeletionManager.deleteSegments(tableNameWithType, segmentNames);
+ TableConfig tableConfig =
ZKMetadataProvider.getTableConfig(_propertyStore, tableNameWithType);
+ String deletedSegmentRetentionString =
tableConfig.getValidationConfig().getDeletedSegmentRetentionInDays();
+ if (deletedSegmentRetentionString != null) {
+ _segmentDeletionManager.deleteSegments(tableNameWithType, segmentNames,
+ Integer.parseInt(deletedSegmentRetentionString));
Review comment:
Add a try catch around this value parsing. Log a warning if the value is
invalid, then fall back to the default retention. We don't want to fail the
segment deletion by an invalid retention config
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
