Jackie-Jiang commented on issue #10951:
URL: https://github.com/apache/pinot/issues/10951#issuecomment-1601955203
Checking the code in `SegmentPurger` and find the following note:
```
// The time column type info is not stored in the segment metadata.
// Keep segment start/end time to properly handle time column type
other than EPOCH (e.g.SIMPLE_FORMAT).
if (segmentMetadata.getTimeInterval() != null) {
config.setTimeColumnName(_tableConfig.getValidationConfig().getTimeColumnName());
config.setStartTime(Long.toString(segmentMetadata.getStartTime()));
config.setEndTime(Long.toString(segmentMetadata.getEndTime()));
config.setSegmentTimeUnit(segmentMetadata.getTimeUnit());
}
```
I think at the time when we added the purge task (in 2018), schema might not
be available in the cluster, and we don't have the time type info (more context
in #2846). Now with #10869 we always use the schema from ZK to generate the new
segment, so we can safely remove these special handling to reflect the actual
time range
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
