vvivekiyer commented on code in PR #9982:
URL: https://github.com/apache/pinot/pull/9982#discussion_r1048857604
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/InvertedIndexAndDictionaryBasedForwardIndexCreator.java:
##########
@@ -218,20 +218,23 @@ public void regenerateForwardIndex()
LoaderUtils.writeIndexToV3Format(_segmentWriter, _columnName,
_forwardIndexFile, ColumnIndexType.FORWARD_INDEX);
- if (!_isTemporaryForwardIndex) {
- // Only update the metadata and cleanup other indexes if the forward
index to be created is permanent. If the
- // forward index is temporary, it is meant to be used only for
construction of other indexes and will be deleted
- // once all the IndexHandlers have completed.
- try {
- LOGGER.info("Created forward index from inverted index and dictionary.
Updating metadata properties for "
- + "segment: {}, column: {}, property list: {}", segmentName,
_columnName, metadataProperties);
- SegmentMetadataUtils.updateMetadataProperties(_segmentMetadata,
metadataProperties);
- } catch (Exception e) {
- throw new IOException(
- String.format("Failed to update metadata properties for segment:
%s, column: %s", segmentName, _columnName),
- e);
- }
+ try {
+ // Update the metadata even for temporary forward index as other
IndexHandlers may rely on the updated metadata
+ // to construct their indexes based on the forward index.
+ LOGGER.info("Created forward index from inverted index and dictionary.
Updating metadata properties for "
+ + "segment: {}, column: {}, property list: {}, is temporary: {}",
segmentName, _columnName,
+ metadataProperties, _isTemporaryForwardIndex);
+ SegmentMetadataUtils.updateMetadataProperties(_segmentMetadata,
metadataProperties);
+ } catch (Exception e) {
+ throw new IOException(
+ String.format("Failed to update metadata properties for segment: %s,
column: %s", segmentName, _columnName),
+ e);
+ }
+ if (!_isTemporaryForwardIndex) {
Review Comment:
Earlier, we weren't updating in-memory and file metadata properties for
temporary forward index.
Now that we are changing this, can you please clarify (and also add a
comment) as to how the metadata property updates would be reverted?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
