Jackie-Jiang commented on code in PR #18496:
URL: https://github.com/apache/pinot/pull/18496#discussion_r3243273664
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandler.java:
##########
@@ -144,6 +142,11 @@ public void updateIndices(SegmentDirectory.Writer
segmentWriter)
String column = entry.getKey();
List<Operation> operations = entry.getValue();
+ // Backfill missing 1.6.0-era stats for this column before dispatching
ops so the per-op handlers can read
+ // them off `ColumnMetadata` when they construct their forward-index
creator. No-op when the column is
+ // fixed-width, already has the stats, or has no forward index on disk.
+ backfillMissingStats(column, segmentWriter);
Review Comment:
Good point. The pre-pass currently runs for any column with an op, including
ones whose only op is `DISABLE_FORWARD_INDEX` (forward index dropped, no
rewrite) or `ENABLE_DICT_FORWARD_INDEX` / `ENABLE_RAW_FORWARD_INDEX` rebuilds
(which use the inv→fwd creator with its own inline backfill). A precise gate
would only run the pre-pass when an op in the list will construct a new
forward-index creator that reads length stats —
`CHANGE_INDEX_COMPRESSION_TYPE`, `DISABLE_DICTIONARY` on a dict-encoded forward
index staying on, or `ENABLE_RAW_FORWARD_INDEX` when the existing forward index
is dict-encoded. Will add the gate in a follow-up commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
