rahulgoswami commented on PR #14607: URL: https://github.com/apache/lucene/pull/14607#issuecomment-3325749486
>This doesnt work. Sometimes we need to force a reindex. That's because the original thing encoded was lossy (e.g. norms) and we need to change the encoding, or because it was indexed incorrectly. all the merging in the world can't fix that. For each older segment which is >=MIN_SUPPORTED_MAJOR but <LATEST, if it can be excluded from merges (eg: via a custom merge policy) and, in parallel, can be read and reindexed outside of Lucene (Eg: in Solr, etc ), the result would be a LOSSLESS index where ALL segments are LATEST version. Of course this only applies to any index where all fields are either stored or docValues enabled. Such indexes can benefit from this kind of in-place reindexing giving the ability to keep "upgrading" even in case of any back incompatible changes like the one that happened in 7.x (as long as Lucene maintains readability for LATEST-1 version segments). I did mention this basic premise for this PR on the mailing list, but in hindsight, could have re-stressed upon it. The Solr side effort is being tracked under https://issues.apache.org/jira/browse/SOLR-17725 . Once Solr can achieve the state as mentioned above (aka all segments have been converted to LATEST version), today the only thing preventing the index from opening is the check in SegmentInfos based on indexCreatedVersionMajor (https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/index/SegmentInfos.java#L349). The three approaches outlined in this PR offer alternatives to this check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
