gautamworah96 opened a new pull request #220: URL: https://github.com/apache/lucene/pull/220
Category documents added in the Lucene 9.0 taxonomy index use a BDV field with a different name Using BDV fields with a different "$full_path_binary$" name ensures that the earlier "$full_path$" StringField does not have the same name as the BDV field and hence they don't violate the field type consistency check (LUCENE-9334). This commit also enables the back-compat check that was disabled earlier. https://issues.apache.org/jira/browse/LUCENE-9450 <!-- _(If you are a project committer then you may remove some/all of the following template.)_ Before creating a pull request, please file an issue in the ASF Jira system for Lucene: * https://issues.apache.org/jira/projects/LUCENE You will need to create an account in Jira in order to create an issue. The title of the PR should reference the Jira issue number in the form: * LUCENE-####: <short description of problem or changes> LUCENE must be fully capitalized. A short description helps people scanning pull requests for items they can work on. Properly referencing the issue in the title ensures that Jira is correctly updated with code review comments and commits. --> # Solution There were two proposed solutions in the JIRA ticket: 1. Add the BDV field with a different name. When we were adding the BDV field with the same `Consts.FULL` name, it was causing a ` java.lang.IllegalArgumentException: cannot change field "$full_path$" from doc values type=NONE to inconsistent doc values type=BINARY ` error because the current logic checks all fields with the same name across segments and ensures that they use the same BinaryDocValues field TYPE. Adding the BDV field with a different name ensures that the [check](https://github.com/apache/lucene/blob/28ba8b77975fc1b5a1a07da373916a2b21ea09aa/lucene/core/src/java/org/apache/lucene/index/FieldInfo.java#L260) does not trip. We are careful here to use the same new name when trying to retrieve `values` in the `DirectoryTaxonomyReader` 2. Perform a check on the index version when we try to add a BDV field. If the index is pre 9.0 we only add the StringField and use only that field when trying to read the value from the index. If the index is newer (>=9.0), we add and read the value from a BDV field. This PR implements the approach described in step 1. # Tests Enabled the back-compat test in `TestBackwardsCompatibility.testCreateNewTaxonomy` # Checklist Please review the following and check all that apply: - [x] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/lucene/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [x] I have created a Jira issue and added the issue ID to my pull request title. - [x] I have given Lucene maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [x] I have developed this patch against the `main` branch. - [x] I have run `./gradlew check`. - [x] I have added tests for my changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
