markrmiller commented on issue #13797:
URL: https://github.com/apache/lucene/issues/13797#issuecomment-3136505419
Hi all,
So the proposal is a simple change: **stop forcing a full re‑index on every
major release unless we actually break the on‑disk format**.
All we need is to maintain one constant manually instead of recomputing
`LATEST.major ‑ 1`.
---------------------------------------------------------------------
## 1 Motivation
---------------------------------------------------------------------
* Today `MIN_SUPPORTED_MAJOR` auto‑advances on every major.
*Example:* Lucene 14 refuses any index **created** by Lucene 12 or
older—even if you’ve merged it under 13.
* Most majors don’t break format, so we make users re‑index for no technical
reason.
* Pinning the constant to the *last* breaking version lets users skip majors
when it’s safe.
---------------------------------------------------------------------
## 2 Two Compatibility Gates
---------------------------------------------------------------------
### A. Index‑Opening Gate (creation version)
`indexCreatedVersionMajor` is written once at the first commit and never
changes.
Lucene opens an index only if
`indexCreatedVersionMajor ≥ MIN_SUPPORTED_MAJOR`
### B. Codec‑Reader Gate (segment format)
Lucene ships codec readers for **current** + **previous** major versions
(e.g. 14 & 13).
If no format break happened between 12 → 13, the “13” reader also parses 12
segments.
Merging rewrites segments to the current codec but **does not** change the
creation version.
> **Key point:** merging can satisfy the codec‑reader gate, but the
creation‑version gate still applies.
> Relaxing that gate is exactly what this proposal does.
---------------------------------------------------------------------
## 3 Policy (“Lazy `MIN_SUPPORTED_MAJOR`”)
---------------------------------------------------------------------
1. **Evaluate `MIN_SUPPORTED_MAJOR` at *every* major release.**
* Lucene 11.0 sets
public static final int MIN_SUPPORTED_MAJOR = 10;
* Lucene 12, 13 & 14 *review* the value but keep it at 10 as long as no
on‑disk break occurs.
* If a breaking change lands in Lucene 15.0, we bump the constant to 14
(the last compatible major).
* **But** if we confirm that *no* format breaks occurred all the way
from 9 → 10 → 11, we _could_ choose to set the constant even lower, e.g. `9`.
(That would give users on Lucene 9 indexes a direct path to 11+, at the
cost of extra testing; opinions welcome.)
2. **Release‑manager checklist**
* “Did any commit since the last major introduce an incompatible on‑disk
change?”
* If **yes** → bump the constant. If **no** → leave it.
3. **Scope unchanged**
* We still ship codec readers for current + previous major only.
* We may bump the constant without a break if support costs spike—no new
promises are made.
---------------------------------------------------------------------
## 4 Upgrade Scenarios (after change)
---------------------------------------------------------------------
| From → To | Format break? | `MIN_SUPPORTED_MAJOR` | Opens? | What you do
|
|-------------|---------------|-----------------------|--------|-----------------------------------|
| **10 → 14** | **No** | 10 | ✅ | Open on 14;
optional `forceMerge` |
| **10 → 14** | Break at 13 | 12 | ❌ | Re‑index
(creation 10 < 12) |
| **13 → 15** | Break at 15 | 14 | ❌ | Re‑index
(creation 13 < 14) |
| **13 → 14** | — | 10 | ✅ | Opens fine
|
---------------------------------------------------------------------
## 5 Code Changes
---------------------------------------------------------------------
* **`Version.java`**
Replace the computed constant with a literal and explanatory Javadoc:
public static final int MIN_SUPPORTED_MAJOR = 10;
* **Tests**
* Update back‑compat tests to use the constant.
* Add a test that checks upgrades from allowed versions.
* **No other core changes needed** — all open‑time checks already funnel
through `Version.MIN_SUPPORTED_MAJOR`.
---------------------------------------------------------------------
## 6 Documentation Changes
---------------------------------------------------------------------
* **`CHANGES.txt`** — record the new policy and initial value (10).
* **`MIGRATE.md`** — add the table above plus a worked example (10 → 14).
* **`dev‑docs/releaseWizard.yaml`** — add the release‑manager checklist.
* **Javadoc** on `Version.MIN_SUPPORTED_MAJOR` — include rationale, bump
criteria, and examples.
* **Error messages** — `IndexFormatTooOldException` shows creation version
vs. minimum and suggest actions.
I've got an initial PR for this up as 15012.
— Mark
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
