Hi Gus, Thanks for taking the time to read the article.
In our case we were able to trace a few contributors. One was similarity changes in Lucene, which shifted score distributions, so we ended up adding a custom similarity to match the previous behavior. We also saw effects from negative score handling in boost functions, and some score differences for primitive fields due to similarity changes. Separately, we ran into a silent subquery parsing behavior change, which affected the candidate set rather than just scoring. The harness, in my original mail, includes a small sample corpus and queries that show these differences side-by-side (even when using the same similarity). Not suggesting these are bugs — just that the combined effects made the upgrade behavior harder to reason about without tooling like this. Regards, Parveen On Thu, Mar 12, 2026 at 5:40 AM Gus Heck <[email protected]> wrote: > Interesting article, though it leaves me wondering which changes or sets of > changes caused the problematic variations in behavior. > > On Thu, Mar 12, 2026 at 3:18 AM Parveen Saini <[email protected] > > > wrote: > > > Hi all, > > > > Following up on the earlier thread about the migration validation > approach > > for major Solr upgrades, I wrote up the full story and lessons learned > from > > the Solr 5 to 8 migration we discussed. > > > > Sharing here in case it’s useful for others planning similar upgrades: > > https://dzone.com/articles/solr5-to-solr8-migration-ads-system > > > > Best, > > Parveen > > > > On Sat, Feb 28, 2026 at 9:17 AM Parveen Saini < > [email protected] > > > > > wrote: > > > > > Hi all, > > > > > > While migrating a production system from Solr 5 to 8, we encountered > > > subtle ranking drift that did not surface in standard upgrade testing. > > The > > > system functioned correctly, but ranking behavior changed in non > obvious > > > ways. We observed score distribution shifts, candidate set differences > > > influenced by negative score handling, and p99 latency regressions. > > > > > > Using that migration as a real world case, I put together a small side > by > > > side validation harness designed to make behavioral differences across > > > major Solr versions observable. The goal is not to provide version > > specific > > > guidance, but to offer a structured approach for detecting ranking and > > > performance drift during major upgrades. > > > > > > The harness compares docset overlap, score distributions, and query > level > > > behavior across versions. > > > > > > Sharing in case it is useful for others planning major Solr upgrades: > > > https://github.com/parveensaini/solr-lucene-migration-correctness > > > > > > Happy to share more details or present the approach at an upcoming > > > community meetup if there is interest. > > > > > > Thanks, > > > Parveen > > > > > > > > -- > http://www.needhamsoftware.com (work) > https://a.co/d/b2sZLD9 (my fantasy fiction book) >
