Hello Kerwin, Firstly, hopefully you've seen the upgrade notes: https://lucene.apache.org/solr/guide/8_7/solr-upgrade-notes.html 8.6 fixes a performance regression found in 8.5; perhaps you are using 8.5?
Missing from the upgrade notes but found in the CHANGES.txt for 8.0 is hl.weightMatches=true is now the default. Try setting it to false. Does that help performance much? It's documented on the highlighting page of the ref guide: https://lucene.apache.org/solr/guide/8_7/highlighting.html#the-unified-highlighter You might want to try toggling hl.requireFieldMatch=true (defaults to false). For a query with dismax, it makes no semantic difference since all clauses target all fields, unless users know how to query only specific fields and do that. It may impact performance significantly when there are many fields. Try a matrix of toggling this and hl.weightMatches (2x2=4 tests). ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Wed, Jan 27, 2021 at 2:20 AM Kerwin <kerwin...@gmail.com> wrote: > Hi, > > While upgrading to Solr 8 from 6 the Unified highlighter begins to have > performance issues going from approximately 100ms to more than 4 seconds > with 76 fields in the hl.q and hl.fl parameters. So I played with > different options and found that the hl.q parameter needs to have any one > field for the performance issue to vanish. I do not know why this would be > so. Could you check if this is a bug or something else? This is not the > case if I use the original highlighter which has same performance on Solr 6 > and Solr 8 of ~ 1.5 seconds. The highlighting payload is also mostly same > in all the cases. > > Prior Solr 8 configuration with bad performance of > 4sec > <str name="hl.q">{!edismax qf="field1 field2 ..field76" v=$qq}</str> > <str name="hl.fl">field1 field2 ..field76</str> > > Solr 8 configuration with original Solr 6 performance of ~ 100 ms > <str name="hl.q">{!edismax qf="field1" v=$qq}</str> > <str name="hl.fl">field1 field2 ..field76</str> > > Other highlighting parameters > <str name="hl">true</str> > <str name="hl.method">unified</str> > <str name="hl.fragsize">200</str> > <str name="f.resume.content.hl.bs.type">WORD</str> > <str name="hl.bs.language">en</str> > <str name="hl.snippets">10</str> > > If I remove the hl.q parameter altogether, the performance time shoots up > to 6-7 seconds, since our user query is quite large with more fields and is > more complicated, I suspect. >
