One thing to note: you can also take on wt=ruby&indent=true it makes the
debug explain data look better for pasting.
But what I am seeing is a score is being all based on that fact that it
found the content you were looking for in an unboosted field, i.e.
*_text_*, so your boosts don't look to be having any value in scoring the
way you are setup currently. Next if you look at what is creating the
score difference you can see its being computed from the tf-Norm values.
But maybe pasting in a cleaning version of the debug cause getting all
scoring lined up is a bit of a pain. You can get it to look something like
this with the wt=ruby&indent=true:
'
10.541302 = (MATCH) sum of:
10.541302 = (MATCH) max plus 0.01 times others of:
10.518621 = (MATCH) weight(ngram_tags_a:"developer group"~1 in 88)
[DefaultSimilarity], result of:
10.518621 = score(doc=88,freq=1.0), product of:
0.64834845 = queryWeight, product of:
16.223717 = idf(), sum of:
9.416559 = idf(docFreq=21, maxDocs=99469)
6.8071575 = idf(docFreq=298, maxDocs=99469)
0.039963003 = queryNorm
16.223717 = fieldWeight in 88, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = phraseFreq=1.0
16.223717 = idf(), sum of:
9.416559 = idf(docFreq=21, maxDocs=99469)
6.8071575 = idf(docFreq=298, maxDocs=99469)
1.0 = fieldNorm(doc=88)
0.32740274 = (MATCH) weight(ngram_content:"developer group"~1 in 88)
[DefaultSimilarity], result of:
0.32740274 = score(doc=88,freq=2.0), product of:
0.38474476 = queryWeight, product of:
9.627523 = idf(), sum of:
6.387304 = idf(docFreq=454, maxDocs=99469)
3.240219 = idf(docFreq=10586, maxDocs=99469)
0.039963003 = queryNorm
0.85096085 = fieldWeight in 88, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = phraseFreq=2.0
9.627523 = idf(), sum of:
6.387304 = idf(docFreq=454, maxDocs=99469)
3.240219 = idf(docFreq=10586, maxDocs=99469)
0.0625 = fieldNorm(doc=88)
1.9406005 = (MATCH) weight(ngram_label:"developer group"~1 in 88)
[DefaultSimilarity], result of:
1.9406005 = score(doc=88,freq=1.0), product of:
0.556964 = queryWeight, product of:
13.936991 = idf(), sum of:
9.3721075 = idf(docFreq=22, maxDocs=99469)
4.5648837 = idf(docFreq=2814, maxDocs=99469)
0.039963003 = queryNorm
3.4842477 = fieldWeight in 88, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = phraseFreq=1.0
13.936991 = idf(), sum of:
9.3721075 = idf(docFreq=22, maxDocs=99469)
4.5648837 = idf(docFreq=2814, maxDocs=99469)
0.25 = fieldNorm(doc=88)
'
Also dont know what Solr version you may be using so you explain data might
look a bit different.
This link is a bit out of date but may help you understand how the scoring
works:
https://wiki.apache.org/solr/SolrRelevancyFAQ#How_are_documents_scored
Nick
On Mon, May 9, 2016 at 8:08 PM, Megha Bhandari <mbhanda...@sapient.com>
wrote:
> To clarify on the debug information given earlier , we changed the query
> factor to the following to ignore title field completely
>
> metatag.description^9 h1^7 h2^6 h3^5 h4^4 _text_^1 id^0.5"
>
> But still title results are coming on top
>
> Full response with debug on:
>
> Full response
>
> {
> "responseHeader":{
> "status":0,
> "QTime":13,
> "params":{
> "mm":"100%",
> "q":"Foo",
> "tie":"0.99",
> "defType":"edismax",
> "q.alt":"Foo",
> "indent":"on",
> "qf":"metatag.description^9 h1^7 h2^6 h3^5 h4^4 _text_^1 id^0.5",
> "wt":"json",
> "debugQuery":"on",
> "_":"1462810987788"}},
> "response":{"numFound":3,"start":0,"maxScore":0.8430033,"docs":[
> {
> "h2":["Looks like your browser is a little out-of-date."],
> "h3":["Already a member?"],
> "title":"Foo Custon",
> "id":"
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/Foo-custon.html
> ",
> "tstamp":"2016-05-09T17:15:57.604Z",
> "metatag.hideininternalsearch":[false],
> "segment":[20160509224553],
> "digest":["844296a63233b3e4089424fe1ec9d036"],
> "boost":[1.4142135],
> "lang":"en",
> "_version_":1533871839698223104,
> "host":"localhost",
> "url":"
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/Foo-custon.html
> ",
> "score":0.8430033},
> {
> "metatag.description":"test",
> "h1":["Health care"],
> "h2":["Looks like your browser is a little out-of-date."],
> "h3":["Already a member?"],
> "title":"Foo",
> "id":"
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/Foo-custon/health-care.html
> ",
> "tstamp":"2016-05-09T17:15:57.838Z",
> "metatag.hideininternalsearch":[false],
> "metatag.topresultthumbnailalt":[","],
> "segment":[20160509224553],
> "digest":["dd4ef8879be2d4d3f28e24928e9b84c5"],
> "boost":[1.4142135],
> "lang":"en",
> "metatag.keywords":",",
> "_version_":1533871839731777536,
> "host":"localhost",
> "url":"
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/Foo-custon/health-care.html
> ",
> "score":0.7009616},
> {
> "metatag.description":"Foo decription testing",
> "h1":["healthcare description"],
> "h2":["Looks like your browser is a little out-of-date."],
> "h3":["Already a member?"],
> "title":"healthcare description",
> "id":"
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/Foo-custon/healthcare-description.html
> ",
> "tstamp":"2016-05-09T17:15:57.682Z",
> "metatag.hideininternalsearch":[false],
> "metatag.topresultthumbnailalt":[","],
> "segment":[20160509224553],
> "digest":["6262795db6aed05a5de7cc3cbe496401"],
> "boost":[1.4142135],
> "lang":"en",
> "metatag.keywords":",",
> "_version_":1533871839739117568,
> "host":"localhost",
> "url":"
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/Foo-custon/healthcare-description.html
> ",
> "score":0.5481102}]
> },
> "debug":{
> "rawquerystring":"Foo",
> "querystring":"Foo",
> "parsedquery":"(+DisjunctionMaxQuery(((metatag.description:Foo)^9.0 |
> (h1:Foo)^7.0 | (h2:Foo)^6.0 | (h3:Foo)^5.0 | (id:Foo)^0.5 | (h4:Foo)^4.0 |
> _text_:Foo)~0.99))/no_coord",
> "parsedquery_toString":"+((metatag.description:Foo)^9.0 | (h1:Foo)^7.0
> | (h2:Foo)^6.0 | (h3:Foo)^5.0 | (id:Foo)^0.5 | (h4:Foo)^4.0 |
> _text_:Foo)~0.99",
> "explain":{
> "
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/Foo-custon.html":"\n0.84300333
> = max plus 0.99 times others of:\n 0.84300333 = weight(_text_:Foo in 0)
> [], result of:\n 0.84300333 = score(doc=0,freq=6.0 = termFreq=6.0\n),
> product of:\n 0.44183275 = idf(docFreq=4, docCount=6)\n 1.9079694
> = tfNorm, computed from:\n 6.0 = termFreq=6.0\n 1.2 =
> parameter k1\n 0.75 = parameter b\n 121.666664 =
> avgFieldLength\n 83.591835 = fieldLength\n",
> "
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/Foo-custon/health-care.html":"\n0.7009616
> = max plus 0.99 times others of:\n 0.7009616 = weight(_text_:Foo in 3) [],
> result of:\n 0.7009616 = score(doc=3,freq=2.0 = termFreq=2.0\n), product
> of:\n 0.44183275 = idf(docFreq=4, docCount=6)\n 1.5864862 =
> tfNorm, computed from:\n 2.0 = termFreq=2.0\n 1.2 = parameter
> k1\n 0.75 = parameter b\n 121.666664 = avgFieldLength\n
> 64.0 = fieldLength\n",
> "
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/Foo-custon/healthcare-description.html":"\n0.5481102
> = max plus 0.99 times others of:\n 0.5481102 = weight(_text_:Foo in 4) [],
> result of:\n 0.5481102 = score(doc=4,freq=1.0 = termFreq=1.0\n), product
> of:\n 0.44183275 = idf(docFreq=4, docCount=6)\n 1.2405376 =
> tfNorm, computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter
> k1\n 0.75 = parameter b\n 121.666664 = avgFieldLength\n
> 64.0 = fieldLength\n"},
> "QParser":"ExtendedDismaxQParser",
> "altquerystring":null,
> "boost_queries":null,
> "parsed_boost_queries":[],
> "boostfuncs":null,
> "timing":{
> "time":12.0,
> "prepare":{
> "time":1.0,
> "query":{
> "time":1.0},
> "facet":{
> "time":0.0},
> "facet_module":{
> "time":0.0},
> "mlt":{
> "time":0.0},
> "highlight":{
> "time":0.0},
> "stats":{
> "time":0.0},
> "expand":{
> "time":0.0},
> "debug":{
> "time":0.0}},
> "process":{
> "time":10.0,
> "query":{
> "time":4.0},
> "facet":{
> "time":0.0},
> "facet_module":{
> "time":0.0},
> "mlt":{
> "time":0.0},
> "highlight":{
> "time":0.0},
> "stats":{
> "time":0.0},
> "expand":{
> "time":0.0},
> "debug":{
> "time":6.0}}}}}
>
>
> -----Original Message-----
> From: Megha Bhandari [mailto:mbhanda...@sapient.com]
> Sent: Tuesday, May 10, 2016 8:10 AM
> To: solr-user@lucene.apache.org
> Subject: RE: Solr edismax field boosting
>
> Hi
>
> Following is the debug information with debug=true
>
> Excerpt of debug information :
>
> "debug":{
> "rawquerystring":"Upendra",
> "querystring":"Upendra",
>
> "parsedquery":"(+DisjunctionMaxQuery(((metatag.description:Upendra)^9.0 |
> (h1:Upendra)^7.0 | (h2:Upendra)^6.0 | (h3:Upendra)^5.0 | (id:Upendra)^0.5 |
> (h4:Upendra)^4.0 | _text_:upendra)~0.99))/no_coord",
> "parsedquery_toString":"+((metatag.description:Upendra)^9.0 |
> (h1:Upendra)^7.0 | (h2:Upendra)^6.0 | (h3:Upendra)^5.0 | (id:Upendra)^0.5 |
> (h4:Upendra)^4.0 | _text_:upendra)~0.99",
> "explain":{
> "
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/upendra-custon.html":"\n0.84300333
> = max plus 0.99 times others of:\n 0.84300333 = weight(_text_:upendra in
> 0) [], result of:\n 0.84300333 = score(doc=0,freq=6.0 = termFreq=6.0\n),
> product of:\n 0.44183275 = idf(docFreq=4, docCount=6)\n 1.9079694
> = tfNorm, computed from:\n 6.0 = termFreq=6.0\n 1.2 =
> parameter k1\n 0.75 = parameter b\n 121.666664 =
> avgFieldLength\n 83.591835 = fieldLength\n",
> "
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/upendra-custon/health-care.html":"\n0.7009616
> = max plus 0.99 times others of:\n 0.7009616 = weight(_text_:upendra in 3)
> [], result of:\n 0.7009616 = score(doc=3,freq=2.0 = termFreq=2.0\n),
> product of:\n 0.44183275 = idf(docFreq=4, docCount=6)\n 1.5864862
> = tfNorm, computed from:\n 2.0 = termFreq=2.0\n 1.2 =
> parameter k1\n 0.75 = parameter b\n 121.666664 =
> avgFieldLength\n 64.0 = fieldLength\n",
> "
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/upendra-custon/healthcare-description.html":"\n0.5481102
> = max plus 0.99 times others of:\n 0.5481102 = weight(_text_:upendra in 4)
> [], result of:\n 0.5481102 = score(doc=4,freq=1.0 = termFreq=1.0\n),
> product of:\n 0.44183275 = idf(docFreq=4, docCount=6)\n 1.2405376
> = tfNorm, computed from:\n 1.0 = termFreq=1.0\n 1.2 =
> parameter k1\n 0.75 = parameter b\n 121.666664 =
> avgFieldLength\n 64.0 = fieldLength\n"},
> "QParser":"ExtendedDismaxQParser",
> "altquerystring":null,
> "boost_queries":null,
>
> ------------------------
> Full response
>
> {
> "responseHeader":{
> "status":0,
> "QTime":13,
> "params":{
> "mm":"100%",
> "q":"Upendra",
> "tie":"0.99",
> "defType":"edismax",
> "q.alt":"Upendra",
> "indent":"on",
> "qf":"metatag.description^9 h1^7 h2^6 h3^5 h4^4 _text_^1 id^0.5",
> "wt":"json",
> "debugQuery":"on",
> "_":"1462810987788"}},
> "response":{"numFound":3,"start":0,"maxScore":0.8430033,"docs":[
> {
> "h2":["Looks like your browser is a little out-of-date."],
> "h3":["Already a member?"],
> "strtitle":"Upendra Custon",
> "id":"
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/upendra-custon.html
> ",
> "tstamp":"2016-05-09T17:15:57.604Z",
> "metatag.hideininternalsearch":[false],
> "segment":[20160509224553],
> "digest":["844296a63233b3e4089424fe1ec9d036"],
> "boost":[1.4142135],
> "lang":"en",
> "_version_":1533871839698223104,
> "host":"localhost",
> "url":"
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/upendra-custon.html
> ",
> "score":0.8430033},
> {
> "metatag.description":"test",
> "h1":["Health care"],
> "h2":["Looks like your browser is a little out-of-date."],
> "h3":["Already a member?"],
> "strtitle":"Upendra",
> "id":"
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/upendra-custon/health-care.html
> ",
> "tstamp":"2016-05-09T17:15:57.838Z",
> "metatag.hideininternalsearch":[false],
> "metatag.topresultthumbnailalt":[","],
> "segment":[20160509224553],
> "digest":["dd4ef8879be2d4d3f28e24928e9b84c5"],
> "boost":[1.4142135],
> "lang":"en",
> "metatag.keywords":",",
> "_version_":1533871839731777536,
> "host":"localhost",
> "url":"
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/upendra-custon/health-care.html
> ",
> "score":0.7009616},
> {
> "metatag.description":"Upendra decription testing",
> "h1":["healthcare description"],
> "h2":["Looks like your browser is a little out-of-date."],
> "h3":["Already a member?"],
> "strtitle":"healthcare description",
> "id":"
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/upendra-custon/healthcare-description.html
> ",
> "tstamp":"2016-05-09T17:15:57.682Z",
> "metatag.hideininternalsearch":[false],
> "metatag.topresultthumbnailalt":[","],
> "segment":[20160509224553],
> "digest":["6262795db6aed05a5de7cc3cbe496401"],
> "boost":[1.4142135],
> "lang":"en",
> "metatag.keywords":",",
> "_version_":1533871839739117568,
> "host":"localhost",
> "url":"
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/upendra-custon/healthcare-description.html
> ",
> "score":0.5481102}]
> },
> "debug":{
> "rawquerystring":"Upendra",
> "querystring":"Upendra",
>
> "parsedquery":"(+DisjunctionMaxQuery(((metatag.description:Upendra)^9.0 |
> (h1:Upendra)^7.0 | (h2:Upendra)^6.0 | (h3:Upendra)^5.0 | (id:Upendra)^0.5 |
> (h4:Upendra)^4.0 | _text_:upendra)~0.99))/no_coord",
> "parsedquery_toString":"+((metatag.description:Upendra)^9.0 |
> (h1:Upendra)^7.0 | (h2:Upendra)^6.0 | (h3:Upendra)^5.0 | (id:Upendra)^0.5 |
> (h4:Upendra)^4.0 | _text_:upendra)~0.99",
> "explain":{
> "
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/upendra-custon.html":"\n0.84300333
> = max plus 0.99 times others of:\n 0.84300333 = weight(_text_:upendra in
> 0) [], result of:\n 0.84300333 = score(doc=0,freq=6.0 = termFreq=6.0\n),
> product of:\n 0.44183275 = idf(docFreq=4, docCount=6)\n 1.9079694
> = tfNorm, computed from:\n 6.0 = termFreq=6.0\n 1.2 =
> parameter k1\n 0.75 = parameter b\n 121.666664 =
> avgFieldLength\n 83.591835 = fieldLength\n",
> "
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/upendra-custon/health-care.html":"\n0.7009616
> = max plus 0.99 times others of:\n 0.7009616 = weight(_text_:upendra in 3)
> [], result of:\n 0.7009616 = score(doc=3,freq=2.0 = termFreq=2.0\n),
> product of:\n 0.44183275 = idf(docFreq=4, docCount=6)\n 1.5864862
> = tfNorm, computed from:\n 2.0 = termFreq=2.0\n 1.2 =
> parameter k1\n 0.75 = parameter b\n 121.666664 =
> avgFieldLength\n 64.0 = fieldLength\n",
> "
> http://localhost:4503/content/uhcdotcom/en/home/waysin/poc/upendra-custon/healthcare-description.html":"\n0.5481102
> = max plus 0.99 times others of:\n 0.5481102 = weight(_text_:upendra in 4)
> [], result of:\n 0.5481102 = score(doc=4,freq=1.0 = termFreq=1.0\n),
> product of:\n 0.44183275 = idf(docFreq=4, docCount=6)\n 1.2405376
> = tfNorm, computed from:\n 1.0 = termFreq=1.0\n 1.2 =
> parameter k1\n 0.75 = parameter b\n 121.666664 =
> avgFieldLength\n 64.0 = fieldLength\n"},
> "QParser":"ExtendedDismaxQParser",
> "altquerystring":null,
> "boost_queries":null,
> "parsed_boost_queries":[],
> "boostfuncs":null,
> "timing":{
> "time":12.0,
> "prepare":{
> "time":1.0,
> "query":{
> "time":1.0},
> "facet":{
> "time":0.0},
> "facet_module":{
> "time":0.0},
> "mlt":{
> "time":0.0},
> "highlight":{
> "time":0.0},
> "stats":{
> "time":0.0},
> "expand":{
> "time":0.0},
> "debug":{
> "time":0.0}},
> "process":{
> "time":10.0,
> "query":{
> "time":4.0},
> "facet":{
> "time":0.0},
> "facet_module":{
> "time":0.0},
> "mlt":{
> "time":0.0},
> "highlight":{
> "time":0.0},
> "stats":{
> "time":0.0},
> "expand":{
> "time":0.0},
> "debug":{
> "time":6.0}}}}}
>
>
>
>
>
>
>
>
> Regards
> Megha
>
> -----Original Message-----
> From: Nick D [mailto:ndrake0...@gmail.com]
> Sent: Tuesday, May 10, 2016 7:58 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr edismax field boosting
>
> You can add the debug flag to the end of the request and see exactly what
> the scoring is and why things are happening.
>
> &debug=ALL will show you everything including the scoring.
>
> Showing the result of the debug query should help you, or adding that into
> your question here, decipher what is going on with your scoring and how the
> boosts are(n't) working.
>
> Nick
>
> On Mon, May 9, 2016 at 7:22 PM, Megha Bhandari <mbhanda...@sapient.com>
> wrote:
>
> > Hi
> >
> > We are trying to boost certain fields with relevancy. However we are not
> > getting results as per expectation. Below is the configuration in
> > solr-config.xml.
> > Even though the title field has a lesser boost than metatag.description
> > results for title field are coming higher.
> >
> > We even created test data that have data only in description in
> > metatag.description and title. Example , page 1 has foo in description
> and
> > page 2 has foo in title. Solr is still returning page 2 before page 1.
> >
> > We are using Solr 5.5 and Nutch 1.11 currently.
> >
> > Following is the configuration we are using. Any ideas on what we are
> > missing to enable correct field boosting?
> >
> > <requestHandler name="/elevate" class="solr.SearchHandler"
> startup="lazy">
> > <lst name="defaults">
> > <str name="qf">
> > metatag.keywords^10 metatag.description^9 title^8 h1^7 h2^6 h3^5
> > h4^4 id _text_^1
> > </str>
> > <str name="echoParams">explicit</str>
> > <int name="rows">10</int>
> >
> > <!-- Below Entry for Search Suggestions -->
> >
> > <str name="echoParams">explicit</str>
> > <str name="df">_text_</str>
> > <str name="spellcheck.dictionary">default</str>
> > <str name="spellcheck">on</str>
> > <str name="spellcheck.extendedResults">false</str>
> > <str name="spellcheck.count">10</str>
> > <str name="spellcheck.alternativeTermCount">5</str>
> > <str name="spellcheck.maxResultsForSuggest">5</str>
> > <str name="spellcheck.collate">false</str>
> > <str name="spellcheck.collateExtendedResults">true</str>
> > <str name="spellcheck.maxCollationTries">10</str>
> > <str name="spellcheck.maxCollations">5</str>
> > <!-- Entry for highlighting of search keyword -->
> > <str name="fl">id title metatag.description itemtype
> > lang metatag.hideininternalsearch metatag.topresultthumbnailalt
> > metatag.topresultthumbnailurl playerid playerkey</str>
> > <str name="hl">on</str>
> > <str name="hl.fragsize">0</str>
> > <str name="hl.fl">title metatag.description</str>
> > <str name="hl.simple.pre"><![CDATA[<strong>]]></str>
> > <str name="hl.simple.post"><![CDATA[</strong>]]></str>
> > </lst>
> > <arr name="last-components">
> > <str>spellcheck</str>
> > <str>elevator</str>
> > </arr>
> > </requestHandler>
> >
> > Thanks
> > Megha
> >
>
