the parsedquery is same when debugging, but when calculating the scores
different fields are being taken into consideration. Why would that be the
case? My guess is that the suggeststopfilterfactory is not working as i
expect it to and causing this weird situation.
Updated field type definition:
<fieldType name="text_no_specialchars" class="solr.TextField"> <analyzer> <
charFilter class="solr.PatternReplaceCharFilterFactory" pattern=
"['!#\$%'\(\)\*+,-\./:;=?@\[\]\^_`{|}~!@#$%^*]" /> <tokenizer class=
"solr.StandardTokenizerFactory"/> <filter class=
"solr.SuggestStopFilterFactory" ignoreCase="true" words=
"lang/stopwords_en.txt" /> <filter class="solr.LowerCaseFilterFactory"/> <
filter class="solr.EnglishPossessiveFilterFactory"/> <filter class=
"solr.KeywordMarkerFilterFactory" protected="protwords.txt"/> </analyzer> </
fieldType>
Debug Query:
*"rawquerystring":"how do i add a field"*,
"querystring":"how do i add a field",
"parsedquery":"(+(DisjunctionMaxQuery((topic_title_plain:how))
DisjunctionMaxQuery((topic_title_plain:do))
DisjunctionMaxQuery((topic_title_plain:i))
DisjunctionMaxQuery((topic_title_plain:add))
DisjunctionMaxQuery((topic_title_plain:a))
DisjunctionMaxQuery((topic_title_plain:field))))/no_coord",
"parsedquery_toString":"+((topic_title_plain:how)
(topic_title_plain:do) (topic_title_plain:i) (topic_title_plain:add)
(topic_title_plain:a) (topic_title_plain:field))",
"explain":{
"1":"
6.1034017 = sum of:
2.0065408 = weight(topic_title_plain:add in 107) [SchemaSimilarity],
result of:
2.0065408 = score(doc=107,freq=1.0 = termFreq=1.0
), product of:
2.1391609 = idf, computed as log(1 + (docCount - docFreq + 0.5) /
(docFreq + 0.5)) from:
32.0 = docFreq
275.0 = docCount
0.9380037 = tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 -
b + b * fieldLength / avgFieldLength)) from:
1.0 = termFreq=1.0
1.2 = parameter k1
0.75 = parameter b
3.4436364 = avgFieldLength
4.0 = fieldLength
4.096861 = weight(topic_title_plain:field in 107) [SchemaSimilarity],
result of:
4.096861 = score(doc=107,freq=1.0 = termFreq=1.0
), product of:
4.367638 = idf, computed as log(1 + (docCount - docFreq + 0.5) /
(docFreq + 0.5)) from:
3.0 = docFreq
275.0 = docCount
0.9380037 = tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 -
b + b * fieldLength / avgFieldLength)) from:
1.0 = termFreq=1.0
1.2 = parameter k1
0.75 = parameter b
3.4436364 = avgFieldLength
4.0 = fieldLength
"},
*rawquerystring":"how do i add a field?",*
"querystring":"how do i add a field?",
"parsedquery":"(+(DisjunctionMaxQuery((topic_title_plain:how))
DisjunctionMaxQuery((topic_title_plain:do))
DisjunctionMaxQuery((topic_title_plain:i))
DisjunctionMaxQuery((topic_title_plain:add))
DisjunctionMaxQuery((topic_title_plain:a))
DisjunctionMaxQuery((topic_title_plain:field))))/no_coord",
"parsedquery_toString":"+((topic_title_plain:how)
(topic_title_plain:do) (topic_title_plain:i) (topic_title_plain:add)
(topic_title_plain:a) (topic_title_plain:field))",
"explain":{
"2":"
3.798876 = sum of:
2.033249 = weight(topic_title_plain:how in 202) [SchemaSimilarity],
result of:
2.033249 = score(doc=202,freq=1.0 = termFreq=1.0
), product of:
2.4634004 = idf, computed as log(1 + (docCount - docFreq + 0.5) /
(docFreq + 0.5)) from:
23.0 = docFreq
275.0 = docCount
0.82538307 = tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1
- b + b * fieldLength / avgFieldLength)) from:
1.0 = termFreq=1.0
1.2 = parameter k1
0.75 = parameter b
3.4436364 = avgFieldLength
5.2244897 = fieldLength
* 1.7656271 = weight(topic_title_plain:add in 202) [SchemaSimilarity],
result of:*
1.7656271 = score(doc=202,freq=1.0 = termFreq=1.0
), product of:
2.1391609 = idf, computed as log(1 + (docCount - docFreq + 0.5) /
(docFreq + 0.5)) from:
32.0 = docFreq
275.0 = docCount
0.82538307 = tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1
- b + b * fieldLength / avgFieldLength)) from:
1.0 = termFreq=1.0
1.2 = parameter k1
0.75 = parameter b
3.4436364 = avgFieldLength
5.2244897 = fieldLength
"},
Thanks
Jay
On Sun, Jan 13, 2019 at 10:32 PM Erick Erickson <erickerick...@gmail.com>
wrote:
> What does adding &debug=query show in both cases?
>
> Best,
> Erick
>
> On Sun, Jan 13, 2019 at 9:30 PM Jay Potharaju <jspothar...@gmail.com>
> wrote:
> >
> > Hi,
> > When searching I get different results when the query contains question
> > mark vs without question mark . The field i am searching on does not
> have
> > any question marks.
> > Any suggestions?
> >
> > <fieldType name="text_no_specialchars" class="solr.TextField">
> <analyzer> <
> > tokenizer class="solr.StandardTokenizerFactory"/> <filter class=
> > "solr.PatternReplaceFilterFactory" pattern=
> > "['!#\$%'\(\)\*+,-\./:;=?@\[\]^_`{|}~]" replacement=" " replace="all" />
> <
> > filter class="solr.LowerCaseFilterFactory"/> <filter class=
> > "solr.SuggestStopFilterFactory" ignoreCase="true" words=
> > "lang/stopwords_en.txt" /> <filter class=
> > "solr.EnglishPossessiveFilterFactory"/> <filter class=
> > "solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
> </analyzer> </
> > fieldType>
> >
> > Thanks
> > Jay
>
