Because the standard query parser treats '?' as a single-character wildcard: https://lucene.apache.org/solr/guide/6_6/the-standard-query-parser.html
So in the case q="how do I add a field", the word "field" in your document matches. In the second case q="how do I add a field?" it is looking for tokens like "fields" or "fielde"; The term without a trailing 1-character suffix doesn't match anymore. That is why it is no longer included in the scoring. https://lucene.apache.org/solr/guide/7_6/the-standard-query-parser.html#wildcard-searches Elizabeth On Mon, Jan 14, 2019 at 2:07 AM Jay Potharaju <jspothar...@gmail.com> wrote: > the parsedquery is same when debugging, but when calculating the scores > different fields are being taken into consideration. Why would that be the > case? My guess is that the suggeststopfilterfactory is not working as i > expect it to and causing this weird situation. > > Updated field type definition: > <fieldType name="text_no_specialchars" class="solr.TextField"> <analyzer> < > charFilter class="solr.PatternReplaceCharFilterFactory" pattern= > "['!#\$%'\(\)\*+,-\./:;=?@\[\]\^_`{|}~!@#$%^*]" /> <tokenizer class= > "solr.StandardTokenizerFactory"/> <filter class= > "solr.SuggestStopFilterFactory" ignoreCase="true" words= > "lang/stopwords_en.txt" /> <filter class="solr.LowerCaseFilterFactory"/> < > filter class="solr.EnglishPossessiveFilterFactory"/> <filter class= > "solr.KeywordMarkerFilterFactory" protected="protwords.txt"/> </analyzer> > </ > fieldType> > > Debug Query: > *"rawquerystring":"how do i add a field"*, > "querystring":"how do i add a field", > "parsedquery":"(+(DisjunctionMaxQuery((topic_title_plain:how)) > DisjunctionMaxQuery((topic_title_plain:do)) > DisjunctionMaxQuery((topic_title_plain:i)) > DisjunctionMaxQuery((topic_title_plain:add)) > DisjunctionMaxQuery((topic_title_plain:a)) > DisjunctionMaxQuery((topic_title_plain:field))))/no_coord", > "parsedquery_toString":"+((topic_title_plain:how) > (topic_title_plain:do) (topic_title_plain:i) (topic_title_plain:add) > (topic_title_plain:a) (topic_title_plain:field))", > "explain":{ > "1":" > 6.1034017 = sum of: > 2.0065408 = weight(topic_title_plain:add in 107) [SchemaSimilarity], > result of: > 2.0065408 = score(doc=107,freq=1.0 = termFreq=1.0 > ), product of: > 2.1391609 = idf, computed as log(1 + (docCount - docFreq + 0.5) / > (docFreq + 0.5)) from: > 32.0 = docFreq > 275.0 = docCount > 0.9380037 = tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - > b + b * fieldLength / avgFieldLength)) from: > 1.0 = termFreq=1.0 > 1.2 = parameter k1 > 0.75 = parameter b > 3.4436364 = avgFieldLength > 4.0 = fieldLength > 4.096861 = weight(topic_title_plain:field in 107) [SchemaSimilarity], > result of: > 4.096861 = score(doc=107,freq=1.0 = termFreq=1.0 > ), product of: > 4.367638 = idf, computed as log(1 + (docCount - docFreq + 0.5) / > (docFreq + 0.5)) from: > 3.0 = docFreq > 275.0 = docCount > 0.9380037 = tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - > b + b * fieldLength / avgFieldLength)) from: > 1.0 = termFreq=1.0 > 1.2 = parameter k1 > 0.75 = parameter b > 3.4436364 = avgFieldLength > 4.0 = fieldLength > "}, > > *rawquerystring":"how do i add a field?",* > "querystring":"how do i add a field?", > "parsedquery":"(+(DisjunctionMaxQuery((topic_title_plain:how)) > DisjunctionMaxQuery((topic_title_plain:do)) > DisjunctionMaxQuery((topic_title_plain:i)) > DisjunctionMaxQuery((topic_title_plain:add)) > DisjunctionMaxQuery((topic_title_plain:a)) > DisjunctionMaxQuery((topic_title_plain:field))))/no_coord", > "parsedquery_toString":"+((topic_title_plain:how) > (topic_title_plain:do) (topic_title_plain:i) (topic_title_plain:add) > (topic_title_plain:a) (topic_title_plain:field))", > "explain":{ > "2":" > 3.798876 = sum of: > 2.033249 = weight(topic_title_plain:how in 202) [SchemaSimilarity], > result of: > 2.033249 = score(doc=202,freq=1.0 = termFreq=1.0 > ), product of: > 2.4634004 = idf, computed as log(1 + (docCount - docFreq + 0.5) / > (docFreq + 0.5)) from: > 23.0 = docFreq > 275.0 = docCount > 0.82538307 = tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 > - b + b * fieldLength / avgFieldLength)) from: > 1.0 = termFreq=1.0 > 1.2 = parameter k1 > 0.75 = parameter b > 3.4436364 = avgFieldLength > 5.2244897 = fieldLength > * 1.7656271 = weight(topic_title_plain:add in 202) [SchemaSimilarity], > result of:* > 1.7656271 = score(doc=202,freq=1.0 = termFreq=1.0 > ), product of: > 2.1391609 = idf, computed as log(1 + (docCount - docFreq + 0.5) / > (docFreq + 0.5)) from: > 32.0 = docFreq > 275.0 = docCount > 0.82538307 = tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 > - b + b * fieldLength / avgFieldLength)) from: > 1.0 = termFreq=1.0 > 1.2 = parameter k1 > 0.75 = parameter b > 3.4436364 = avgFieldLength > 5.2244897 = fieldLength > "}, > Thanks > Jay > > > > On Sun, Jan 13, 2019 at 10:32 PM Erick Erickson <erickerick...@gmail.com> > wrote: > > > What does adding &debug=query show in both cases? > > > > Best, > > Erick > > > > On Sun, Jan 13, 2019 at 9:30 PM Jay Potharaju <jspothar...@gmail.com> > > wrote: > > > > > > Hi, > > > When searching I get different results when the query contains > question > > > mark vs without question mark . The field i am searching on does not > > have > > > any question marks. > > > Any suggestions? > > > > > > <fieldType name="text_no_specialchars" class="solr.TextField"> > > <analyzer> < > > > tokenizer class="solr.StandardTokenizerFactory"/> <filter class= > > > "solr.PatternReplaceFilterFactory" pattern= > > > "['!#\$%'\(\)\*+,-\./:;=?@\[\]^_`{|}~]" replacement=" " replace="all" > /> > > < > > > filter class="solr.LowerCaseFilterFactory"/> <filter class= > > > "solr.SuggestStopFilterFactory" ignoreCase="true" words= > > > "lang/stopwords_en.txt" /> <filter class= > > > "solr.EnglishPossessiveFilterFactory"/> <filter class= > > > "solr.KeywordMarkerFilterFactory" protected="protwords.txt"/> > > </analyzer> </ > > > fieldType> > > > > > > Thanks > > > Jay > > >
