Re: Problem with extended dismax, minus prefix (to mean NOT) and interaction with mm?

Tue, 25 May 2010 01:55:43 -0700

This looks like a case where the extended dismax parser is creating a Lucene QueryParser parsed query rather than a disjunction maximum query.
A case of "too much magic" maybe? Looks like this one should be parsed quite differently. Try dismax and see what you get, it'll be quite different. 

        Erik

On May 24, 2010, at 11:33 AM, Bill Dueber wrote:


I'm running edismax (on both a 1.4 with patch and a branch_3x  
version) and

I'm seeing something I don't expect.


We have our mm set such that 2/2 must match and 2/3 must match  
(mm=2<-1

5<67%

A query of
  dog cat

...gets interpreted as
 dog AND cat

But a query of
 dog cat -mouse

...gets interpreted as

 (dog AND cat) OR (dog AND NOT mouse) OR (cat AND NOT mouse)


In other words, the "-mouse" is being interpreted as a single token  
(NOT

mouse) to be counted for mm.

I would expect the query to interpret as:

 (dog AND cat) AND (NOT mouse)

Are my expectations out of whack? Or is this unexpected behavior?


[I've pasted the debugQuery info for a similar search below, though  
I freely

admit to not knowing how to read it]

Any thoughts on what I'm seeing here?


-Bill-

<lst name="debug">
<str name="rawquerystring">dog cat -trilogy</str>
<str name="querystring">dog cat -trilogy</str>
<str name="parsedquery">allfields:dog allfields:cat
-allfields:trilogi</str>
<str name="parsedquery_toString">allfields:dog allfields:cat
-allfields:trilogi</str>
<lst name="explain">
 <str name="000107098">
2.1741915 = (MATCH) sum of:
 1.2620605 = (MATCH) weight(allfields:dog in 3187), product of:
   0.7618881 = queryWeight(allfields:dog), product of:
     8.744003 = idf(docFreq=64, maxDocs=150000)
     0.08713264 = queryNorm
   1.6564907 = (MATCH) fieldWeight(allfields:dog in 3187), product of:
     1.7320508 = tf(termFreq(allfields:dog)=3)
     8.744003 = idf(docFreq=64, maxDocs=150000)
     0.109375 = fieldNorm(field=allfields, doc=3187)
 0.912131 = (MATCH) weight(allfields:cat in 3187), product of:
   0.64770865 = queryWeight(allfields:cat), product of:
     7.4335938 = idf(docFreq=240, maxDocs=150000)
     0.08713264 = queryNorm
   1.4082427 = (MATCH) fieldWeight(allfields:cat in 3187), product of:
     1.7320508 = tf(termFreq(allfields:cat)=3)
     7.4335938 = idf(docFreq=240, maxDocs=150000)
     0.109375 = fieldNorm(field=allfields, doc=3187)
</str>
 <str name="000036695">
2.1518915 = (MATCH) sum of:
 1.249116 = (MATCH) weight(allfields:dog in 36426), product of:
   0.7618881 = queryWeight(allfields:dog), product of:
     8.744003 = idf(docFreq=64, maxDocs=150000)
     0.08713264 = queryNorm

   1.6395006 = (MATCH) fieldWeight(allfields:dog in 36426), product  
of:

     2.0 = tf(termFreq(allfields:dog)=4)
     8.744003 = idf(docFreq=64, maxDocs=150000)
     0.09375 = fieldNorm(field=allfields, doc=36426)
 0.9027756 = (MATCH) weight(allfields:cat in 36426), product of:
   0.64770865 = queryWeight(allfields:cat), product of:
     7.4335938 = idf(docFreq=240, maxDocs=150000)
     0.08713264 = queryNorm

   1.3937988 = (MATCH) fieldWeight(allfields:cat in 36426), product  
of:

     2.0 = tf(termFreq(allfields:cat)=4)
     7.4335938 = idf(docFreq=240, maxDocs=150000)
     0.09375 = fieldNorm(field=allfields, doc=36426)
</str>
 <str name="000038137">
1.4345944 = (MATCH) sum of:
 0.832744 = (MATCH) weight(allfields:dog in 37852), product of:
   0.7618881 = queryWeight(allfields:dog), product of:
     8.744003 = idf(docFreq=64, maxDocs=150000)
     0.08713264 = queryNorm

   1.0930004 = (MATCH) fieldWeight(allfields:dog in 37852), product  
of:

     1.0 = tf(termFreq(allfields:dog)=1)
     8.744003 = idf(docFreq=64, maxDocs=150000)
     0.125 = fieldNorm(field=allfields, doc=37852)
 0.6018504 = (MATCH) weight(allfields:cat in 37852), product of:
   0.64770865 = queryWeight(allfields:cat), product of:
     7.4335938 = idf(docFreq=240, maxDocs=150000)
     0.08713264 = queryNorm

   0.9291992 = (MATCH) fieldWeight(allfields:cat in 37852), product  
of:

     1.0 = tf(termFreq(allfields:cat)=1)
     7.4335938 = idf(docFreq=240, maxDocs=150000)
     0.125 = fieldNorm(field=allfields, doc=37852)
</str>
 <str name="000134898">
1.2629167 = (MATCH) sum of:
 0.624558 = (MATCH) weight(allfields:dog in 30673), product of:
   0.7618881 = queryWeight(allfields:dog), product of:
     8.744003 = idf(docFreq=64, maxDocs=150000)
     0.08713264 = queryNorm

   0.8197503 = (MATCH) fieldWeight(allfields:dog in 30673), product  
of:

     1.0 = tf(termFreq(allfields:dog)=1)
     8.744003 = idf(docFreq=64, maxDocs=150000)
     0.09375 = fieldNorm(field=allfields, doc=30673)
 0.6383587 = (MATCH) weight(allfields:cat in 30673), product of:
   0.64770865 = queryWeight(allfields:cat), product of:
     7.4335938 = idf(docFreq=240, maxDocs=150000)
     0.08713264 = queryNorm

   0.9855646 = (MATCH) fieldWeight(allfields:cat in 30673), product  
of:

     1.4142135 = tf(termFreq(allfields:cat)=2)
     7.4335938 = idf(docFreq=240, maxDocs=150000)
     0.09375 = fieldNorm(field=allfields, doc=30673)
</str>
 <str name="000029964">
1.25527 = (MATCH) sum of:
 0.728651 = (MATCH) weight(allfields:dog in 29736), product of:
   0.7618881 = queryWeight(allfields:dog), product of:
     8.744003 = idf(docFreq=64, maxDocs=150000)
     0.08713264 = queryNorm

   0.95637536 = (MATCH) fieldWeight(allfields:dog in 29736), product  
of:

     1.0 = tf(termFreq(allfields:dog)=1)
     8.744003 = idf(docFreq=64, maxDocs=150000)
     0.109375 = fieldNorm(field=allfields, doc=29736)
 0.5266191 = (MATCH) weight(allfields:cat in 29736), product of:
   0.64770865 = queryWeight(allfields:cat), product of:
     7.4335938 = idf(docFreq=240, maxDocs=150000)
     0.08713264 = queryNorm

   0.8130493 = (MATCH) fieldWeight(allfields:cat in 29736), product  
of:

     1.0 = tf(termFreq(allfields:cat)=1)
     7.4335938 = idf(docFreq=240, maxDocs=150000)
     0.109375 = fieldNorm(field=allfields, doc=29736)
</str>
 <str name="000100541">
1.25527 = (MATCH) sum of:
 0.728651 = (MATCH) weight(allfields:dog in 99929), product of:
   0.7618881 = queryWeight(allfields:dog), product of:
     8.744003 = idf(docFreq=64, maxDocs=150000)
     0.08713264 = queryNorm

   0.95637536 = (MATCH) fieldWeight(allfields:dog in 99929), product  
of:

     1.0 = tf(termFreq(allfields:dog)=1)
     8.744003 = idf(docFreq=64, maxDocs=150000)
     0.109375 = fieldNorm(field=allfields, doc=99929)
 0.5266191 = (MATCH) weight(allfields:cat in 99929), product of:
   0.64770865 = queryWeight(allfields:cat), product of:
     7.4335938 = idf(docFreq=240, maxDocs=150000)
     0.08713264 = queryNorm

   0.8130493 = (MATCH) fieldWeight(allfields:cat in 99929), product  
of:

     1.0 = tf(termFreq(allfields:cat)=1)
     7.4335938 = idf(docFreq=240, maxDocs=150000)
     0.109375 = fieldNorm(field=allfields, doc=99929)
</str>
 <str name="000036787">
0.9310362 = (MATCH) product of:
 1.8620723 = (MATCH) sum of:
   1.8620723 = (MATCH) weight(allfields:dog in 36523), product of:
     0.7618881 = queryWeight(allfields:dog), product of:
       8.744003 = idf(docFreq=64, maxDocs=150000)
       0.08713264 = queryNorm

     2.4440234 = (MATCH) fieldWeight(allfields:dog in 36523),  
product of:

       2.236068 = tf(termFreq(allfields:dog)=5)
       8.744003 = idf(docFreq=64, maxDocs=150000)
       0.125 = fieldNorm(field=allfields, doc=36523)
 0.5 = coord(1/2)
</str>
 <str name="000024903">
0.89662147 = (MATCH) sum of:
 0.520465 = (MATCH) weight(allfields:dog in 24680), product of:
   0.7618881 = queryWeight(allfields:dog), product of:
     8.744003 = idf(docFreq=64, maxDocs=150000)
     0.08713264 = queryNorm

   0.68312526 = (MATCH) fieldWeight(allfields:dog in 24680), product  
of:

     1.0 = tf(termFreq(allfields:dog)=1)
     8.744003 = idf(docFreq=64, maxDocs=150000)
     0.078125 = fieldNorm(field=allfields, doc=24680)
 0.37615648 = (MATCH) weight(allfields:cat in 24680), product of:
   0.64770865 = queryWeight(allfields:cat), product of:
     7.4335938 = idf(docFreq=240, maxDocs=150000)
     0.08713264 = queryNorm

   0.5807495 = (MATCH) fieldWeight(allfields:cat in 24680), product  
of:

     1.0 = tf(termFreq(allfields:cat)=1)
     7.4335938 = idf(docFreq=240, maxDocs=150000)
     0.078125 = fieldNorm(field=allfields, doc=24680)
</str>
 <str name="000002131">
0.728651 = (MATCH) product of:
 1.457302 = (MATCH) sum of:
   1.457302 = (MATCH) weight(allfields:dog in 2108), product of:
     0.7618881 = queryWeight(allfields:dog), product of:
       8.744003 = idf(docFreq=64, maxDocs=150000)
       0.08713264 = queryNorm

     1.9127507 = (MATCH) fieldWeight(allfields:dog in 2108), product  
of:

       2.0 = tf(termFreq(allfields:dog)=4)
       8.744003 = idf(docFreq=64, maxDocs=150000)
       0.109375 = fieldNorm(field=allfields, doc=2108)
 0.5 = coord(1/2)
</str>
 <str name="000007984">
0.728651 = (MATCH) product of:
 1.457302 = (MATCH) sum of:
   1.457302 = (MATCH) weight(allfields:dog in 7901), product of:
     0.7618881 = queryWeight(allfields:dog), product of:
       8.744003 = idf(docFreq=64, maxDocs=150000)
       0.08713264 = queryNorm

     1.9127507 = (MATCH) fieldWeight(allfields:dog in 7901), product  
of:

       2.0 = tf(termFreq(allfields:dog)=4)
       8.744003 = idf(docFreq=64, maxDocs=150000)
       0.109375 = fieldNorm(field=allfields, doc=7901)
 0.5 = coord(1/2)
</str>
</lst>




--
Bill Dueber
Library Systems Programmer
University of Michigan Library
























					Reply via email to