At what point do you draw the line? 0.01 is too low, but what about
0.5 or 0.3? In fact, there may be queries where 0.01 is relevant.
Relevance is a tricky thing and putting in arbitrary cutoffs is
usually not a good thing. An alternative might be to instead look at
the difference between scores and see if the gap is larger than some
delta, but even that is subject to the vagaries of scoring.
What kind of relevance testing have you done so far to come up with
those values? See also http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Debugging-Relevance-Issues-in-Search/
On Feb 11, 2009, at 10:16, Cheng Zhang <zhangyongji...@yahoo.com> wrote:
Hi Grant,
In my case, for example searching a book. Some of the returned
documents are with high relevance (score > 3), but some of document
with low score (<0.01) are useless.
Without a "score filter", I have to go through each document to find
out the number of documents I'm interested (score > nnn). This
causes some problem for pagination. For example if I only need to
display the first 10 records I need to retrieve all 1000 documents
to figure out the number of meaningful documents which have score >
nnn.
Thx,
Kevin
----- Original Message ----
From: Grant Ingersoll <gsing...@apache.org>
To: solr-user@lucene.apache.org
Sent: Wednesday, February 11, 2009 6:47:11 AM
Subject: Re: score filter
What's the motivation for wanting to do this? The reason I ask, is
score is a relative thing determined by Lucene based on your index
statistics. It is only meaningful for comparing the results of a
specific query with a specific instance of the index. In other
words, it isn't useful to filter on b/c there is no way of knowing
what a good cutoff value would be. So, you won't be able to do
score:[1.2 TO *] because score is a not an actual Field.
That being said, you probably could implement a HitCollector at the
Lucene level and somehow hook it into Solr to do what you want. Or,
of course, just stop processing the results in your app after you
see a score below a certain value. Naturally, this still means you
have to retrieve the results.
-Grant
On Feb 10, 2009, at 10:01 PM, Cheng Zhang wrote:
Hello,
Is there a way to set a score filter? I tried "+score:[1.2 TO *]"
but it did not work.
Many thanks,
Kevin
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search
