Spellchecker delivers far too few suggestions

[Martin Dietze]

I recently upgraded to SOLR 4.10.1 and after that set up the spell
checker which I use for returning suggestions after searches with few
or no results.
When the spellchecker is active, this request handler is used (most of
which is taken from examples I found in the net):

  
 
   explicit
   true
   false
   10
   false
   *:*
   explicit
   50
   *,score
 
 
   spellcheck
 
  

The search component is configured as follows (again most of it copied
from examples in the net):

  
text

  default
  text
  solr.DirectSolrSpellChecker
  internal
  0.3
  2
  1
  5
  4
  0.01
  .01

  

With this setup I can get suggestions for misspelled words. The
results on my developer machine were mostly fine, but on the test
system (much larger database, much larger search index) I found it
very hard to get suggestions at all. If for instance I misspell “bank”
as “bnak” I’d expect to get a suggestion for “bank” (since that word
can be found in the index very often).

I’ve played around with maxQueryFrequency and maxQueryFrequency with
no success.

Does anyone see any obvious misconfiguration? Anything that I could try?

Any way I can debug this? (problem is that my application uses the
core API which makes trying out requests through the web interface
does not work)

Any help would be greatly appreciated!

Cheers,

Martin


-- 
-- mdie...@gmail.com --/-- mar...@the-little-red-haired-girl.org 
- / http://herbert.the-little-red-haired-girl.org / -

Re: Spellchecker delivers far too few suggestions

[Martin Dietze]

On 17 December 2014 at 16:41, Erick Erickson  wrote:
> First, I'd look in your corpus for "bnak". The problem with index-based
> suggestions is that if your index contains garbage, they're "correctly
> spelled" since they're in the index. TermsComponent is very useful for this.
>
> You can also loosen up the match criteria, and as I remember the collations
> parameter does some permutations of the word (but my memory of how that
> works is shaky).

Thank you for your response. I now set up a TermsComponent for this
case as follows:

  

  true
  text


  termsComponent

  

… constructed a MapSolrParams from which I create my
SolrQueryRequestBase object using these params (“text” is the name of
my catch-all-field):

{{params(terms.prefix="bnak"),defaults(terms.fl=text&terms=true)}}

… and call my core with it, yielding the following:

{responseHeader={status=0,QTime=5416},terms={text={}}}

That seems to imply that indeed the term “bnak” is not in my index, or
am I using the TermsComponent the wrong way?

Cheers,

Martin

-- 
-- mdie...@gmail.com --/-- mar...@the-little-red-haired-girl.org 
- / http://herbert.the-little-red-haired-girl.org / -

Re: Spellchecker delivers far too few suggestions

[Martin Dietze]

On 17 December 2014 at 18:08, Erick Erickson  wrote:
> This is seeming like a puzzler...

I’ve got to the point that I do get suggestions if I find no document
at all. The problem was seemingly caused by the way I quoted my search
queries.

Still I don’t get suggestions for terms that are in the index. For
instance, if I create a document that contains the term “bnak”, I
would like to display a result like: “found one occurrence of ‘bnak’,
but did you mean: ”.

Is there a setting I’ve missed?


-- 
-- mdie...@gmail.com --/-- mar...@the-little-red-haired-girl.org 
- / http://herbert.the-little-red-haired-girl.org / -