Re: spellcheck index blown away during rebuild

2010-08-27 Thread Lance Norskog
To make a dictionary with a 'minimum document count' you need to make the dictionary from the facets. Facets will create this for you; but will allocate memory for every last term. The last N facets will have the smallest # of terms. To get term counts for hundreds of millions of terms, I think yo

Re: spellcheck index blown away during rebuild

2010-08-26 Thread Chris Hostetter
: What you're talking about is effectively promoting the spellcheck : index to a first-class Solr index, instead of an appendage bolted on : the side of an existing core. Given sharding and distributed search, : this may be a better design. even w/o promoting the spell index to be a "main" index,

Re: spellcheck index blown away during rebuild

2010-08-22 Thread Shawn Heisey
On 8/20/2010 8:56 PM, Lance Norskog wrote: The first question is about your use cases. How many words are in the eventual 3GB spelling index? Do you really need that many? Spell-checking is a more controllable UI if you make it from a dictionary. It's built from an index-only field that combin

Re: spellcheck index blown away during rebuild

2010-08-20 Thread Lance Norskog
The first question is about your use cases. How many words are in the eventual 3GB spelling index? Do you really need that many? Spell-checking is a more controllable UI if you make it from a dictionary. What you're talking about is effectively promoting the spellcheck index to a first-class Solr

spellcheck index blown away during rebuild

2010-08-20 Thread Shawn Heisey
I am just delving into the spellcheckcomponent on a test server running a 3.1 build from June 29th. I have noticed that when you ask for a rebuild of the spell check index, it deletes it before starting the rebuild. It takes about 39 minutes to build one (3GB), which is a long time to do wit