Re: How to optimize Index Process?

2009-03-28 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Sat, Mar 28, 2009 at 7:38 AM, Otis Gospodnetic wrote: > > Hi, > > Answers inlined. > > > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > - Original Message >>   We have a distributed Solr system (2-3 boxes with each running 2 >> instances of Solr and each Solr instan

Re: optimization advice?

2009-03-28 Thread Steve Conover
Otis, That's an interesting suggestion. I'm curious about the thought process behind it though - we currently don't have memory problems, and in fact our max memory setting is below where it could be. Does your suggestion imply that something could be gained by throwing more memory at the proble

Re: optimization advice?

2009-03-28 Thread Otis Gospodnetic
OK, how about this trick then. Do you really need the full string for sorting? Could you get by (cheat) sorting only on the first N characters? If so, you could create a separate field for that (copyField will come handy) and that should consume a little less memory. Otis -- Sematext -- ht

Re: spellcheck.onlyMorePopular

2009-03-28 Thread David Smiley @MITRE.org
I know your issue has already been addressed but you may want to consider "gran" being a synonym for "grand" and then analyzing it as such. ~ David Smiley Marcus Stratmann wrote: > > Hello, > > I have another question concerning the spell checking mechanism. > Setting onlyMorePopular=true and

Re: How to optimize Index Process?

2009-03-28 Thread vivek sar
Thanks Otis. This is very useful. I'll try all your suggestions and post my findings (and improvements). Thanks, -vivek On Fri, Mar 27, 2009 at 7:08 PM, Otis Gospodnetic wrote: > > Hi, > > Answers inlined. > > > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > - Original Me

Re: Issue with Unique Key

2009-03-28 Thread Shalin Shekhar Mangar
As a best practice, use a unique key which is not tokenized e.g. a string type. On Sat, Mar 28, 2009 at 10:42 PM, prerna07 wrote: > > > Issue: Unique Key not working. > > We defined unique key in schema.xml: > > > IndexId_s > > Still multiple index are getting craete with the same Index_id. > >

Issue with Unique Key

2009-03-28 Thread prerna07
Issue: Unique Key not working. We defined unique key in schema.xml: IndexId_s Still multiple index are getting craete with the same Index_id. Please suggest if we need to add/ change something in schema.xml. Thanks, prerna -- View this message in context: http://www.nabble.com/Issue-wit

RE: [solr-user] Upgrade from 1.2 to 1.3 gives 3x slowdown

2009-03-28 Thread Grant Ingersoll
Hey Fergus, Finally got a chance to run your scripts, etc. per the thread: http://www.lucidimagination.com/search/document/5c3de15a4e61095c/upgrade_from_1_2_to_1_3_gives_3x_slowdown_script#8324a98d8840c623 I can reproduce your slowdown. One oddity with rev 643465 is: On the old version, there

Re: OOM at MultiSegmentReader.norms

2009-03-28 Thread Michael McCandless
Still, 1024M ought to be enough to load one field's norms (how many fields have norms?). If you do things requiring FieldCache that'll also consume RAM. It's also possible you're hitting this bug (false OOME) in Sun's JRE: http://issues.apache.org/jira/browse/LUCENE-1566 Feel free to go vote