Re: Solr Cluster - Is it wise to run optimize() on the master after each update

2012-01-23 Thread Erick Erickson
My first reaction is that, unless you have a specific use-case, this is unnecessary. When using a slave the Solr replication goes on in the background. Autowarming also is carried out in the background. Only when the autowarming is done are queries sent to the new (internal-to-solr) searcher. All w

Re: Solr Cluster - Is it wise to run optimize() on the master after each update

2012-01-23 Thread Maxim Veksler
Wonderful input. Thank you very much Erick. One question, I've been told that Solr supports an operation mode of multi core where you build the index on the master (optimize or not) then pass it to the "stand by" core on the slaves. Once the synchronization is complete you switch on the slave betw

Re: Solr Cluster - Is it wise to run optimize() on the master after each update

2012-01-23 Thread Erick Erickson
In general, do not optimize unless you 1> have a very static index 2> actually test the search performance afterwards. First, as Andrew says, optimizing will force a complete copy of the entire index at replication. If you do NOT optimize, only the most recent segments to be written are copied. S

Re: Solr Cluster - Is it wise to run optimize() on the master after each update

2012-01-23 Thread Andrew Harvey
We found that optimising too often killed our slave performance. An optimise will cause you to merge and ship the whole index rather than just the relevant portions when you replicate. The change on our slaves in terms of IO and CPU as well as RAM was marked. Andrew Sent on the run. On 23/

Solr Cluster - Is it wise to run optimize() on the master after each update

2012-01-23 Thread Maxim Veksler
I'm planning on having 1 Master and multiple slaves (cloud based, slaves are going up / down randomly). The slaves should be constantly available, meaning searching performance should optimally not be affected by the updates at all. It's unclear to me how the Cluster based replication works, does