You polling interval is much too short. 1 second
is probably getting you into resource contention
issues.
A more reasonable interval is on the order of several
minutes. If you really need near real time
searching, consider 4.0 which supports NRT
Best
Erick
On Fri, Aug 31, 2012 at 10:02 AM, Damien Dudognon
<damien.dudog...@ebuzzing.com> wrote:
> Hi,
>
> We get some troubles with the solr replication after cache optimizations. We
> use a lot the facet features.
>
> We have increased the cache size and its initial size. We have also change
> the queryResultCache from LRU to FastLRU and the fieldValueCache was
> activated (see below the detailed configuration). This optimization step
> allows to divide the average time per request by 50 (from 260ms to 5ms).
>
> However, with these modifications we noticed an important replication lag.
> This issue is not troublesome for smaller indexes (about 300.000 elements -
> 3Gb), but it becomes critical if the index size is significant (30 million
> elements - 70Gb). In fact, the slaves can't make up for lag and they become
> out-of-date (and consequently unusable).
>
> Has anyone ever been faced this kind of problems?
>
> Our environment :
> - Solr 3.4.0
> - Java 1.6.0_26
> - Debian 6.0.3
>
> Best regards,
> Damien
>
> ------------------------------------------------------------------
> My previous cache settings (fieldValueCache was disabled):
> ------------------------------------------------------------------
> <filterCache class="solr.FastLRUCache" size="512" initialSize="512"
> autowarmCount="0" />
> <queryResultCache class="solr.LRUCache" size="512" initialSize="512"
> autowarmCount="0" />
> <documentCache class="solr.FastLRUCache" size="512" initialSize="512"
> autowarmCount="0" />
> <!--
> <fieldValueCache class="solr.FastLRUCache" size="512" autowarmCount="128"
> showItems="32" />
> -->
>
> ------------------------------------------------------------------
> The settings now used:
> ------------------------------------------------------------------
> <filterCache class="solr.FastLRUCache" size="16384" initialSize="4096"
> autowarmCount="0" />
> <queryResultCache class="solr.FastLRUCache" size="16384" initialSize="4096"
> autowarmCount="0" />
> <documentCache class="solr.FastLRUCache" size="16384" initialSize="4096"
> autowarmCount="0" />
> <fieldValueCache class="solr.FastLRUCache" size="16384" autowarmCount="1024"
> showItems="32" />
>
> ------------------------------------------------------------------
> The replication config:
> ------------------------------------------------------------------
> <requestHandler name="/replication" class="solr.ReplicationHandler" >
> <lst name="master">
> <str name="enable">${solr.enable.master:false}</str>
> <str name="replicateAfter">commit</str>
> <str name="replicateAfter">startup</str>
> <str name="confFiles">schema.xml,stopwords.txt</str>
> </lst>
> <lst name="slave">
> <str name="enable">${solr.enable.slave:false}</str>
> <str
> name="masterUrl">http://solrmaster:${jetty.port:8083}/solr/en/replication</str>
> <str name="pollInterval">00:00:01</str>
> </lst>
> </requestHandler>
