Thanks for your answer Erick.
For the polling interval, we use 1 second for the small index and 1 minute
for the big one. I'll try to increase it up to 5 minutes and see if it
solve the problem.The issue doesn't occur with the default cache settings
(i.e. cache size=512).
Indeed, we strive for near real time searching, so I'll take a look at 4.0
with NRT support, hoping it is sufficiently stable for production
environment.
Kind regards,
Damien
2012/9/2 Erick Erickson <erickerick...@gmail.com>
> You polling interval is much too short. 1 second
> is probably getting you into resource contention
> issues.
>
> A more reasonable interval is on the order of several
> minutes. If you really need near real time
> searching, consider 4.0 which supports NRT
>
> Best
> Erick
>
> On Fri, Aug 31, 2012 at 10:02 AM, Damien Dudognon
> <damien.dudog...@ebuzzing.com> wrote:
> > Hi,
> >
> > We get some troubles with the solr replication after cache
> optimizations. We use a lot the facet features.
> >
> > We have increased the cache size and its initial size. We have also
> change the queryResultCache from LRU to FastLRU and the fieldValueCache was
> activated (see below the detailed configuration). This optimization step
> allows to divide the average time per request by 50 (from 260ms to 5ms).
> >
> > However, with these modifications we noticed an important replication
> lag. This issue is not troublesome for smaller indexes (about 300.000
> elements - 3Gb), but it becomes critical if the index size is significant
> (30 million elements - 70Gb). In fact, the slaves can't make up for lag and
> they become out-of-date (and consequently unusable).
> >
> > Has anyone ever been faced this kind of problems?
> >
> > Our environment :
> > - Solr 3.4.0
> > - Java 1.6.0_26
> > - Debian 6.0.3
> >
> > Best regards,
> > Damien
> >
> > ------------------------------------------------------------------
> > My previous cache settings (fieldValueCache was disabled):
> > ------------------------------------------------------------------
> > <filterCache class="solr.FastLRUCache" size="512" initialSize="512"
> autowarmCount="0" />
> > <queryResultCache class="solr.LRUCache" size="512" initialSize="512"
> autowarmCount="0" />
> > <documentCache class="solr.FastLRUCache" size="512" initialSize="512"
> autowarmCount="0" />
> > <!--
> > <fieldValueCache class="solr.FastLRUCache" size="512"
> autowarmCount="128" showItems="32" />
> > -->
> >
> > ------------------------------------------------------------------
> > The settings now used:
> > ------------------------------------------------------------------
> > <filterCache class="solr.FastLRUCache" size="16384" initialSize="4096"
> autowarmCount="0" />
> > <queryResultCache class="solr.FastLRUCache" size="16384"
> initialSize="4096" autowarmCount="0" />
> > <documentCache class="solr.FastLRUCache" size="16384" initialSize="4096"
> autowarmCount="0" />
> > <fieldValueCache class="solr.FastLRUCache" size="16384"
> autowarmCount="1024" showItems="32" />
> >
> > ------------------------------------------------------------------
> > The replication config:
> > ------------------------------------------------------------------
> > <requestHandler name="/replication" class="solr.ReplicationHandler" >
> > <lst name="master">
> > <str name="enable">${solr.enable.master:false}</str>
> > <str name="replicateAfter">commit</str>
> > <str name="replicateAfter">startup</str>
> > <str name="confFiles">schema.xml,stopwords.txt</str>
> > </lst>
> > <lst name="slave">
> > <str name="enable">${solr.enable.slave:false}</str>
> > <str name="masterUrl">http://solrmaster:
> ${jetty.port:8083}/solr/en/replication</str>
> > <str name="pollInterval">00:00:01</str>
> > </lst>
> > </requestHandler>
>
--
Damien DUDOGNON
R&D Engineer - PhD Student
- +33 (0)6 62 79 34 10
- damien.dudog...@ebuzzing.com
-
-
- linkedin.com/in/damiendudognon<http://www.linkedin.com/in/damiendudognon>
- www.irit.fr/~Damien.Dudognon
- 1, avenue Jean Rieux
- 31500 Toulouse
- France
- +33 (0)5 62 48 33 90
-
