no, Solr 4.0.0, I wanted to update to Solr 4.1 but I read that there was an
issue with the replication, so I decided not to try it for now
Mark Miller-3 wrote
> Are you using Solr 4.1?
>
> - Mark
>
> On Mar 11, 2013, at 1:53 PM, Victor Ruiz <
> bik1979@
> > wrote:
>
>> Hi guys,
>>
>> I have a problem with Solr replication. I have 2 solr servers (Solr
>> 4.0.0) 1
>> master and 1 slave (8 processors,16GB RAM ,Ubuntu 11, ext3, each). In
>> every server, there are 2 independent instances of solr running (I tried
>> also multicore config, but having independent instances has for me better
>> performance), every instance having a differente collection. So, we have
>> 2
>> masters in server 1, and 2 slaves in server 2.
>>
>> Index size is currently (for the biggest collection) around 17 million
>> documents, with a total size near 12 GB. The files transferred every
>> replication cycle are typically not more than 100, with a total size not
>> bigger than 50MB. The other collection is not that big, just around 1
>> million docs and not bigger than 2 GB and not a high update ratio. The
>> big
>> collection has a load around 200 queries per second (MoreLikeThis,
>> RealTimeGetHandler , TermVectorComponent mainly), and for the small one
>> it
>> is below 50 queries per second
>>
>> Replication has been working for long time with any problem, but in the
>> last
>> weeks the replication cycles started to take long and long time for the
>> big
>> collection, even more than 2 minutes, some times even more. During that
>> time, slaves are so overloaded, that many queries are timing out, despite
>> the timeout in my clients is 30 seconds.
>>
>> The servers are in same LAN, gigabit ethernet, so the broadband should
>> not
>> be the bottleneck.
>>
>> Since the index is receiving frequents updates and deletes (update
>> handler
>> receives more than 200 request per second for the big collection, but not
>> more than 5 per second for the small one), I tried to use the
>> maxCommitsToKeep attribute, to ensure that no file was deleted during
>> replication, but it has no effect.
>>
>> My solrconfig.xml in the big collection is like that:
>>
>> <?xml version="1.0" encoding="UTF-8" ?>
>>
>>
> <config>
>>
>>
> <luceneMatchVersion>
> LUCENE_40
> </luceneMatchVersion>
>>
>>
> <directoryFactory name="DirectoryFactory"
>>
>
> class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
>>
>>
>>
> <indexConfig>
>>
> <mergeFactor>
> 3
> </mergeFactor>
>>
>>
> <deletionPolicy class="solr.SolrDeletionPolicy">
>>
>>
> <str name="maxCommitsToKeep">
> 10
> </str>
>>
> <str name="maxOptimizedCommitsToKeep">
> 1
> </str>
>>
>>
> <str name="maxCommitAge">
> 6HOUR
> </str>
>>
>>
> </deletionPolicy>
>>
>>
> </indexConfig>
>>
>>
> <jmx/>
>>
>>
> <updateHandler class="solr.DirectUpdateHandler2">
>>
>>
> <autoCommit>
>>
> <maxDocs>
> 2000
> </maxDocs>
>>
> <maxTime>
> 30000
> </maxTime>
>>
> </autoCommit>
>>
>>
> <autoSoftCommit>
>>
> <maxTime>
> 500
> </maxTime>
>>
> </autoSoftCommit>
>>
>>
> <updateLog>
>>
> <str name="dir">
> ${solr.data.dir:}
> </str>
>>
> </updateLog>
>>
>>
> </updateHandler>
>>
>>
> <query>
>>
> <maxBooleanClauses>
> 2048
> </maxBooleanClauses>
>>
>>
> <filterCache
>>
> class="solr.FastLRUCache"
>> size="2048"
>> initialSize="1024"
>> autowarmCount="1024"/>
>>
>>
> <queryResultCache
>>
> class="solr.LRUCache"
>> size="2048"
>> initialSize="1024"
>> autowarmCount="1024"/>
>>
>>
>>
> <documentCache
>>
> class="solr.LRUCache"
>> size="2048"
>> initialSize="1024"
>> autowarmCount="1024"/>
>>
>>
> <enableLazyFieldLoading>
> true
> </enableLazyFieldLoading>
>>
>>
> <queryResultWindowSize>
> 50
> </queryResultWindowSize>
>>
>>
> <queryResultMaxDocsCached>
> 50
> </queryResultMaxDocsCached>
>>
>>
> <listener event="newSearcher" class="solr.QuerySenderListener">
>>
> <arr name="queries">
>>
> <lst>
>>
> <str name="q">
> *:*
> </str>
>>
> <str name="fq">
> date:[NOW/DAY-7DAY TO NOW/DAY+1DAY]
> </str>
>>
> <str name="rows">
> 1000
> </str>
>>
> </lst>
>>
> </arr>
>>
> </listener>
>>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
>>
> <arr name="queries">
>>
> <lst>
>>
> <str name="q">
> *:*
> </str>
>>
> <str name="fq">
> date:[NOW/DAY-7DAY TO NOW/DAY+1DAY]
> </str>
>>
> <str name="rows">
> 1000
> </str>
>>
> </lst>
>>
> </arr>
>>
> </listener>
>>
>>
> <useColdSearcher>
> true
> </useColdSearcher>
>>
>>
> <maxWarmingSearchers>
> 4
> </maxWarmingSearchers>
>>
> </query>
>>
>>
>>
>>
> <requestHandler name="/replication" class="solr.ReplicationHandler">
>>
> <lst name="master">
>>
> <str name="enable">
> ${enable.master:false}
> </str>
>>
> <str name="replicateAfter">
> commit
> </str>
>>
> <str name="replicateAfter">
> startup
> </str>
>>
> <str name="backupAfter">
> startup
> </str>
>>
> <str
>>
>
> name="confFiles">schema.xml,solrconfig.xml,stopwords_de.txt,stopwords_en.txt,mapping-FoldToASCII.txt,mapping-FoldToASCII_de.txt
> </str>
>>
> </lst>
>>
> <lst name="slave">
>>
> <str name="enable">
> ${enable.slave:false}
> </str>
>>
> <str
>>
> name="masterUrl">http://${MASTER_HOST}:${MASTER_PORT}/solr/${MASTER_CORE}
> </str>
>>
> <str name="pollInterval">
> 05:00
> </str>
>>
> <str name="httpBasicAuthUser">
> ${MASTER_HTTP_USER}
> </str>
>>
> <str name="httpBasicAuthPassword">
> ${MASTER_HTTP_PWD}
> </str>
>>
> </lst>
>>
> </requestHandler>
>>
>>
> <admin>
>>
> <defaultQuery>
> *:*
> </defaultQuery>
>>
> </admin>
>>
>>
> </config>
>>
>> Poll interval is now set to 5 min, I tried to reduce it to 2 min and to
>> increase it up to 10, with no effect, the replication is always taking so
>> long., even with a poll time of 2 minutes, when there are only a few
>> megas
>> to replicate.
>>
>> Any idea suggestion about what could be the problem?
>>
>> Thanks in advance,
>> Victor
>>
>>
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/Solr-replication-takes-long-time-tp4046388.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-replication-takes-long-time-tp4046388p4046392.html
Sent from the Solr - User mailing list archive at Nabble.com.
