On 8/6/10 5:03 PM, Chris Hostetter wrote:
: We have an index around 25-30G w/ 1 master and 5 slaves. We perform
: replication every 30 mins. During replication the disk I/O obviously shoots up
: on the slaves to the point where all requests routed to that slave take a
: really long time... sometimes to the point of timing out.
:
: Is there any logical or physical changes we could make to our architecture to
: overcome this problem?
If the problem really is disk I/O then perhaps you don't have enough RAM
set asside for the filesystem cache to keep the "current" index in memory?
I've seen people have this type of problem before, but usually it's
network I/O that is the bottleneck, in which case using multiple NICs on
your slaves (one for client requests, one for replication)
I think at one point there was also talk about leveraging an rsync option
to force snappuller to throttle itself an only use a max amount of
bandwidth -- but then we moved away from script based replication to java
based replication and i don't think the Java Network/IO system suports
that type of throttling. However: you might be able to configure it in
your switches/routers (ie: only let the slaves use X% of their total
badwidth to talk to the master)
-Hoss
Thanks for the suggestions. Our slaves have 12G with 10G dedicated to
the JVM.. too much?
Are the rysnc snappuller featurs still available in 1.4.1? I may try
that to see if helps. Configuration of the switches may also be possible.
Also, would you mind explaining your second point... using dual NIC
cards. How can this be accomplished/configured. Thanks for you help
