Hmmm, the first thing I'd look at is why you are having long GC pauses. Here's a great place to start:
http://www.lucidimagination.com/blog/2011/03/27/garbage-collection-bootcamp-1-0/ and: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html I've wondered about a similar approach, but by firing off the same query to multiple nodes in your cluster, you'll be effectively doubling (at least) the load on your system. Leading to more memory issues perhaps in a "non-virtuous cycle". FWIW, Erick On Fri, Jan 18, 2013 at 5:41 AM, Phil Hoy <p...@brightsolid.com> wrote: > Hi, > > I would like to experiment with some custom load balancers to help with query > latency in the face of long gc pauses and the odd time-consuming query that > we need to be able to support. At the moment setting the socket timeout via > the HttpShardHandlerFactory does help, but of course it can only be set to a > length of time as long as the most time consuming query we are likely to > receive. > > For example perhaps a load balancer that sends multiple queries concurrently > to all/some replicas and only keeps the first response might be effective. Or > maybe a load balancer which takes account of the frequency of timeouts would > be able to recognize zombies more effectively. > > To use alternative load balancer implementations cleanly and without having > to hack solr directly, I would need to be able to make the existing > LBHttpSolrServer and HttpShardHandlerFactory more amenable to extension, I > can then override the default load balancer using solr's plugin mechanism. > > So my question is, if I made a patch to make the load balancer more > pluggable, is this something that would be acceptable and if so what do I do > next? > > Phil > > ______________________________________________________________________ > "brightsolid" is used in this email to collectively mean brightsolid online > innovation limited and its subsidiary companies brightsolid online publishing > limited and brightsolid online technology limited. > findmypast.co.uk is a brand of brightsolid online publishing limited. > brightsolid online innovation limited, Gateway House, Luna Place, Dundee > Technology Park, Dundee DD2 1TP. Registered in Scotland No. SC274983. > brightsolid online publishing limited, The Glebe, 6 Chapel Place, Rivington > Street, London EC2A 3DQ. Registered in England No. 04369607. > brightsolid online technology limited, Gateway House, Luna Place, Dundee > Technology Park, Dundee DD2 1TP. Registered in Scotland No. SC161678. > > Email Disclaimer > > This message is confidential and may contain privileged information. You > should not disclose its contents to any other person. If you are not the > intended recipient, please notify the sender named above immediately. It is > expressly declared that this e-mail does not constitute nor form part of a > contract or unilateral obligation. Opinions, conclusions and other > information in this message that do not relate to the official business of > brightsolid shall be understood as neither given nor endorsed by it. > ______________________________________________________________________ > This email has been scanned by the brightsolid Email Security System. Powered > by MessageLabs > ______________________________________________________________________
