Yes, we have plan to eventually have to shard the clusters - that will go hand in hand with how rest of the system gets partitioned as well (swim lanes). The other considerations for these lanes will be geo location etc (in a AWS context, zones in east coast will be used for swim lanes that cater to customer in that region.
p.s we are not yet in AWS, but it is part of medium term strategy. On Tue, Oct 7, 2014 at 9:12 AM, Jack Krupansky <j...@basetechnology.com> wrote: > You'll have to do a proof of concept test to determine how many > collections Solr/SolrCloud can handle. > > With a very large number of customers you may have to do sharding of the > clusters themselves - limit each cluster to however many > customers/colllections work well (100? 250?) and then have separate > clusters for larger groups of customers, maybe with a smaller cluster with > a collection that maps the customer ID to a Solr cluster, and then the > application layer can direct requests to the Solr cluster that owns that > customer. > > -- Jack Krupansky > > -----Original Message----- From: Manoj Bharadwaj > Sent: Tuesday, October 7, 2014 8:27 AM > To: solr-user@lucene.apache.org > Subject: Advise on an architecture with lot of cores > > > Hi folks, > > My team inherited a SOLR setup with an architecture that has a core for > every customer. We have a few different types of cores, say "A", "B", C", > and for each one of this there is a core per customer - namely "A1", > "A2"..., "B1", "B2"... Overall we have over 600 cores. We don't know the > history behind the current design - the exact reasons why it was done the > way it was done - one probable consideration was to ensure a customer data > separate from other. > > We want to go to a single core per type architecture, and move on to SOLR > cloud as well in near future to achieve sharding via the features cloud > provides. > > Further aspects such as monitoring become easier as well. We will need to > watch and tune the caches for the different pattern of hits that we see. > > Is there anything else to evaluate before we move to a single core per type > setup? > > We are using 4.4.0 currently and will be moving to latest 4.10.1 as a part > of the redesign as well. > > Regards > Manoj >
