Re: scale

2016-06-02 Thread John Blythe
sure. the processes we run to do linkage take hours. we're processing ~600k records, bouncing our users data up against a few data sources that act as 'sources of truth' for us for the sake of this linkage. we get the top 3 results and run some quick checks on it algorithmically to determine if we

Re: scale

2016-06-02 Thread Erick Erickson
Without having a lot more data it's hard to say anything helpful. _What_ is slow? What does "data linkage" mean exactly? Etc. Best, Erick On Thu, Jun 2, 2016 at 9:33 AM, John Blythe wrote: > hi all, > > having lots of processing happening using multiple solr cores to do some > data linkage with

Re: scale with 3 shards on single server

2012-11-08 Thread Erick Erickson
Yep, that's the usual process for growth planning. Best Erick On Wed, Nov 7, 2012 at 4:01 AM, SuoNayi wrote: > Hi all, >Because we cannot add or remove shard when solrcloud cluster has been > set up, > so we have to predict a precise shard size at first, says we need 3 shards. > but now we

Re: Scale out design patterns

2011-02-03 Thread Ganesh
nt: Friday, January 21, 2011 12:04 PM Subject: Re: Scale out design patterns > Hi Ganesh, > I'd suggest, if you have a particular dimension/field on which you could > shard your data such that the query/data breakup gets predictable, that > would be a good way to scale out e.g. if you h