I third that request. Would greatly appreciate taking a look at that diagram!
Regards, Jonathan On Wed, Apr 6, 2011 at 9:12 AM, Isan Fulia <isan.fu...@germinait.com> wrote: > Hi Ephraim/Jen, > > Can u share that diagram with all.It may really help all of us. > Thanks, > Isan Fulia. > > On 6 April 2011 10:15, Tirthankar Chatterjee <tchatter...@commvault.com > >wrote: > > > Hi Jen, > > Can you please forward the diagram attachment too that Ephraim sent. :-) > > Thanks, > > Tirthankar > > > > -----Original Message----- > > From: Jens Mueller [mailto:supidupi...@googlemail.com] > > Sent: Tuesday, April 05, 2011 10:30 PM > > To: solr-user@lucene.apache.org > > Subject: Re: FW: Very very large scale Solr Deployment = how to do > (Expert > > Question)? > > > > Hello Ephraim, > > > > thank you so much for the great Document/Scaling-Concept!! > > > > First I think you really should publish this on the solr wiki. This > > approach is nowhere documented there and not really obvious for newbies > and > > your document is great and explains this very well! > > > > Please allow me to further questions regarding your document: > > 1.) Is it correct, that you mean by "DB" the Origin-Data-Source of the > data > > that is fed into the Solr "Cloud" for searching? > > > > 2.) Solr Aggregator: This term did not yeald any google results, but is a > > very important aspect of your design (and this was the missing piece for > me > > when thinking about solr architectures): Is it cocrrec that the > > "aggregators" are simply tomcat instances, with the solr webapp deployed? > > These Aggregators do not have their own index but only run the solr > webapp > > and I access them via the ?shard= parameter giving the shards I want to > > query? (So in the end they aggreate the data of the shards but do not > have > > their own data). This is really an important aspect that is not > documented > > well enough in the solr documentation. > > > > Thank you very much! > > Jens > > > > > > 2011/4/5 Ephraim Ofir <ephra...@icq.com> > > > > > of course the attachment didn't get to the list, so here it is if you > > > want it... > > > > > > Ephraim Ofir > > > > > > > > > -----Original Message----- > > > From: Ephraim Ofir > > > Sent: Tuesday, April 05, 2011 10:20 AM > > > To: 'solr-user@lucene.apache.org' > > > Subject: RE: Very very large scale Solr Deployment = how to do (Expert > > > Question)? > > > > > > I'm not sure about the scale you're aiming for, but you probably want > > > to do both sharding and replication. There's no central server which > > > would be the bottleneck. The guidelines should probably be something > > like: > > > 1. Split your index to enough shards so it can keep up with the update > > > rate. > > > 2. Have enough replicates of each shard master to keep up with the > > > rate of queries. > > > 3. Have enough aggregators in front of the shard replicates so the > > > aggregation doesn't become a bottleneck. > > > 4. Make sure you have good load balancing across your system. > > > > > > Attached is a diagram of the setup we have. You might want to look > > > into SolrCloud as well. > > > > > > Ephraim Ofir > > > > > > > > > -----Original Message----- > > > From: Jens Mueller [mailto:supidupi...@googlemail.com] > > > Sent: Tuesday, April 05, 2011 4:25 AM > > > To: solr-user@lucene.apache.org > > > Subject: Very very large scale Solr Deployment = how to do (Expert > > > Question)? > > > > > > Hello Experts, > > > > > > > > > > > > I am a Solr newbie but read quite a lot of docs. I still do not > > > understand what would be the best way to setup very large scale > > > deployments: > > > > > > > > > > > > Goal (threoretical): > > > > > > A.) Index-Size: 1 Petabyte (1 Document is about 5 KB in Size) > > > > > > B) Queries: 100000 Queries/ per Second > > > > > > C) Updates: 100000 Updates / per Second > > > > > > > > > > > > > > > Solr offers: > > > > > > 1.) Replication => Scales Well for B) BUT A) and C) are not > > > satisfied > > > > > > > > > 2.) Sharding => Scales well for A) BUT B) and C) are not satisfied > > > (=> As > > > I understand the Sharding approach all goes through a central server, > > > that dispatches the updates and assembles the quries retrieved from > > > the different shards. But this central server has also some capacity > > > limits...) > > > > > > > > > > > > > > > What is the right approach to handle such large deployments? I would > > > be thankfull for just a rough sketch of the concepts so I can > > > experiment/search further... > > > > > > > > > Maybe I am missing something very trivial as I think some of the "Solr > > > Users/Use Cases" on the homepage are that kind of large deployments. > > > How are they implemented? > > > > > > > > > > > > Thanky very much!!! > > > > > > Jens > > > > > ******************Legal Disclaimer*************************** > > "This communication may contain confidential and privileged > > material for the sole use of the intended recipient. Any > > unauthorized review, use or distribution by others is strictly > > prohibited. If you have received the message in error, please > > advise the sender by reply email and delete the message. Thank > > you." > > ********************************************************* > > > > > > -- > Thanks & Regards, > Isan Fulia. >
