Re: how to replicate Solr Cloud

2013-06-26 Thread Erick Erickson
On the lengthy TODO list is making SolrCloud nodes "rack aware" that should help with this, but it's not real high in the priority queue as I recall. The current architecture sends updates and requests all over the cluster, so there are lots of messages that go across the presumably expensive pipe

Re: how to replicate Solr Cloud

2013-06-25 Thread Otis Gospodnetic
Uh, I remember that email, but can't recall where we did it will try to recall it some more and reply if I can manage to dig it out of my brain... Otis -- Solr & ElasticSearch Support -- http://sematext.com/ Performance Monitoring -- http://sematext.com/spm On Tue, Jun 25, 2013 at 2:24 PM,

Re: how to replicate Solr Cloud

2013-06-25 Thread Walter Underwood
Also, you have to track two sets of batches, failures, and retries. --wunder On Jun 25, 2013, at 11:30 AM, Kevin Osborn wrote: > Jason, > > My initial reluctance to indexing directly to both data centers is that we > are doing a lot of bulk loading through CSV handler. We never get just 1 > do

Re: how to replicate Solr Cloud

2013-06-25 Thread Kevin Osborn
Jason, My initial reluctance to indexing directly to both data centers is that we are doing a lot of bulk loading through CSV handler. We never get just 1 document at a time. It comes in large batch updates. And now we would have to send the batch updates twice. That is not to say that we won't g

Re: how to replicate Solr Cloud

2013-06-25 Thread Kevin Osborn
Otis, I did actually stumble upon this link. http://comments.gmane.org/gmane.comp.jakarta.lucene.solr.user/74870 This was from you. You were attempting to replicate data from SolrCloud to some other slaves for heavy-duty queries. You said that you accomplished this. Can you provide a few pointer

Re: how to replicate Solr Cloud

2013-06-25 Thread Jason Hellman
Kevin, I can imagine this working if you consider your second data center a pure slave relationship to your SolrCloud cluster. I haven't tried it, but I don't see why the solrconfig.xml can't identify as a master allowing you to call any of your cores in the cluster to replicate out. That bei

Re: how to replicate Solr Cloud

2013-06-25 Thread Otis Gospodnetic
I think what is needed is a Leader that, while being a Leader for its own Slice in its local Cluster and Collection (I think I'm using all the latest terminology correctly here), is at the same time a Replica of its own Leader counterpart in the "Primary Cluster". Not currently possible, AFAIK. Or