Re: Sharding and Replication

2018-01-02 Thread Shawn Heisey
On 12/27/2017 3:02 AM, Gopesh Sharma wrote: > We had two system where we were doing Master Slave Replication, we used to do > delta-import every 24 hours since we did not want the near real-time data. > Now since our data is increasing we thought of adding one more machine to the > master slave

RE: Sharding and Replication

2013-08-09 Thread Alexey Kozhemiakin
ific cases and environments. Alex -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Sunday, June 23, 2013 20:41 To: solr-user@lucene.apache.org Subject: Re: Sharding and Replication Asif: Thanks, this is great info and may add to the priority of m

Re: Sharding and Replication

2013-06-23 Thread Erick Erickson
Asif: Thanks, this is great info and may add to the priority of making this configurable. I raised a JIRA, see: https://issues.apache.org/jira/browse/SOLR-4956 and feel free to add anything you'd like or correct anything I didn't get right. Best Erick On Sat, Jun 22, 2013 at 10:16 PM, Asif wrot

Re: Sharding and Replication

2013-06-22 Thread Asif
Erick, Its a completely practical problem - we are exploring Solr to build a real time analytics/data solution for a system handling about 1000 qps. We have various metrics that are stored as different collections on the cloud, which means very high amount of writes. The cloud also needs to suppor

Re: Sharding and Replication

2013-06-22 Thread Erick Erickson
Yeah, there's been talk of making this configurable, but there are more pressing priorities so far. So just to be clear, is this theoretical or practical? I know of several very high-performance situations where 1,000 updates/sec (and I'm assuming that it's 1,000 docs/sec not 1,000 batches of 1,00

Re: Sharding and Replication

2013-06-21 Thread Asif
Erick, Thanks for your reply. You are right about 10 updates being batch up - It was hard to figure out due to large number of updates/logging that happens in our system. We are batching 1000 updates every time. Here is my observation from leader and replica - 1. Leader logs are clearly indica

Re: Sharding and Replication

2013-06-21 Thread Erick Erickson
Update are batched, but it's on a per-request basis. So, if you're sending one document at a time you'll won't get any batching. If you send 10 docs at a time and they happen to go to 10 different shards, you'll get 10 different update requests. If you're sending 1,000 docs per update you' should

Re: Sharding and Replication clarification

2013-06-20 Thread Shalin Shekhar Mangar
On Wed, Jun 19, 2013 at 11:12 PM, Asif wrote: > Hi, > > I had questions on implementation of Sharding and Replication features of > Solr/Cloud. > > 1. I noticed that when sharding is enabled for a collection - individual > requests are sent to each node serving as a shard. Yes, search requests ar