Re: Rule-based replication or sharing

On 10/2/2018 9:11 AM, Chuck Reynolds wrote: Until we move to Solr 7.5 is there a way that we can control sharding with the core.properties file? It seems to me that you use to be able to put a core.properties file in the Solr home path with something like the following. coreNodeName=bts_shard

Re: Rule-based replication or sharing

Thanks Varun, Until we move to Solr 7.5 is there a way that we can control sharding with the core.properties file? It seems to me that you use to be able to put a core.properties file in the Solr home path with something like the following. coreNodeName=bts_shard3_01 shard=shard3 collection=BT

Re: Rule-based replication or sharing

Hi Chuck, I was chatting with Noble offline and he suggested we could use this starting 7.5 {replica:'#EQUAL', shard : ''#EACH' , sysprop.az :'#EACH'} where "az" is a sysprop while starting each solr instance ( -Daz=us-east-1 ) It's documented https://lucene.apache.org/solr/guide/7_5/solrcloud-

Re: Rule-based replication or sharing

Hi Chuck, I was chatting with Noble offline and he suggested we could use this starting 7.5 *{replica:'#EQUAL', shard : ''#EACH' , sysprop.az :'#EACH'}* where "az" is a sysprop while starting each solr instance ( -Daz=us-east-1 ) It's documented https://lucene.apache.org/sol

Re: Rule-based replication or sharing

Yes, it uses a the autoscaling policies to achieve the same. Please refer to the documentation here https://lucene.apache.org/solr/guide/7_5/solrcloud-autoscaling-policy-preferences.html On Thu, Sep 27, 2018, 02:11 Chuck Reynolds wrote: > Noble, > > Are you saying in the latest version of Solr t

Re: Rule-based replication or sharing

Noble, Are you saying in the latest version of Solr that this would work with three instances of Solr running on each server? If so how? Thanks again for your help. On 9/26/18, 9:11 AM, "Noble Paul" wrote: I'm not sure if it is pertinent to ask you to move to the latest Solr which h

Re: Rule-based replication or sharing

I'm not sure if it is pertinent to ask you to move to the latest Solr which has the policy based replica placement. Unfortunately, I don't have any other solution I can think of On Wed, Sep 26, 2018 at 11:46 PM Chuck Reynolds wrote: > > Noble, > > So other than manually moving replicas of shard d

Re: Rule-based replication or sharing

Noble, So other than manually moving replicas of shard do you have a suggestion of how one might accomplish the multiple availability zone with multiple instances of Solr running on each server? Thanks On 9/26/18, 12:56 AM, "Noble Paul" wrote: The rules suggested by Steve is correct. I

Re: Rule-based replication or sharing

The rules suggested by Steve is correct. I tested it locally and I got the same errors. That means a bug exists probably. All the new development efforts are invested in the new policy feature .https://lucene.apache.org/solr/guide/7_4/solrcloud-autoscaling-policy-preferences.html The old one is go

Re: Rule-based replication or sharing

Shawn, Thanks for the info. We’ve been running this way for the past 4 years. We were running on very large hardware, 20 physical cores with 256 gigs of ram with 3 billion document and it was the only way we could take advantage of the hardware. Running 1 Solr instance per server never gave

Re: Rule-based replication or sharing

On 9/25/2018 9:21 AM, Chuck Reynolds wrote: Each server has three instances of Solr running on it so every instance on the server has to be in the same replica set. You should be running exactly one Solr instance per server.  When evaluating rules for replica placement, SolrCloud will treat e

Re: Rule-based replication or sharing

Steve, Sorry must have omitted it from a past response. Here is what came back from the response. 40091org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Could not identify nodes matching the rules [{ "shard":"*", "replica":"1", "sysprop.AWSAZ":"AZ1"}, { "shard

Re: Rule-based replication or sharing

Chuck, see my responses inline below: > On Sep 25, 2018, at 12:50 PM, Chuck Reynolds wrote: > The bottom line is I guess I'm confused by the documentation and the > reference to replicas. Normally when referring to replicas in the > documentation it is referring to the number of times you want

Re: Rule-based replication or sharing

Steve, No doubt I confused you. I'm confused myself__ When I said replica set what I was referring to was one of the three replicas of the data. Each replica needing to be in a different AZ. What is a "replica set”? And why does each instance of Solr (referred to in the reference guide as a

Re: Rule-based replication or sharing

Hi Chuck, see my replies inline below: > On Sep 25, 2018, at 11:21 AM, Chuck Reynolds wrote: > > So we have 90 server in AWS, 30 servers per AZ's. > 90 shards for the cluster. > Each server has three instances of Solr running on it so every instance on > the server has to be in the same replica

Re: Rule-based replication or sharing

Steve, Yes I set the set system property AWSAZ and I've checked the java properties in Solr and I can see them. It maybe the way we are configuring Solr so let me explain that first. So we have 90 server in AWS, 30 servers per AZ's. 90 shards for the cluster. Each server has three instances of

Re: Rule-based replication or sharing

Chuck, The default Snitch is the one that’s used if you don’t specify one in a rule. The sysprop.* tag is provided by the default Snitch. The only thing that seems wrong to me in your rules is “replica:1”, “replica:2”, and “replica:3” - these say that exactly one, two, and three replicas of e

Re: Rule-based replication or sharing

Steve, I wasn't able to get the sysprop to work. I think maybe there is a disconnect on my part. From the documentation it looks like I can only use the sysprop tag if I'm using a Snitch. Is that correct. I can't find any example of anyone using the default Snitch. Here is what I have for m

Re: Rule-based replication or sharing

Thanks Steve, I saw this option but it will mean a bit of re-working our automation to implement. The documentation talks about an EC2Snitch. I wish it worked like Cassandra where you just say that's the one you're using and it figures out how to replicate the data On 9/21/18, 2:40 PM, "Stev

Re: Rule-based replication or sharing

On 9/21/2018 2:07 PM, Chuck Reynolds wrote: I'm using Solr 6.6 and I want to create a 90 node cluster with a replication factor of three. I'm using AWS EC2 instances and I have a requirement to replicate the data into 3 AWS availability zones. So 30 servers in each zone and I don't see a create

Re: Rule-based replication or sharing

Hi Chuck, One way to do it is to set a system property on the JVM running each Solr node, corresponding to the the AWS availability zone on which the node is hosted. For example, you could use sysprop “AWSAZ”, then use rules like: replica:<2,sysprop.AWSAZ:us-east-1 replica:<2,sysprop.AWSA

Rule-based replication or sharing

I'm using Solr 6.6 and I want to create a 90 node cluster with a replication factor of three. I'm using AWS EC2 instances and I have a requirement to replicate the data into 3 AWS availability zones. So 30 servers in each zone and I don't see a create collection rule that will put one replica i