Re: Solr 4.0 SolrCloud with AWS Auto Scaling

2013-01-04 Thread Bill Au
thanks for pointing me to Solr's Zookeeper servlet. I will look at the source to see how I can use to fulfill my needs. Bill On Thu, Jan 3, 2013 at 6:43 PM, Mark Miller wrote: > Technically, you want to make sure zookeeper reports the node as live and > active. > > You could use the same api

Re: Solr 4.0 SolrCloud with AWS Auto Scaling

2013-01-03 Thread Mark Miller
Technically, you want to make sure zookeeper reports the node as live and active. You could use the same api that the UI uses for that - the localhost:port/solr/zookeeper (I think?) servlet. If you can't reach it for a node, it's obviously down - if you can reach it, parse the json and see if

Re: Solr 4.0 SolrCloud with AWS Auto Scaling

2013-01-03 Thread Bill Au
Thanks, Mark. That does remove the node. And it seems to do so permanently. Even when I restart Solr after unloading, it does not join the SolrCloud cluster. And I can get it to re-join the cluster by creating the core. Anyone know if there is an API to determine the state of a node. When AWS

Re: Solr 4.0 SolrCloud with AWS Auto Scaling

2013-01-03 Thread Mark Miller
http://wiki.apache.org/solr/CoreAdmin#UNLOAD - Mark On Jan 3, 2013, at 9:06 AM, Bill Au wrote: > Mark, > What do you mean by "unload them"? > > I am using an AWS load balancer with my auto scaling group in stead of > using Solr's built-in load balancer. I am no sharding my index. I am >

Re: Solr 4.0 SolrCloud with AWS Auto Scaling

2013-01-03 Thread Bill Au
Mark, What do you mean by "unload them"? I am using an AWS load balancer with my auto scaling group in stead of using Solr's built-in load balancer. I am no sharding my index. I am using SolrCloud for replication only. I am doing local search on each instance and sending all updates to the

Re: Solr 4.0 SolrCloud with AWS Auto Scaling

2013-01-03 Thread Bill Au
With AWS auto scaling, one can specify a minimum number of instances for an auto scaling group. So there should never be an insufficient number of replicas. Once can also specify a termination policy so that the newly added nodes are removed first. But with SolrCloud as long as there are enough

Re: Solr 4.0 SolrCloud with AWS Auto Scaling

2013-01-02 Thread Otis Gospodnetic
We've considered using AWS Beanstalk (hmm, what's the difference between AWS auto scaling and elastic beanstalk? not sure.) for search-lucene.com , but the idea of something adding and removing nodes seems scary. The scariest part to me is automatic removal of wrong nodes that ends up in data loss

Re: Solr 4.0 SolrCloud with AWS Auto Scaling

2013-01-02 Thread Mark Miller
On Jan 2, 2013, at 5:51 PM, Bill Au wrote: > Is anyone running Solr 4.0 SolrCloud with AWS auto scaling? > > My concern is that as AWS auto scaling add and remove instances to > SolrCloud, the number of nodes in SolrCloud Zookeeper config will grow > indefinitely as removed instances will never