Hi, I am trying to come up with a repeatable process for deploying a Solr Cloud cluster from scratch along with the appropriate security groups, auto scaling groups, and custom Solr plugin code. I saw that LucidWorks created a Solr Scale Toolkit but that seems to be more of a one-shot deal than really setting up your environment for the long-haul. Here is were we are at right now:
1. ZooKeeper ensemble is easily brought up via a Cloud Formation Script 2. We have an RPM built to lay down the Solr distribution + Custom plugins + Configuration 3. Solr machines come up and connect to ZK Now, we are using Puppet which could easily create the core.properties file for the corresponding core and have ZK get bootstrapped but that seems to be a no-no these days... So, can anyone think of a way to get ZK bootstrapped automatically with pre-configured Collection configurations? Also, is there a recommendation on how to deal with machines that are coming/going? As I see it machines will be getting spun up and terminated from time to time and we need to have a process of dealing with that, the first idea was to just use a common node name so if a machine was terminated a new one can come up and replace that particular node but on second thought it would seem to require an auto scaling group *per* node (so it knows what node name it is). For a large cluster this seems crazy from a maintenance perspective, especially if you want to be elastic with regard to the number of live replicas for peak times. So, then the next idea was to have some outside observer listen to when new ec2 instances are created or terminated (via CloudWatch SQS) and make the appropriate API calls to either add the replica or delete it, this seems doable but perhaps not the simplest solution that could work. I was hoping others have already gone through this and have valuable advice to give, we are trying to setup Solr Cloud the "right way" so we don't get nickel-and-dimed to death from an O&M perspective. Thanks, -Steve
