The AMIs are Red Hat (not Amazon's) and the instances are properly sized for the environment (t1.micro for ZK, m3.xlarge for Solr). I do plan to add hooks for a clean shutdown of Solr when the VM is shut down, but if Solr takes too long, AWS may clobber it anyway. One frustrating part of auto scaling shutdown is that you can't log into the 'vanishing machine' to view the logs.
Peter On Fri, Dec 12, 2014 at 5:21 PM, Chris Hostetter <hossman_luc...@fucit.org> wrote: > > > : No, I wasn't aware of these. I will give that a try. If I stop the Solr > : jetty service manually, things recover fine, but the hang occurs when I > : 'stop' or 'terminate' the EC2 instance. The Zookeeper leader reports a > > I don't know squat about AWS Auto-Scaling, (and barely anything about AWS) > but what you describe makes it sound like maybe your "machine" (ie AMI?) > isn't really configured very well? > > Do you have some init.d/systemd type scripts to ensure a clean shutdown of > Solr when the machine is shutdown/rebooted? That seems like a pretty good > idea in general (in dependent of wether you are using Auto-Scaling) and -- > assuming AWS auto-scaling does clean OS shutdowns when terminating > instances -- would probably solve your problem. It would help ensure you > would never have to wait on the timeouts -- the nodes will each explicitly > tell ZK they are going bye-bye. > > if you do have things setup so that *manually* shutting down your > instances executes a clean shutdown of solr, but AWS Auto-Scaling is > actaully totally brutal and doesn't even do a clean shutdown of your > virtual machines -- just yanks the virtual power cord -- perhaps you could > implement one of these "LifecycleHook" options that poped up when i did > some googling for "AWS Auto-Scale termination" to explicitly do a clean > shutdown of the Solr process" before the machine vanishes into thin air? > > > > -Hoss > http://www.lucidworks.com/ >
