Just my two cents, but why not put your data on EBS volumes and decouple from the AMI? This way you're storing the collections in the "amazon suggested" way: https://aws.amazon.com/premiumsupport/knowledge-center/instance-store-vs-ebs/
Also saves the (potentially error-prone) step of creating a new AMI when data has changed. Kyle On Fri, Jun 28, 2019 at 8:27 AM chris <ch...@yeeplusplus.com> wrote: > I have a cluster of 100 shards on 100 nodes, with solr 7.5, running in > AWS.The use case is read-dominant, with ingestion performed about once per > week. There are about 84 billion documents in the cluster. It is unused on > weekends and only used during normal business hours M-F.What I do now is > after each round of ingestion, create a new set of AMIs, then terminate > each instance.The next morning, the cluster is restarted by creating a new > set of spot requests, using the most recent AMIs. At the end of the day, > the cluster is turned off by terminating the instances (if no data was > changed), or by creating a new set of AMIs and then terminating the > instances.Is there a better way to do this? I'm not facing any real > problems with this setup, but I want to make sure I'm not missing something > obvious.Thanks,Chris -- Kyle Fransham Vice President, Research & Development *Business Protection You Can Count On* 104 Schneider Road | Kanata, Ontario K2K 1Y2 <https://www.google.ca/maps/place/104+Schneider+Rd,+Kanata,+ON+K2K+1Y2/@45.340043,-75.9036994,17z/data=!3m1!4b1!4m2!3m1!1s0x4cd1fffa70d4d3f3:0xddf444ae3573dcc5> tel 613-729-1100 | mobile 613-897-9414 <613-xxx-yyyy> | www.supernaeyeglass.com <https://www.facebook.com/SupernaNET> <https://twitter.com/SupernaNet> -- CONFIDENTIALITY NOTICE: The information contained in this email is privileged and confidential and intended only for the use of the individual or entity to whom it is addressed. If you receive this message in error, please notify the sender immediately at 613-729-1100 and destroy the original message and all copies. Thank you.
