The point was that a lot of people think server_spec is somehow required to test "behind" Ansible to see what it did what it said it did.
It's not. The reason is because this is the whole purpose of a configuration management system - to say "the state shall be X" and make it so. Thus what you really want to do is run integration tests. In Ansible, especially in doing a rolling update, calling a test script prior to your "post_tasks" section where you add something back to a load balancer is a GREAT way to make sure failed nodes don't hop back in. (set max_failed_percentage: X first to choose how many nodes can fail before the update is stopped). assert, wait_for, script, and stat all make good modules to drive some basic testing. I definitely would disagree that anyone would have to look at infrastructure *like code*, that's the whole point of Ansible is saying this is over complication. Ansible playbooks aren't code, they are more lists of things that should be done, or true, in order. The best way of testing is often just to do a full playbook run against a stage environment, and then run your test battery against that stage environment. That is often going to be something a lot more involved that your QA team might write - check to make sure the app actually works, rather than what server_spec does, which is usually (in all examples I've seen) usually weak stuff like "is this port open". Which doesn't prove your app actually works -- and has the added overhead of (A) being a different language (B) being a duplication of effort, etc. I owe a blog post on this, which I hope to get done this week. TBD. So, that's not saying no to testing, but it's saying no to writing tests that are a mirror of the low level descriptions of your infrastructure that you already wrote playbooks for. Integration testing is where it's at, and having ansible playbooks call those -- especially in a rolling update -- is a GREAT idea, since ansible is a fail-fast system and it will show up in your server report at the bottom. There are some systems that will keep configuring as much as you can, but Ansible's going to raise it to your attention and make you fix it :) On Fri, May 23, 2014 at 9:43 AM, Guy Matz <[email protected]> wrote: > Dear Ansible (Michael deHaan?), > Hello! I've searched the google & the mailing list for playbook testing > strategies and haven't found anything that looks a consensus on how this > should be done. Perhaps there are some projects, e.g. ansible-spec or > ansible-cucumber, in the pipeline, but until then, is there a "best > practice" for ansible testing? > > At the NYC ansible-fest the other day there was a talk about testing, but > the method discussed involved using servermaint, which is ruby and seems to > irritate python people . . . the moderator of the fest (Adm. deHaan) said > to instead use the scripts & fail modules - at least I think that's what > was said - but I haven't been able to find anything about them pertaining > to testing. Does anyone have a doc about this? An example? > > Thanks a lot!!! > Guy > > -- > You received this message because you are subscribed to the Google Groups > "Ansible Project" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/ansible-project/e6672400-886f-42c0-b1f4-7d5bc7322992%40googlegroups.com<https://groups.google.com/d/msgid/ansible-project/e6672400-886f-42c0-b1f4-7d5bc7322992%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Ansible Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CA%2BnsWgyME%2BPbWu8J%3DS%2BM435uSXzHdU-hvX6PdHeOp3C29ubJVA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
