Hey guys, quick note -- if you are sharing something, link to github, please don't attach files to the mailing list. (I really need to turn this off).
Also makes it easier to browse and look around, etc :) On Thu, Dec 12, 2013 at 2:34 AM, Stephan Buys <[email protected]>wrote: > Hi Aaron, > > Please find attached a small (and definitely imperfect/ugly/hacky) > callback plugin that I wrote for ansible, It produces xUnit output from > ansible playbook runs and integrates very nicely into Jenkins (xUnit -> > Check Version N/A http://check.sourceforge.net) > > It should be copied to {{ ansible_dir > }}/ansible/lib/ansible/callback_plugins/ > > I use it for TDD of my own infrastructure/code and use it in a pattern > very similar to what John linked to below. Complex tests are implemented as > scripts (rolename/files), and "test suites" are bundled into roles > (roles/test_server_live, roles/test_server_logon, etc). > > I've found that it works really well when integrating it into my build > processes on Jenkins to catch errors and test for expected behaviour at an > infrastructure/functional level. Of course it is up to you to creatively > use ansible to do the tests. I use a combination of modules and scripts > wrapping 3rd party tools, and rely on the return code to trigger "success" > and "failure" in ansible. > > Another note, I have ignore_errors peppered throughout the playbooks so > that all relevant tests can run, ansible's conditionals, etc make this a > very powerful setup, here is a short example: > > - name: Check vagrant user logon (should fail) > local_action: shell roles/test_auth_local/files/check_vv.sh > ignore_errors: yes > > I hope that this is a step in the right direction, with any luck someone > can polish the plugin and integrate it into the ansible codebase. Otherwise > if there is interest, please indicate as much and I'll try to get it into > github for a proper pull request or to maintain it separately. > > Kind regards, > Stephan > > > -- > You received this message because you are subscribed to the Google Groups > "Ansible Project" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > > > p.s. Humble apologies to all the Python developers for the ugly hack :) > > > On 11 Dec 2013, at 8:00 PM, John Dewey <[email protected]> wrote: > > Hi Aaron - > > I too would find it useful to have the ability to “unit test” my tasks. > However, I have opted to create a testing playbook [1] which handles > integration testing. It is not perfect, but allows for TDD/BDD, and > integrate into CI gating. > > [1] > https://github.com/blueboxgroup/ursula/tree/master/playbooks/tests/tasks > > On Wednesday, December 11, 2013 at 9:54 AM, Aaron Hunter wrote: > > I see your point but I'm not sure I agree. "Unit testing" may not be the > best term for it but it's not too far off. Consider this case instead: > installing and configuring a DHCP server. I have my senior admin write down > exactly what the DHCP deployment should look like. This is how success will > be measured. Testing this will include checking file permissions (which is > also security issue), processes are running and with the right settings, > logging , and a whole set of functionality based on the config file of the > DHCP server. > > Another admin (let's say a junior admin) writes the Ansible code to > accomplish this. When the code passes the test, it is done. Certainly this > crosses into what is traditionally called functional testing. Testing IT > infrastructure is somewhat different from testing a new software > development. I would still test what is in the Ansible script for the DHCP > server because the junior admin may have gotten it wrong and plus its still > code and you always test code. Plus I would have the Ansible scripts > initially written in a development environment with one suite of tests. > These tests will be very obtrusive and are not the same I would use to test > a successful deployment. > > To be sure, the most important part is testing that the DHCP server works > as expected (i.e., testing the config file) which is not an Ansible issue. > Still I would test everything. The issue for me is what to use to write > the tests? Cucumber? JBehave? Plain Python? I know plenty of tools designed > to test custom apps/code, I don't think the IT testing tools have caught up > yet. > > Maybe this isn't even an Ansible issue. I was just speculating on how it > might include automated TDD-like test process in the framework. > > --- > Aaron > DevOps Blog: http://www.sharknet.us > > On Wednesday, December 11, 2013 11:55:40 AM UTC-5, Michael DeHaan wrote: > > > Yes, this is super easy to already do today, basically just call your > tests at the end as the last step of your playbook. > > Executing arbitrary python code is possible, but you can use ansible > modules like get_url and fail and so on. > > If you want to push a python script, the 'script' module is awesome for > that. > > Many of users have tests integrated with their continuous deployment > process so it will fail the rolling update block before moving on the to > next, thus not taking more machines out of rotation. > > However, if you feel you have to test the file module, seriously, you're > wasting time -- if the file module doesn't work for you, how good is the > product? It would be much better to test instead for something functional, > like whether your web service is operational, rather than duplicating all > the basics of Ansible in, as you say, arbitrary python code just to make > sure Ansible works. > > There's a difference between unit and integration testing, and also in > testing a live deployment. > > Unit tests are things you run on development code. > > > > > On Wed, Dec 11, 2013 at 11:39 AM, Aaron Hunter <[email protected]>wrote: > > I come from an Agile software development background in which test driven > development (TDD) is the norm. As I write Ansible scripts, I'd like some > way of testing them. In principle, I want to test every command in a > playbook. For example, if one of my command changes the user permissions on > a file, I want a test that independently confirms that it has in fact done > so. I don't see a "test" module but I may have missed it. > > Is that something that Ansible may offer some day? I'm thinking of the > Ansible equivalent to unit testing. I believe it would require the ability > to execute arbitrary Python code in the test. The Java tests I have written > could certainly be very complex. > > I'm also curious what others do for testing using Ansible. What > frameworks, etc. > > Thanks, > Aaron > DevOps Blog: http://www.sharknet.us > > > -- > You received this message because you are subscribed to the Google Groups > "Ansible Project" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > > > > > -- > Michael DeHaan <[email protected]> > CTO, AnsibleWorks, Inc. > http://www.ansibleworks.com/ > > > -- > You received this message because you are subscribed to the Google Groups > "Ansible Project" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > > > > -- > You received this message because you are subscribed to the Google Groups > "Ansible Project" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > > > > -- Michael DeHaan <[email protected]> CTO, AnsibleWorks, Inc. http://www.ansibleworks.com/ -- You received this message because you are subscribed to the Google Groups "Ansible Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
