Edit: s/along with a failed_when/along with wait_for/ On Tue, 31 Oct 2023 at 23:58, Will McDonald <[email protected]> wrote:
> I don't entirely understand your approach, constraints or end-to-end > requirements here, but trying to read between the lines... > > 1. You have a cluster of zookeeper nodes (presumably 2n+1 so 3, 5 or more > nodes) > 2. You want to do a rolling restart of these nodes 1 at a time, wait for > the node to come back up, check it's functioning, and if that doesn't work, > fail the run > 3. With your existing approach you can limit the restart of a service > using throttle at the task level, but then don't know how to handle failure > in a subsequent task > 4. You don't think wait_for will work because you only throttle on the > restart task > > (Essentially you want your condition "has the service restarted > successfully" to be in the task itself.) > > Again some thoughts that might help you work through this... > > 1. Any reason you couldn't just use serial at a playbook level? If so, > what is that? > 2. If you must throttle rather than serial, consider using it in a block > along with a failed_when > 3. Try and avoid using shell and use builtin constructs like service, > it'll save you longer term pain > > Read through the links I posted earlier and explain what might stop you > using the documented approach. > > This post from Vladimir on Superuser might be useful too: > https://superuser.com/questions/1664197/ansible-keyword-throttle (loads > of other 2n+1 rolling update/restart examples out there too: > https://stackoverflow.com/questions/62378317/ansible-rolling-restart-multi-cluster-environment > ) > > > > > On Tue, 31 Oct 2023 at 17:54, Sameer Modak <[email protected]> > wrote: > >> Hello Will, >> >> I have used throttle so that part is sorted. But i dont think wait_for >> works here for example. >> task 1 restart. <--- now in this task already he has restarted all hosts >> one by one >> task 2 wait_for <-- this will fail if port does not come up but no use >> because restart is triggered. >> >> we just want to know if in one task it restarts and checks if fails >> aborts play thats it. Now we got the results but used shell module. >> >> >> >> >> On Tuesday, October 31, 2023 at 7:53:31 PM UTC+5:30 Will McDonald wrote: >> >>> I'd suggest reading up on rolling updates using serial: >>> >>> >>> https://docs.ansible.com/ansible/latest/playbook_guide/guide_rolling_upgrade.html#the-rolling-upgrade >>> >>> https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_strategies.html#setting-the-batch-size-with-serial >>> >>> You can use wait_for or wait_for_connection to ensure service >>> availability before continuing: >>> >>> >>> https://docs.ansible.com/ansible/latest/collections/ansible/builtin/wait_for_module.html >>> >>> https://docs.ansible.com/ansible/latest/collections/ansible/builtin/wait_for_connection_module.html >>> >>> >>> >>> >>> >>> On Tue, 31 Oct 2023 at 14:08, Sameer Modak <[email protected]> >>> wrote: >>> >>>> restart service, check if service is ready to accept connection because >>>> it takes time to come up. Once we sure its listening on port then only move >>>> to next host. unless dont move because we can only afford to have one >>>> service down at a time. >>>> >>>> is there any to short hand or ansible native way to handle this using >>>> ansible module. >>>> >>>> >>>> code: >>>> >>>> name: Restart zookeeper followers >>>> >>>> throttle: 1 >>>> >>>> any_errors_fatal: true >>>> >>>> shell: | >>>> >>>> systemctl restart {{zookeeper_service_name}} >>>> >>>> timeout 22 sh -c 'until nc localhost {{zookeeper_server_port}}; do >>>> sleep 1; done' >>>> >>>> when: not zkmode.stdout_lines is search('leader') >>>> >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Ansible Project" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/ansible-project/67ca5f13-855d-4d40-a47a-c0fbe11ea3b5n%40googlegroups.com >>>> <https://groups.google.com/d/msgid/ansible-project/67ca5f13-855d-4d40-a47a-c0fbe11ea3b5n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Ansible Project" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/ansible-project/3370b143-050a-4a14-a858-f5abe60c2678n%40googlegroups.com >> <https://groups.google.com/d/msgid/ansible-project/3370b143-050a-4a14-a858-f5abe60c2678n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Ansible Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAKtKohTkSPe3d9cyi_sCLRsbv4ARv3tSDv%2Bxk8OP-xWw6zXzoQ%40mail.gmail.com.
