Hello will,
i tried to do it with block and serial no it does not work say's block cant
have serial
tasks:
- name: block check
block:
- name: run this shell
shell: 'systemctl restart "{{zookeeper_service_name}}"'
- name: debug
debug:
msg: "running my task"
- name: now run this task
shell: timeout -k 3 1m sh -c 'until nc -zv localhost
{{hostvars[inventory_hostname].zk_port}}; do sleep 1; done'
when:
- not zkmode is search('leader')
serial: 1
~
On Wednesday, November 1, 2023 at 3:39:54 PM UTC+5:30 Sameer Modak wrote:
> Let me try with block and serial and get back to you
>
> On Wednesday, November 1, 2023 at 5:33:14 AM UTC+5:30 Will McDonald wrote:
>
>> Edit: s/along with a failed_when/along with wait_for/
>>
>> On Tue, 31 Oct 2023 at 23:58, Will McDonald <[email protected]> wrote:
>>
>>> I don't entirely understand your approach, constraints or end-to-end
>>> requirements here, but trying to read between the lines...
>>>
>>> 1. You have a cluster of zookeeper nodes (presumably 2n+1 so 3, 5 or
>>> more nodes)
>>> 2. You want to do a rolling restart of these nodes 1 at a time, wait for
>>> the node to come back up, check it's functioning, and if that doesn't work,
>>> fail the run
>>> 3. With your existing approach you can limit the restart of a service
>>> using throttle at the task level, but then don't know how to handle failure
>>> in a subsequent task
>>> 4. You don't think wait_for will work because you only throttle on the
>>> restart task
>>>
>>> (Essentially you want your condition "has the service restarted
>>> successfully" to be in the task itself.)
>>>
>>> Again some thoughts that might help you work through this...
>>>
>>> 1. Any reason you couldn't just use serial at a playbook level? If so,
>>> what is that?
>>> 2. If you must throttle rather than serial, consider using it in a block
>>> along with a failed_when
>>> 3. Try and avoid using shell and use builtin constructs like service,
>>> it'll save you longer term pain
>>>
>>> Read through the links I posted earlier and explain what might stop you
>>> using the documented approach.
>>>
>>> This post from Vladimir on Superuser might be useful too:
>>> https://superuser.com/questions/1664197/ansible-keyword-throttle (loads
>>> of other 2n+1 rolling update/restart examples out there too:
>>> https://stackoverflow.com/questions/62378317/ansible-rolling-restart-multi-cluster-environment
>>> )
>>>
>>>
>>>
>>>
>>> On Tue, 31 Oct 2023 at 17:54, Sameer Modak <[email protected]>
>>> wrote:
>>>
>>>> Hello Will,
>>>>
>>>> I have used throttle so that part is sorted. But i dont think wait_for
>>>> works here for example.
>>>> task 1 restart. <--- now in this task already he has restarted all
>>>> hosts one by one
>>>> task 2 wait_for <-- this will fail if port does not come up but no use
>>>> because restart is triggered.
>>>>
>>>> we just want to know if in one task it restarts and checks if fails
>>>> aborts play thats it. Now we got the results but used shell module.
>>>>
>>>>
>>>>
>>>>
>>>> On Tuesday, October 31, 2023 at 7:53:31 PM UTC+5:30 Will McDonald wrote:
>>>>
>>>>> I'd suggest reading up on rolling updates using serial:
>>>>>
>>>>>
>>>>> https://docs.ansible.com/ansible/latest/playbook_guide/guide_rolling_upgrade.html#the-rolling-upgrade
>>>>>
>>>>> https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_strategies.html#setting-the-batch-size-with-serial
>>>>>
>>>>> You can use wait_for or wait_for_connection to ensure service
>>>>> availability before continuing:
>>>>>
>>>>>
>>>>> https://docs.ansible.com/ansible/latest/collections/ansible/builtin/wait_for_module.html
>>>>>
>>>>> https://docs.ansible.com/ansible/latest/collections/ansible/builtin/wait_for_connection_module.html
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, 31 Oct 2023 at 14:08, Sameer Modak <[email protected]>
>>>>> wrote:
>>>>>
>>>>>> restart service, check if service is ready to accept connection
>>>>>> because it takes time to come up. Once we sure its listening on port
>>>>>> then
>>>>>> only move to next host. unless dont move because we can only afford to
>>>>>> have
>>>>>> one service down at a time.
>>>>>>
>>>>>> is there any to short hand or ansible native way to handle this using
>>>>>> ansible module.
>>>>>>
>>>>>>
>>>>>> code:
>>>>>>
>>>>>> name: Restart zookeeper followers
>>>>>>
>>>>>> throttle: 1
>>>>>>
>>>>>> any_errors_fatal: true
>>>>>>
>>>>>> shell: |
>>>>>>
>>>>>> systemctl restart {{zookeeper_service_name}}
>>>>>>
>>>>>> timeout 22 sh -c 'until nc localhost {{zookeeper_server_port}};
>>>>>> do sleep 1; done'
>>>>>>
>>>>>> when: not zkmode.stdout_lines is search('leader')
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Ansible Project" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to [email protected].
>>>>>> To view this discussion on the web visit
>>>>>> https://groups.google.com/d/msgid/ansible-project/67ca5f13-855d-4d40-a47a-c0fbe11ea3b5n%40googlegroups.com
>>>>>>
>>>>>> <https://groups.google.com/d/msgid/ansible-project/67ca5f13-855d-4d40-a47a-c0fbe11ea3b5n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>> .
>>>>>>
>>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Ansible Project" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/ansible-project/3370b143-050a-4a14-a858-f5abe60c2678n%40googlegroups.com
>>>>
>>>> <https://groups.google.com/d/msgid/ansible-project/3370b143-050a-4a14-a858-f5abe60c2678n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/ansible-project/69417f84-b761-4008-8284-ac644d3384f7n%40googlegroups.com.
