Can you share the Vagrant file? And ideally playbook.yml? I have this working precisely as expected, you just need to ensure that the if statement is nested at just the right point in the Vagrantfile.
On Mon, 27 Nov 2023 at 17:44, Evan Hisey <[email protected]> wrote: > Will- > Looks like even with the cluster limit I still get 3 discrete runs, when > using the cluster example. I did a very simple play book and you can see > the gathering_facts stages gets run in triplicate: > [image: image.png] > Definitely changed the behavior but not quite were I need it to go. > However, it has given me an interesting ideas to try. > > On Sun, Nov 26, 2023 at 3:06 PM Evan Hisey <[email protected]> wrote: > >> Will- >> That was exactly the issue. I will give the bottom solution a go. I >> think that will work, I will need to play with generating the group, but I >> think t I can make it work. Thanks for the help, will update when I get >> something working or fail :) >> >> >> On Sun, Nov 26, 2023 at 11:25 AM Will McDonald <[email protected]> >> wrote: >> >>> OK, after some experimentation, I think I see what your problem might >>> be? If you do something like: >>> >>> BOX_IMAGE = "fedora/37-cloud-base" >>> NODE_COUNT = 2 >>> >>> Vagrant.configure("2") do |config| >>> >>> (1..NODE_COUNT).each do |i| >>> config.vm.define "node#{i}" do |subconfig| >>> subconfig.vm.box = BOX_IMAGE >>> subconfig.vm.hostname = "node#{i}" >>> >>> if i == NODE_COUNT >>> config.vm.provision :ansible do |ansible| >>> # Disable default limit to connect to all the machines >>> ansible.limit = "all" >>> ansible.playbook = "playbook.yml" >>> end >>> end >>> >>> end >>> end >>> >>> end >>> >>> The Vagrant Ansible provisioner fires for every VM causing multiple >>> discrete runs, you can control that to a degree with ansible.limit, the >>> hosts statement in the playbook and/or delegate_to but it would be hard to >>> do stateful cross-cluster config. >>> >>> If you do something like the following instead, this will provision all >>> 3 Vagrant boxes and then fire the provisioner* once *triggering an >>> Ansible run just for the final box: >>> >>> wmcdonald@fedora:~/working/vagrant/fedora-multi$ cat Vagrantfile >>> Vagrant.configure(2) do |config| >>> #Define the number of nodes to spin up >>> N = 3 >>> >>> #Iterate over nodes >>> (1..N).each do |node_id| >>> nid = (node_id - 1) >>> >>> config.vm.define "node#{nid}" do |node| >>> node.vm.box = "fedora/37-cloud-base" >>> node.vm.provider "virtualbox" do |vb| >>> vb.memory = "1024" >>> end >>> node.vm.hostname = "node#{nid}" >>> >>> if node_id == N >>> node.vm.provision "ansible" do |ansible| >>> ansible.limit = "all" >>> ansible.groups = { >>> "cluster-nodes" => [ >>> "node0", >>> "node1", >>> "node2", >>> ] >>> } >>> ansible.playbook = "playbook.yml" >>> end >>> end >>> >>> end >>> end >>> end >>> >>> wmcdonald@fedora:~/working/vagrant/fedora-multi130$ cat playbook.yml >>> - name: Vagrant post-provision >>> hosts: cluster_nodes >>> >>> tasks: >>> - name: Debug vars for hosts >>> debug: >>> var: ansible_play_hosts >>> >>> Note that the provisioner will run once but still parallelise like a >>> normal Ansible run would and hit each node because we're setting the hosts >>> to the group members. You could further limit with delegate_to or have one >>> cluster node in its own 'primary_node' group in addition to the >>> cluster_nodes. >>> >>> See: >>> https://everythingshouldbevirtual.com/automation/virtualization/vagrant-ansible-provisioning-multi-nodes/ >>> And another variant with per-box behaviour here: >>> https://stackoverflow.com/questions/54468546/how-to-run-an-ansible-playbook-on-a-specific-vagrant-host >>> >>> >>> >>> On Sun, 26 Nov 2023 at 00:22, Will McDonald <[email protected]> wrote: >>> >>>> There are a couple of ways you could exercise "control over the process >>>> to pull data from host 1 to be used on host 2 and 3". >>>> >>>> If you look at >>>> https://manski.net/2016/09/vagrant-multi-machine-tutorial/#multi-machine.3A-the-clever-way >>>> 3 nodes are provisioned, one as primary, then two as secondary nodes and >>>> it'd be relatively trivial to use this to key off the 'primary' node to do >>>> what you needed, I imagine. >>>> >>>> Where I've had scenarios provisioning 3 nodes of something in a 2n+1 >>>> cluster (basically anything like Mongo, Etcd, Zookeeper etc. etc.) and you >>>> need to at least temporarily choose a semi-deterministic primary I've used >>>> logic like: >>>> >>>> pre_tasks: >>>> - name: pre_tasks | cluster member role setup for multiple hosts >>>> block: >>>> - name: pre_tasks | set cluster role to primary when >>>> inventory_hostame matches random seed >>>> set_fact: >>>> cluster_role: primary >>>> when: inventory_hostname == >>>> ansible_play_hosts|random(seed=ansible_play_hosts | join()) >>>> >>>> - name: pre_tasks | set mongo replication role to secondary when >>>> inventory_hostame does not match random seed >>>> set_fact: >>>> cluster_role: secondary >>>> when: inventory_hostname != >>>> ansible_play_hosts|random(seed=ansible_play_hosts | join()) >>>> >>>> - name: pre_tasks | create a custom facts.d directory on the target >>>> host >>>> file: >>>> state: directory >>>> recurse: true >>>> path: /etc/ansible/facts.d >>>> >>>> - name: pre_tasks | persist the cluster membership role as a custom >>>> fact >>>> copy: >>>> content: | >>>> {'cluster_role':'{{ cluster_role }}'} >>>> dest: /etc/ansible/facts.d/cluster.fact >>>> mode: 0644 >>>> owner: root >>>> group: root >>>> >>>> *Warning! *This sets a *transient value* in facts.d. Which in my cases >>>> is fine for our purposes. If your cluster membership state changes >>>> post-setup, the fact would be misleading. (i.e. a node flaps and another >>>> cluster member assumes leader/primary.) >>>> >>>> You would want to replace cluster.fact with something that dynamically >>>> pulls out the cluster role membership state of a node once the >>>> cluster/replicaset/whatever topology is provisioned and configured. >>>> >>>> >>>> On Sat, 25 Nov 2023 at 23:25, Evan Hisey <[email protected]> wrote: >>>> >>>>> Definitely an edge case. Not an issue in my file atleast as written >>>>> based on my understanding of the process, but possibly an issue in my >>>>> understanding of how vagrant is executing ansible as it looks like vagrant >>>>> runs on each vm as a separate job in either case, just in parallel on each >>>>> the second time. I still need control over the process to pull data from >>>>> host 1 to be used on host 2 and 3, which if it is running in parallel as >>>>> multiple jobs would still be an issue. If it in fact runs a single ansible >>>>> playbook across the inventory, then that could work, and be the opposite >>>>> of >>>>> how I am understanding vagrant ansible provider works. I would need to >>>>> refactor a large chunk of the application code to support that, but that >>>>> can be easily done. >>>>> >>>>> On Sat, Nov 25, 2023 at 4:44 PM Will McDonald <[email protected]> >>>>> wrote: >>>>> >>>>>> I think you may be misunderstanding me, or I'm misunderstanding you. >>>>>> >>>>>> Just for clarity's sake, the flow you would like is: >>>>>> >>>>>> 1. An Ansible control node runs a playbook (or role) on >>>>>> <controlnode> targeting a machine, <targetmachine> >>>>>> 2. The <targetmachine> is configured to run as a Vagrant host >>>>>> with a virtualisation provider (Virtualbox, Libvirt or whatever) in >>>>>> order >>>>>> to support Vagrant box creation >>>>>> 3. You then have a Vagrantfile which runs on <targetmachine> and >>>>>> configures multiple Vagrant boxes <vb1>, <vb2>, <vb3> >>>>>> 4. Once <vb1>, <vb2>, <vb3> are UP* and only then,* you want to >>>>>> run some Ansible which needs the primary and 2 secondaries to be up >>>>>> >>>>>> That being the case, then that is the behaviour that >>>>>> https://developer.hashicorp.com/vagrant/docs/provisioning/ansible#ansible-parallel-execution >>>>>> describes. It's slightly poorly worded but to me: >>>>>> >>>>>> # Only execute once the Ansible provisioner, # when all the >>>>>> machines are up and ready. >>>>>> >>>>>> Is equivalent to: >>>>>> >>>>>> # Provision all Vagrant boxes in the multi-machine setup. >>>>>> # Only once all the machines are up and ready, run the Ansible >>>>>> provisioner >>>>>> >>>>>> If that's not what's happening, that's likely a Vagrant configuration >>>>>> or provisioner misbehaviour? >>>>>> >>>>>> That's why I'm saying this isn't necessarily an Ansible thing. That >>>>>> wording, the boxes should all spin up before any Vagrant Ansible >>>>>> provisioner runs, you're saying that's not the case. That sounds like >>>>>> either your Vagrantfile is wrong, or your Vagrant VM provisioner or >>>>>> something else isn't working as expected. >>>>>> >>>>>> I'm spinning this up on a test but if you already have a test >>>>>> case/reproducer, or can provide more info on your Vagrant setup then this >>>>>> would collectively help people help you. If there's an obvious error in >>>>>> your Vagrantfile it could be a simple fix rather than an edge case. >>>>>> >>>>>> cf: >>>>>> - >>>>>> https://manski.net/2016/09/vagrant-multi-machine-tutorial/#multi-machine.3A-the-clever-way >>>>>> - >>>>>> https://developer.hashicorp.com/vagrant/docs/provisioning/ansible#ansible-parallel-execution >>>>>> >>>>>> >>>>>> On Sat, 25 Nov 2023 at 21:55, Evan Hisey <[email protected]> wrote: >>>>>> >>>>>>> Vagrant is behaving fine, so not a vagrant specific problem. It is a >>>>>>> task problem. I need the vagrant hosts fully installed first because I >>>>>>> have >>>>>>> to collect data from all 3 at once before deploying the software, and >>>>>>> during software deployment I have to install the master first, collect >>>>>>> keys >>>>>>> and then install the slaves. Vagrant provider provisions does provide >>>>>>> this >>>>>>> kind of control as it assumes the each provisioned VM is self >>>>>>> contained. A >>>>>>> more typical solution would be to directly remote in to the VM's for >>>>>>> ansible to run after deployment from the remote controller, but that is >>>>>>> not >>>>>>> an available option. Only the vagrant host will have access to the >>>>>>> vagrant >>>>>>> vms, and really only as the vagrant user. The last limitation is not >>>>>>> hard >>>>>>> to deal with, as vagrant provides everything an ansible job would need >>>>>>> if >>>>>>> run from the vagrant host. >>>>>>> >>>>>>> That is why I need to trigger to a vagrant host ansible playbook, >>>>>>> since it can't not run from the initial ansible controller. Yes it is a >>>>>>> bit >>>>>>> of an odd edge case, as the vagrant provider normally would be plenty. >>>>>>> >>>>>>> On Sat, Nov 25, 2023 at 2:08 PM Will McDonald <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> It sounds like a Vagrant issue rather than an Ansible issue. Or >>>>>>>> possibly a niche Vagrant provider problem. >>>>>>>> >>>>>>>> Can you share a sample Vagrantfile that's not behaving as it should >>>>>>>> and details of the target OS of the Vagrant host, and the >>>>>>>> virtualisation >>>>>>>> provider you're using? >>>>>>>> >>>>>>>> >>>>>>>> On Sat, 25 Nov 2023 at 19:30, Evan Hisey <[email protected]> wrote: >>>>>>>> >>>>>>>>> Already tried it and it does not work, which was why I explicitly >>>>>>>>> referenced that behavior as not working as not working in this >>>>>>>>> scenario.While vagrant can run playbooks at provisioning time. it >>>>>>>>> does not >>>>>>>>> really proivde a way to to control when the provisioin runs. All 3 >>>>>>>>> hosts >>>>>>>>> need to be up be for the first host can be provisioned since it >>>>>>>>> requires >>>>>>>>> the ips of the later hosts. Second option does not work, as the remote >>>>>>>>> control node does not have access to the VMs, as mentioned. Which is >>>>>>>>> what >>>>>>>>> lead to the need to trigger a second playbook. otherwise could lust >>>>>>>>> load >>>>>>>>> the vagrant generated inventory with add_host module. >>>>>>>>> >>>>>>>>> IC ould do some ugly sequencing of the "vagrant up --provision" >>>>>>>>> from a playbook to control the ansible provisioning sequence of the >>>>>>>>> vms, >>>>>>>>> but I am trying to avoid using ugly shell commands as much as I can. >>>>>>>>> If I >>>>>>>>> uses a shell command I could also just trigger an ansible playbook >>>>>>>>> that >>>>>>>>> way, but feels wrong. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, Nov 25, 2023 at 12:40 PM Will McDonald < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Quickly skimming the Vagrant Ansible provisioner docs, isn't this >>>>>>>>>> precisely the behaviour you're looking for: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://developer.hashicorp.com/vagrant/docs/provisioning/ansible#ansible-parallel-execution >>>>>>>>>> >>>>>>>>>> # Only execute once the Ansible provisioner, # when all the >>>>>>>>>> machines are up and ready. >>>>>>>>>> >>>>>>>>>> So you would spin up all your Vagrant boxes from your control >>>>>>>>>> node, wait for that to complete, template out a static inventory of >>>>>>>>>> your >>>>>>>>>> Vagrant boxes then run your subsequent Vagrant Ansible provisioner >>>>>>>>>> automation? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Sat, 25 Nov 2023 at 18:20, Evan Hisey <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I am working on a scenario where the first playbook executes >>>>>>>>>>> commands on a remote host to create a vagrant host and spins up >>>>>>>>>>> multiple >>>>>>>>>>> vms. Vagrant can triggers it's own ansible provisioning runs but >>>>>>>>>>> they are >>>>>>>>>>> only single host aware and run when the host is provisioned. That >>>>>>>>>>> does not >>>>>>>>>>> work in this case, as I need all VM's running BEFORE the deployment >>>>>>>>>>> playbook can be triggered. Added wrinkle is the VMs are accessible >>>>>>>>>>> at this >>>>>>>>>>> time from outside the vagrant host. If they were, I could simply >>>>>>>>>>> import the >>>>>>>>>>> vagrant host list into the controller inventory and refresh. >>>>>>>>>>> >>>>>>>>>>> Right now I am looking at the possibility of using >>>>>>>>>>> ansible.builtin.shell to trigger the new ansible-playbook command >>>>>>>>>>> on the >>>>>>>>>>> vagrant host to run the vagrant VM application configuration. But >>>>>>>>>>> while >>>>>>>>>>> this works it is not exactly ansible clean. Suggestions on >>>>>>>>>>> approaches? >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Evan Hisey >>>>>>>>>>> [email protected] >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "Ansible Project" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to >>>>>>>>>>> [email protected]. >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/ansible-project/CAEcFzYwsmYmvh%3DWJwSNmJWertkxFRDiKkumnwhzAFupggP58Vg%40mail.gmail.com >>>>>>>>>>> <https://groups.google.com/d/msgid/ansible-project/CAEcFzYwsmYmvh%3DWJwSNmJWertkxFRDiKkumnwhzAFupggP58Vg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "Ansible Project" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/ansible-project/CAKtKohS2LdL4qtTyQF8VdV6pu2tjL3mx868TgvkwvAEUkRbF%3Dg%40mail.gmail.com >>>>>>>>>> <https://groups.google.com/d/msgid/ansible-project/CAKtKohS2LdL4qtTyQF8VdV6pu2tjL3mx868TgvkwvAEUkRbF%3Dg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "Ansible Project" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/ansible-project/CAEcFzYxdk%3DfA13__d1wEJTiTkpDnaR2eqEpGm%2BXy3v3L1M%3DJ9A%40mail.gmail.com >>>>>>>>> <https://groups.google.com/d/msgid/ansible-project/CAEcFzYxdk%3DfA13__d1wEJTiTkpDnaR2eqEpGm%2BXy3v3L1M%3DJ9A%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Ansible Project" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/ansible-project/CAKtKohQ8PP4n7h-ERfE4iONYxZK-nZRLaoMwvRATzSj1hnD%3Dbw%40mail.gmail.com >>>>>>>> <https://groups.google.com/d/msgid/ansible-project/CAKtKohQ8PP4n7h-ERfE4iONYxZK-nZRLaoMwvRATzSj1hnD%3Dbw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Ansible Project" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/ansible-project/CAEcFzYwxGLhF275kMbBaMMUx6FNQoduaxO7wa_j9AVkUw5ZQVg%40mail.gmail.com >>>>>>> <https://groups.google.com/d/msgid/ansible-project/CAEcFzYwxGLhF275kMbBaMMUx6FNQoduaxO7wa_j9AVkUw5ZQVg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Ansible Project" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/ansible-project/CAKtKohS9rcVZ0rOe_MH9bdb2EgFL7jeBmqwLgyRbZMG2NjmrmQ%40mail.gmail.com >>>>>> <https://groups.google.com/d/msgid/ansible-project/CAKtKohS9rcVZ0rOe_MH9bdb2EgFL7jeBmqwLgyRbZMG2NjmrmQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Ansible Project" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/ansible-project/CAEcFzYzr7mYhJJimRKsyoV_ikYQoxjHYgOs-pABj4WEPUfoMbg%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/ansible-project/CAEcFzYzr7mYhJJimRKsyoV_ikYQoxjHYgOs-pABj4WEPUfoMbg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Ansible Project" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/ansible-project/CAKtKohSGMsrKNf9rAqafEk3UoLf9AdWK9G491JB87gaAUhUEmg%40mail.gmail.com >>> <https://groups.google.com/d/msgid/ansible-project/CAKtKohSGMsrKNf9rAqafEk3UoLf9AdWK9G491JB87gaAUhUEmg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "Ansible Project" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/ansible-project/CAEcFzYyhyZ%2BzxG2QX0NGySx4iDgydg41Mbeubu5JEpKpLOOccg%40mail.gmail.com > <https://groups.google.com/d/msgid/ansible-project/CAEcFzYyhyZ%2BzxG2QX0NGySx4iDgydg41Mbeubu5JEpKpLOOccg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Ansible Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAKtKohQdOPApxAcPAW_cSvYTRrZPCdGcnFjE-7K6XLujOaG0sQ%40mail.gmail.com.
