Hi Michael,
Your idea triggered the best solution I have found. Thanks.
This is my solution - for people who discover this later.
io.hcs holds the backup we are trying to restore.
sinope.hcs is where we are restoring to.
callisto.hcs is where ansible is run.
There is a (read only) samba share on io.hcs that is permanently open on
callisto.hcs, but can't be used to copy the files over, because the
permissions get altered. But it can be used to copy the archive over.
tar maintains ownership and permissions as default behaviour.
Therefore the tasks are:
- name: get IO to tar up the website
community.general.archive:
path: "/home/ian/BackupFiles/hobson42/var/www/ianhobson.com/"
dest: /home/ian/BackupFiles/ianhobson.com.gz # will appear in
share as /home/ian/backup/ianhobson.com.gz
delegate_to: io.hcs
- name: create {{ location }} directory
ansible.builtin.file:
path: "{{ location }}"
state: directory
- name: move archive over to target, and extract it.
ansible.builtin.unarchive:
src: /home/ian/backup/ianhobson.com.gz
dest: "{{ location }}"
- name: remove archive
ansible.builtin.file:
path: /home/ian/BackupFiles/ianhobson.com.gz
state: absent
delegate_to: io.hcs
Result is a speed up of between 60 and 100 times!
The files are restored with the correct permissions and ownership. I
think it is necessary that all users are set up on all machines with the
same user numbers in /etc/passwd.
Regards
Ian
On 30/07/2022 17:51, 'Michael Ströder' via Ansible Project wrote:
On 7/30/22 12:44, Ian Hobson wrote:
I need advice on speeding up this task:
- name: "restore files to {{ location }}"
ansible.builtin.copy:
src: "/home/ian/backup/hobson42/var/www/ianhobson.com/"
dest: "{{ location }}"
It copies 5022 files, totalling 109.2Mb, and even though both machines
have NVME SSDs and the network is quiet,
the task takes over 30 minutes! Note - some of the files are owned by
root, and have "600" permissions, which need to be preserved.
I'd recommend to avoid copying thousands of files.
Ideas to try:
- Trigger rsync [1] if most of the files are static
- Pack the files into a single tar.gz and transfer and extract that [2][3]
Ciao, Michael.
[1]
https://docs.ansible.com/ansible/latest/collections/ansible/posix/synchronize_module.html
[2]
https://docs.ansible.com/ansible/latest/collections/community/general/archive_module.html
[3]
https://docs.ansible.com/ansible/latest/collections/ansible/builtin/unarchive_module.html#ansible-collections-ansible-builtin-unarchive-module
--
Ian Hobson
Tel (+66) 626 544 695
--
You received this message because you are subscribed to the Google Groups "Ansible
Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/ansible-project/07e5ac29-bf6e-dc22-83ec-c14478621e05%40gmail.com.
