Hi Lars,
Thanks for the quick follow up. First of all, since it was not clear,
in my previous mail, the files are not actually included in the
backup, see just the doubled processing time and found files.
On Sat, Nov 07, 2015 at 09:10:09PM +0000, Lars Wirzenius wrote:
> I don't seem to be able to reproduce this, so I would like to ask some
> questions.
Sure.
> On Sat, Nov 07, 2015 at 09:42:18PM +0100, Salvatore Bonaccorso wrote:
> > I observed the following behaviour after updating to the recent
> > 1.18.1-1 from 1.17-1. I have put in several directories a CACHEDIR.TAG
> > to skip processing of these direcories, e.g. ~/.cache.
> >
> > I can reproduce that, say by the following configuration (only a test
> > setup, but it's relevant for my own backup where I have directories
> > with many files where I have put a CACHEDIR.TAG which are now
> > processed as well):
>
> Can you attach a copy of your CACHEDIR.TAG file, please?
yes, it's attached to this mail, but the content is as follows:
dummy@sid:~$ sha256sum files/.cache/CACHEDIR.TAG
a6cadd53399872fa9aca8a92b2a96e111a30203e4e8a21f4068d0b960031b3af
files/.cache/CACHEDIR.TAG
dummy@sid:~$ cat files/.cache/CACHEDIR.TAG
Signature: 8a477f597d28d172789f06886806bc55
dummy@sid:~$
> Can you run "obnam --config obnam.config ls" and attach the output,
> please? If the filenames are sensitive, you can replace them with
> dummies by adding "| tr a-z x" to the command, or only send the output
> to me.
I tried to distil a "test case", well it's not a proper one but the
doubled processing time is seen. I'm as well attaching the ls output.
Create some directories and put in each a 100M file there:
dummy@sid:~$ mkdir -p files/{foo,.cache}/{foo,bar}
dummy@sid:~$ cp CACHEDIR.TAG files/.cache
dummy@sid:~$ dd if=/dev/zero of=test.img bs=100M count=1
and copy the test.img to all above created leaf directories.
Configuration:
[config]
repository = /home/dummy/backups
root = /home/dummy/files
exclude-caches = yes
Now with 1.17-1 (I delete the backup directory after each of these run):
dummy@sid:~$ time obnam --config ./obnam.config backup
Backed up 6 files (of 6 found), containing 200.0 MiB.
Uploaded 1.0 MiB file data in 1s at 965.9 KiB/s average speed.
Total download amount 193.0 KiB.
Total upload amount 1.0 MiB. Overhead was 586.0 KiB (41.4 %).
real 0m1.222s
user 0m1.092s
sys 0m0.088s
dummy@sid:~$ obnam --config ./obnam.config ls
Generation 2 (2015-11-07 22:34:52 - 2015-11-07 22:34:53)
drwxr-xr-x 22 root root 4096 2015-10-28 07:31:36 /
drwxr-xr-x 3 root root 4096 2015-11-07 20:13:43 /home
drwxr-xr-x 5 dummy dummy 4096 2015-11-07 21:34:52 /home/dummy
drwxr-xr-x 4 dummy dummy 4096 2015-11-07 21:30:15
/home/dummy/files
drwxr-xr-x 4 dummy dummy 4096 2015-11-07 21:30:47
/home/dummy/files/foo
drwxr-xr-x 2 dummy dummy 4096 2015-11-07 21:27:28
/home/dummy/files/foo/bar
-rw-r--r-- 1 dummy dummy 104857600 2015-11-07 21:22:09
/home/dummy/files/foo/bar/test.img
drwxr-xr-x 2 dummy dummy 4096 2015-11-07 21:30:47
/home/dummy/files/foo/foo
-rw-r--r-- 1 dummy dummy 104857600 2015-11-07 21:27:34
/home/dummy/files/foo/foo/test.img
dummy@sid:~$
Note the 6 files, 200 MiB and processing time 2s.
After updating to 1.18.1-1 the content of the backup would be the
same, but the backuping process processes as well the subdirectories
of the directory which contains the CACHEDIR.TAG file, note the 11
files processed, 400MiB in total in 2s:
dummy@sid:~$ time obnam --config ./obnam.config backup
Backed up 11 files (of 11 found), containing 400.0 MiB.
Uploaded 1.0 MiB file data in 2s at 500.5 KiB/s average speed.
Total download amount 193.0 KiB.
Total upload amount 1.0 MiB. Overhead was 594.0 KiB (41.7 %).
real 0m2.111s
user 0m1.888s
sys 0m0.164s
dummy@sid:~$ obnam --config ./obnam.config ls
Generation 2 (2015-11-07 22:35:58 - 2015-11-07 22:36:00)
drwxr-xr-x 22 root root 4096 2015-10-28 07:31:36 /
drwxr-xr-x 3 root root 4096 2015-11-07 20:13:43 /home
drwxr-xr-x 5 dummy dummy 4096 2015-11-07 21:35:58 /home/dummy
drwxr-xr-x 4 dummy dummy 4096 2015-11-07 21:30:15
/home/dummy/files
drwxr-xr-x 4 dummy dummy 4096 2015-11-07 21:30:47
/home/dummy/files/foo
drwxr-xr-x 2 dummy dummy 4096 2015-11-07 21:27:28
/home/dummy/files/foo/bar
-rw-r--r-- 1 dummy dummy 104857600 2015-11-07 21:22:09
/home/dummy/files/foo/bar/test.img
drwxr-xr-x 2 dummy dummy 4096 2015-11-07 21:30:47
/home/dummy/files/foo/foo
-rw-r--r-- 1 dummy dummy 104857600 2015-11-07 21:27:34
/home/dummy/files/foo/foo/test.img
dummy@sid:~$
So the ending result in the backup is correct, but the backup
processing time can increased since the direcories are nevertheless
traversed and checked. In my real home I have e.g. put CACHEDIR.TAG in
e.g. ~/.cache were there are caches for browsers, apt-file, etc.
Does this helps? Please let me know if I need/can provide more
information.
Regards,
Salvatore
