On 08/02/2022 06:59, [email protected] wrote:
Hi,
just because I became curious, the numbers are probably not to be
compared:
- iperf3 tells me ~170Mbit/s
- I transferred initially my Downloads repository from laptop to
server (which has roughly as much disk as you have RAM :-P), 2.7GBytes
in 3.45minutes, a mixture of not too big files and quite some small
files (see the size distribution below).
- this makes it roughly 90-100Mbit/s
As said, nothing comparable, just for comparaison. Even on a local
backup, the initial backup takes quite a while and becomes better in
delta mode.
I have the plan to improve performance, without knowing if there is
really potential, but at this stage, I'm focussing on re-structuring
the code and understanding it better.
KR, Eric
Neat one liner I found at
https://superuser.com/questions/565443/generate-distribution-of-file-sizes-from-the-command-prompt
(added as utility at
https://github.com/rdiff-backup/rdiff-backup/pull/668)
$ find Downloads -type f -print0 | xargs -0 ls -l | awk
'{size[int(log($5)/log(2))]++}END{for (i in size) printf("%10d %3d\n",
2^i, size[i])}' | sort -n
0 3
4 4
8 3
16 6
32 5
128 4
256 8
512 14
1024 27
2048 29
4096 30
8192 18
16384 15
32768 9
65536 16
131072 15
262144 12
524288 6
1048576 1
2097152 3
4194304 7
8388608 5
16777216 2
33554432 4
67108864 2
268435456 1
536870912 1
1073741824 1
Love the one-liner, here is my variation on it (analysing all the
current data in my repos, excluding some extraneous stuff and the actual
reverse-diff data (i.e. the rdiff-backup-data subdirectories) - sorry no
data re speed:
find /home -type f \( \( -path "/home/lost+found/*" -or -path
"/home/timedicer/*" -or -path "/home/backup/*" -or -path /home/swapfile
-or -path "/home/*/.*" -or -path "*/rdiff-backup-data/*" \) -prune -or
-printf "%s\n" \)|awk '{size[int(log($1)/log(1024))]++}END{for (i in
size) print 1024^i, size[i]}' | sort -n | numfmt --to=iec
--format='%.0fB'| sed '2,${s/ /+ /};1iSize Count' | column -t
Size Count
0B 24284
1B+ 88907
1KB+ 494536
1MB+ 64411
1GB+ 59
