Hi,
just because I became curious, the numbers are probably not to be compared:
- iperf3 tells me ~170Mbit/s
- I transferred initially my Downloads repository from laptop to server
(which has roughly as much disk as you have RAM :-P), 2.7GBytes in
3.45minutes, a mixture of not too big files and quite some small files
(see the size distribution below).
- this makes it roughly 90-100Mbit/s
As said, nothing comparable, just for comparaison. Even on a local
backup, the initial backup takes quite a while and becomes better in
delta mode.
I have the plan to improve performance, without knowing if there is
really potential, but at this stage, I'm focussing on re-structuring the
code and understanding it better.
KR, Eric
Neat one liner I found at
https://superuser.com/questions/565443/generate-distribution-of-file-sizes-from-the-command-prompt
(added as utility at https://github.com/rdiff-backup/rdiff-backup/pull/668)
$ find Downloads -type f -print0 | xargs -0 ls -l | awk
'{size[int(log($5)/log(2))]++}END{for (i in size) printf("%10d %3d\n",
2^i, size[i])}' | sort -n
0 3
4 4
8 3
16 6
32 5
128 4
256 8
512 14
1024 27
2048 29
4096 30
8192 18
16384 15
32768 9
65536 16
131072 15
262144 12
524288 6
1048576 1
2097152 3
4194304 7
8388608 5
16777216 2
33554432 4
67108864 2
268435456 1
536870912 1
1073741824 1
On 07/02/2022 18:39, EricZolf wrote:
Can you send me your hardware that I do some tests?
Seriously, no clue. It might be that you have many very small files but even
then the drop in performance is impressive...
Perhaps someone with a bigger estate can be more helpful.
Eric
On 7 February 2022 03:20:15 UTC, Eric Robinson <[email protected]> wrote:
I have two RHEL 8.5 servers with 48 cores, 1TB RAM, and NVME disks connected
over a 25 Gbit network. I confirmed with iperf that I'm betting 20+ Gbits/sec
bandwidth. Using rdiff-backup 2.0.5, I am only getting 15-25 MB/sec transfer
speed. Why is rdiff-backup so slow? The target server is empty. This is the
first backup attempt.
-Eric
Disclaimer : This email and any files transmitted with it are confidential and
intended solely for intended recipients. If you are not the named addressee you
should not disseminate, distribute, copy or alter this email. Any views or
opinions presented in this email are solely those of the author and might not
represent those of Physician Select Management. Warning: Although Physician
Select Management has taken reasonable precautions to ensure no viruses are
present in this email, the company cannot accept responsibility for any loss or
damage arising from the use of this email or attachments.
