https://bugs.kde.org/show_bug.cgi?id=296526
Patrick <m...@patrick-nagel.net> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |m...@patrick-nagel.net
--- Comment #40 from Patrick <m...@patrick-nagel.net> ---
Alright, I was trying to clean up my disk and move stuff to the file server,
and this time I wanted to do it with Dolphin instead of with rsync commands.
Call me lazy...
My setup is decent: [ Desktop: 12 core, 64 GB, fast SSD ] -- Gbit LAN -- [
Server: 8 core, mediocre SSD as bcache + big HDD ]
With scp I usually see 80+ MB/s (>650 MBit/s) transfer rates. However with
Dolphin / kio-extras/sftp I only got 14.6 MB/s (a bit over 100 MBit/s). That
might feel ok if you transfer some pictures, but if there are many 60+ GB
videos this is really slow. And many 60+ GB videos are definitely a reality in
2024. I'll attach a screenshot of the Dolphin transfer to back this up.
So that's when I started googling for what's wrong, and I found this ticket.
Having seen Harald's complaint that apples are being compared with oranges, I
set out to make an apples-to-apples comparison:
I cloned the libssh repo (commit 48d474f78c5f68471bf412a7dbf508ef52f77664),
configured with CMake (mkdir build && cd build && cmake
-DCMAKE_BUILD_TYPE=Release -DWITH_SFTP=True -DWITH_BENCHMARKS=True ..),
installed some missing deps (cmocka, socket_wrapper, nss_wrapper, uid_wrapper,
pam_wrapper) and built (make -j12). That gave me a neat
"tests/benchmarks/benchmarks" executable. Its help text says "-9,
--async-sftp-aio-upload Upload data using asynchronous SFTP AIO api (fast)",
so I thought that must be what Dolphin / kio is using. But when I ran it, I got
3x the speed - 46 MB/s / 370 MBit/s:
$ ./benchmarks --host pn-test@data --size 1000 --async-sftp-aio-upload
ping RTT : 0.489000 ms
SSH request times : 3.508000 ms ; 3.955000 ms ; 3.974000 ms
SSH RTT : 3.812333 ms. Theoretical max BW (win=128K) : 33.575241 Mbps
pn-test@data : benchmark_async_sftp_aio_upload : 370.660278 Mbps
That was already much better than expected. But when I started tinkering with
the "--chunk" size and "--prequests" (parallel requests) parameters I found
that I could essentially saturate my Gigabit link by doubling either parameter
(chunk size is 32k by default, found that in the source code, parallel requests
is 20 by default):
$ ./benchmarks --host pn-test@data --size 1000 --chunk=64000
--async-sftp-aio-upload
ping RTT : 0.600000 ms
SSH request times : 3.452000 ms ; 3.984000 ms ; 3.995000 ms
SSH RTT : 3.810333 ms. Theoretical max BW (win=128K) : 33.592865 Mbps
pn-test@data : benchmark_async_sftp_aio_upload : 767.680969 Mbps
$ ./benchmarks --host pn-test@data --size 1000 --prequests=40
--async-sftp-aio-upload
ping RTT : 1.421000 ms
SSH request times : 3.476000 ms ; 3.996000 ms ; 3.987000 ms
SSH RTT : 3.819667 ms. Theoretical max BW (win=128K) : 33.510777 Mbps
pn-test@data : benchmark_async_sftp_aio_upload : 750.160156 Mbps
Hope that helps narrowing down the issue. I might dig deeper into the source
some time in winter and maybe I can find it, but since I've never done serious
KDE development that will be a steep learning curve for me. I'd be happy to
assist in any tests and benchmarking endeavours, if someone tries a fix and
tells me how to build it. :)
--
You are receiving this mail because:
You are watching all bug changes.
