{"id":808,"date":"2024-11-26T23:33:19","date_gmt":"2024-11-26T22:33:19","guid":{"rendered":"https:\/\/patrick-nagel.net\/blog\/?p=808"},"modified":"2024-11-26T23:33:19","modified_gmt":"2024-11-26T22:33:19","slug":"fast-sftp-transfers-with-kde-kio-extras-dolphin","status":"publish","type":"post","link":"https:\/\/patrick-nagel.net\/blog\/archives\/808","title":{"rendered":"Fast SFTP transfers with KDE \/ kio-extras \/ Dolphin"},"content":{"rendered":"\n

For a few years now, I’ve used Dolphin \/ Konqueror \/ Krusader for transferring files to or from my home server via SFTP. However when there were larger amounts to be transferred, I opened the shell and used rsync. That’s because I felt it was always somewhat slower with SFTP. I blamed it on the weak file server, on the network, or whatever.<\/p>\n\n\n\n

Recently I again wanted to clean up my desktop’s home directory, where a bunch of raw or edited videos were sitting. All in all ~250 GiB. Because I wanted to sort the videos into different sub-directories on the server, I used Dolphin and SFTP once again. While I was watching the transfer, I began to doubt whether the network between my desktop and the home server was actually Gigabit, because I only saw 10-15 MiB\/s. Quick check with rsync and scp: nope, 80+ MB\/s is possible, just not with Dolphin.<\/p>\n\n\n\n

That’s when I decided to get to the bottom of the issue. I discovered KDE bug 296526 – Dolphin is too slow when upload a file on a SSH server<\/a>. It fit exactly what I was observing, even though it’s from 2012. That’s 14 years<\/strong>! I went on to read all the comments, and it became apparent that none of the users that had commented had spent the time to do a proper side-by-side comparison, and document it properly. So I took up that task and posted comment 40<\/a>, where I wrote pretty much what I wrote here, but also the results of some tests that I had done with ‘libssh’. That’s the underlying library which kio-extras\/sftp uses to interact with SSH\/SFTP servers. And Dolphin \/ Konqueror \/ Krusader in turn use kio-extras\/sftp to do SFTP transfers.<\/p>\n\n\n\n

These tests showed very promising results: I could actually saturate my Gigabit link and transfer >750 Mbps. What I didn’t know at that time (September): I had tested a brand new version of libssh (0.11.0), released in August, that came with major changes. Namely a new async I\/O API had been added. The transfers with Dolphin had still used libssh 0.10.x though.<\/p>\n\n\n\n

Upon learning about these important changes<\/a>, I opened a version bump request for libssh 0.11.0 in Gentoo’s bug tracker<\/a> to make the Gentoo devs aware of this new libssh version. Fast forward two months, libssh 0.11.1 was available in Gentoo. I then erroneously tested with kio-extras linked against libssh 0.10.x (and thus disabled new async I\/O API), even though I had 0.11.1 on my system. The reason was, that I hadn’t rebuilt kio-extras. That resulted in comment 45<\/a>. A few minutes later I realized my error, rebuilt kio-extras (actually most of KDE, because an update was coming in anyway), and voil\u00e0: ~230 MiB\/s or ~1840 Mbps (with peaks going >2 Gbps)! <\/strong>Hurray!<\/p>\n\n\n\n

\"\"<\/a><\/figure>\n\n\n\n

Side note: I had upgraded my network from Gigabit to 2.5 Gigabit in the meantime, otherwise that would obviously not be possible. But with the old libssh only around ~90 MiB\/s or ~720 Mbps were possible over the same network, 2.5x slower than with the new libssh 0.11.x.<\/p>\n\n\n\n

So, if you find that your SFTP transfers in KDE are slower than they should be, check which libssh version comes with your distribution. If it’s <0.11.0, you know you need to upgrade. With non-rolling binary distributions, you’ll probably have to wait a bit and then upgrade your whole distribution. For example Ubuntu will only get libssh 0.11.1 in “Plucky” aka version 25.04<\/a>. For rolling distributions like Gentoo<\/a> or Arch<\/a> it’s already available.<\/p>\n\n\n\n

This is probably the biggest single improvement to my Linux on the Desktop experience of the last years… hence this blog post \ud83d\ude42<\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

For a few years now, I’ve used Dolphin \/ Konqueror \/ Krusader for transferring files to or from my home server via SFTP. However when there were larger amounts to be transferred, I opened the shell and used rsync. That’s because I felt it was always somewhat slower with SFTP. I blamed it on the … Continue reading “Fast SFTP transfers with KDE \/ kio-extras \/ Dolphin”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[32,25],"class_list":["post-808","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-gentoo","tag-kde"],"_links":{"self":[{"href":"https:\/\/patrick-nagel.net\/blog\/wp-json\/wp\/v2\/posts\/808","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/patrick-nagel.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/patrick-nagel.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/patrick-nagel.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/patrick-nagel.net\/blog\/wp-json\/wp\/v2\/comments?post=808"}],"version-history":[{"count":2,"href":"https:\/\/patrick-nagel.net\/blog\/wp-json\/wp\/v2\/posts\/808\/revisions"}],"predecessor-version":[{"id":811,"href":"https:\/\/patrick-nagel.net\/blog\/wp-json\/wp\/v2\/posts\/808\/revisions\/811"}],"wp:attachment":[{"href":"https:\/\/patrick-nagel.net\/blog\/wp-json\/wp\/v2\/media?parent=808"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/patrick-nagel.net\/blog\/wp-json\/wp\/v2\/categories?post=808"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/patrick-nagel.net\/blog\/wp-json\/wp\/v2\/tags?post=808"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}