Topic "Slow downloads using SFTP"

Author Message
aliengerm1
[View user's profile]

Joined: 2011-01-19
Posts: 8
This is a repeat and summary of a related post.

Situation:
PC on one coast of USA
Unix (solaris) server on the other coast of the USA, running OpenSSH_5.6p1-hpn13v10
Ping: 80ms
Network connectivity: WAN network speed - 40mbps

Over long distance (WAN) on a tuned system, WinSCP 4.3.1 using SFTP (protocol ssh-2) caps around 690kbytes** on download and upload. Cipher AES or blowfish does not make a significant difference. (Ciphers make a large difference during local performance where CPU is more of an issue.)

We found that switching to Filezilla removed the download cap, reaching up to 4.4MB/s (40mbps - which was near the limit of the line). Upload was capped at 650kbytes, which has since been removed by the Filezilla devs (Fixed as of 3.3.5)

Speed difference:
Filezilla results: Transfer of 600mb file in 3minutes. 4.4MB/s

WinSCP 4.3.1 beta results: Transfer of the same 600mb file in 14minutes. 689KiB/s
Advertisements
aliengerm1
[View user's profile]

Joined: 2011-01-19
Posts: 8
I can only make an educated guess and say that I think the problem lies with a too small socket receive buffer.

More research:
I found another tool that has adjustable send/receive buffers. The default it negotiated was 1024kbytes, which was still slow. The ideal for this particular line seems to be 2048kbytes. My guess is that WinSCP has 512 send/receive buffers. Those should at minimum be upped to 1024 in general, and consider makeing that a user adjustable parameter since its useful particularly on large and long lines, aka WAN.
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
Thanks for your investigation. I'll check it.
_________________
Martin Prikryl
aliengerm1
[View user's profile]

Joined: 2011-01-19
Posts: 8
Thank YOU.

Update: I had a bit of software that had a default of 128kbytes, which autotuned to 1024kbytes for socket receive buffer. I then set default to 2048, then it autotuned to 4096. Autotuned = tcp sliding window adjustment, in the socket receive buffer. That's for download only, that particular bit of software performed poorly when it came to upload, so no good numbers.

I hate to put you on the spot but do you consider this a priority item or a low level item?

Thanks again.
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
aliengerm1 wrote:
I hate to put you on the spot but do you consider this a priority item or a low level item?

Rather of priority. Which means I'll take a look at it in a week or two.
_________________
Martin Prikryl
aliengerm1
[View user's profile]

Joined: 2011-01-19
Posts: 8
Another dev mentioned it might have to do with sending more packets without waiting on ACK, they called it "pipelining".

I'd guess you've already seen this but if not, its a interesting read.

http://www.psc.edu/index.php/hpn-ssh
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
aliengerm1 wrote:
Another dev mentioned it might have to do with sending more packets without waiting on ACK, they called it "pipelining".

I'd guess you've already seen this but if not, its a interesting read.

http://www.psc.edu/index.php/hpn-ssh

This is partly implemented already:
https://winscp.net/tracker/show_bug.cgi?id=17
aliengerm1
[View user's profile]

Joined: 2011-01-19
Posts: 8
Do you have any update on this?

Thanks!
aliengerm1
[View user's profile]

Joined: 2011-01-19
Posts: 8
I just tested the latest version, WinSCP 4.3.2.

(600mb test file)

California PC download from East Coast OpenSSH server using WinSCP 4.3.2 ~15minutes, high speed around KiB/s

California PC upload to East Coast OpenSSH using WinSCP 4.3.2 ~16minutes, high speed around 640 KiB/s


Sorry Sad
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
By coincidence I have done some tests yesterday too. But I had no luck so far.
ErikSorensen

Guest


Hi - came across this thread while googling this issue. Just to add my .02 -- I have 3.8.2 installed side by side with 4.3.2 on the same machine - 3.8.2 gets 4300 KB/s while 4.3.2 gets 30 B/s while trying to sftp upload to the same server.
bmorgan

Guest


i too am seeing very slow speeds when uploading files from my winxp pc to a redhat 5 linux machine (also slow to aix and solaris machines) (14mb in 24 minutes)
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
ErikSorensen wrote:
Hi - came across this thread while googling this issue. Just to add my .02 -- I have 3.8.2 installed side by side with 4.3.2 on the same machine - 3.8.2 gets 4300 KB/s while 4.3.2 gets 30 B/s while trying to sftp upload to the same server.

Can you try 4.2.9 for comparison?
_________________
Martin Prikryl
bmorgan

Guest


4.29 is also slow for me, getting ready to uninstall and try 4.28 again
Guest




4.28 was now also slow... checked my speed setting on the NIC and it was set to "Auto" - changed this to 100 Full and am now working normally again (3mb sec)
andilao
[View user's profile]

Joined: 2007-07-16
Posts: 4
Today tested on a new Server 2008 R2, 2x Xeon E5620:
Code:
        DSL 10MBit/s   LAN 1GBits/s
WinSCP      208 KiB/s    2650 KiB/s
Tunnelier   210 kB/s     --
FileZilla   675 KB/s     13 MB/s !!


FTP over LAN 1GBits/s for comparison:
Code:
     
WinSCP      ~60 MB/s
FileZilla   ~75 MB/s
Tranton
[View user's profile]

Joined: 2011-08-12
Posts: 2
Location: USA
I get just the same problem when I try and login to my school's server to transfer something,
Error: ssh_init: Host does not exist
Error: Could not connect to server
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
Tranton wrote:
I get just the same problem when I try and login to my school's server to transfer something,
Error: ssh_init: Host does not exist
Error: Could not connect to server

Can you connect using any other SSH (SCP/SFTP) client?

Also read this article:
http://winscp.net/eng/docs/message_host_does_not_exist
Guest




I did little test here:

Server: CentOS 5.6 x64 + SSH 5.8p1-hpn13v11 / vsftpd 2.0.5
Network: GigaLan
Client: Windows 7.1 x64 + WinSCP 5.0beta / FileZilla 3.5.1
File: 524 MB

WinSCP (SFTP-3 arcfour) download 22.5 MB/s
WinSCP (SCP arcfour) download 16 MB/s
WinSCP (FTP) download 105 MB/s
Guest




Continue to the previous post.

WinScp (SFTP-3 AES-256) download 10.5 MB/s

FileZilla (SFTP AES-256) download 12.6 MB/s
FileZilla (FTP) download 105 MB/s

Tunnelier 4.4

Tunnelier (SFTP AES-256) download 15 MB/s
Tunnelier (SFTP arcfour) download 24 M/s
Advertisements

You can post new topics in this forum






Search Site

What is WinSCP?

It is award-winning SFTP client, SCP client, FTPS client and FTP client integrated into one software program for file transfer to FTP server or secure SFTP server. [More]

And it's free!

Donate

About donations

$9   $19   $49   $99

About donations

Recommend

WinSCP Privacy Policy

WinSCP License