Topic "Slow file transfer for many small files using FTP"

Author Message
tomb
[View user's profile]

Joined: 2013-07-05
Posts: 19
WinSCP version 5.1.6

When transferring many small files the transfer speed is 4-5 files per second at most. Files size is about 250 bytes. When doing the same test with SFTP to the same host I get to 20-25 files /sec.

The SFTP speed is fine. Can the FTP transfer do it at the same speed?
log_slow_ftp.txt (195.38 KB) [Download]

Description: This is the output from the slow FTP session

script_slow_ftp.txt (271 Bytes) [Download]

Description: this is the FTP script that is used

Advertisements
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
I'd expect FTP to be slower as it has to open new TCP/IP connection for each file. Do you experience a better performance with other FTP clients than with WinSCP for small files?
_________________
Martin Prikryl
tomb
[View user's profile]

Joined: 2013-07-05
Posts: 19
OK, so I can understand why FTP is slower than SFTP.

I compared WinSCP with FileZilla using FTP with small files: Filezilla is about 2x faster than Winscp doing 8-9 files/sec.
Getting Winscp up to this speed would be nice, but the current speed is no show stopper Wink
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
Can you attach comparable logs both using WinSCP and FileZilla?
tomb
[View user's profile]

Joined: 2013-07-05
Posts: 19
I did the same load test with Filezilla instead of Winscp, tranferring 160 files of 276 bytes each. The speed is 17 sec for 160 files, about 9 files/sec. WinSCP did 4-5 files/sec.
I included the log file.
The WinSCP log file is in my post of 20 sep.

The Filezilla log file suggests that it is setting up two connections in parallel. Maybe that is a clue?
filezilla_log.txt (94.13 KB) [Download]

Description: (none)

martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
tomb wrote:
The Filezilla log file suggests that it is setting up two connections in parallel. Maybe that is a clue?

You can do the same with WinSCP:
http://winscp.net/eng/docs/transfer_queue
_________________
Martin Prikryl
tomb
[View user's profile]

Joined: 2013-07-05
Posts: 19
Can I do that in scripts too? By using -rawsettings option?
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
tomb wrote:
Can I do that in scripts too? By using -rawsettings option?

No. But you can run multiple scripts in parallel.
_________________
Martin Prikryl
Advertisements

You can post new topics in this forum






Search Site

What is WinSCP?

It is award-winning SFTP client, SCP client, FTPS client and FTP client integrated into one software program for file transfer to FTP server or secure SFTP server. [More]

And it's free!

Donate

About donations

$9   $19   $49   $99

About donations

Recommend

WinSCP Privacy Policy

WinSCP License