Transfer Speed of Numerous Small Files

Hello,

When putting files onto a SFTP server with the .NET assembly, what is the expected speed when it comes to small files (1 KB or less)? For example, I had 18 1KB files take around 4 minutes to upload, and that is with multiple Session Tasks running. It seems to vary in the amount of time, but it's generally longer than 2 minutes.

I watched the files process a few times, and it seems like it's 1 file every 10 seconds or so, which feels like a long time, even for small files.

Is there anything I can look at to improve this speed? The files, once uploaded, do get copied to an archive folder, and that does have over 100,000 files in it already if that makes a difference.

Let me know if you need me to provide more information, and I will try to get that to you.

Thanks

Your server takes long time to respond to every request. Do you have a different experience with any other SFTP client?

Hi Martin, there are a couple different SFTP clients. One of them completes uploads of a similar size in about half the time with the same number of files. If the issue is server response time, is there anything that can be done? I have noticed that the download side also appears to suffer from slowness.

At the moment this issue isn't a huge issue, but in the past it has conflicted with other tasks that run at the same time.

So please post verbose log file from the other SFTP client too.

Here is a session log for another SFTP client. The files being upload are roughly the same size, so this is a good comparison.

That log is from WinSCP too.

I'm sorry, I must've misunderstood what you were looking for. Do you want a log from the code that I have creating a session to the client?

You have again attached a log from WinSCP:

I understand now, sorry about that. We do not currently use any other SFTP clients besides WinSCP, sorry for that confusion. We have numerous connections that all use WinSCP, and the speed appears to vary between them, so I was curious if there is any way to resolve the slower connections.

OK, so can you try any other SFTP client?

At the moment I don't believe I can, but as of late I believe the issues have been on the other end of the connection. What I do have is one of our tasks uses a Windows batch file to conduct the downloads and uploads, and that typically takes 10 seconds to run, even with over 300 files to download and a few larger files to upload (3KB to 10KB). In that task, the C#/.NET project creates the files and puts them in a folder for the batch file to pick up and upload. The downloads and uploads still go through WinSCP. Could the batch file be more efficient than creating sessions through .NET?

Could the batch file be more efficient than creating sessions through .NET?

No. It should be the same.

Okay, I thought so. We are going to be restructuring our project in the very near future to break out some of the larger tasks, so I think for now this won't be a huge issue. Thank you for your help.