Post a reply

Options
Add an Attachment

If you do not want to add an Attachment to your Post, please leave the Fields blank.

(maximum 10 MB; please compress large files; only common media, archive, text and programming file formats are allowed)

Options

Topic review

jacobgralinski

Okay, I thought so. We are going to be restructuring our project in the very near future to break out some of the larger tasks, so I think for now this won't be a huge issue. Thank you for your help.
martin

Could the batch file be more efficient than creating sessions through .NET?

No. It should be the same.
jacobgralinski

At the moment I don't believe I can, but as of late I believe the issues have been on the other end of the connection. What I do have is one of our tasks uses a Windows batch file to conduct the downloads and uploads, and that typically takes 10 seconds to run, even with over 300 files to download and a few larger files to upload (3KB to 10KB). In that task, the C#/.NET project creates the files and puts them in a folder for the batch file to pick up and upload. The downloads and uploads still go through WinSCP. Could the batch file be more efficient than creating sessions through .NET?
martin

OK, so can you try any other SFTP client?
jacobgralinski

I understand now, sorry about that. We do not currently use any other SFTP clients besides WinSCP, sorry for that confusion. We have numerous connections that all use WinSCP, and the speed appears to vary between them, so I was curious if there is any way to resolve the slower connections.
martin

You have again attached a log from WinSCP:
. 2021-09-20 11:17:09.261 --------------------------------------------------------------------------
. 2021-09-20 11:17:09.261 WinSCP Version 5.18.1 beta (Build 11089) (OS 6.2.9200 - Windows Server 2012 Standard)
jacobgralinski

I'm sorry, I must've misunderstood what you were looking for. Do you want a log from the code that I have creating a session to the client?
martin

That log is from WinSCP too.
jacobgralinski

Here is a session log for another SFTP client. The files being upload are roughly the same size, so this is a good comparison.
martin

So please post verbose log file from the other SFTP client too.
jacobgralinski

Hi Martin, there are a couple different SFTP clients. One of them completes uploads of a similar size in about half the time with the same number of files. If the issue is server response time, is there anything that can be done? I have noticed that the download side also appears to suffer from slowness.

At the moment this issue isn't a huge issue, but in the past it has conflicted with other tasks that run at the same time.
martin

Re: Transfer Speed of Numerous Small Files

Your server takes long time to respond to every request. Do you have a different experience with any other SFTP client?
Guest

Transfer Speed of Numerous Small Files

Hello,

When putting files onto a SFTP server with the .NET assembly, what is the expected speed when it comes to small files (1 KB or less)? For example, I had 18 1KB files take around 4 minutes to upload, and that is with multiple Session Tasks running. It seems to vary in the amount of time, but it's generally longer than 2 minutes.

I watched the files process a few times, and it seems like it's 1 file every 10 seconds or so, which feels like a long time, even for small files.

Is there anything I can look at to improve this speed? The files, once uploaded, do get copied to an archive folder, and that does have over 100,000 files in it already if that makes a difference.

Let me know if you need me to provide more information, and I will try to get that to you.

Thanks