c# - Bad download speed when writing to NAS (network drive) - solved?

hi,

I'm using the C# package to build an automatic downloader (from an FTP server on my LAN)

Everything works right out of the box, but...

I noticed that download speed is about 10x slower when I download to a network (NAS) folder, than when I download to a local (SSD) folder

NAS folder: 0.5MB/s
SSD folder: 5MB/s

(FileZilla does 5MB/s in both cases)

While looking at the file that was being downloaded by WinSCP, in Windows Explorer, I noticed that the size of the file changes very frequently.

I assumed that the difference in download speed might be caused by WinSCP doing many, many small writes to the file - and that this might freak out the NAS and degrade overall throughput performance (whereas the local SSD is able to keep up)

To confirm, I added some debug logic to my executable. This debug logic monitors the frequency with which the size of the file (that is being downloaded by WinSCP) changes.

If my calculations are correct I saw between 30 and 50 updates to the file PER SECOND.

At 5MB/s this means that WinSCP's (write) buffer size is roughly 125KB.

If the above is correct, can I suggest changing the size of the internal write buffer to 8MB or so?

Btw, it would be nice if we had more control over file IO in general in the WinSCP libraries.

Thanks,
Lambert

Thanks for your investigation.
WinSCP does not do any buffering atm, it leaves that on OS/filesystem. So adding buffering is quite a change.
How is your NAS connected/mapped to your machine? This actually looks like a badly implemented driver to me.

hi,

I'm using a consumer-grade NAS, WDMyCloud 4TB, hooked up to a router in my home network.

You're right, the WDMyCloud devices are indeed somewhat underpowered in terms of CPU/memory.

But I would like to mention that FileZilla doesn't have this "problem" - it downloads with 5MB/s to the same NAS or the local SSD (no difference), whereas WinSCP reaches only 10% of that speed (when writing to the NAS).

cheers,
Lambert

I understand that. But as I've wrote above, this is a huge change. I will see if more people ask for this.

martin wrote:

I understand that. But as I've wrote above, this is a huge change. I will see if more people ask for this.

I don't know why this should be *huge* change, I guess it would be sufficient to add something like this:

file.SetBufferSize(32000);

(on the output file handle)

Or am I missing the point entirely?

Thanks, Lambert

As I wrote above, there's no buffering implemented in WinSCP. WinSCP writes all data directly to WriteFile:
https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-writefile
All buffering is done by Windows OS inside the function. Afaik, there no API to control the OS buffer.