Post a reply

Options
Add an Attachment

If you do not want to add an Attachment to your Post, please leave the Fields blank.

(maximum 10 MB; please compress large files; only common media, archive, text and programming file formats are allowed)

Options

Topic review

martin

Re: Synchronize as "stream" to speed up?

With each file there is some overhead for initiating the transfer. And that's why transferring small files (or even copying them locally) is always slower than transferring one huge file, as possibly that overhead has more impact than the actual transfer.

We have a script that can synchronize using multiple connections:
Automating transfers or synchronization in parallel connections over SFTP/FTP protocol
Waruwaru

Synchronize as "stream" to speed up?

Perhaps a false assumption, but I have noticed that the data transfer speed is extremely slow when a large number of small files are to be synchronized and much faster for larger files. At least it seems as if the speed depends on the file size. In other words, the speed corresponds to the maximum file size. Otherwise I can't explain why thumbnails synchronize with 10-20 KB/s and I need years and at the same time synchronize a folder with larger graphics with values between 500 KB and 1 MB. At least 10 MB/s would be possible with my connection and the server can also deliver this. So why is it so slow do download?

Is it not possible to send a continuous "stream" here so that the speed does not have to be increased for each new file? Is this just an observation that may be completely wrong?