Synchronize as "stream" to speed up?

Advertisement

Waruwaru
Joined:
Posts:
44
Location:
Deutschland

Synchronize as "stream" to speed up?

Perhaps a false assumption, but I have noticed that the data transfer speed is extremely slow when a large number of small files are to be synchronized and much faster for larger files. At least it seems as if the speed depends on the file size. In other words, the speed corresponds to the maximum file size. Otherwise I can't explain why thumbnails synchronize with 10-20 KB/s and I need years and at the same time synchronize a folder with larger graphics with values between 500 KB and 1 MB. At least 10 MB/s would be possible with my connection and the server can also deliver this. So why is it so slow do download?

Is it not possible to send a continuous "stream" here so that the speed does not have to be increased for each new file? Is this just an observation that may be completely wrong?

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
40,605
Location:
Prague, Czechia

Re: Synchronize as "stream" to speed up?

With each file there is some overhead for initiating the transfer. And that's why transferring small files (or even copying them locally) is always slower than transferring one huge file, as possibly that overhead has more impact than the actual transfer.

We have a script that can synchronize using multiple connections:
Automating transfers or synchronization in parallel connections over SFTP/FTP protocol

Reply with quote

Advertisement

You can post new topics in this forum