Sync with Google Cloud

Advertisement

rupertito
Guest

Sync with Google Cloud

Hello

I use this script to connect to Google Cloud:
option batch continue
option confirm off
option reconnecttime 5
open s3://%id%:%secret%@storage.googleapis.com/
synchronize remote -delete -preservetime -mirror -filemask=%mask% %dirLocal% %bucket%
close
exit
Everything is fine, except that it doesn't sync, rather it copies all the content, even if the files are identical.
I've also tried it from PowerShell and the result is the same.
If I use the FTP on my synology, the sync works correctly.

What am I doing wrong?

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
40,852
Location:
Prague, Czechia

Re: Sync with Google Cloud

Afaik, S3 protocol does not support preserving remote file timestamps.
(Definitely, WinSCP does not preserve the timestamps when uploading to S3).
So the -preservetime has no effect and indeed, your command would result in all files being uploaded every time, as all remote files have different timestamps (times of the upload). So WinSCP sees all of them as modified.
You can remove "time" from synchronization criteria. And you might want to add "size":
synchronize remote -delete -criteria=size -mirror -filemask=%mask% %dirLocal% %bucket%
https://winscp.net/eng/docs/scriptcommand_synchronize#criteria
Though obviously, modified files, that do not change their size, won't be synchronized.

It might be useful to support checksum-based synchronization with S3 protocol. Though it has limitations on AWS side for large files. So it may not work in the end.
Issue 2295 – Checksum support for S3 protocol

Reply with quote

rupertito
Guest

Sync with Google Cloud

Thanks, Martin
I had to make a decision about which ftp server to use, and obviously S3 is not suitable for me, because using -criteria=size is not feasible in my case. I will continue with ftp on synology

Reply with quote

Advertisement

You can post new topics in this forum