Topic "Synchronize -mirror took 3 runs to complete"

Author Message
3K
[View user's profile]
Donor
Joined: 2012-01-20
Posts: 34
Location: Geneva
Hello,

I am synchronizing my test site which contains ~72GB. The script is show below. It took me 3 times to run to be able to fully download the files. On the first run, it downloaded 98%. On the second run, 99%, then on the third run, 100%. During download, it shows often that it's disconnected from the server or lost connection?

winscp.com /script="syncScript.txt"

syncScript.txt:

option batch continue
option confirm off
option reconnecttime 2
open ftp://.....
option transfer automatic
synchronize -mirror local "C:\Mirror_Site" "/"
close
exit

How can I improve it? Thanks.
Advertisements
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
Please attach a full log file showing the problem (using the latest version of WinSCP) for all runs.

To generate log file, use /log=path_to_log_file command-line argument. Submit the log with your post as an attachment. Note that passwords and passphrases not stored in the log. You may want to remove other data you consider sensitive though, such as host names, IP addresses, account names or file names (unless they are relevant to the problem). If you do not want to post the log publicly, you can mark the attachment as private.
_________________
Martin Prikryl
3K
[View user's profile]
Donor
Joined: 2012-01-20
Posts: 34
Location: Geneva
Hi,

I'm attaching the log file as private. I noticed that the lost connections is every minute or 2.

Thanks.
synclog-2015-02-05 10-32-01-reduced.rar (5.46 MB) Private file

Description: The original txt file is 391mb. I deleted many lines in the middle, now when compressed, it's now 5.59mb.

martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
"for all runs" please. Or at least for any two consecutive runs (where the latter run synchronizes the same files again).

Last edited by martin on 2015-02-18; edited 1 time in total
3K
[View user's profile]
Donor
Joined: 2012-01-20
Posts: 34
Location: Geneva
It seems some files were downloaded again. I checked their XML log, searched 6 filenames randomly and 4 were re-downloaded.

I've also other batch sync using mirror command that after few months that I re-run them, it downloads all the files again.

Regards,
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
Sorry, I do not understand your comment.

I asked you for a logs from two consecutive runs (where the latter run synchronizes the same files again). I got just one log from you. I cannot see from one log what files were downloaded again.
_________________
Martin Prikryl
3K
[View user's profile]
Donor
Joined: 2012-01-20
Posts: 34
Location: Geneva
I'm going to regenerate the log files. I've deleted all the files and run again the batch sync. However, it will take 2 days to complete this. Then, I will post the txt and xml logs.

Regards,
3K
[View user's profile]
Donor
Joined: 2012-01-20
Posts: 34
Location: Geneva
Hi Martin,

I am posting the Text and XML log files. I randomly checked some downloaded files on the first XML and they are present again on the second XML.

Some files downloaded twice:

C:\ITU\ITU_Mirror_Site\ITU_Publications\ITU_Menu.html
C:\ITU\ITU_Mirror_Site\ITU_Publications\dms\pages\itu-t\rec\g\T-REC-G.1080.xml
C:\ITU\ITU_Mirror_Site\ITU_Publications\dms\pages\itu-t\rec\g\T-REC-G.108-200309-I!Amd1.html
C:\ITU\ITU_Mirror_Site\ITU_Publications\dms\pages\itu-t\rec\g\T-REC-G.1091-201410-P.xml
C:\ITU\ITU_Mirror_Site\ITU_Publications\dms\pay\itu-t\rec\e\T-REC-E.502-198811-S!!ZWD-E.zip

FYI. I couldn't post the second log file as it's the 4th file.

Regards.
synclog-2015-02-18 10-19-18 - Copy.rar (8.33 MB) Private file

Description: first text log file.

syncxml-2015-02-19 08-14-11.rar (444.5 KB) Private file

Description: second xml log file.

syncxml-2015-02-18 10-19-18.rar (757.14 KB) Private file

Description: first xml log file.

3K
[View user's profile]
Donor
Joined: 2012-01-20
Posts: 34
Location: Geneva
Hello Martin,

In case you need the second log file, I have uploaded it here. The original file size is 328mb, so I deleted many lines in the middle and compress it. The first log file is around 370mb. FYI.

Regards.
synclog-2015-02-19 08-14-11 - Copy.rar (7.81 MB) Private file

Description: The second log file.

martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
Thanks for the logs!

I have sent you an email with a debug version of WinSCP to address you have used to register on this forum.
_________________
Martin Prikryl
3K
[View user's profile]
Donor
Joined: 2012-01-20
Posts: 34
Location: Geneva
Hi Martin,

Thanks, however I could not find your email with a debug version of WinSCP. Is it possible to attached it here as private file? Or otherwise, resend it again to my email.

Regards,
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
I've re-sent it.
_________________
Martin Prikryl
Advertisements

You can post new topics in this forum






Search Site

What is WinSCP?

It is award-winning SFTP client, SCP client, FTPS client and FTP client integrated into one software program for file transfer to FTP server or secure SFTP server. [More]

And it's free!

Donate

About donations

$9   $19   $49   $99

About donations

Recommend

WinSCP Privacy Policy

WinSCP License