Synchronize -mirror took 3 runs to complete

Advertisement

3K
Donor
Joined:
Posts:
32
Location:
Geneva

Synchronize -mirror took 3 runs to complete

Hello,

I am synchronizing my test site which contains ~72GB. The script is show below. It took me 3 times to run to be able to fully download the files. On the first run, it downloaded 98%. On the second run, 99%, then on the third run, 100%. During download, it shows often that it's disconnected from the server or lost connection?

winscp.com /script="syncScript.txt"

syncScript.txt:

option batch continue
option confirm off
option reconnecttime 2
open ftp://.....
option transfer automatic
synchronize -mirror local "C:\Mirror_Site" "/"
close
exit

How can I improve it? Thanks.

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
40,476
Location:
Prague, Czechia

Re: Synchronize -mirror took 3 runs to complete

Please attach a full log file showing the problem (using the latest version of WinSCP) for all runs.

To generate log file, use /log=path_to_log_file command-line argument. Submit the log with your post as an attachment. Note that passwords and passphrases not stored in the log. You may want to remove other data you consider sensitive though, such as host names, IP addresses, account names or file names (unless they are relevant to the problem). If you do not want to post the log publicly, you can mark the attachment as private.

Reply with quote

3K
Donor
Joined:
Posts:
32
Location:
Geneva

Hi,

I'm attaching the log file as private. I noticed that the lost connections is every minute or 2.

Thanks.
  • synclog-2015-02-05 10-32-01-reduced.rar (5.46 MB, Private file)
Description: The original txt file is 391mb. I deleted many lines in the middle, now when compressed, it's now 5.59mb.

Reply with quote

martin
Site Admin
martin avatar

"for all runs" please. Or at least for any two consecutive runs (where the latter run synchronizes the same files again).
Last edited by martin on 2015-02-18; edited 1 time in total

Reply with quote

3K
Donor
Joined:
Posts:
32
Location:
Geneva

It seems some files were downloaded again. I checked their XML log, searched 6 filenames randomly and 4 were re-downloaded.

I've also other batch sync using mirror command that after few months that I re-run them, it downloads all the files again.

Regards,

Reply with quote

Advertisement

martin
Site Admin
martin avatar

Sorry, I do not understand your comment.

I asked you for a logs from two consecutive runs (where the latter run synchronizes the same files again). I got just one log from you. I cannot see from one log what files were downloaded again.

Reply with quote

3K
Donor

I'm going to regenerate the log files. I've deleted all the files and run again the batch sync. However, it will take 2 days to complete this. Then, I will post the txt and xml logs.

Regards,

Reply with quote

3K
Donor
Joined:
Posts:
32
Location:
Geneva

Hi Martin,

I am posting the Text and XML log files. I randomly checked some downloaded files on the first XML and they are present again on the second XML.

Some files downloaded twice:

C:\ITU\ITU_Mirror_Site\ITU_Publications\ITU_Menu.html
C:\ITU\ITU_Mirror_Site\ITU_Publications\dms\pages\itu-t\rec\g\T-REC-G.1080.xml
C:\ITU\ITU_Mirror_Site\ITU_Publications\dms\pages\itu-t\rec\g\T-REC-G.108-200309-I!Amd1.html
C:\ITU\ITU_Mirror_Site\ITU_Publications\dms\pages\itu-t\rec\g\T-REC-G.1091-201410-P.xml
C:\ITU\ITU_Mirror_Site\ITU_Publications\dms\pay\itu-t\rec\e\T-REC-E.502-198811-S!!ZWD-E.zip

FYI. I couldn't post the second log file as it's the 4th file.

Regards.
  • synclog-2015-02-18 10-19-18 - Copy.rar (8.33 MB, Private file)
Description: first text log file.
  • syncxml-2015-02-19 08-14-11.rar (444.5 KB, Private file)
Description: second xml log file.
  • syncxml-2015-02-18 10-19-18.rar (757.14 KB, Private file)
Description: first xml log file.

Reply with quote

3K
Donor
Joined:
Posts:
32
Location:
Geneva

Hello Martin,

In case you need the second log file, I have uploaded it here. The original file size is 328mb, so I deleted many lines in the middle and compress it. The first log file is around 370mb. FYI.

Regards.
  • synclog-2015-02-19 08-14-11 - Copy.rar (7.81 MB, Private file)
Description: The second log file.

Reply with quote

Advertisement

martin
Site Admin
martin avatar

Thanks for the logs!

I have sent you an email with a debug version of WinSCP to address you have used to register on this forum.

Reply with quote

3K
Donor
Joined:
Posts:
32
Location:
Geneva

Hi Martin,

Thanks, however I could not find your email with a debug version of WinSCP. Is it possible to attached it here as private file? Or otherwise, resend it again to my email.

Regards,

Reply with quote

Advertisement

You can post new topics in this forum