Synchronize failing for large directories

Advertisement

naasking
Joined:
Posts:
7

Synchronize failing for large directories

Using WinSCP 4.3.7 on Windows 7, connecting over straight FTP to a Windows 2008 server. Synchronize always fails when the directories are very out of sync. For example, perhaps more than X files exist remotely but not locally. I don't know what X is exactly, but I know it's more than 15. If it's a small number of differences, the error does not occur.

The file sizes range from 200KB up to 3MB, and the size of the directory is currently around 5,500 items, although I experienced the error as early as 4,500 items.

I use the synchronize function in a backup script like this:
option batch on
option confirm off
open "user@host"

cd Backups\Diffs
lcd Diffs
synchronize local

exit

Edit: to be more specific, the error is this:
---------------------------
Error
---------------------------
Timeout waiting for external console to complete the command.
---------------------------
OK Help
---------------------------

Reply with quote

Advertisement

naasking

Sorry for the noise. I had read that page, but didn't think it applied because I was calling it from a batch file. Turns out the program with the problem wasn't the batch file program, it was indeed a .NET program, and I had confused the two. Thanks!

Reply with quote

naasking
Joined:
Posts:
7

I adjusted the program using the recommendations in the documentation, but now something strange is going on. It seems to be copying over all the files that it had already copied. I have thousands of differential backup sets and winscp seems to start at the beginning every time I run it. Synchronizations are now taking forever.

They're both Windows machines, and I've checked the timestamps and they're the same on both ends as far as I can tell (even when viewed via FTP). Since this happened so recently, I suspect the recent DST change March 11th, but I really don't know how to proceed from here.

Reply with quote

Advertisement

naasking

Yes, they were off by an hour. There was no setting I could find on the FTP server to match the DST change, so I just manually adjusted the offset by an hour in the WinSCP settings. Thanks.

Reply with quote

naasking
Joined:
Posts:
7

The manual offset of one hour seems to have solved resyncing all existing files, however when WinSCP detected a new file out of sync, it downloaded it and then gave it the wrong timestamp. This causes it to resync the same file over and over again.

I had to remove the 1 hour offset, download the file manually, then add the manual offset back. Then WinSCP stopped resyncing the file.

WinSCP 4.3.7 build 1679.

Reply with quote

Advertisement

GaelenBurns
Joined:
Posts:
2
Location:
United States

only 4500 files causing trouble? I have 500,000

Has this been resolved, or should I just stop pursuing Winscp? My directory that I want to synchronize has 500,000 files, not just 5,000. About 100-300 files are added per day.

Do I have a chance, or is this an inappropriate use?

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
40,476
Location:
Prague, Czechia

Re: only 4500 files causing trouble? I have 500,000

GaelenBurns wrote:

Has this been resolved, or should I just stop pursuing Winscp? My directory that I want to synchronize has 500,000 files, not just 5,000. About 100-300 files are added per day.

Do I have a chance, or is this an inappropriate use?
There's no limit set. Definitely worth trying. Let us know.

Reply with quote

Advertisement

You can post new topics in this forum