Synchronize failing for large directories

Advertisement

naasking
Joined:
Posts:
7

Synchronize failing for large directories

Using WinSCP 4.3.7 on Windows 7, connecting over straight FTP to a Windows 2008 server. Synchronize always fails when the directories are very out of sync. For example, perhaps more than X files exist remotely but not locally. I don't know what X is exactly, but I know it's more than 15. If it's a small number of differences, the error does not occur.

The file sizes range from 200KB up to 3MB, and the size of the directory is currently around 5,500 items, although I experienced the error as early as 4,500 items.

I use the synchronize function in a backup script like this:
option batch on
option confirm off
open "user@host"

cd Backups\Diffs
lcd Diffs
synchronize local

exit    

Edit: to be more specific, the error is this:
---------------------------
Error
---------------------------
Timeout waiting for external console to complete the command.
---------------------------
OK Help
---------------------------

Reply with quote

Advertisement

naasking
Joined:
Posts:
7

Sorry for the noise. I had read that page, but didn't think it applied because I was calling it from a batch file. Turns out the program with the problem wasn't the batch file program, it was indeed a .NET program, and I had confused the two. Thanks!

Reply with quote

naasking
Joined:
Posts:
7

I adjusted the program using the recommendations in the documentation, but now something strange is going on. It seems to be copying over all the files that it had already copied. I have thousands of differential backup sets and winscp seems to start at the beginning every time I run it. Synchronizations are now taking forever.

They're both Windows machines, and I've checked the timestamps and they're the same on both ends as far as I can tell (even when viewed via FTP). Since this happened so recently, I suspect the recent DST change March 11th, but I really don't know how to proceed from here.

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
27,271
Location:
Prague, Czechia

So if you check the timestamps from WinSCP GUI, do you see the same for local and remote files?
_________________
Martin Prikryl

Reply with quote

naasking
Joined:
Posts:
7

Yes, they were off by an hour. There was no setting I could find on the FTP server to match the DST change, so I just manually adjusted the offset by an hour in the WinSCP settings. Thanks.

Reply with quote

naasking
Joined:
Posts:
7

The manual offset of one hour seems to have solved resyncing all existing files, however when WinSCP detected a new file out of sync, it downloaded it and then gave it the wrong timestamp. This causes it to resync the same file over and over again.

I had to remove the 1 hour offset, download the file manually, then add the manual offset back. Then WinSCP stopped resyncing the file.

WinSCP 4.3.7 build 1679.

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
27,271
Location:
Prague, Czechia

I have sent you an email with a debug version of WinSCP to address you have used to register on this forum.
_________________
Martin Prikryl

Reply with quote

GaelenBurns
Joined:
Posts:
2
Location:
United States

only 4500 files causing trouble? I have 500,000

Has this been resolved, or should I just stop pursuing Winscp? My directory that I want to synchronize has 500,000 files, not just 5,000. About 100-300 files are added per day.

Do I have a chance, or is this an inappropriate use?

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
27,271
Location:
Prague, Czechia

Re: only 4500 files causing trouble? I have 500,000

GaelenBurns wrote:

Has this been resolved, or should I just stop pursuing Winscp? My directory that I want to synchronize has 500,000 files, not just 5,000. About 100-300 files are added per day.

Do I have a chance, or is this an inappropriate use?
There's no limit set. Definitely worth trying. Let us know.

Reply with quote

Advertisement

You can post new topics in this forum