synchronize command stopped working in console environment

Hi

I've got some time under my belt with WinSCP, so I'm not what i'd call a newbie.

I've had a script that backs up a remote SSH server to my local machine using SFTP. Its worked before, but it stopped working at some point. I'm not sure when exactly it stopped working. The script itself has not changed

I've been making regular upgrades to WinSCP. I was just using 4.0.7, but I've uninstalled and reinstalled back to 4.0.4 to see if it make any difference. It doesn't.

To troubleshoot, I've run my script line by line by hand in a "WinSCP.exe /console" window. There are no errors. Heres the script. Its really very simple

option batch continue
option confirm off
option synchdelete on
open HOST
synchronize local "D:\JDTEST\" "/D/WebDir"

Heres how it looks on in the console window

winscp> option batch continue
batch continue
winscp> option confirm off
confirm off
winscp> option synchdelete on
synchdelete on
winscp> open HOST
Searching for host...
Connecting to host...
Authenticating...
Using username "USER".
Authenticating with pre-entered password.
Authenticated.
Starting the session...
Reading remote directory...
Session started.
Active session: [1] HOST
winscp> synchronize local "D:\JDTEST\" "/D/WebDir"
Comparing 'D:\JDTEST' with '/D/WebDir'...
winscp>

For the purposes of this test, the local directory i'm using is completely empty.

when the synchronise command is entered, despite the message saying 'comparing....', the prompt returns immediately.
Nothing is copied from the remote to the local dir. Its as if WinSCP see's nothing in the remote directory - no files and no subdirectories, so it has nothing to do!
But I know there is data there because - from the same console window - i can manually traverse the remote directly structures using 'cd' and 'ls' commands, seeing everying
So, I know that the paths are correct and that there are files there. I also know that WinSCP is logged in and communicating.

Incidentally, i've also tried carrying out the same operation from the GUI and it works just fine!

It doesnt matter what i do, I'm unable to get this script, which once worked perfectly, to work anymore.
In fact i'm no longer able to get Synchronisation command to work in the console in any way at all.
Once again, *nothing* in the script itself has changed.

Can Anyone shed any light on this? I'm at a totall loss.

Here's the log file output from 4.0.7, and from 4.0.4

. 2008-03-13 13:00:23.419 --------------------------------------------------------------------------
. 2008-03-13 13:00:23.419 WinSCP Version 4.0.7 (Build 370) (OS 5.1.2600 Service Pack 2)
. 2008-03-13 13:00:23.419 Login time: 13 March 2008 13:00:23
. 2008-03-13 13:00:23.419 --------------------------------------------------------------------------
. 2008-03-13 13:00:23.419 Session name: HOST
. 2008-03-13 13:00:23.419 Host name: HOST (Port: 22)
. 2008-03-13 13:00:23.434 User name: USER (Password: Yes, Key file: No)
. 2008-03-13 13:00:23.434 Tunnel: No
. 2008-03-13 13:00:23.434 Transfer Protocol: SFTP (SCP)
. 2008-03-13 13:00:23.434 Ping type: -, Ping interval: 30 sec; Timeout: 15 sec
. 2008-03-13 13:00:23.434 Proxy: none
. 2008-03-13 13:00:23.434 SSH protocol version: 2; Compression: No
. 2008-03-13 13:00:23.434 Agent forwarding: No; TIS/CryptoCard: No; KI: Yes; GSSAPI: No
. 2008-03-13 13:00:23.434 Ciphers: aes,blowfish,3des,WARN,des; Ssh2DES: No
. 2008-03-13 13:00:23.434 SSH Bugs: -,-,-,-,-,-,-,-
. 2008-03-13 13:00:23.434 SFTP Bugs: -,-,-
. 2008-03-13 13:00:23.434 Return code variable: Autodetect; Lookup user groups: Yes
. 2008-03-13 13:00:23.434 Shell: default, EOL: 0
. 2008-03-13 13:00:23.434 Clear aliases: Yes, Unset nat.vars: Yes, Resolve symlinks: Yes
. 2008-03-13 13:00:23.434 Alias LS: No, Ign LS warn: Yes, Scp1 Comp: No
. 2008-03-13 13:00:23.434 Local directory: default, Remote directory: home, Update: No, Cache: Yes
. 2008-03-13 13:00:23.434 Cache directory changes: Yes, Permanent: Yes
. 2008-03-13 13:00:23.434 DST mode: 1
. 2008-03-13 13:00:23.434 --------------------------------------------------------------------------
. 2008-03-13 13:00:23.434 Looking up host "HOST"
. 2008-03-13 13:00:23.434 Connecting to UP_ADDR port 22
. 2008-03-13 13:00:23.434 Server version: SSH-2.0-VShell_3_0_3_569 VShell
. 2008-03-13 13:00:23.434 We claim version: SSH-2.0-WinSCP_release_4.0.7
. 2008-03-13 13:00:23.434 Using SSH protocol version 2
. 2008-03-13 13:00:23.434 Doing Diffie-Hellman group exchange
. 2008-03-13 13:00:23.559 Doing Diffie-Hellman key exchange
. 2008-03-13 13:00:23.981 Host key fingerprint is:
. 2008-03-13 13:00:23.981 ssh-dss 1024 3c:a3:5b:8b:32:72:1e:96:11:92:c1:84:31:69:19:c0
. 2008-03-13 13:00:23.981 Initialised AES-256 client->server encryption
. 2008-03-13 13:00:23.981 Initialised HMAC-SHA1 client->server MAC algorithm
. 2008-03-13 13:00:23.981 Initialised AES-256 server->client encryption
. 2008-03-13 13:00:23.981 Initialised HMAC-SHA1 server->client MAC algorithm
! 2008-03-13 13:00:24.169 Using username "USER".
. 2008-03-13 13:00:24.169 Session password prompt (USER password: )
. 2008-03-13 13:00:24.169 Using stored password.
. 2008-03-13 13:00:24.169 Sent password
. 2008-03-13 13:00:24.169 Access granted
. 2008-03-13 13:00:24.169 Opened channel for session
. 2008-03-13 13:00:24.169 Started a shell/command
. 2008-03-13 13:00:24.169 --------------------------------------------------------------------------
. 2008-03-13 13:00:24.169 Using SFTP protocol.
. 2008-03-13 13:00:24.169 Doing startup conversation with host.
> 2008-03-13 13:00:24.169 Type: SSH_FXP_INIT, Size: 5, Number: -1
< 2008-03-13 13:00:24.200 Type: SSH_FXP_VERSION, Size: 768, Number: -1
. 2008-03-13 13:00:24.200 SFTP version 4 negotiated.
. 2008-03-13 13:00:24.200 Unknown server extension newline@vandyke.com=0x000000020D0A
. 2008-03-13 13:00:24.200 Server requests EOL sequence 0x0D0A.
. 2008-03-13 13:00:24.200 Unknown server extension default-fs-attribs@vandyke.com=....
. 2008-03-13 13:00:24.200 Server support information:
. 2008-03-13 13:00:24.200 Attribute mask: 800003F9, Attribute bits: 17, Open flags: 83F
. 2008-03-13 13:00:24.200 Access mask: 1F01FF, Open block masks: 0, Block masks: 0, Max read size: 32768
. 2008-03-13 13:00:24.200 Attribute extensions (0)
. 2008-03-13 13:00:24.200 Extensions (5)
. 2008-03-13 13:00:24.200 text-seek
. 2008-03-13 13:00:24.200 fs-multiple-roots-supported@vandyke.com
. 2008-03-13 13:00:24.200 space-available
. 2008-03-13 13:00:24.200 version-select
. 2008-03-13 13:00:24.200 vendor-id
. 2008-03-13 13:00:24.200 Server support information:
. 2008-03-13 13:00:24.200 Attribute mask: 800003F9, Attribute bits: 17, Open flags: 83F
. 2008-03-13 13:00:24.200 Access mask: 1F01FF, Open block masks: FFFFFFFF, Block masks: 0, Max read size: 32768
. 2008-03-13 13:00:24.200 Attribute extensions (0)
. 2008-03-13 13:00:24.200 Extensions (5)
. 2008-03-13 13:00:24.200 text-seek
. 2008-03-13 13:00:24.200 fs-multiple-roots-supported@vandyke.com
. 2008-03-13 13:00:24.200 space-available
. 2008-03-13 13:00:24.200 version-select
. 2008-03-13 13:00:24.200 vendor-id
. 2008-03-13 13:00:24.200 Server software: VShell 3.0.3 (build 569 Unicode) (569) by VanDyke Software, Inc.
. 2008-03-13 13:00:24.200 SFTP versions supported by the server: 3,4,draft-ietf-secsh-filexfer-11@vandyke.com,partial-v6@vandyke.com
> 2008-03-13 13:00:24.200 Type: SSH_FXP_EXTENDED, Size: 63, Number: 200
. 2008-03-13 13:00:24.200 We will use UTF-8 strings when appropriate
. 2008-03-13 13:00:24.200 Getting current directory name.
. 2008-03-13 13:00:24.200 Getting real path for '.'
> 2008-03-13 13:00:24.200 Type: SSH_FXP_REALPATH, Size: 10, Number: 272
< 2008-03-13 13:00:24.200 Type: SSH_FXP_STATUS, Size: 57, Number: 200
. 2008-03-13 13:00:24.200 Discarding reserved response
< 2008-03-13 13:00:24.200 Type: SSH_FXP_NAME, Size: 19, Number: 272
. 2008-03-13 13:00:24.200 Real path is '/'
. 2008-03-13 13:00:24.200 Startup conversation with host finished.
. 2008-03-13 13:00:24.200 Collecting synchronization list for local directory 'D:\JDTEST\' and remote directory '/D/WebDir', mode = 1, params = 1031
. 2008-03-13 13:00:24.200 Listing directory "D:\JDTEST\".
> 2008-03-13 13:00:24.200 Type: SSH_FXP_OPENDIR, Size: 19, Number: 523
< 2008-03-13 13:00:24.200 Type: SSH_FXP_HANDLE, Size: 13, Number: 523
> 2008-03-13 13:00:24.200 Type: SSH_FXP_READDIR, Size: 13, Number: 780
< 2008-03-13 13:00:24.200 Type: SSH_FXP_NAME, Size: 154, Number: 780
> 2008-03-13 13:00:24.200 Type: SSH_FXP_READDIR, Size: 13, Number: 1036
< 2008-03-13 13:00:24.200 Type: SSH_FXP_STATUS, Size: 44, Number: 1036
< 2008-03-13 13:00:24.200 Status/error code: 1
> 2008-03-13 13:00:24.200 Type: SSH_FXP_CLOSE, Size: 13, Number: 1284
. 2008-03-13 13:00:24.200 Synchronizing local directory 'D:\JDTEST\' with remote directory '/D/WebDir', params = 1031
. 2008-03-13 13:00:24.200 Closing connection.


Heres the log file after i reinstalled back to v4.0.4

. 2008-03-13 18:12:08.160 --------------------------------------------------------------------------
. 2008-03-13 18:12:08.160 WinSCP Version 4.0.4 (Build 346) (OS 5.1.2600 Service Pack 2)
. 2008-03-13 18:12:08.160 Login time: 13 March 2008 18:12:08
. 2008-03-13 18:12:08.160 --------------------------------------------------------------------------
. 2008-03-13 18:12:08.160 Session name: HOST
. 2008-03-13 18:12:08.160 Host name: HOST (Port: 22)
. 2008-03-13 18:12:08.160 User name: USER (Password: Yes, Key file: No)
. 2008-03-13 18:12:08.160 Tunnel: No
. 2008-03-13 18:12:08.160 Transfer Protocol: SFTP (SCP)
. 2008-03-13 18:12:08.160 Ping type: -, Ping interval: 30 sec; Timeout: 15 sec
. 2008-03-13 18:12:08.160 Proxy: none
. 2008-03-13 18:12:08.160 SSH protocol version: 2; Compression: No
. 2008-03-13 18:12:08.160 Agent forwarding: No; TIS/CryptoCard: No; KI: Yes; GSSAPI: No
. 2008-03-13 18:12:08.160 Ciphers: aes,blowfish,3des,WARN,des; Ssh2DES: No
. 2008-03-13 18:12:08.160 SSH Bugs: -,-,-,-,-,-,-,-
. 2008-03-13 18:12:08.160 SFTP Bugs: -,-,-
. 2008-03-13 18:12:08.160 Return code variable: Autodetect; Lookup user groups: Yes
. 2008-03-13 18:12:08.160 Shell: default, EOL: 1
. 2008-03-13 18:12:08.160 Clear aliases: Yes, Unset nat.vars: Yes, Resolve symlinks: Yes
. 2008-03-13 18:12:08.160 Alias LS: No, Ign LS warn: Yes, Scp1 Comp: No
. 2008-03-13 18:12:08.160 Local directory: default, Remote directory: home, Update: No, Cache: Yes
. 2008-03-13 18:12:08.160 Cache directory changes: Yes, Permanent: Yes
. 2008-03-13 18:12:08.160 DST mode: 0
. 2008-03-13 18:12:08.160 --------------------------------------------------------------------------
. 2008-03-13 18:12:08.160 Looking up host "HOST"
. 2008-03-13 18:12:08.160 Connecting to IP_ADDR port 22
. 2008-03-13 18:12:08.160 Server version: SSH-2.0-VShell_3_0_3_569 VShell
. 2008-03-13 18:12:08.160 We claim version: SSH-2.0-WinSCP_release_4.0.4
. 2008-03-13 18:12:08.160 Using SSH protocol version 2
. 2008-03-13 18:12:08.176 Doing Diffie-Hellman group exchange
. 2008-03-13 18:12:08.269 Doing Diffie-Hellman key exchange
. 2008-03-13 18:12:08.706 Host key fingerprint is:
. 2008-03-13 18:12:08.706 ssh-dss 1024 3c:a3:5b:8b:32:72:1e:96:11:92:c1:84:31:69:19:c0
. 2008-03-13 18:12:08.706 Initialised AES-256 client->server encryption
. 2008-03-13 18:12:08.706 Initialised HMAC-SHA1 client->server MAC algorithm
. 2008-03-13 18:12:08.706 Initialised AES-256 server->client encryption
. 2008-03-13 18:12:08.706 Initialised HMAC-SHA1 server->client MAC algorithm
! 2008-03-13 18:12:08.878 Using username "USER".
. 2008-03-13 18:12:08.878 Session password prompt (USERS password: )
. 2008-03-13 18:12:08.878 Using stored password.
. 2008-03-13 18:12:08.878 Sent password
. 2008-03-13 18:12:08.878 Access granted
. 2008-03-13 18:12:08.878 Opened channel for session
. 2008-03-13 18:12:08.878 Started a shell/command
. 2008-03-13 18:12:08.878 --------------------------------------------------------------------------
. 2008-03-13 18:12:08.878 Using SFTP protocol.
. 2008-03-13 18:12:08.878 Doing startup conversation with host.
> 2008-03-13 18:12:08.878 Type: SSH_FXP_INIT, Size: 5, Number: -1
< 2008-03-13 18:12:08.893 Type: SSH_FXP_VERSION, Size: 768, Number: -1
. 2008-03-13 18:12:08.893 SFTP version 4 negotiated.
. 2008-03-13 18:12:08.893 Unknown server extension newline@vandyke.com=0x000000020D0A
. 2008-03-13 18:12:08.893 Server requests EOL sequence 0x0D0A.
. 2008-03-13 18:12:08.893 Unknown server extension default-fs-attribs@vandyke.com=....
. 2008-03-13 18:12:08.893 Server support information:
. 2008-03-13 18:12:08.893 Attribute mask: 800003F9, Attribute bits: 17, Open flags: 83F
. 2008-03-13 18:12:08.893 Access mask: 1F01FF, Open block masks: 0, Block masks: 0, Max read size: 32768
. 2008-03-13 18:12:08.893 Attribute extensions (0)
. 2008-03-13 18:12:08.893 Extensions (5)
. 2008-03-13 18:12:08.893 text-seek
. 2008-03-13 18:12:08.893 fs-multiple-roots-supported@vandyke.com
. 2008-03-13 18:12:08.893 space-available
. 2008-03-13 18:12:08.893 version-select
. 2008-03-13 18:12:08.893 vendor-id
. 2008-03-13 18:12:08.909 Server support information:
. 2008-03-13 18:12:08.909 Attribute mask: 800003F9, Attribute bits: 17, Open flags: 83F
. 2008-03-13 18:12:08.909 Access mask: 1F01FF, Open block masks: FFFFFFFF, Block masks: 0, Max read size: 32768
. 2008-03-13 18:12:08.909 Attribute extensions (0)
. 2008-03-13 18:12:08.909 Extensions (5)
. 2008-03-13 18:12:08.909 text-seek
. 2008-03-13 18:12:08.909 fs-multiple-roots-supported@vandyke.com
. 2008-03-13 18:12:08.909 space-available
. 2008-03-13 18:12:08.909 version-select
. 2008-03-13 18:12:08.909 vendor-id
. 2008-03-13 18:12:08.909 Server software: VShell 3.0.3 (build 569 Unicode) (569) by VanDyke Software, Inc.
. 2008-03-13 18:12:08.909 SFTP versions supported by the server: 3,4,draft-ietf-secsh-filexfer-11@vandyke.com,partial-v6@vandyke.com
> 2008-03-13 18:12:08.909 Type: SSH_FXP_EXTENDED, Size: 63, Number: 200
. 2008-03-13 18:12:08.909 We will use UTF-8 strings when appropriate
. 2008-03-13 18:12:08.909 Getting current directory name.
. 2008-03-13 18:12:08.909 Getting real path for '.'
> 2008-03-13 18:12:08.909 Type: SSH_FXP_REALPATH, Size: 10, Number: 272
< 2008-03-13 18:12:08.909 Type: SSH_FXP_STATUS, Size: 57, Number: 200
. 2008-03-13 18:12:08.909 Discarding reserved response
< 2008-03-13 18:12:08.909 Type: SSH_FXP_NAME, Size: 19, Number: 272
. 2008-03-13 18:12:08.909 Real path is '/'
. 2008-03-13 18:12:08.909 Startup conversation with host finished.
. 2008-03-13 18:12:08.909 Collecting synchronization list for local directory 'D:\JDTEST\' and remote directory '/D/WebDir', mode = 1, params = 1031
. 2008-03-13 18:12:08.909 Listing directory "/D/InetPub".
> 2008-03-13 18:12:08.909 Type: SSH_FXP_OPENDIR, Size: 19, Number: 523
< 2008-03-13 18:12:08.909 Type: SSH_FXP_HANDLE, Size: 13, Number: 523
> 2008-03-13 18:12:08.909 Type: SSH_FXP_READDIR, Size: 13, Number: 780
< 2008-03-13 18:12:08.909 Type: SSH_FXP_NAME, Size: 154, Number: 780
> 2008-03-13 18:12:08.909 Type: SSH_FXP_READDIR, Size: 13, Number: 1036
< 2008-03-13 18:12:08.909 Type: SSH_FXP_STATUS, Size: 44, Number: 1036
< 2008-03-13 18:12:08.909 Status/error code: 1
> 2008-03-13 18:12:08.909 Type: SSH_FXP_CLOSE, Size: 13, Number: 1284
. 2008-03-13 18:12:08.909 Synchronizing local directory 'D:\JDTEST\' with remote directory '/D/WebDir/', params = 1031

I can see several strange things in the log. I cannot tell which one or more of them is causing the trouble.

1) You have the Existing files only turned on. Hence WinSCP won't synchronize anything in local directory is empty.

2) The 4.0.7 log says 'Listing directory "D:\JDTEST\"', when it is listing the remote directory.

3) The 4.0.4 log says 'Listing directory "/D/InetPub".' in the same situation.

Thanks for the insight. I see what you're getting at.
please ignore the transposition of source and destination Dirs in the log file. I manually edited them. It was late - I was tired. Probably a manual mistake. I'll check this again and post back

Regarding 'Existing files only' synch option. I can see this in the GUI, in the synch dialogue. It is indeed checked.

Question: If i change this options in the GUI, and check 'use same option next time', are these options then used as my defaults in the console?

I guess I have a few questions:

1. I know that 'option synchdelete' is something i can set at concole level. Are the other synch GUI options adjustable from inside the console environment? such as 'Existing files only'. If not, how do i control them? Are they 'inherited' fom the GUI? Documentation and command help show only limited switches for the synchronise command (I did notice more flexibility with synch command coming in next version - very much looking forward to that)

2. Are synch options that are set from inside the GUI then stored in the registry/INI file?

4. Are synch options GUI settings 'global' or 'per session'

5. If I remove WinSCP and ensure all files and registry settings are cleared, would default install of 4.0.7 clear any difficulty, do you think? I didnt clear the registry when i uninstalled 4.0.7 / reinstalled 4.0.4. Are there any files or reg keys (beyond the obvious ones) that need to be manually removed to ensure that all trace of the app is gone?

phew!

Cheers :)

J

OK

I think i answered my own questions.

'Existing files only' option seems to be global, not per session.
If i turn it off, and check the option to use 'same options next time' the synchronise console command now seems to work as expected.

So i guess at least some of the GUI settings are 'inherited' by the console.

So, many thanks for your assistance, I think I'm up and running again now.

Looking forward to the new version with the extended synch command functionality.

Many thanks again. Keep up the excellent work!!

J