Synchronisation scripting headache

I've been trying to get a script for synchronising working over the last few days and its proving a huge headache. It just doesn't seem to work as expected.

I'm synchronising a directory where there is multiple subdirectories. I only want some of these subdirectories so I'm using either option include or exclude.

Here's the script using option exclude;

============
option confirm off
open sftp://budgetscp:<passwd>@<ip>
cd /www_backup/OpenJaw
option exclude "E:\budget\Copy of tomcat\"
option exclude "E:\budget\tomcat\logs\*"
option transfer binary
synchronize remote -criteria=both E:\budget\
close
exit
=============

And its output;

=============
E:\budget>winscp.com /command /script=ftp.txt
batch on
confirm off
Searching for host...
Connecting to host...
Authenticating...
Using username "budgetscp".
Authenticating with pre-entered password.
Authenticated.
Starting the session...
Reading remote directory...
Session started.
Active session: [1] budgetscp@<ip>
/www_backup/OpenJaw
exclude E:\budget\Copy of tomcat\
exclude E:\budget\tomcat\logs\*
transfer binary
Comparing...
Local 'E:\budget' => Remote '/www_backup/OpenJaw'
Synchronizing...
Local 'E:\budget' => Remote '/www_backup/OpenJaw'
E:\budget\Copy of tomcat | 0 KiB | 0.0 KiB/s | binary | 0%
E:\...\Copy of tomcat\bin | 0 KiB | 0.0 KiB/s | binary | 0%
E:\...\bin\bootstrap.jar | 23 KiB | 1246.4 KiB/s | binary | 100%
E:\...\bin\catalina.bat | 7 KiB | 134.1 KiB/s | binary | 100%
E:\...\bin\catalina.sh | 9 KiB | 87.3 KiB/s | binary | 100%
E:\...\commons-daemon.jar | 9 KiB | 75.8 KiB/s | binary | 100%
E:\...\bin\cpappend.bat | 0 KiB | 53.3 KiB/s | binary | 100%
E:\...\bin\digest.bat | 1 KiB | 34.1 KiB/s | binary | 100%
E:\...\bin\digest.sh | 0 KiB | 25.8 KiB/s | binary | 100%
E:\...\bin\jasper.bat | 2 KiB | 21.1 KiB/s | binary | 100%
E:\...\bin\jasper.sh | 0 KiB | 18.8 KiB/s | binary | 0%
Terminated by user.
===========

As you can see its including the directory I told it to exclude. I've used lots of different combinations in the exclude;
E:\budget\web\Copy of tomcat*
E:\budget\web\Copy of tomcat\
E:\budget\web\Copy of tomcat\*
E:\budget\web\Copy of tomcat\*.*
The "Copy of tomcat" directory is still included no matter what.



I've also tried using the option include method, with slightly more success, but only slightly. Here's the script;

============
option confirm off
open sftp://budgetscp:<passwd>@<ip>
cd /www_backup/OpenJaw
option include "E:\budget\web\"
option transfer binary
synchronize remote -criteria=both E:\budget\
close
exit
=============


And here's the output;

=============

E:\budget>winscp.com /command /script=ftp.txt
batch on
confirm off
Searching for host...
Connecting to host...
Authenticating...
Using username "budgetscp".
Authenticating with pre-entered password.
Authenticated.
Starting the session...
Reading remote directory...
Session started.
Active session: [1] budgetscp@<IP>
/www_backup/OpenJaw
include E:\budget\web\
transfer binary
Comparing...
Local 'E:\budget' => Remote '/www_backup/OpenJaw'
Synchronizing...
Local 'E:\budget' => Remote '/www_backup/OpenJaw'
E:\budget\web | 0 KiB | 0.0 KiB/s | binary | 0%
E:\budget\web\book.jsp | 0 KiB | 0.0 KiB/s | binary | 0%
E:\...\web\book.jsp_new | 0 KiB | 0.0 KiB/s | binary | 0%
E:\...\brochureSearch.jsp | 0 KiB | 0.0 KiB/s | binary | 0%
E:\budget\web\config | 0 KiB | 0.0 KiB/s | binary | 0%
E:\budget\web\css | 0 KiB | 0.0 KiB/s | binary | 0%
E:\budget\web\error.jsp | 0 KiB | 0.0 KiB/s | binary | 0%
E:\budget\web\home.jsp | 0 KiB | 0.0 KiB/s | binary | 0%
E:\budget\web\images | 0 KiB | 0.0 KiB/s | binary | 0%
E:\budget\web\index.jsp | 0 KiB | 0.0 KiB/s | binary | 0%
E:\budget\web\old | 0 KiB | 0.0 KiB/s | binary | 0%
E:\...\packageResults.jsp | 0 KiB | 0.0 KiB/s | binary | 0%
E:\...\packageSearch.jsp | 0 KiB | 0.0 KiB/s | binary | 0%
E:\budget\web\pages | 0 KiB | 0.0 KiB/s | binary | 0%
E:\budget\web\popups | 0 KiB | 0.0 KiB/s | binary | 0%
E:\budget\web\scripts | 0 KiB | 0.0 KiB/s | binary | 0%
E:\...\SessionAdmin.jsp | 0 KiB | 0.0 KiB/s | binary | 0%
E:\...\waitForResults.jsp | 0 KiB | 0.0 KiB/s | binary | 0%
E:\budget\web\WEB-INF | 0 KiB | 0.0 KiB/s | binary | 0%
E:\...\web\xRezError.jsp | 0 KiB | 0.0 KiB/s | binary | 0%
E:\...\web\xRezReset.jsp | 0 KiB | 0.0 KiB/s | binary | 0%
Session 'budgetscp@193.95.150.26' closed.
No session.

============

This files should not be 0 KB and nothing shows up on the remote server except for the "web" subdirectory which is empty.



Is there any chance somebody could help me out here because I'm not sure of the syntax for the option commands? Shoud I be using windows directory notation or unix?

Some further examples of those would be helpful in the documentation, like having multiple "option exclude <dir>" in a row, or how the include/exclude pipe works mentioned on the last line of the File Mask section.

Thanks for any help,
R

I just realise by reading here that the second option exclude was overwriting the first. How can I exclude multiple directories? Do I seperate them with semi colons?

I'm still stuck with the include end of things tho.

I've managed to get some kind of functionality from the include statement.
I've added the following line;

option include "E:\budget\web\; E:\budget\web\*; E:\budget\web\*\*;

The way this seems to be working is that it synchronises the first directory (not the files in it but the directory itself)
It then synchronises the files in the first directory using the wildcard. This means if there's sub directories they will be sync'd but the files in them wont be.
The last entry will then sync everything 1 directory down.
If there were further sub directories you would need to continue to add entries with extra \* wildcards until you've reached the lowest level of sub directory.

If this is the expected way that the include statment works I think it should be changed. When a directory is included in a synchronisation I think the transfer should work recurrsively ... the way the exclude statement works.

Using the option Exclude is not a good way to do things because the contents of a directory may change over time ... a large log archive for example for grow by several GBs over a matter of a few days ... and these changes would be included in the sync. The include will only ever transfer what you want regardless of the other directory content.

Does anybody disagree with me?

Actually your last mask should include everything under "E:\budget\web\". Let me know if it does not.

The problem with 0kB files looks like a bug, I'll check it.

I just tested it using only the last mask.
Here's my script;

=======================
option batch on
option confirm off
open sftp://budgetscp:<pwd>@<IP>
cd /www_backup/OpenJaw/test
option include " E:\budget\dpsearch\*\*; E:\budget\golf\*\*; E:\budget\web\*\*;"
option transfer binary
synchronize remote E:\budget\
close
exit
=======================

All the directories on the local server exist, containing non-zero files, and the remote directory is empty.
When I execute this I get the following result;

=======================
E:\budget>winscp /script=FTP_Commands.txt
batch on
confirm off
Searching for host...
Connecting to host...
Authenticating...
Using username "budgetscp".
Authenticating with pre-entered password.
Authenticated.
Starting the session...
Reading remote directory...
Session started.
Active session: [1] budgetscp@<IP>
/www_backup/OpenJaw/test
include E:\budget\dpsearch\*\*; E:\budget\golf\*\*; E:\budget\web\*\*;
transfer binary
Comparing...
Local 'E:\budget' => Remote '/www_backup/OpenJaw/test'
Nothing to synchronize.
Session 'budgetscp@<IP>' closed.
No session.
======================

Maybe somebody could try to replicate this?

You have to do:
option include "E:\budget\dpsearch\; E:\budget\dpsearch\*; E:\budget\dpsearch\*\*; E:\budget\golf\; E:\budget\golf\*; E:\budget\golf\*\*; E:\budget\web\; E:\budget\web\*; E:\budget\web\*\*"

martin wrote:

The problem with 0kB files looks like a bug, I'll check it.

This bug has been added to tracker.