Forum » Scripting / Automation »

Sync with files exclusions

ehm01
Joined:: 2016-10-30
Posts:: 8
Location:: USA

Sync with files exclusions

2016-10-30 23:45

I'm having a hard to time to figure this out using the resources here, so, if I overlook something, bear with me.
Using WinSCP 5.9.2

I need to copy the whole web site back to my PC, with exclusion of couple folders that are 'sessions' on a database. If I try to sync everything, include that /sessions folder, sometimes I get error when a user leaves the database, so I figure out that the best option will be to EXCLUDE that.

This is my .txt file:

synchronize local c:\MySites\Site1\ /public_html -delete -preservetime -filemask=""*/*|*/sessions""

This work on the sense that /sessions IS NOT being copied, but also, any sub-folder from my root. What I'm doing wrong?
I have this structure:

/
index.html
manyfiles.html
/folder1
/folder1/index.html
/folder1/manyfiles.html
/folder2
/folder2/index.html
/sessions/
/sessions/manysessions.html

Using the instructions above, I copy all files under / but none under any /folder1 or /folder2.

So, what is wrong?
Thank you very much

Reply with quote

jrp78
Joined:: 2016-10-07
Posts:: 22
Location:: SoVA

2016-10-31 01:10

Try:

/public_html -delete -preservetime -filemask=""| sessions/""

Reply with quote

ehm01
Joined:: 2016-10-30
Posts:: 8
Location:: USA

2016-10-31 01:39

Try it.
the return was:
all sub-folder (included /sessions)
but any file. The sub-folders are empty on my PC (and they are not on the server)
and any file from the root was downloaded too.

Just to make easy, this is what I already try:

-filemask=""|*/sessions/""
-filemask=""|/sessions/""
-filemask=""*/|*/sessions/""
-filemask=""*.*/|*/sessions/""
-filemask=""*/*|*/sessions/""

The results vary, but any gives me ALL files from root, All folder, All sub-folders, All files from sub-folders BUT not /sessions.

Other ideas please?

Reply with quote

jrp78
Joined:: 2016-10-07
Posts:: 22
Location:: SoVA

2016-10-31 01:58

No, I do not unfortunately. The code I sent should work. You can use the GUI to build a filemask and that's what I did for you in my example. I'm not sure why it's still syncing the sessions folder. Also, */ means to not sync any subfolders so you don't want that.

ysRiYmK.png (11.44 KB)

Reply with quote

ehm01

2016-10-31 02:10

@jrp78: Where did you find this 'File Mask Edit'???

Reply with quote

jrp78
Joined:: 2016-10-07
Posts:: 22
Location:: SoVA

2016-10-31 02:17

@ehm01

GWDhBMT.png (80.59 KB)

Reply with quote

ehm01
Joined:: 2016-10-30
Posts:: 8
Location:: USA

2016-10-31 02:22

That is very odd.

I use the mask tool you point to create this:

-filemask=""*.*; *.*/|/sessions/""

and then I got error:

Too many parameters for command 'synchronize'.

maybe I have a defective version? let me download again...

Reply with quote

ehm01
Joined:: 2016-10-30
Posts:: 8
Location:: USA

2016-10-31 02:24

One more info:
If I use the "sync' from the program, and not from my script, IT WORKS!

So, what could be wrong?

Reply with quote

jrp78

2016-10-31 02:31

I'm not sure. I don't use the txt file method so I have no experience with it. I will say I don't think you need the *.* part of the filemask. I just leave it blank on mine and I have just the exclusions set. I sync tons of folders and subfolders and have no issues. I'll look through the forums. There should be an example of what you need somewhere.

Reply with quote

ehm01
Joined:: 2016-10-30
Posts:: 8
Location:: USA

2016-10-31 02:37

You were right again. If I just place the exclusion, on the GUI version (| sessions/), works exactly the way I need.
It just do not work on the Script:

synchronize local c:\MySites\Site1\ /public_html -delete -preservetime -filemask=""| sessions/""

Give me the Too Many Parameters err.

:(

Reply with quote

ehm01
Joined:: 2016-10-30
Posts:: 8
Location:: USA

2016-10-31 08:18

O GOT IT!!!!!!!!!!

synchronize local c:\MySites\Site1\ /public_html -delete -preservetime -filemask="| sessions/"

ONLY ONE quotation mark on each side.

All info on the forum always mention that you have to have two quotations!
The manual says two (double)
It's working.
Thank you for pointing the Mask File Editor. That was a key for me to find the right way.

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,506
Location:: Prague, Czechia

2016-10-31

ehm01 wrote:

All info on the forum always mention that you have to have two cotations!
The manual says two (double)

Only, if you provide commands on command line (as opposite to a script file).

Reply with quote

jrp78

2016-10-31 19:33

@ehm01: Good deal. I use command line instead of a script file which requires the double quotes so why I would not have guessed that :D

Reply with quote

Guest

2016-11-01 22:46

OK. I was to fast.
I was doing tests in a demo base, since the real one is about 1.2Gb large.
What happens with that code:

synchronize local c:\MySites\Site1\ /public_html -delete -preservetime -filemask="| sessions/"

is that if the /sessions is empty, they will not transfer.
if it's not, then it will try to read files and transfer it, even do it's marked as not to.
end then the error happens.
I found it after about 4 hours downloading the site.
Back to the testing mode... :(
Any other help people?
PS: if I use the GUI version, it works. Only on the Script it's not.

Reply with quote

ehm01
Joined:: 2016-10-30
Posts:: 8
Location:: USA

2016-11-02 00:06

Double cheked and this is real:
If I use the GUI, with this command:

*; */ | sessions/

All files and folders are copied to my PC, but the files and folders under /sessions.

If I use the Script with this command:

synchronize local c:\MySites\Site1\ /public_html -delete -preservetime -filemask="*; */ | sessions/"

All files and folder are copied to my PC, including sessions folder and files underneath.

Reply with quote

jrp78
Joined:: 2016-10-07
Posts:: 22
Location:: SoVA

2016-11-03 17:10

You could try a different format. I use a windows batch file (.bat)

Here is an example of my code inside the .bat file which works without issue. One caveat, if you copy this code, you must remove any blank spaces AFTER the ^ symbol. The ^ must be the final character on each line.

@echo off
 
"C:\Program Files (x86)\WinSCP\WinSCP.com" ^
/log="C:\scripts\WinSCP-5.9.1-Portable\log_!Y_!M_!D_!T.log" /loglevel=0 /ini=nul ^
/command ^
"open sftp://username@serverip/ -hostkey=""ssh-rsa 2048 xx:xx:xx:xx"" -privatekey=""C:\scripts\Puttygen\privkey.ppk""" ^
"option batch abort" ^
"option transfer binary" ^
"option synchdelete on" ^
"synchronize local d:\inetpub\web\ /web/ -criteria=both -filemask=""| alert.xml; emergency.xml; .htaccess; notice.xml; sitemap.xml; web.config; wincache.php; fileshare/; xml-sitemap-generator/""" ^
"close" ^
"quit" ^
"exit"

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,506
Location:: Prague, Czechia

2016-11-04

@ehm01: Your code works for me.
Please attach a full session log file showing the problem (using the latest version of WinSCP).

To generate the session log file, use /log=path_to_log_file command-line argument. Submit the log with your post as an attachment. Note that passwords and passphrases not stored in the log. You may want to remove other data you consider sensitive though, such as host names, IP addresses, account names or file names (unless they are relevant to the problem). If you do not want to post the log publicly, you can mark the attachment as private.

Reply with quote

martin◆ Site Admin

2016-11-04

jrp78 wrote:

You could try a different format. I use a windows batch file(.bat)

Using a different format does not change a meaning of the commands.

Reply with quote

You can post new topics in this forum