Forum » Scripting / Automation »

Synchronize -filemask and exclude empty directories (command line)

Advertisement

ttop
Joined:
Posts:
4
Location:
Ireland

Synchronize -filemask and exclude empty directories (command line)

Hi,
I realise the below script is not correct as I have a few queries about how to move forward.
  • I am writing a script for file transfer synchronisation
  • I wish to sync my local directory with the remote directory but I don't want all the remote directory empty folders?
  • I also only want the most recent files available from the non-empty directories of the Remote end sync's into me local end?
  • I will schedule this .bat file once complete to run every 15 minutes
  • The below synchronisation still downloads the empty directories? 
for /f "tokens=2 delims==" %%a in ('wmic OS Get localdatetime /value') do set "dt=%%a"
set "YY=%dt:~2,2%" & set "YYYY=%dt:~0,4%" & set "MM=%dt:~4,2%" & set "DD=%dt:~6,2%"
set "HH=%dt:~8,2%" & set "Min=%dt:~10,2%" & set "Sec=%dt:~12,2%"
set datestamp=%YYYY%%MM%%DD%
 
set CONN_SFTP_SVR="ftp://user:password!@10.10.10.10 hostkey=* -rawtransfersettings ExcludeEmptyDirectories=1"  
::::::I tried the '-rawtransfersettings ExcludEmptyDirectories=1' here, but I am not sure how to include this?:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
 
set LOCAL_DIR1="D:\A\B\C\D\E\"
set LOCAL_DIR2=""D:\A\B\C\D\A1\"
set LOCAL_DIR3=""D:\A\B\C\D\A2\"
set LOCAL_DIR4=""D:\A\B\C\D\A3\"
 
set REMOTE_DIR1="D:\A\B\C\D\E\"
set REMOTE_DIR2=""D:\A\B\C\D\A1\"
set REMOTE_DIR3=""D:\A\B\C\D\A2\"
set REMOTE_DIR4=""D:\A\B\C\D\A3\"
 
set file_pattern=*%datestamp%*statsfile.xml
    
"C:\Program Files (x86)\WinSCP\WinSCP.exe" /xmllog=D:\Logs\!Y!M!D_!T_LOGS.xml 
"/script=D:\LOACTION\TheCommands.txt"

###################################This is my transfer#######################
##A new file arrives on the remote server every 15 minutes and I wan to collect the most recent one. I use >15N but maybe there is a better way?########
 
open %CONN_SFTP_SVR%
option batch abort
######Example of dir 2 only##########
synchronize local -filemask=%file_pattern%>=15N %LOCAL_DIR2% %REMOTE_DIR2%
 
close

Reply with quote

Advertisement

ttop
Joined:
Posts:
4
Location:
Ireland

Synchronize -filemask and exclude empty directories (command line)

Thank you, I realise it is part of the synchronise command, but I could not get it to work that way. See below 
open %CONN_SFTP_SVR%
option batch abort
cd %REMOTE_DIR2%
lcd %LOCAL_DIR2%
synchronize -rawtransfersettings ExcludeEmptyDirectories=1 -filemask=*.xml>=15N local
I get the below error 
<?xml version="1.0" encoding="UTF-8"?>
<session xmlns="http://winscp.net/schema/session/1.0" name="tleacy@10.147.21.34 -hostkey=*" start="2020-12-11T09:48:14.646Z">
  <failure>
    <message>Unknown option 'ExcludeEmptyDirectories=1'.</message>
  </failure>
</session>
I tried this 
open %CONN_SFTP_SVR%
option batch abort
get -rawtransfersettings ExcludeEmptyDirectories=1 -filemask=*.xml>=15N %LOCAL_DIR2% %REMOTE_DIR2%
and I got the below error 
<?xml version="1.0" encoding="UTF-8"?>
<session xmlns="http://winscp.net/schema/session/1.0" name="tleacy@10.147.21.34 -hostkey=*" start="2020-12-11T11:26:16.740Z">
  <failure>
    <message>Can't get attributes of file 'ExcludeEmptyDirectories=1'.</message>
    <message>File or folder 'ExcludeEmptyDirectories=1' does not exist.</message>
  </failure>
</session>

Reply with quote

martin◆
Site Admin
martin avatar

Re: Synchronize -filemask and exclude empty directories (command line)

You are probably using an old version of WinSCP that does not support the -rawtransfersettings. Make sure you use the latest version of WinSCP. If this does not help, please post a session log file, not the XML log.

Reply with quote

Advertisement

Ashim
Guest

filemask error

Dear Martin,

I am also trying to synchronise using the WinSCP commands, but I do not succeed in synchronising the files from the server to my local drive: 
REM Step2: Connect to FTP and synchronise all the files
"C:\Users\giyash\AppData\Local\Programs\WinSCP\WinSCP.com" ^
  /log="z:\Projekte\109797-TestfeldBHV\30_Technical_execution_Confidential\TP3\AP2_Aufbau_Infrastruktur\Infrastruktur_Windmessung\02_Equipment\01_Wartung_Messmast_GE-NET_DWG_20190226\Data\UpgradeData\sync_Metmast_upgrade.log" /loglevel=-1 /logsize=5*10M /ini=nul ^
  /command ^
    "open ftp://iwes_bhv:iwes2020#@172.25.5.231:48489" ^
    "synchronize local -filemask="*>60D" "z:\Projekte\109797-TestfeldBHV\30_Technical_execution_Confidential\TP3\AP2_Aufbau_Infrastruktur\Infrastruktur_Windmessung\02_Equipment\01_Wartung_Messmast_GE-NET_DWG_20190226\Data\UpgradeData\" ""/Intenso/LoggerRaw/"" "  ^
    "exit"
If I replace the filemask with *.* it works, but synchronises the whole folder, which I want to restrict to 60 days for example.

Reply with quote

martin◆
Site Admin
martin avatar
Joined:
Posts:
41,442
Location:
Prague, Czechia

Re: filemask error

@Ashim: You have the quotes wrong. The simplest solution is to remove all intern quotes in the synchronize command, as none of them are needed: 
    "synchronize local -filemask=*>60D z:\Projekte\109797-TestfeldBHV\30_Technical_execution_Confidential\TP3\AP2_Aufbau_Infrastruktur\Infrastruktur_Windmessung\02_Equipment\01_Wartung_Messmast_GE-NET_DWG_20190226\Data\UpgradeData\ /Intenso/LoggerRaw/"  ^
See also:
https://winscp.net/eng/docs/commandline#syntax

Reply with quote

Advertisement

ashimgiyanani@yahoo.com
Joined:
Posts:
2

Filemask

Hi,

Thank you for the prompt reply.

The quotes are placed rightly as they form a part of a batch script and the lines access WinSCP from the cmd command.

I got the script to work eventually, by enclosing the filemask in additional quotes. For example 
-filemask=““*>60D““...

Hopefully, you can add this somewhere in the documentation as I couldn’t find this information anywhere

Reply with quote

martin◆
Site Admin
martin avatar

Re: Filemask

No, you do not have the quotes right. Though indeed, doubling the all internal quotes helps too. And the link I've posted in my previous response shows that! But again, the quotes are unnecessary as there are no spaces in your parameters.

Reply with quote

ashimgiyanani@yahoo.com
Joined:
Posts:
2

Re: Filemask

Hi Martin,

Right said. It works with double quotes, but is unnecessary. I am not sure why, but my batch file keeps stopping at certain junctions after winscp is implemented. Do you have any recommendations, how I can resolve the issue? 
REM syncing to z: drive
cd "c:\Program Files\OpenVPN\bin"
REM Step1: Connect to VPN
openvpn-gui --connect "mdex fixed.IP20 TCP.ovpn" --config_dir "c:\Program Files\OpenVPN\config" --silent_connection 1
 
    "open ftp://xxxx_abc:xxxxxxx@172.xx.x.xx:xxxxx" ^
    "synchronize local -filemask=*.* -neweronly "server_mirror\" ""/Raw/"" "  ^
   "close" ^
    "exit"
set WINSCP_RESULT=%ERRORLEVEL%
if %WINSCP_RESULT% equ 0 (
  echo WinSCP finished:Success
) else (
  echo Error
)
 
REM Step3- Disconnect openVPN
"c:\Program Files\OpenVPN\bin\openvpn-gui.exe" --command disconnect_all
 
REM Step 4- Copy newer files to a to_convert folder, delete 0kB files
rmdir "Path"

Reply with quote

martin◆
Site Admin
martin avatar

Re: Filemask

Sorry, no idea what you mean by "keeps stopping at certain junctions". Can you be more specific?

Though your batch file does even run WinSCP. So no surprise it does not work. There's no WinSCP.com anywhere.

Reply with quote

Advertisement

MAXOHDA
Joined:
Posts:
1

Hi 
synchronize local -transfer=binary -filemask=>1K "E:\Backup\SxHotSrv\Documents\"
it gets stuck on every file that has a size of 0.
how to solve the problem? 
A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied.
П(р)ервать, П(о)вторить, (П)ропустить, Пропустить (в)се: Пропустить
5dd1fcc9-102e-11ed-80d5-0 |         147 KB |   15,4 KB/s | binary | 100%
5dd880e1-c1da-11ec-80d4-0 |            0 B |   15,4 KB/s | binary |   0%

Reply with quote

Advertisement

You can post new topics in this forum

Documentation

Support

Associations

Follow Us