Get including subfolders :: Support Forum

Guest

Get including subfolders

2012-10-04 17:38

First: thx a lot for that great tool! It works fine except a little problem I cannot manage.

I'm getting in trouble using mask and it seems that I got the exact opposite problem that other reported.

Using a shell command, I just need to download all files of some type from an FTP folder, including its subfolders.
But I can't get the files located in the subfolders.

Acc. to the doc:

Path Mask
When the mask selects files and it makes sense to select them based on directory, you can extend the mask with a path mask. You should separate the path mask from the filename mask by a slash. For example mask /home/martinp/*.txt matches all text files within the directory. To match all text files within subtree, use mask /home/martinp/*.txt; /home/martinp/*/*.txt.

I tried many configuration like this:

/command "option batch abort" "option confirm off" "option include /*/*.txt" "open ftp://myftp.com" "get -delete -resume -preservetime /data/* C:\temp\" "exit"

or that:

/command "option batch abort" "option confirm off" "option include */; *.txt" "open ftp://myftp.com" "get -delete -resume -preservetime /data/* C:\temp\" "exit"

Or without using the mask:

/command "option batch abort" "option confirm off" "open ftp://myftp.com" "get /data/*/*.txt C:\temp\" "exit"

But I cannot get it working.

What I'm I doing wrong?
Thanks in advance

Reply with quote

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,454
Location:: Prague, Czechia

Re: Get including subfolders

2012-10-08

What version of WinSCP are you using? Make sure, you have the latest one.

Use just simple mask *.txt.
Also note that you should use -filemask switch instead of the option include (although it is still supported for backwards compatibility).

get -delete -resume -preservetime -filemask="*.txt" /data/* C:\temp\

If it does not help, please attach a complete log file.

Reply with quote

plubarj2 Guest

2012-10-08 09:16

Thank you @martin
Apparently, the version I was running 4.3.9 doesn't like the filemask switch:

Unknown switch 'filemask'

I installed the 5.1 and it works great!

Thanks once more for the project and your support. I will donate.

Reply with quote

Hushham Guest

Getting specific file extension from all subfolders

2019-01-08 12:49

Hi Martin,
You have explained everything wonderfully but I am having a trouble in getting specific file from all subfolders.
I am using 5.13.6 & I am using below command to get csv file from the listed folder & its subfolder but it only gets file from the specified folder.

get -delete *.csv C:\Users\hash\Desktop\BI\

I have checked all changes like */*.csv etc.
I have also checked

get -filemask="*.csv" -delete *.*  C:\Users\hash\Desktop\BI\

but of no use.
I want to take all .csv files from folder/subfolder & delete it later on.
Thanks

Reply with quote

Hushham
Joined:: 2019-01-09
Posts:: 7
Location:: Pakistan

2019-01-09 10:38

Just to tell you, I have used all below

get -delete "*.csv" C:\Users\hash\Desktop\BI\

result = No file matching '*.csv' found

get -delete "*\*.csv" C:\Users\hash\Desktop\BI\

result = No file matching '*\*.csv' found

get -filemask="*.csv" -delete *.* C:\Users\hash\Desktop\BI\

result = it transferred the complete folder including subfolder.

Reply with quote E-mail

Hushham
Joined:: 2019-01-09
Posts:: 7
Location:: Pakistan

2019-01-09 10:51

Referring to your below post, I have tried this as well but it didn't work as well:
Get files from subfolders
I tried

get * C:\Users\hash\Desktop\BI\ -filemask="*.csv"

result = it copied all complete folder again.

Reply with quote E-mail

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,454
Location:: Prague, Czechia

2019-01-10

Hushham wrote:
get -filemask="*.csv" -delete *.* C:\Users\hash\Desktop\BI\
result = it transferred the complete folder including subfolder.

This is correct. If it does not work for you, please attach a full session log file showing the problem (using the latest version of WinSCP).

To generate the session log file, use /log=path_to_log_file command-line argument. Submit the log with your post as an attachment. Note that passwords and passphrases not stored in the log. You may want to remove other data you consider sensitive though, such as host names, IP addresses, account names or file names (unless they are relevant to the problem). If you do not want to post the log publicly, you can mark the attachment as private.

Reply with quote

Hushham
Joined:: 2019-01-09
Posts:: 7
Location:: Pakistan

2019-01-10 12:59

Dear Martin,

Thanks for the reply, the log of the below command is attached privately to you. Please check.
Command used is as below. It copied the folder named 1 & 2 as well.

get -filemask="*.csv" -delete *.* C:\Users\hash\Desktop\BI\

Thanks

logfile.log (14.44 KB, Private file)

Reply with quote E-mail

Hushham

2019-01-10 13:01

I forgot to tell you, I used the most latest version 5.13.7

Reply with quote E-mail

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,454
Location:: Prague, Czechia

2019-01-15

Hushham wrote:

It copied the folder named 1 & 2 as well.

Sorry, I got lost.
This is what you wanted, isn't it?

Previously your wrote:

I am using 5.13.6 & I am using below command to get csv file from the listed folder & its subfolder but it only gets file from the specified folder.

What sounds like the files in subfolders were not downloaded, but you wanted them to. So it seems like now you have what you wanted.

Reply with quote

Hushham
Joined:: 2019-01-09
Posts:: 7
Location:: Pakistan

2019-01-15 11:39

Dear Martin,

Thanks for the reply, actually I want the files only and not the folders.
Is there a command who can pick '.csv' files only and not the subfolder?

right now I am getting the subfolders as well.

Reply with quote E-mail

martin◆ Site Admin
Joined:: 2002-12-10
Posts:: 41,454
Location:: Prague, Czechia

2019-01-18

An easy way is to select only *.csv files for the transfer (this assumes you subfolder names don't have .csv extension):

get -delete *.csv C:\Users\hash\Desktop\BI\

Or you can explicitly exclude subfolders:

get -filemask="*.csv|*\" -delete *.* C:\Users\hash\Desktop\BI\

See How do I transfer (or synchronize) directory non-recursively?

Reply with quote

jwatson151 Guest

trying to move files within subfolders without taking the subfolders

2024-10-11 22:45

Trying to get PDF's only, from unknown folders being created and placed on the FTP. I was successful using -filemask but it transfers the folders as well which I do not want. I am currently trying this syntax but it does nothing and returns no errors. Your help is greatly appreciated.

cd /Test/Exports/
mv * /Staging/
cd /Staging/
get -filemask="*.pdf|*\"  *.* "X:\Workflow\Scans\Documents\Generic"
cd /Staging/
rm *.*
exit

attempted these others as well:
I get no pdf's found error on this one...

cd /Test/Exports/
mv * /Staging/
cd /Staging/
lcd \\1.2.3.4\Data\Workflow\Scans\Documents\Generic
mget */*.pdf
cd /Staging/
rm *.*
exit

**this one executes but no files move and errors are generated.

cd /Test/Exports/
mv * /Staging/
cd /Staging/
get *.pdf "X:\Workflow\Scans\Documents\Generic"
cd /Staging/
rm *.*
exit

**this one executes but no files move and errors are generated.

cd /Test/Exports/
mv * /Staging/
cd /Staging/
get -filemask="*.pdf|*\" -delete *.* 
exit

Reply with quote

martin◆ Site Admin

Re: trying to move files within subfolders without taking the subfolders

2024-10-14

@jwatson151: Your task description is rather vague. Is this what you are after?
Downloading all files from FTP/SFTP to the same local folder
If not, please give us more detail/examples.

Reply with quote

jwatson151 Guest

trying to move files within subfolders without taking the subfolders

2024-10-14 21:50

@Martin. Thanks. I generally stay away from .NET assembly..as it may leave a process running at the database level. My task is to import pdf files. They are being dropped in a /test folder. File A.pdf is within /test/A, file b.pdf is in /test/B and so on. So I'm trying to get the pdf's from within the unknown folders that will be dropped in /test then delete. In order not to get dup's or delete files I have not received yet, I am using a staging folder to move the folders within /test to then move the pdfs out of and then delete the folders and files.

Reply with quote

martin◆ Site Admin

Re: trying to move files within subfolders without taking the subfolders

2024-10-17

@jwatson151: I'm not aware of any such problems with .NET assembly. Are you?
Anyway, the link I've posted shows a solution using plain scripting and a batch file.

Reply with quote