Topic "How to always filter some files and folders during transfer following sym links Linux -> Windows?"

Author Message
jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
I need to transfer all folders, except for one where only a subset of files and folders are needed.
I tried the following filter mask, but it does not work as expected: nothing is transferred from the exception folder.
My understanding is that the file search is not recursive in the source folder: how can the mask be modified to attain the goal?

For example, the Ubuntu/*.sh files are located within numerous subfolders.

Another issue is that the mask is not automatically set for the next transfers: we have to reload it each time.

Last edited by jcmanciot on 2016-05-31 21:56; edited 2 times in total
WinSCP transfer options.JPG (65.58 KB) [Download]

Description: WinSCP transfer options

WinSCP filter mask.JPG (57.68 KB) [Download]

Description: WinSCP transfer filter mask

Advertisements
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
This should do: |Ubuntu/*.sh;Ubuntu/*/*.sh

To set the mask permanently, set the defaults in the Preferences:
https://winscp.net/eng/docs/ui_pref_transfer
(or use Transfer Settings > Set as default on Upload/Download dialog)
jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
prikryl wrote:
This should do: |Ubuntu/*.sh;Ubuntu/*/*.sh

With the following mask ("Filter mask"), the whole subfolders match, instead of a collection of files within some of them:
To give you a glimpse of what is available in the source folder, we have:
find Ubuntu -name "build*.sh" | more
Ubuntu/Libguestfs/build-libguestfs.sh
Ubuntu/Google/Authenticator/build-google-authenticator.sh
Ubuntu/Celt-Codec/build-celt.sh
Ubuntu/Libpcap/build-libpcap.sh
Ubuntu/Git/build-git.sh
Ubuntu/Spice/GTK/build-spice-gtk-stable.sh
Ubuntu/Spice/GTK/build-spice-gtk.sh
Ubuntu/Spice/VD-Agent/build-spice-vd-agent.sh
Ubuntu/Spice/Server/build-spice-server.sh
Ubuntu/Spice/Protocol/build-spice-protocol-dev.sh
...

As you can see, some *.sh are within "subsubfolders"; do they have a chance to get on board? Wink
I believe there should be a simple "recursive" option for the files instead, like a classic "-R" in Linux.

prikryl wrote:
[/m]
To set the mask permanently, set the defaults in the Preferences:
https://winscp.net/eng/docs/ui_pref_transfer
(or use Transfer Settings > Set as default on Upload/Download dialog)

I saw the defaults for all sessions, but I could not locate the "default[/i] on Upload/Download dialog" ("Sync Checklist")
Sync Checklist.JPG (113.97 KB) [Download]

Description: (none)

Filter Mask.JPG (54.32 KB) [Download]

Description: (none)

jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
Two more issues with filtering.
1) Timestamps
There are some discrepancies between source and dest ("Date & Time Discrepancies").

2) Excluding a folder or not
Excluding "Ubuntu" folder leads to nothing to do ("Without Ubuntu Filter Mask" & "Without Ubuntu Sync Results").
However, including all folders lead to some job outside of "Ubuntu" folder ("With Ubuntu Filter Mask" & "With Ubuntu Sync Results", cf. next post).
Without Ubuntu Sync Results.JPG (16.94 KB)

Description: (none)

Without Ubuntu Sync Results.JPG

Without Ubuntu Filter Mask.JPG (39.85 KB) [Download]

Description: (none)

Date & Time Discrepancies.JPG (133.6 KB) [Download]

Description: (none)

jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
Last 2 screenshots:
With Ubuntu Sync Results.JPG (176.12 KB) [Download]

Description: (none)

With Ubuntu Filter Mask.JPG (39.42 KB) [Download]

Description: (none)

martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
Now, I'm not sure what you want to do.
Do you want exclude the .sh files or transfer only the .sh files?

For the timestamp problem, start here:
http://winscp.net/eng/docs/troubleshooting#timestamps
If that does not help, please collect the information mentioned in the article and start a new thread about this issue.

The Ubuntu directory exclude mask excludes all Ubuntu folders. So even the .\Issues\Ubuntu, not only the .\Ubuntu. If you want to exclude a specific folder only, use a full path.
See http://winscp.net/eng/docs/file_mask#path
jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
prikryl wrote:
Now, I'm not sure what you want to do.
Do you want exclude the .sh files or transfer only the .sh files?
The Ubuntu directory exclude mask excludes all Ubuntu folders. So even the .\Issues\Ubuntu, not only the .\Ubuntu. If you want to exclude a specific folder only, use a full path.
See http://winscp.net/eng/docs/file_mask#path

I'm trying to recursively transfer all *.sh which are located inside ./Ubuntu
Whether I put the source whole path from the root or the partial path from the current sync folder, the results are the same: WinSCP tries to transfer all subdirectories, instead of just *.sh located inside all subdirectories and all subsubdirectories and so on ("Whole source path" and "partial source path"). At least, that's how the results are shown on the "synchronization checklist".

prikryl wrote:
For the timestamp problem, start here:
http://winscp.net/eng/docs/troubleshooting#timestamps
If that does not help, please collect the information mentioned in the article and start a new thread about this issue.

New issue then.
current-path.jpg (80.52 KB) [Download]

Description: partial source path

whole-path.jpg (93.01 KB) [Download]

Description: Whole source path

jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
BTW, does WinSCP recognize the escape "\" as in "/home/actionmystique/Google\ Drive/Ubuntu/"?
jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
OKm I removed the escape character; however, the issue remains: WinSCP wants to sync the whole "Ubuntu" folder even when specifying the whole path: /home/actionmystique/Google Drive/Ubuntu/*/*.sh
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
It seems that you have started a new thread:
http://winscp.net/forum/viewtopic.php?t=22804
_________________
Martin Prikryl
jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
I started a new thread for the timestamp issue as you requested..
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
No WinSCP does not use the \ as an escape character. + There's no need to escape spaces. The spaces does not have any special meaning in the file mask.
See the documentation http://winscp.net/eng/docs/file_mask

Show us a log file without the \.
jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
OK.

Please note that I could not perform the transfer (I cancelled at the sync checklist) because WinSCP is not able to apply the filter mask correctly.
Since the whole "Ubuntu" folder is selected by WinSCP instead of the filter, the destination space would not have been sufficient to hold all the data.
WinSCP log.zip (353.99 KB) [Download]

Description: (none)

Transfer Settings.JPG (60.39 KB) [Download]

Description: (none)

Filter Mask.JPG (45.08 KB) [Download]

Description: (none)

martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
WinSCP updates the folder timestamp only, when it touches the folder. There's was no change in the Missings folder, so the timestamp was not updated.
jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
It should sync all folders timestamps, since the transfer type is "mirror".
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
jcmanciot wrote:
It should sync all folders timestamps, since the transfer type is "mirror".

Is that a feature request? Will see if more users ask for it.
jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
I don't get your point: mirror is mirror, isn't it?
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
I understand that you want to have the timestamps synchronized too.
But we never claimed to mirror directory timestamps. WinSCP mirrors files only, atm.
jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
I've tried the filter feature with symbolic links with the latest 5.8.3 RC (build 6701), and the issue is still there.

BTW,
    "Hekp -> Check for updates" does not show new RCs when we already have a dev version installed.
    Also, opting in "log automatically on at each visit" does not work after a reboot.
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
jcmanciot wrote:
I've tried the filter feature with symbolic links with the latest 5.8.3 RC (build 6701), and the issue is still there.

What issue do you mean specifically? I believe there was no issue discovered in this thread.

Quote:
"Hekp -> Check for updates" does not show new RCs when we already have a dev version installed.

Do you mean that you had dev version 5.8.3 installed and it did not show the official 5.8.3?

Quote:
Also, opting in "log automatically on at each visit" does not work after a reboot.

Do you mean the option on this forum?
jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
prikryl wrote:
What issue do you mean specifically? I believe there was no issue discovered in this thread.

https://winscp.net/forum/viewtopic.php?p=68865#68865; here, /Ubuntu is a Linux symbolic link.
Quote:
"Do you mean that you had dev version 5.8.3 installed and it did not show the official 5.8.3?

I had 5.8.2 RC and it did not show 5.8.3 RC
Quote:
Do you mean the option on this forum?

Yes

Also, login in when on a thread makes us land on the forum home page when the login is successful.
At last, I have changed the title of this thread in the first post to make it obvious that we are dealing with Linux sym links here. I wish it could have changed the title of all subsequent posts so that the situation is clear to everyone jumping in at any point of the thread.
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
jcmanciot wrote:
prikryl wrote:
What issue do you mean specifically? I believe there was no issue discovered in this thread.

https://winscp.net/forum/viewtopic.php?p=68865#68865; here, /Ubuntu is a Linux symbolic link.

OK, but we ended up the discussion with me stating that it's not a bug, but maybe a missing feature. And I never claimed it to be implemented. Or am I missing something?

Quote:
Quote:
"Do you mean that you had dev version 5.8.3 installed and it did not show the official 5.8.3?

I had 5.8.2 RC and it did not show 5.8.3 RC

The official 5.8.2 beta?
jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
prikryl wrote:
OK, but we ended up the discussion with me stating that it's not a bug, but maybe a missing feature. And I never claimed it to be implemented. Or am I missing something?

Yes you are mixing up 2 separate issues (I should have not written about the second one in the same thread, but I redirected it to its own separate thread https://winscp.net/forum/viewtopic.php?t=22804 afterwards):

1) filtering files during transfer from Linux symbolic links -> Windows:
Not correctly done

2) mirroring folders timestamps:
You said it is so far an unsupported feature, unless some other people ask about it.

I'm focusing only on the first issue here.

Quote:
The official 5.8.2 beta?

I received a RC following a bug report. Nevermind, this is not very important.[/list]
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
jcmanciot wrote:
prikryl wrote:
OK, but we ended up the discussion with me stating that it's not a bug, but maybe a missing feature. And I never claimed it to be implemented. Or am I missing something?

Yes you are mixing up 2 separate issues (I should have not written about the second one in the same thread, but I redirected it to its own separate thread https://winscp.net/forum/viewtopic.php?t=22804 afterwards):

1) filtering files during transfer from Linux symbolic links -> Windows:
Not correctly done

Well, you are mixing up two separate issues.

Anyway, so it did work in the dev version, I've sent you, but it does not work in the released 5.8.3?
jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
prikryl wrote:
Anyway, so it did work in the dev version, I've sent you, but it does not work in the released 5.8.3?

It never worked, neither in 5.8.2 nor in 5.8.3.
jcmanciot
[View user's profile]
Donor
Joined: 2016-02-27
Posts: 26
Location: Paris, France
The issue is still there with 5.8.4 RC.
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24512
Location: Prague, Czechia
Sorry for not responding yet. It's on my TODO list. I was busy with the release recently.
Advertisements

You can post new topics in this forum






Search Site

What is WinSCP?

It is award-winning SFTP client, SCP client, FTPS client and FTP client integrated into one software program for file transfer to FTP server or secure SFTP server. [More]

And it's free!

Donate

About donations

$9   $19   $49   $99

About donations

Recommend

WinSCP Privacy Policy

WinSCP License