Forum » Support and Bug Reports »

BUG REPORT: Duplicate option copies directories incorrectly [does not overwrite existing directory]

Advertisement

NewUser1
Joined:
Posts:
10
Location:
Europe/London

BUG REPORT: Duplicate option copies directories incorrectly [does not overwrite existing directory]

Example of error:
WinSCP on Windows 7 SP1 (all latest Updates), WinSCP version 5.13.2 (Build 8455).
2 directories '/mnt/usb-data/entware/lib/' & '/mnt/usb-data/entware/[BACKUP]-lib/'

Want to copy contents of '/mnt/usb-data/entware/lib/' to '/mnt/usb-data/entware/[BACKUP]-lib/'.
Target directory already has files & directory structure from previous duplicate action.
When the files/dirs are duplicated the files copy to the correct place and overwrite the existing files but the directory tree ('opkg/' in this instance) is copied within the existing 'opkg' directory

[See Screen captures]

Steps to recreate:
Navigate Directory tree to '/mnt/usb-data/entware/lib/' and select all files/Directories using left click & drag down.
Right click on selection and click 'Duplicate' [Same as Shift-F5]
Set Target Remote Path: to '/mnt/usb-data/entware/[BACKUP]-lib/'
Click OK.
Files/Dirs are copied to 'target remote path:'.
When the '/mnt/usb-data/entware/[BACKUP]-lib/' tree is selected/refreshed the 'opkg' directory contains the new copy of the 'opkg' directory instead of being overwritten.
All other files are overwritten as expected.

Description: Before / Duplicate command / After performing Duplicate command / WinSCP Version

Screen Captures [Duplicate Bug].png

Reply with quote

Advertisement

NewUser1
Joined:
Posts:
10
Location:
Europe/London

Please could this be moved to userid 'NewUser1'

For some reason I could not post this on my new account 'NewUser1'!!!???

I posted it as 'NewUser2', if possible could you change the ID so I get the replies sent to the correct user.

NewUser2 does not exist as a account.

Thanks.

Reply with quote

martin◆
Site Admin
martin avatar
Joined:
Posts:
41,552
Location:
Prague, Czechia

Re: BUG REPORT: Duplicate option copies directories incorrectly [does not overwrite existing directory]

Thanks for your report. What protocol are you using?

Please attach a full session log file showing the problem (using the latest version of WinSCP).

To generate the session log file, enable logging, log in to your server and do the operation and only the operation that causes the error. Submit the log with your post as an attachment. Note that passwords and passphrases not stored in the log. You may want to remove other data you consider sensitive though, such as host names, IP addresses, account names or file names (unless they are relevant to the problem). If you do not want to post the log publicly, you can mark the attachment as private.

Reply with quote

NewUser1
Joined:
Posts:
10
Location:
Europe/London

Session Log for Error.

Protocol info as follows from top of log file: 
 2018-05-25 19:10:39.929 Tunnel: No
. 2018-05-25 19:10:39.929 Transfer Protocol: SCP
. 2018-05-25 19:10:39.929 Ping type: Null, Ping interval: 20 sec; Timeout: 30 sec
. 2018-05-25 19:10:39.929 Disable Nagle: No
. 2018-05-25 19:10:39.929 Proxy: None
. 2018-05-25 19:10:39.929 Send buffer: 262144
. 2018-05-25 19:10:39.929 SSH protocol version: 2; Compression: Yes
. 2018-05-25 19:10:39.929 Bypass authentication: No
. 2018-05-25 19:10:39.929 Try agent: Yes; Agent forwarding: No; TIS/CryptoCard: No; KI: Yes; GSSAPI: Yes
. 2018-05-25 19:10:39.929 GSSAPI: Forwarding: Yes; Libs: gssapi32,sspi,custom; Custom: 
. 2018-05-25 19:10:39.929 Ciphers: aes,blowfish,3des,chacha20,WARN,arcfour,des; Ssh2DES: No
. 2018-05-25 19:10:39.929 KEX: ecdh,dh-gex-sha1,dh-group14-sha1,dh-group1-sha1,rsa,WARN
. 2018-05-25 19:10:39.929 SSH Bugs: Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto
. 2018-05-25 19:10:39.929 Simple channel: Yes
. 2018-05-25 19:10:39.929 Return code variable: Autodetect; Lookup user groups: Auto
. 2018-05-25 19:10:39.929 Shell: default
. 2018-05-25 19:10:39.929 EOL: LF, UTF: Auto
. 2018-05-25 19:10:39.929 Clear aliases: Yes, Unset nat.vars: Yes, Resolve symlinks: Yes; Follow directory symlinks: Yes
. 2018-05-25 19:10:39.929 LS: ls -la, Ign LS warn: Yes, Scp1 Comp: No
. 2018-05-25 19:10:39.929 Local directory: C:\Downloads, Remote directory: /jffs, Update: Yes, Cache: Yes
. 2018-05-25 19:10:39.929 Cache directory changes: Yes, Permanent: Yes
. 2018-05-25 19:10:39.929 Recycle bin: Delete to: Yes, Overwritten to: No, Bin path: /tmp
. 2018-05-25 19:10:39.929 DST mode: Unix;Timezone offset: 0h 0m

The copy of 'opkg' starts at line 8629
I marked it with:
##################################################
# STARTS COPYING 'opkg' HERE ====>
##################################################

Attached .zip file marked as 'Private File'.
Many thanks for your help.
  • Asus-Router-(sftp)[duplicate].zip (136.15 KB, Private file)
Description: Removed some data such as passwords IP Addresses etc

Reply with quote

Advertisement

martin◆
Site Admin
martin avatar

Re: Session Log for Error.

Sorry for my late answer. Though I cannot reproduce the problem on a standard Linux server with a bash shell. What is your server like? What shell are you using?

Reply with quote

NewUser1
Joined:
Posts:
10
Location:
Europe/London

Re: Session Log for Error.

martin wrote:

Sorry for my late answer. Though I cannot reproduce the problem on a standard Linux server with a bash shell. What is your server like? What shell are you using?
Thanks for your response.
It is an ASUS Router RT-AC56U running an ash shell (Busybox).
[ash & sh are the same 'busybox' binary]

uname -a gives:
'Linux RT-AC56U 2.6.36.4brcmarm #1 SMP PREEMPT Sat May 12 22:03:01 EDT 2018 armv7l ASUSWRT-Merlin'

I would expect ash/sh & bash to work the same way for a file copy !!!

To eliminate ash vs bash:
Installed bash version [GNU bash, version 4.4.18(1)-release-(arm-openwrt-linux-gnu)]
Set the shell to '/mnt/usb-data/entware/bin/bash' in WinSCP.
Tried same steps and the problem repeated itself exactly the same result.

Tried running 'cp' command to replicate steps.
Ran the following multiple times under the 'bash' & 'ash' shell:

cp -r /mnt/usb-data/entware/lib/* /mnt/usb-data/entware/[Backup]-lib/

Files & directories copied as expected .... overwriting the previous files & directories.
Problem looks to me like it is something that WinSCP is doing, unless I am missing something !!!???

Additional Note:
Tried older version of WinSCP Ver 5.9.5 and it also has the same problem.

Reply with quote

martin◆
Site Admin
martin avatar

Re: Session Log for Error.

WinSCP does this:
cp -p -r -f -T "opkg" "/mnt/usb-data/entware/[BACKUP]-lib/opkg"
Can you test it?

Reply with quote

NewUser1
Joined:
Posts:
10
Location:
Europe/London

Re: Session Log for Error.

WinSCP does this:
cp -p -r -f -T "opkg" "/mnt/usb-data/entware/[BACKUP]-lib/opkg"
Can you test it?
The cp command on the Router does not have a -T flag.
Could that be the problem ?

Without the -T flag I get: 
User@56U:/tmp/mnt/usb-data/entware/lib# cp -p -r -f "opkg" "/mnt/usb-data/entware/[Backup\]-lib/opkg"
cp: can't create directory '/mnt/usb-data/entware/[Backup\]-lib/opkg': No such file or directory

Googled Man Page for cp and got: 
-T, --no-target-directory
    treat DEST as a normal file
Sorry, not sure what the -T flag is supposed to do when destination is a Directory !!!???

Reply with quote

Advertisement

NewUser1
Joined:
Posts:
10
Location:
Europe/London

Solved the problem with a 'Workaround'

I took the hint from the 'missing flag on cp' and installed just 'cp' from the coreutils package.
[opkg install coreutils-cp] <--- Installation command for Routers Firmware.

I now have a cp command that has all the flags and the 'Duplicate problem' is solved.
Files copy as they should.

Interesting that 'cp -r' using the original 'Busybox' cp works BUT with all the flags (while missing -T) did not !!!???

As it is not WinSCP that is the problem, probably need to include a note that non-GNU or 'busybox' derived commands may be incompatible with WinSCP.

Many thanks for your time & help in solving this.

Reply with quote

NewUser1
Joined:
Posts:
10
Location:
Europe/London

Alternative Solution

Problem has been replicated & confirmed by another WinSCP user on an ASUS Router running the same firmware.*
Found alternative solution that requires NO new s/w or install.

*https://www.snbforums.com/threads/fyi-winscp-may-work-incorrectly-when-using-the-standard-busybox-cp.47311/#post-412881

If you tick the "Duplicate via local temporary copy" box the problem doesn't occur.
Seems odd that the temporary copy makes a difference !!!???
Can only assume that the 'Temporary copy' method uses a different 'cp' commandline !!!???

There is now a simple solution that you may need to document if others have the problem.
As you will know Busybox is used in a lot of kit, so problem may be more common than it initially appears. i.e. they will all have cp without '-T flag'.
Apparently, it is also missing from BSD's native 'cp' as well.
Also I have beeen advised that it is not posix compliant, ref Opengroup base spec here:
cp -T (no target directory) is part of GNU's coreutils package, but the -T flag itself is outside of posix....
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/cp.html#tag_20_24


Problem Solved via 2 methods ...... thanks again for you help.

Reply with quote

Advertisement

You can post new topics in this forum

Documentation

Support

Associations

Follow Us