SCP and auto detection of UTF-8 encoding for filenames

Advertisement

bjmi
Joined:
Posts:
2

SCP and auto detection of UTF-8 encoding for filenames

I haven't encounting problems with directory and filename encoding so far.
But recently it was necessary to change UTF-8 encoding for filenames to "On" instead of the default "Auto" to retain right filenames while copying files to SSH server running Unraid OS using SCP method.
Filenames result in Gebühr or Beitrittserklärung but should be Gebühr and Beitrittserklärung.

According to
With SCP protocol this means that UTF-8 will be used when a LANG environment variable is set to “UTF-8” encoding.
https://winscp.net/eng/docs/ui_login_environment#utf
utf8 should be detected.
LANG is set to en_US.UTF-8.
root@unraid:~# locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE=C
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

root@unraid:~# printenv LANG
en_US.UTF-8
Any ideas why utf8 auto detection doesn't work or do other checks exist that aren't mentioned in FAQ?
Description: Debug 1 log to Unraid OS

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
41,276
Location:
Prague, Czechia

Re: SCP and auto detection of UTF-8 encoding for filenames

Your LANG variable is probably set for interactive sessions only. You need to fix your startup scripts to set it for all (even non-interactive) sessions.

Typically you need to move the relevant commands from .bash_profile to .bashrc.

This is actually the opposite of this problem:
https://winscp.net/eng/docs/message_large_packet

Reply with quote

bjmi
Joined:
Posts:
2

I tried following locations
/root/.bash_profile
/root/.bashrc
/etc/environment
/root/.ssh/rc
but without success. It doesn't detect UTF-8 on Unraid 6.8.2 (Linux 4.19.98-Unraid).

I end up with the global setting to On in winscp.ini
[Sessions\Default%20Settings]
Utf=1

Reply with quote

martin
Site Admin
martin avatar

Well, the point is that in WinSCP session the variable is not set. So it's not a problem on WinSCP side. I do not know your system, nor I'm a *nix expert, so I cannot advice you how to properly configure your system to set the variable for all sessions. Sorry.

Reply with quote

tomftp
Guest

FTP detection of UTF8

Here https://winscp.net/eng/docs/ui_login_environment#utf it is stated that "With FTP protocol this means that UTF-8 will be used when the server announces support for it."

But on our VSFTPD Linux server the WinSCP app does not detect UTF8 from this FTP server when set to Auto. Everyone needs to remember to change the Environment option to On.

What is expected from the server by WinSCP to detect it? Or perhaps WinSCP detection mechanism from FTP is not good enough solution and WinSCP should be changed?

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
41,276
Location:
Prague, Czechia

Re: FTP detection of UTF8

tomftp wrote:

What is expected from the server by WinSCP to detect it? Or perhaps WinSCP detection mechanism from FTP is not good enough solution and WinSCP should be changed?
WinSCP looks for UTF8 in the FEAT response. What the latest version of vsftpd does.

If it does not work for your, please attach a full session log file showing the problem (using the latest version of WinSCP).

To generate the session log file, enable logging. Submit the log with your post as an attachment. Note that passwords and passphrases not stored in the log. You may want to remove other data you consider sensitive though, such as host names, IP addresses, account names or file names (unless they are relevant to the problem). If you do not want to post the log publicly, you can mark the attachment as private.

Reply with quote

Advertisement

You can post new topics in this forum