Unicode text file downloads

I encountered this weird problem of which I am not sure why and how it happens, if it is a bug or just due to undefined behaviour as WinSCP is not a Unicode application.

I have uploaded plain text files encoded in UTF-8 via standard FTP to my webserver. When I retrieved them again, I discovered that the local copies were messed up where there should be non-Unicode characters. Instead of the actual characters, there are a lot of seemingly random (but also Unicode) characters.

I figured that this has to do with Text/Binary upload modes, and that maybe my uploaded files were definitely messed up as I uploaded them in the default Text mode for .txt files, when I should have chosen Binary for the Unicode files. However, I have since figured out through testing that it does not matter if the files are uploaded in text or binary mode, but how they are downloaded. If I download them again in Text mode, they are messed up. If I download them again in Binary mode, the files are fine.

Is there a reasonable explanation behind this behaviour?

Can you send me an email, so I can send you back a debug version of WinSCP to track the problem? Please include link back to this topic in your email. Also note in this topic that you have sent the email. Thanks.

E-mail is sent :)

I've sent you a debug version.

We had a similar issue. When we copied UTF-16 xml files via standard mode the files get messed up.

The problem is that the linefeed (hex 00 0a) will be changed to carriage return, linefeed (hex 00 0d 0a). The text would be readable if it would change it to: hex 00 0d 00 0a

TomB wrote:

We had a similar issue. When we copied UTF-16 xml files via standard mode the files get messed up.

The problem is that the linefeed (hex 00 0a) will be changed to carriage return, linefeed (hex 00 0d 0a). The text would be readable if it would change it to: hex 00 0d 00 0a

Thanks for your post. This issue is being tracked already.