File transfer creates 3 lead characters

Advertisement

appleton98
Joined:
Posts:
6
Location:
Nottingham

File transfer creates 3 lead characters

On revision 5.1 build 3799 this did not occur but on version 5.5.1 build 3970 it does and its frustrating.
We copy a file that Business Objects creates from one server to a SFTP server in ASCII and all appears fine. If you then open the text file within WINSCP and select ENCODING from the menu and change it from UTF-8 to 1252 ANSI latin 1 the file suddenly has these three symbols at the start of the file. 
Files transferred in the old version do not have this issue.

Unfortunately due to the nature of the file content i can not post it on here.

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
28,542
Location:
Prague, Czechia

Re: File transfer creates 3 lead characters

The three characters are UTF-8 BOM mark:
https://en.wikipedia.org/wiki/Byte_order_mark#UTF-8

I suppose the file has the BOM included on its own. When you open the file in WinSCP internal editor, it autodetects the UTF-8 encoding by its presence (and does not show the mark as it's not part of a contents). If you force WinSCP to interpret the file in ANSI encoding, it displays these characters as if they were part of the contents. But this has not changed between 5.1 and 5.5.1. Can you please test this once again using 5.1?

If the above does not explain what you experience, I have further questions:

What encoding do you have configured as default in Preferences?
https://winscp.net/eng/docs/ui_pref_editor

Am I right assuming you are using WinSCP internal editor, right?

What protocol? FTP or SFTP?

Is the BOM present in the file before upload? (the file is 3 bytes longer than its contents if BOM is included).

Reply with quote

appleton98
Joined:
Posts:
6
Location:
Nottingham

Re: File transfer creates 3 lead characters

Having checked the original file before WINSCP sees it appears it does indeed have the BOM included on its own. To do this i had to use the internal editor of WINSCP from version 5.5.1
I am using SFTP protocol in both cases however i had made a mistake in the revisions. The new version is indeed 5.5.1 however the old version is not 5.1 but is in fact version 4.3.5 build 1463, appologies
If i transfer the same file from the same location using version 4.3.5 and then open it with the internal editior of 5.5.1 then the three characters do not appear when the encoding is changed between UTF-8 to 1252 ANSI latin 1 and back again. However do exactly the same with version 5.5.1 and they do.

I have enclosed two images from both versions showing the Editor page.
In version 5.5.1 i have tried the transfer with both 1252 ANSI chosen as default and then UTF-8 with the same result.

Is there anything else you could think of i could try?
We upgraded from version 4.3.5 to 5.5.1 in order to fix an authentication problem and the new verion allowed us to use the switch -hostkey="*" after the login process to solve that.

editors 5.5.1.jpg

editors 4.3.5 1463.JPG

Reply with quote

appleton98
Joined:
Posts:
6
Location:
Nottingham

Re: File transfer creates 3 lead characters

I had previously set winscp to remove the BOM remarks but to no avail. The system basically picks up a file from a server and places it into a folder on a sftp server. IS there something i'm doing wrong?

transfer settings 5.5.1.JPG

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
28,542
Location:
Prague, Czechia

Re: File transfer creates 3 lead characters

It's difficult to debug this remotely.
So when you have the BOM removing enabled, does the BOM disappear from the file on upload or not?

Also, what is the actual problem? Why are you even trying to change the encoding in WinSCP from UTF-8 to Ansi, when the file is (probably) in UTF-8 (providing it does have UTF-8 BOM)?
_________________
Martin Prikryl

Reply with quote

appleton98
Joined:
Posts:
6
Location:
Nottingham

Hello,
With remove BOM enables it does not appear to remove the BOM characters from the file.
I am trying to change the encoding from UTF-8 to Ansi to see whether the three control characters have been removed.
The customer who i am trying to help requires that the characters be removed however uploading their exported file is automated and it sees the three characters and rejects the file accordingly. I have changed the encoding on files from version 4.3.5 and they do not have this issue. Currently we have moved this customer to our old server running 4.3.5 but this is not realistically a permanent solution.

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
28,542
Location:
Prague, Czechia

Please attach a full log file showing upload of a file using 5.5.2 with BOM removing enabled (but not happening).
_________________
Martin Prikryl

Reply with quote

appleton98
Joined:
Posts:
6
Location:
Nottingham

scripting question

Hi,
unfortunately this issue was resolved but has occurred once again. I have included a screen shot of the transfer settings. When i manually copy one file from the source to the SFTP server from within WINSCP GUI interface these settings are utilised and i am happy, however when i use a script it ignores these settings and the file contains the three characters once again (as the original also does) Is there a better way to transfer files though a script that will force it to use the settings listed? I have not only added a new settings config but also changed all the others that come with WINSCP to match thinking it must use one of them, but it appears not.
My current transfer script is below

open f***r:q******r@sftp**.********.co.uk
cd /incoming/BO
put -delete "C:\SFTP\*****\Business Objects\*.*"

due to the nature of the company i work for i have had to * out some information.

Is there a way to force the sript to use the transfer settings or should i be using a different command to complete file transfer?
Many thanks

Description: screen shot showing transfer settings

winscp transfer settings.JPG

Reply with quote

Advertisement

Advertisement

You can post new topics in this forum