Topic "Preserve line ending (LF vs. CR/LF) when editing"

Author Message
dandvd
[View user's profile]

Joined: 2009-12-02
Posts: 15
Location: Silicon Valley
I'm not sure how this bug hasn't been observed yet:

When the user saves a file they are editing in WinSCP, the line ending type is forced to CR/LF. Since the vast majority of files on Linux systems use LF only, this leads to all sorts of errors.

WinSCP should either detect and preserve the line ending type (best option), or (easy) force it to line feed only, which is supported on Windows by most applications, except toy ones like Notepad.
Advertisements
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
This would happen only if you have configured WinSCP to use Windows line endings in session options.
_________________
Martin Prikryl
dandvd
[View user's profile]

Joined: 2009-12-02
Posts: 15
Location: Silicon Valley
I forgot to mention: the file I was editing is on the local (Windows) machine. When editing remote files, LF is preserved.

I checked Environment -> "End-of-line characters (if not indicated by server)" and the setting was "LF".
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
OK then. The internal editor of WinSCP does not preserve line endings. You can use any external editor you like instead.
_________________
Martin Prikryl
dandvd
[View user's profile]

Joined: 2009-12-02
Posts: 15
Location: Silicon Valley
prikryl wrote:
OK then. The internal editor of WinSCP does not preserve line endings. You can use any external editor you like instead.


It would be really nice if the internal editor preserved line endings, or at least if it warned that it doesn't. Right now, users who assume that it does, might have odd surprises when they upload the file and line endings are corrupted.
Brian_nairB

Guest


prikryl wrote:
OK then. The internal editor of WinSCP does not preserve line endings. You can use any external editor you like instead.


Hello, I am using an editor that doesn't convert the line endings, and I'm still having a similar issue. I have a file in UNIX that originated in Windows, so we expect each line to end with CRLF. However, we have found in some cases, the data contains a CR, so the end of the line actually looks like CRCRLF. With my transfer mode set to Binary, when I download the file, it preserves the CRCRLF, and that's what I see in my editor. Then if I change something in the file, and save it, the end of line in UNIX after upload is just CRLF. I verified in the temp\scp### directory, the updated file still has the CRCRLF, so my editor is not changing the end of line. Is it possible that only the download is in binary mode, but the upload is occurring in ASCII mode?
sample_CRCRLF.txt (54 Bytes) [Download]

Description: this is a UTF-8 file that ends with CRCRLF. Depending on how you view it, it might appears as two lines.

martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
Brian_nairB wrote:
Is it possible that only the download is in binary mode, but the upload is occurring in ASCII mode?

It shouldn't be the case. But please post a full session log file to verify.
_________________
Martin Prikryl
clfischer123@...

Guest


If CRLF is still an issue after transferring, there are several UNIX commands to remove the carriage returns on the UNIX side - especially in KSH scripts.

Add this alias to your .profile:

alias remove_cr="perl -p -i -e 's/\r\n$/\n/g'"
### remove any carriage control at end of lines ###

then at a prompt > remove_cr bld_score.ksh
to remove cr's.

To verify, check the size before and after.

Curt
Nobilis

Guest


Hi,

This a little old I know but I've been going through a similar problem editing Unix files in notepad++ through WinSCP without those being converted to a Windows EOL format.

Every time I opened I a file from WinSCP with Np++ (not transferring it but opening it directly) I found that it had silently added CRs even though I absolutely did not want this. However the transfer was working fine.

The key is to un-tick an option in the Editors tab that reads 'Force text file mode for files edited in external editor' which can be found when you click 'Edit' on your external text editor. Hope it helps others Smile
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
Nobilis wrote:
Every time I opened I a file from WinSCP with Np++ (not transferring it but opening it directly) I found that it had silently added CRs even though I absolutely did not want this. However the transfer was working fine.

What protocol are you using?
_________________
Martin Prikryl
sameer

Guest


I have a text file in windows that I am transferring to Linux using batch script. I have transfer option binary set but still a empty newline is added at the end of the file. Not sure how to deal with this?
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
sameer wrote:
I have a text file in windows that I am transferring to Linux using batch script. I have transfer option binary set but still a empty newline is added at the end of the file. Not sure how to deal with this?

So is the uploaded file on the server larger than the source file? Do you have the same problem in GUI? Did you try other FTP client (even GUI one)?
Please attach a full log file showing the problem (using the latest version of WinSCP).

To generate log file, use /log=path_to_log_file command-line argument. Submit the log with your post as an attachment. Note that passwords and passphrases not stored in the log. You may want to remove other data you consider sensitive though, such as host names, IP addresses, account names or file names (unless they are relevant to the problem). If you do not want to post the log publicly, you can mark the attachment as private.
TeejMonster

Guest


Just chiming in to say that this solved my issue. I used to have trouble with files opened from WinSCP in an external editor (notepad2, sublimetext3) having the line endings double-up. Never would have checked this setting.

Nobilis wrote:
Hi,

This a little old I know but I've been going through a similar problem editing Unix files in notepad++ through WinSCP without those being converted to a Windows EOL format.

Every time I opened I a file from WinSCP with Np++ (not transferring it but opening it directly) I found that it had silently added CRs even though I absolutely did not want this. However the transfer was working fine.

The key is to un-tick an option in the Editors tab that reads 'Force text file mode for files edited in external editor' which can be found when you click 'Edit' on your external text editor. Hope it helps others Smile
now happy!!!!!!

Guest


Yes!!!!! this solves the problem - was causing havoc with files esp. php
Perhaps the default should be un-checked - this setting is very hard to find!!

Nobilis wrote:

The key is to un-tick an option in the Editors tab that reads 'Force text file mode for files edited in external editor' which can be found when you click 'Edit' on your external text editor. Hope it helps others Smile
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
This issue has been added to the tracker:
http://winscp.net/tracker/show_bug.cgi?id=1239
BackuPs

Guest


Hi

i installed 5.6.3 and i am still having the issue. even if i uncheck the option in the Editors tab that reads 'Force text file mode for files edited in external editor'

When notepad++ opens the file all lf are crlf

I have to select binary transfer mode in order to get this right.

The moment i transfer text mode i have the crlf again.

How to do a textmode transfer but not have the conversion?

Thnx
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
BackuPs wrote:
How to do a textmode transfer but not have the conversion?

The purpose of the text mode transfer is to have files converted. If you do not want to have it converted, use binary mode.
tazosmr
[View user's profile]

Joined: 2014-07-09
Posts: 2
===My Only SOLUTIONS===
(other solutions: http://superuser.com/questions/165539/winscp-screws-up-line-breaks-in-files-how-do-avoid/ )

-in "Preferences>Transer>Default" choose "Automatic(Transfer Mode)";
-Then in "Text" window, set such conditions (but i havent tested them well):
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
See Why are text file line breaks wrong after the file is transferred or edited?
DevarB

Guest


Why not make this a simple option in the config?

"[ ] Convert line endings for target system"

As it is now, it's very different from the norm and was a huge surprise when I discovered it.
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24555
Location: Prague, Czechia
Can you show me an example of software that uses this "norm"? Any FTP software I know has a text/binary switch.
Advertisements

You can post new topics in this forum






Search Site

What is WinSCP?

It is award-winning SFTP client, SCP client, FTPS client and FTP client integrated into one software program for file transfer to FTP server or secure SFTP server. [More]

And it's free!

Donate

About donations

$9   $19   $49   $99

About donations

Recommend

WinSCP Privacy Policy

WinSCP License