This issue has been added to the tracker:
https://winscp.net/tracker/971

I believe it's misunderstanding.
The selection of encoding does not re-encode the file it only allows you to choose, what encoding to use when loading the file.
https://winscp.net/eng/docs/ui_editor

Yes, problem appears when text is encoded in "1257 (ANSI-Baltic)" and is not converted to LV characters.

This happens when using "1257 (ANSI-Baltic)" encoding (encoded LV characters for example - ļ»æÄ€ A Å ā) there is "UTF-8" characters saved, but the problem is that we cannot switch between encoding without corrupting or loosing data.

So you:
1) Create file in 1257 encoding
2) Open the file in internal editor of WinSCP
3) Characters are displayed correctly (encoding is correctly detected as Ansi), is this correct?
4) You switch encoding to UTF-8 => You force WinSCP to interpret the file as, if it were encoded in UTF-8 encoding.
5) File is shown corrupted.

Did I understand right?
Then it behaves as expected. What do you find wrong about it?

Opening file into an WinSCP Internal editor with "UTF-8" encoding, which contains Latvian (LV) characters, text is not displayed.

Steps to reproduce problem:
Create file into Internal editor with "1257 (ANSI - Baltic)" encoding, fill text with Latvian charaters (for example: "Ziemassvētki Latvijā") and changing to "UTF-8" encoding text disappears and vice versa - saving file with "UTF-8" encoding, which contains Latvian characters and switching to "1257 (ANSI - Baltic)" encoding, characters are corrupted.

Problem occurs, when files whith LV characters are coppied to remote machine and edited with UTF-8 encoding enabled.

Using WinSCP Version 5.1.2 (Build 2816) on Windows 7 x64 SP1.