I have the answer from support PowershellServer:
Thank you for providing this information. Are you trying to use WinSCP to connect to PowerShell Server over SCP? Unfortunately, WinSCP is a special case which will not work with PowerShell Server (on any version of Windows, PowerShell, etc). The functionality for SCP that is built into PowerShell Server is strictly SCP, and WinSCP won't work with it because of some tricks it uses underneath which are not standard to SCP. There are alternatives to SCP, but they are mostly command line tools. One common one is PSCP (PuTTY's SCP client), but again, it is a command line tool.
However, I've confirmed that SFTP works from WinSCP to PowerShell Server. If you're attempting to connect over something other than SCP, let me know and we can troubleshoot the issue.
If you have any other questions or concerns, please let me know.
/n software, inc.