Choice of protocol (SFTP/SCP vs FTP) changes case (upper/lower) of ExecuteCommand string...

Hi,

I've written a Visual Studio 2013 C# program that uses the WinSCP NET assembly (version 5.9.1) to log into an AIX Unix server, execute a remote korn shell and then downloads a file (it does more than that, but for the sake of this issue I've run into I didn't want to complicate the post).

I had initially used the SFTP protocol and every step worked as expected with no errors, but I wasn't happy with the download speed (and even if that isn't the particular issue I will describe below, I'm wondering if I can do anything about that).

I decided to give both SCP and FTP a try just to see if I could improve on the download speed.

However, for some unknown reason, when I use FTP, part of my ExecuteCommand string (a C# string variable) seems to get automatically converted from lowercase to uppercase (at least that's how the error message shows it).

Original string (works with protocols SFTP and SCP):

"$CCCS_EXE/ccc85000.ksh qc 99009 ic20140130-1.0.5.5 N/A N/A"

After a change to use the protocol FTP the string seems to become:

"$CCCS_EXE/CCC85000.KSH qc 99009 ic20140130-1.0.5.5 N/A N/A"

It's converting the name of the korn shell to uppercase (very odd behaviour since it's only a certain portion of the string) and in Unix this is obviously an issue as it will not find said korn shell (filenames are case sensitive).

I get the following exception error message:

'$CCCS_EXE/CCC85000.KSH qc 99009 ic20140130-1.0.5.5 N/A N/A' : command not understood.

Again, if I change nothing in the code except for the protocol it works with SFTP and SCP, but fails if it's FTP, in as far as the ExecuteCommmand goes.

Just to add, if I am running the program in the Visual Studio 2013 environment, if I set the protocol to FTP and I purposely skip the ExecuteCommand that would otherwise make the program fail, the file does transfer via FTP successfully, not only that, but it is also about 4 times faster than SFTP or SCP.

Bottom line, I really need to solve this issue in order to use FTP, at least for the moment, because of the speed improvement, but it is a must for me to be able to properly fire off remote korn shells as well.

Please help me! :)

Edit #1: I just tried version 5.9.2 and the issue is still there.

Edit #2: I'm working on trying to solve the problem at the moment and I just gave this test a whirl. I changed the string to:

"/home/parcccs/exe/ccc85000.ksh qc 99009 ic20140130-1.0.5.5 N/A N/A"

hence removing the environment variable $CCCS_EXE and it returns an error message because the string ends up being:

"/HOME/PARCCCS/EXE/CCC85000.KSH qc 99009 ic20140130-1.0.5.5 N/A N/A"

It's basically uppercase'ing anything at the start of the command line while respecting the case after the first space. As an example, if I uppercase the QC in the above string it stays in uppercase.

Edit #3: Just another particular result. I changed the string to (notice the space at the front of the string):

" /home/parcccs/exe/ccc85000.ksh qc 99009 ic20140130-1.0.5.5 N/A N/A"

and it returns an error message because the string ends up being:

" /HOme/parcccs/exe/ccc85000.ksh qc 99009 ic20140130-1.0.5.5 N/A N/A"

The only characters that got converted to uppercase were the letters H and O.

Thank you.

It's misunderstanding.

With the SFTP protocol, the Session.ExecuteCommand opens a separate shell session to execute the command.
While with the FTP protocol, the Session.ExecuteCommand runs the commands as an FTP command.
See https://winscp.net/eng/docs/library_session_executecommand#remarks

As the FTP commands are upper case, your FTP server probably internally "uppercases" the command. And uses that altered command name in the later error message (as the $CCCS_EXE/CCC85000.KSH is obviously not an FTP command).

What you can do is to use SFTP (or actually SCP) session to execute the command.
And when open FTP session (or better FTPES) for the transfer.

martin wrote:

It's misunderstanding.

With the SFTP protocol, the Session.ExecuteCommand opens a separate shell session to execute the command.
While with the FTP protocol, the Session.ExecuteCommand runs the commands as an FTP command.
See https://winscp.net/eng/docs/library_session_executecommand#remarks

As the FTP commands are upper case, your FTP server probably internally "uppercases" the command. And uses that altered command name in the later error message (as the $CCCS_EXE/CCC85000.KSH is obviously not an FTP command).

What you can do is to use SFTP (or actually SCP) session to execute the command.
And when open FTP session (or better FTPES) for the transfer.

Hi Martin,

You are correct in that I misunderstood what I had read about using the FTP protocol as my session option. I thought it just INCLUDED the ability to run true FTP commands.

I have two questions based on your response:

A. If I understand your suggestion, I can easily switch protocol in the same session? If so, are there any examples of that in C# on the site?
B. You mention FTPES. I don't see anything on the site that mentions it? Is it possibly this: https://winscp.net/eng/docs/ftps#methods

Thank you.

Chris David wrote:

A. If I understand your suggestion, I can easily switch protocol in the same session? If so, are there any examples of that in C# on the site?

You can call Close to close one "session" and call Open to start a new "session" withing the same Session.
Or you can simply use two Session instances (even in parallel).

B. You mention FTPES. I don't see anything on the site that mentions it? Is it possibly this: https://winscp.net/eng/docs/ftps#methods

Yes, that's it. Explicit FTP over TLS/SSL.

martin wrote:

Chris David wrote:

A. If I understand your suggestion, I can easily switch protocol in the same session? If so, are there any examples of that in C# on the site?

You can call Close to close one "session" and call Open to start a new "session" withing the same Session.
Or you can simply use two Session instances (even in parallel).

B. You mention FTPES. I don't see anything on the site that mentions it? Is it possibly this: https://winscp.net/eng/docs/ftps#methods

Yes, that's it. Explicit FTP over TLS/SSL.

If I close a session, doesn't that perform a "disconnect"? I need to be able to do the following in this particular order and without ever disconnecting:

- Login
- Execute korn shell
- Perform some uploads
- Execute korn shell
- Perform some downloads
- Execute korn shell
- Execute korn shell

Is that possible based on what you said?

Yes, it disconnects.

But note that even with your original SFTP solution, you actually did not perform the upload and execution on the same connection. As with FTP, there's no way to execute a shell command over SFTP protocol. WinSCP just opens a separate SSH connection behind the scenes. So if that worked for you, using two parallel FTP and SCP connections must work too.

martin wrote:

Yes, it disconnects.

But note that even with your original SFTP solution, you actually did not perform the upload and execution on the same connection. As with FTP, there's no way to execute a shell command over SFTP protocol. WinSCP just opens a separate SSH connection behind the scenes. So if that worked for you, using two parallel FTP and SCP connections must work too.

I will consider doing this as FTP seems to be about 400% faster than SFTP at the moment. I'm wondering if there is a possibility of that speed gap being reduced by doing something else? Any suggestions? If not, I will just use FTP for now if the code change isn't too complex to perform parallel sessions.

Thank you.

Do you get faster speed with any other SFTP client?

martin wrote:

Do you get faster speed with any other SFTP client?

Hi Martin,

Sorry for the late reply.

I didn't try any other SFTP client at the moment. However, I had already decided to build something using WinSCP's .NET assembly, thinking that SFTP would surely be an improvement over Windows FTP.EXE, even in speed. To my dismay, it was about as fast if not slower in some scenarios. However, I had already finished the code by this time and it was only at that time that I read on the forums about some other people having speed issues with SFTP. They even said using other SFTP clients gave them better results (and it seemed by a good margin as well). That is the only reason why I gave FTP a try and it seems to work a lot faster. I do know that there is some overhead with SFTP, but I didn't realize it was going to make THAT big a difference. For now, I will use FTP over SFTP until the latter is faster.

Thank you.

In general, the SFTP cannot ever be faster than a plain FTP.

martin wrote:

In general, the SFTP cannot ever be faster than a plain FTP.

Absolutely. However, should my gain be, at least based on my initial test, 400% using FTP over SFTP? It seems a bit high to me. I also ran into some posts on the forum talking about how the WinSCP SFTP seems to be slower in comparison to other SFTP implementations. I don't know if this is the case today, but I just can't give up a 400% loss in speed using SFTP hence why I chose FTP for the time being (it was FTP before). I do hope to use SFTP in WinSCP in the future, but that will depend on the speed issue at that time.

Thanks.