The command was meant as a placeholder for your actual command. Not as a command/keyword.
In other words, where's no special keyword you need to use. Just type in your command!
If this still won't help, please post screenshot of an actual standalone SSH terminal, where you successfully execute your command. And equivalent screenshot of WinSCP Console window, where the same command fails.

It's also possible that your server's shell is not compatible with WinSCP. Post a session log file if you want us to check.
What are you actually trying to achieve? Why are you executing commands in WinSCP and not in an SSH terminal?

Tried calling the following way, still the same error:

will throw syntax error

error: -bsh: syntax error: `(' unexpected

This looks like a misunderstanding. The RCMD is a proprietary FTP command of some IBM FTP servers.
It has nothing to do with SFTP. SFTP protocol does not support executing remote commands. And it does not have textual commands either.
But WinSCP can open a secondary shell session behind the scenes to allow you to execute arbitrary shell commands. But there's no special command involved, you type your actual shell command in the Console window. So not RCMD command, but just command.

When I send in the Console, I get:

-bsh: rcmd: not found

I can login fine and transfer files.

What I am trying to do is use the SFTP connection and call AS/400 program. Calling any program fails, so I tried RCMD? to see what commands are available, but I get the above error.