WinSCP and z/OS FTP Server

Hi all,

WinSCP is our company's (semi-)official FTP client of choice. As a member of the mainframe department, I connect to FTP servers running on IBM's z/OS most of the time.

All works well when using WinSCP in the interactive GUI-based mode. However, when trying to automate file transfers using WinSCP's batch interface, things turn out to be a little complicated.
While uploading files to the mainframe still works as expected, scripted downloads are restricted. It seems as if the main problem is that, before issuing the actual RETR command, WinSCP sends a LIST command to the server. Apparently, it then attempts to find the name of the file to be downloaded within the received listing. This works as long as one does not specify a fully qualified z/OS dataset name (that is, a dataset name enclosed within single quotes) in the GET batch statement. Trying to do so makes the operations fail with error messages of the type

Can't get attributes of file ''HLQ.DATASET.NAME''

On plain z/OS, there is basically no such concept as directories in terms of hierarchically organized folders which one can specify paths to. Rather, z/OS dataset names consist of multiple qualifiers all separated by periods. To account for this, on an FTP server running on z/OS, the CWD command does not actually change into the specified directory, but rather sets a certain dataset name prefix. In GET or PUT operations, in case they do not use a fully qualified dataset name, this dataset name prefix is prepended to the involved dataset names. For LIST commands, only datasets with a name matching this prefix are returned. (As a side note and reflecting on this for a moment, one comes to the conclusion that, in the context of an FTP server, this behavior is actually not that far from and somewhat compatible to the concept of directories.)

Now, since after login the dataset name prefix is set to the name of the current user, the LIST command issued at the beginning of a GET operation returns only those datasets which are under the HLQ of the user. In the interactive GUI-based mode, this is not a problem, since using the GUI the user is unlikely to specify a fully qualified dataset name, but will rather either download a dataset under his/her HLQ or CWD to the appropriate prefix before starting a download operation.
In both cases, WinSCP will find the requested dataset name in the listing returned by the LIST command.

In batch mode, however, one can specify a fully qualified dataset name as described above. Apparently, WinSCP issues a LIST rather than a LIST 'HLQ.DATASET.NAME' command in this case. Consequently, the received listing will contain only those datasets matching the current dataset name prefix (i.e. the name of the current user) and not the requested one.

As a workaround, one can manually add an appropriate CD command and then use the dataset name without the specified prefix in the subsequent GET operation. Apparently, this is also what WinSCP does automatically when specifying a fully qualified UNIX-style filename (i.e. one starting with /).
Manually splitting the operations like this, however, makes things unnecessarily complicated.

Based on this, I would like to ask whether it is somehow possible to make WinSCP identify fully qualified z/OS dataset names by means of the leading ' and then craft and issue the appropriate CD commands.

Thanks in advance!

Regards,
gustav_heinrich

Thanks for your detailed post. Can you include few log files illustrating the various scenarios?

Hi Martin,

we have run five tests and included the server and client logs as well as the scripts that were used (see the attached archive).

-) get_mvs__abs_no_cd.txt: FTP GET on an MVS dataset. The name of this dataset was specified as a fully qualified name.
-) get_mvs__rel_cd.txt: FTP GET on an MVS dataset. The name was specified as a "relative" dataset name. An appropriate preceding CD command was added.
-) get_mvs__rel_no_cd.txt: FTP GET on an MVS dataset. The name was specified as a "relative" dataset name. NO preceding CD command was coded.
-) get_uss__abs_no_cd.txt: FTP GET on an Unix dataset. The name was specified as a fully qualified path. NO preceding CD command was coded.
-) get_uss__rel_cd.txt FTP GET on an Unix dataset. The name was specified as a relative path. An appropriate preceding CD command was added.

Regards,
gustav_heinrich

Thanks. It may take few days, before I'll be able to dig into this. I'll get back to you.

Can you do RETR 'OTHERHLQ.FTPTEST.TXT' on the MVS FTP server?

Yes, this is possible. The MVS FTP server will then send the dataset with name OTHERHLQ.FTPTEST.TXT and will not prepend the current dataset name prefix. Specifying a name enclosed in single quotes on an MVS FTP server is somewhat similar to specifying the name with a leading slash on a UNIX-style server.

This issue has been added to the tracker:
https://winscp.net/tracker/1579

Thanks for your report.

Can you send me an email, so I can send you back a development version of WinSCP for testing? Please include link back to this topic in your email. Also note in this topic that you have sent the email. Thanks.

Hi Martin,

sorry for the delay.

Unfortunately, specifying a fully qualified dataset name does not work yet.
We have run some of the tests again and included an additional one (see the attached file).

Regards,
gustav_heinrich

Thanks for the tests.
The get_mvs__abs_no_cd__client.log is the test I've asked for, right?
But it shows get 'OTHERHLQ.BARTEST.TXT', while there's FOOTEST.TXT in the listing.

Hi Martin,

thanks for your response!

Yes, that's correct. This test intends to illustrate downloading a dataset whose name is specified by a fully qualified name (OTHERHLQ.BARTEST.TXT). The fact that you see 'FOOTEST.TXT' in the listing illustrates the problem we have pretty well. Since the GET command in our test is not preceded by a CD command, the current dataset prefix is still 'USER.'. Hence, among others, the server returns 'USER.FOOTEST.TXT' as 'FOOTEST.TXT' in the listing. Consequently, WinSCP is unable to lookup the requested 'BARTEST.TXT' (or 'OTHERHLQ.BARTEST.TXT') in the listing.

I've included all the files once again, as there were some inconsistencies between the logs and the scripts.

Regards,
gustav_heinrich

I understand that "server returns 'USER.FOOTEST.TXT' as 'FOOTEST.TXT' in the listing".

But you have asked for BARTEST.TXT and there's FOOTEST.TXT in the listing.

There are two datasets involved in the tests I've posted: USER.FOOTEST.TXT and OTHERHLQ.BARTEST.TXT.

The test get_mvs__abs_no_cd tries to download the dataset OTHERHLQ.BARTEST.TXT by specifying a fully qualified dataset name in the GET command. Due to the absence of a preceding CD command, the dataset prefix is still set to USER. and the LIST command returns all datasets whose names start with this qualifier. Among others, this includes USER.FOOTEST.TXT. Since BARTEST.TXT has a different high level qualifier (OTHERHLQ), it is not contained in the listing.

OK, I get it now.
But than it's so complicated, that I just cannot solve it without an access to MVS server. What I do not have.

Hi Martin,

unfortunately, I cannot give you access to our mainframe...
However, I can provide you all types of logs and listings. Would that be of any help for your development?

Kind regards,
gustav_heinrich

gustav_heinrich wrote:

However, I can provide you all types of logs and listings. Would that be of any help for your development?

Thanks for you offer, but as you see, we didn't get anywhere with this approach for almost three months.

i am able to see the file but cannot run the get on it.

smijoss wrote:

i am able to see the file but cannot run the get on it.

And did you try using simple file name?

Don't know if this helps.
I have done a number of test and for me this works, using z/OS datasetname DVL.REXXSORT.XMI where DVL is not my userid:
Note that the name used in both the cd and the get command must be in uppercase,
Put works too, but only if the datasetname exists.
It is a bit cumbersome but I am scripting all my transfers, so not too bad.
I have attached the log from the successful get test. You can see the WinScp and Windows version in there.

Thanks for sharing your solution.

Hi,

Ran into same problem as other users here and though I would share my solution. As I could not find one easily on other forums.

Instead of using FTP to connect to ZOS box use the default SFTP connection WinSCP creates to connect to the UNIX system on the ZOS box.

Code below:
Use GET command as follows:
The MY.MVS.FILE.ON.ZOS.BOX is your fully qualified MVS file name under ZOS. The UNIX OS will be able to retrieve it for you and pass it to your local directory. Be sure to include the // and ' as prefix and suffix to your MVS file name.

I specified -transfer=ascii since natively on ZOS box it uses EBCDIC codepages and I was transferring to a Windows box.

Hope this helps someone.

Good luck!

Thanks for sharing this.

Also note that 5.18.1 has improved support for non-Unix-like FTP servers.
https://winscp.net/tracker/49
It may need some configuration to make it working against MVS. If anyone is willing to test it, let me know.