WinSCP and z/OS FTP Server

Advertisement

gustav_heinrich
Guest

WinSCP and z/OS FTP Server

Hi all,

WinSCP is our company's (semi-)official FTP client of choice. As a member of the mainframe department, I connect to FTP servers running on IBM's z/OS most of the time.

All works well when using WinSCP in the interactive GUI-based mode. However, when trying to automate file transfers using WinSCP's batch interface, things turn out to be a little complicated.
While uploading files to the mainframe still works as expected, scripted downloads are restricted. It seems as if the main problem is that, before issuing the actual RETR command, WinSCP sends a LIST command to the server. Apparently, it then attempts to find the name of the file to be downloaded within the received listing. This works as long as one does not specify a fully qualified z/OS dataset name (that is, a dataset name enclosed within single quotes) in the GET batch statement. Trying to do so makes the operations fail with error messages of the type

Can't get attributes of file ''HLQ.DATASET.NAME''

On plain z/OS, there is basically no such concept as directories in terms of hierarchically organized folders which one can specify paths to. Rather, z/OS dataset names consist of multiple qualifiers all separated by periods. To account for this, on an FTP server running on z/OS, the CWD command does not actually change into the specified directory, but rather sets a certain dataset name prefix. In GET or PUT operations, in case they do not use a fully qualified dataset name, this dataset name prefix is prepended to the involved dataset names. For LIST commands, only datasets with a name matching this prefix are returned. (As a side note and reflecting on this for a moment, one comes to the conclusion that, in the context of an FTP server, this behavior is actually not that far from and somewhat compatible to the concept of directories.)

Now, since after login the dataset name prefix is set to the name of the current user, the LIST command issued at the beginning of a GET operation returns only those datasets which are under the HLQ of the user. In the interactive GUI-based mode, this is not a problem, since using the GUI the user is unlikely to specify a fully qualified dataset name, but will rather either download a dataset under his/her HLQ or CWD to the appropriate prefix before starting a download operation.
In both cases, WinSCP will find the requested dataset name in the listing returned by the LIST command.

In batch mode, however, one can specify a fully qualified dataset name as described above. Apparently, WinSCP issues a LIST rather than a LIST 'HLQ.DATASET.NAME' command in this case. Consequently, the received listing will contain only those datasets matching the current dataset name prefix (i.e. the name of the current user) and not the requested one.

As a workaround, one can manually add an appropriate CD command and then use the dataset name without the specified prefix in the subsequent GET operation. Apparently, this is also what WinSCP does automatically when specifying a fully qualified UNIX-style filename (i.e. one starting with "/").
Manually splitting the operations like this, however, makes things unnecessarily complicated.


Based on this, I would like to ask whether it is somehow possible to make WinSCP identify fully qualified z/OS dataset names by means of the leading ' and then craft and issue the appropriate CD commands.


Thanks in advance!

Regards,
gustav_heinrich

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
27,459
Location:
Prague, Czechia

Re: WinSCP and z/OS FTP Server

Thanks for your detailed post. Can you include few log files illustrating the various scenarios?

Reply with quote

gustav_heinrich
Guest

Hi Martin,

we have run five tests and included the server and client logs as well as the scripts that were used (see the attached archive).

-) get_mvs__abs_no_cd.txt: FTP GET on an MVS dataset. The name of this dataset was specified as a fully qualified name.
-) get_mvs__rel_cd.txt: FTP GET on an MVS dataset. The name was specified as a "relative" dataset name. An apropriate preceding CD command was added.
-) get_mvs__rel_no_cd.txt: FTP GET on an MVS dataset. The name was specified as a "relative" dataset name. NO preceding CD command was coded.
-) get_uss__abs_no_cd.txt: FTP GET on an Unix dataset. The name was specified as a fully qualified path. NO preceding CD command was coded.
-) get_uss__rel_cd.txt FTP GET on an Unix dataset. The name was specified as a relative path. An appropriate preceding CD command was added.


Regards,
gustav_heinrich
  • logs.zip (20.7 KB, Private file)
Description: The passphrase is *W1nSCP*

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
27,459
Location:
Prague, Czechia

Thanks. It may take few days, before I'll be able to dig into this. I'll get back to you.

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
27,459
Location:
Prague, Czechia

Can you do RETR 'OTHERHLQ.FTPTEST.TXT' on the MVS FTP server?

Reply with quote

gustav_heinrich
Guest

Yes, this is possible. The MVS FTP server will then send the dataset with name OTHERHLQ.FTPTEST.TXT and will not prepend the current dataset name prefix. Specifying a name enclosed in single quotes on an MVS FTP server is somewhat similar to specifying the name with a leading slash on a UNIX-style server.

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
27,459
Location:
Prague, Czechia

Re: WinSCP and z/OS FTP Server

This issue has been added to the tracker:
https://winscp.net/tracker/1579

Thanks for your report.

Can you send me an email, so I can send you back a development version of WinSCP for testing? Please include link back to this topic in your email. Also note in this topic that you have sent the email. Thanks.

You will find my address (if you log in) in my forum profile.

Reply with quote

gustav_heinrich
Joined:
Posts:
4

Hi Martin,

sorry for the delay.

Unfortunately, specifying a fully qualified dataset name does not work yet.
We have run some of the tests again and included an additional one (see the attached file).


Regards,
gustav_heinrich
  • logs_02.zip (15.96 KB, Private file)
Description: Same password as for the previous archive.

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
27,459
Location:
Prague, Czechia

Thanks for the tests.
The get_mvs__abs_no_cd__client.log is the test I've asked for, right?
But it shows get 'OTHERHLQ.BARTEST.TXT', while there's FOOTEST.TXT in the listing.

Reply with quote

gustav_heinrich
Joined:
Posts:
4

Hi Martin,

thanks for your response!

Yes, that's correct. This test intends to illustrate downloading a dataset whose name is specified by a fully qualified name (OTHERHLQ.BARTEST.TXT). The fact that you see 'FOOTEST.TXT' in the listing illustrates the problem we have pretty well. Since the GET command in our test is not preceded by a CD command, the current dataset prefix is still 'USER.'. Hence, among others, the server returns 'USER.FOOTEST.TXT' as 'FOOTEST.TXT' in the listing. Consequently, WinSCP is unable to lookup the requested 'BARTEST.TXT' (or 'OTHERHLQ.BARTEST.TXT') in the listing.

I've included all the files once again, as there were some inconsistencies between the logs and the scripts.


Regards,
gustav_heinrich

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
27,459
Location:
Prague, Czechia

gustav_heinrich wrote:

Yes, that's correct. This test intends to illustrate downloading a dataset whose name is specified by a fully qualified name (OTHERHLQ.BARTEST.TXT). The fact that you see 'FOOTEST.TXT' in the listing illustrates the problem we have pretty well. Since the GET command in our test is not preceded by a CD command, the current dataset prefix is still 'USER.'. Hence, among others, the server returns 'USER.FOOTEST.TXT' as 'FOOTEST.TXT' in the listing. Consequently, WinSCP is unable to lookup the requested 'BARTEST.TXT' (or 'OTHERHLQ.BARTEST.TXT') in the listing.
I understand that "server returns 'USER.FOOTEST.TXT' as 'FOOTEST.TXT' in the listing".

But you have asked for "BARTEST.TXT" and there's "FOOTEST.TXT" in the listing.

Reply with quote

gustav_heinrich
Joined:
Posts:
4

martin wrote:

But you have asked for "BARTEST.TXT" and there's "FOOTEST.TXT" in the listing.

There are two datasets involved in the tests I've posted: USER.FOOTEST.TXT and OTHERHLQ.BARTEST.TXT.

The test "get_mvs__abs_no_cd" tries to download the dataset OTHERHLQ.BARTEST.TXT by specifying a fully qualified dataset name in the GET command. Due to the absence of a preceding CD command, the dataset prefix is still set to USER. and the LIST command returns all datasets whose names start with this qualifier. Among others, this includes USER.FOOTEST.TXT. Since BARTEST.TXT has a different high level qualifier (OTHERHLQ), it is not contained in the listing.

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
27,459
Location:
Prague, Czechia

OK, I get it now.
But than it's so complicated, that I just cannot solve it without an access to MVS server. What I do not have.

Reply with quote

gustav_heinrich
Joined:
Posts:
4

martin wrote:

But than it's so complicated, that I just cannot solve it without an access to MVS server. What I do not have.
Hi Martin,

unfortunately, I cannot give you access to our mainframe...
However, I can provide you all types of logs and listings. Would that be of any help for your development?


Kind regards,
gustav_heinrich

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
27,459
Location:
Prague, Czechia

gustav_heinrich wrote:

However, I can provide you all types of logs and listings. Would that be of any help for your development?
Thanks for you offer, but as you see, we didn't get anywhere with this approach for almost three months.

Reply with quote

Advertisement

You can post new topics in this forum