Cannot parse LIST line for z/OS member with no statstics

Advertisement

mschmitt
Joined:
Posts:
9

Cannot parse LIST line for z/OS member with no statstics

WinSCP version: 5.19.6
Windows version: Windows 10 Enterprise 20H2
Transfer protocol: FTP

I am attempting use WinSCP to connect to a z/OS Communications Server 2.3 FTP server, to the "MVS" side of the system (as opposed to the Open Edition MVS aka Unix side).

One problem I'm having is with files that are members of a Partitioned Data Set (aka Library) in MVS. The FTP server responds to the LIST command by listing each member name along with its statistics. FTP clients interpret this as if each member is a unique file. WinSCP works in that case.

But, statistics are OPTIONAL in MVS. They are often created by default, such as if the member is a source (text) file that was edited by the standard text editor. But there are a zillion ways to have members that do not have any associated statistics.

For example, if I have these members in a z/OS library:
 Name     Prompt       Size   Created          Changed          ID   
LOADLIST                                                             
MAGPCM0A               1570  2001/01/30  2022/02/07 02:28:02  TTPJMD1
MAGPCM1A               2663  2000/08/18  2022/02/07 02:28:53  TTPJMD1
MAGPCM2A               2494  2000/11/30  2022/03/03 12:27:47  TTPJMD1
MAGYCM13                 26  1988/11/01  2022/02/07 02:25:11  TTPJMD1
Notice that the LOADLIST member has no statistics. This is a valid situation.

The LIST command returns this as:
LOADLIST
MAGPCM0A  01.00 2001/01/30 2022/02/07 02:28  1570  1520     0 TTPJMD1 
MAGPCM1A  01.00 2000/08/18 2022/02/07 02:28  2663  2549     0 TTPJMD1 
MAGPCM2A  01.01 2000/11/30 2022/03/03 12:27  2494  1823     0 TTPJMD1 
MAGYCM13  01.00 1988/11/01 2022/02/07 02:25    26    17     0 TTPJMD1
again, the LOADLIST member has no statistics.

But, WinSCP can't parse that line. It either interprets it as some kind of directory or doesn't return it at all:
D---------   0                           0              ..
----------   0                        1570 Feb  7  2:28 MAGPCM0A
----------   0                        2663 Feb  7  2:28 MAGPCM1A
----------   0                        2494 Mar  3 12:27 MAGPCM2A
----------   0                          26 Feb  7  2:25 MAGYCM13

The result is that you can not use WinSCP to GET such a file.

I recreated this in the attached log, as follows:

1. I am using the command line, because I haven't been able to get WinSCP GUI to work with z/OS at all. This may be do to problems in directory caching.

2. I used RegEdit to manually delete the Computer\HKEY_CURRENT_USER\SOFTWARE\Martin Prikryl\WinSCP 2\Configuration\History key. This is because until I realized that the directory cache was at fault, I was having about a 1% success rate with directory traversal in WinSCP on z/OS. It was bizarre: I might be able to get it to work one time, but then the identical set of commands would no longer work, and it would get to the point where it wouldn't permit me to perform ANY CD command, even after quitting the session and restarting.

I manually deleted the key because I couldn't figure out how to do it from the command line. For example, invoking with /command refresh gave a an error "Unknown command 'refresh'.".

3. I invoked WinSCP as: winscp.com ftp://plp312.sys.eds.com /log=winscp.log.

I am using the host with the protocol because otherwise it tries to make a secure connection, which will not work on this host. I couldn't figure out any other way to get the command line to just connect with FTP.

4. I changed to the desired directory, and then did...

5. LS

the result was as shown above.

6. I did MGET *, to show that it only downloads the files that have statistics. It downloaded the 4 members with statistics, but not member LOADLIST.

One question would be, what should WinSCP set the file stats to if you were able to download a member without server side statistics? Windows command-line FTP just sets the created & modified dates to the current date & time.
Description: Log of session as described in the bug report

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
38,209
Location:
Prague, Czechia

Re: Cannot parse LIST line for z/OS member with no statstics

Thanks for your detailed report. Indeed WinSCP cannot parse listings entries with a name only. With such entry, WinSCP cannot even know if it is a file or a folder. I'll see if more people have need to work with such z/OS listings.

Reply with quote

mschmitt
Joined:
Posts:
9

In z/OS the current directory is either in a library or it is not. If it is in a library, then the LIST can't have any subdirectories, all of the entries represent files.

One way you can tell that you're in a library is that the LIST returns a header line. I didn't show it in the original post. It looks like this, where the first line is the header:

Name     VV.MM   Created       Changed      Size  Init   Mod   Id
LOADLIST
MAGPCM0A  01.00 2001/01/30 2022/02/07 02:28  1570  1520     0 TTPJMD1 
MAGPCM1A  01.00 2000/08/18 2022/02/07 02:28  2663  2549     0 TTPJMD1 
MAGPCM2A  01.01 2000/11/30 2022/03/03 12:27  2494  1823     0 TTPJMD1 
MAGYCM13  01.00 1988/11/01 2022/02/07 02:25    26    17     0 TTPJMD1

So, if you get that header, then the remaining entries are files.

But I'm kind of thinking that WinSCP must know this at some level, because notice that the LIST results are formatted completely differently when the current directory is not a library:

Volume Unit    Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname
V9W024 3390   2022/03/24  1    1  FBA    203 27811  PS  DIR.LIST
V9W032 3390   2022/03/24  2    2  FBA    203 27811  PS  LIST
VKP116 3390   2022/03/24  1    1  FB      80  1600  PS  STMTS

When it is not a library the file name is at the far right of the result. When it is a library the entry name is at the far left of the result.

But anyway, my point is that if the LIST is returning a library, it can not contain any subdirectories, every entry is a file.

Reply with quote

martin
Site Admin
martin avatar

Sure, I can see that. But so far WinSCP directory parser is stateless. It parses each line individually, without any context. It even identify the server-type (directory listing format style) for each and every line individually. So making this work would be a huge change.

Reply with quote

mschmitt

I think in order for WinSCP to actually work on z/OS you'll have to know that the server is z/OS, due to severe directory change problems that result if you don't. I plan to open a different bug report to explain what I mean, as soon as I can get a better idea of what's going wrong. (I think I may know, but not sure yet.)

Reply with quote

Advertisement

Advertisement

You can post new topics in this forum