"Error listing directory" when opening a folder with many files in S3

Advertisement

roland
Donor+
Joined:
Posts:
3
Location:
Munich

"Error listing directory" when opening a folder with many files in S3

Version: 6.3.3, GUI
Protocol: S3

When opening a folder with many files (in this case: 18.000 but happens already with less) the files get loaded obviously alright, but with a modal with the message "error listing directory...file or folder does not exist" (see screenshot).

When clicking OK you can continue working but the error will happen again on every refresh of the folder.
I've checked the "common error messages", where it says this could happen due to a server timeout. I tried to change the setting from 15 to 30 seconds, but the error persists (and comes just after a few seconds, not anything close to 15 or 30 seconds).

error-listing.PNG

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
41,852
Location:
Prague, Czechia

Re: "Error listing directory" when opening a folder with many files in S3

Please attach a full session log file showing the problem (using the latest version of WinSCP).

To generate the session log file, enable logging, log in to your server and do the operation and only the operation that causes the error. Submit the log with your post as an attachment. Note that passwords and passphrases not stored in the log. You may want to remove other data you consider sensitive though, such as host names, IP addresses, account names or file names (unless they are relevant to the problem). If you do not want to post the log publicly, you can mark the attachment as private.

Reply with quote

roland
Donor+
Joined:
Posts:
3
Location:
Munich

Logfile

OK, here is the logfile. I've set it to private though, as it contains potentially sensitive URLs and filenames.
  • winscp-log.log (6.02 MB, Private file)

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
41,852
Location:
Prague, Czechia

Re: Logfile

Thanks. You seem to be connecting to some proprietary server using S3 protocol.
The server pages the directory contents by 1000 objects/files.
The directory contains exactly 18000 files, so exactly 18 pages.
With the actual Amazon S3 server, the last (18th) page would say:
<IsTruncated>false</IsTruncated>
while your server says:
<IsTruncated>true</IsTruncated>
So WinSCP reads one more "page".
The server returns an empty page.
That's another difference to the actual Amazon S3 server, which never returns an empty page, unless you are asking for listing of directory/prefix that actually does not exist. WinSCP gets confused here by these two differences and hence the "File or folder ... does not exist" error.

I can improve WinSCP not to error, if it gets an empty page, unless it is the first page.
But I'm not sure how your server responds to an actual empty directory/prefix. Can you send a log for that?

Reply with quote

roland
Donor+
Joined:
Posts:
3
Location:
Munich

Logfile again

Sorry for not getting back to you earlier, but here it comes. The logfile shows WinSCP opening an empty folder called Sonstiges. Hope that is what you need. I set the logfile to private as last time.
  • winscp-log.log (25.47 KB, Private file)

Reply with quote

Advertisement

Advertisement

You can post new topics in this forum