Post a reply

Before posting, please read how to report bug or request support effectively.

Bug reports without an attached log file are usually useless.

Username

Subject

Message body

Font colour:

Font size:

[quote="bendecko"]I'm uploading a lot of individual small files to Google Cloud Storage (S3 type) using Version: 5.19.4 – see attachment.

I started it Wednesday, and it is now Sunday.

There are a lot of files, and it should take approximately 1 day I guess.

The reason it is taking so long is that apparently randomly and periodically Google servers return
[quote]503 service unavailable[/quote]
6gb WinSCP log file so far.... I've just copied the end of it:
[code]
. 2021-11-21 04:04:57.621 Header Name: [content-length], Value: [19]
. 2021-11-21 04:04:57.621 Header Name: [date], Value: [Sun, 21 Nov 2021 04:04:57 GMT]
. 2021-11-21 04:04:57.621 Header Name: [server], Value: [UploadServer]
. 2021-11-21 04:04:57.621 Header Name: [connection], Value: [close]
. 2021-11-21 04:04:57.621 End of headers.
. 2021-11-21 04:04:57.622 sess: Closing connection.
. 2021-11-21 04:04:57.622 sess: Connection closed.
. 2021-11-21 04:04:57.622 Request ends, status 503 class 5xx, error line:
. 2021-11-21 04:04:57.622 503 Service Unavailable
< 2021-11-21 04:04:57.622 Service Unavailable
. 2021-11-21 04:04:57.622 Request ends.
. 2021-11-21 04:04:57.622 sess: Destroying session.
* 2021-11-21 04:04:57.622 (ExtException) **XmlParseFailure**
. 2021-11-21 04:04:57.622 Asking user:
. 2021-11-21 04:04:57.622 Error transferring file 'D:\landcycle\Tiles\landregistry\16\32392\21852.pbf'. (XmlParseFailure)
[/code]
So WinSCP then halts the upload process with a dialogue (attached)

If I click [i]"Retry"[/i], it always works on this attempt and thousand more files are uploaded until Google throws the 503 again.    This, of course, happens at 2am and I don't see the error until 9am.   So those intervening hours have been lost.

Eventually I will be able to upload all these files.  But what would be better if there was a setting and countdown.  [i]"Automatically retry failed uploads after X seconds"[/i] and WinSCP would continue and automatically close the dialogue and retry itself.

I've attached a short video of the clicking [i]"Retry"[/i], and that the file that previously failed then works no problem on the reattempt.[/quote]

Options

Disable BBCode in this post

Notify me when a reply is posted (registered users only)

Add an Attachment

If you do not want to add an Attachment to your Post, please leave the Fields blank.

Filename

(maximum 10 MB; please compress large files; only common media, archive, text and programming file formats are allowed)

File Comment

Options

Private file (available to author and site moderators only)

Topic review

martin

Re: Multi million file upload to Google Cloud Storage occasionally file fails stalls process.

2021-11-30

I have added this issue to the tracker:
https://winscp.net/tracker/2047

I'm sending you an email with a development version of WinSCP to the address you have used to register on this forum.

martin

Re: Multi million file upload to Google Cloud Storage occasionally file fails stalls process.

2021-11-24

Thanks for your report.
I will take a look at this issue and will contact you in several days.

bendecko

Multi million file upload to Google Cloud Storage occasionally file fails stalls process.

2021-11-21 11:41

I'm uploading a lot of individual small files to Google Cloud Storage (S3 type) using Version: 5.19.4 – see attachment.

I started it Wednesday, and it is now Sunday.

There are a lot of files, and it should take approximately 1 day I guess.

The reason it is taking so long is that apparently randomly and periodically Google servers return

503 service unavailable

6gb WinSCP log file so far.... I've just copied the end of it:

. 2021-11-21 04:04:57.621 Header Name: [content-length], Value: [19]

. 2021-11-21 04:04:57.621 Header Name: [date], Value: [Sun, 21 Nov 2021 04:04:57 GMT]

. 2021-11-21 04:04:57.621 Header Name: [server], Value: [UploadServer]

. 2021-11-21 04:04:57.621 Header Name: [connection], Value: [close]

. 2021-11-21 04:04:57.621 End of headers.

. 2021-11-21 04:04:57.622 sess: Closing connection.

. 2021-11-21 04:04:57.622 sess: Connection closed.

. 2021-11-21 04:04:57.622 Request ends, status 503 class 5xx, error line:

. 2021-11-21 04:04:57.622 503 Service Unavailable

< 2021-11-21 04:04:57.622 Service Unavailable

. 2021-11-21 04:04:57.622 Request ends.

. 2021-11-21 04:04:57.622 sess: Destroying session.

* 2021-11-21 04:04:57.622 (ExtException) **XmlParseFailure**

. 2021-11-21 04:04:57.622 Asking user:

. 2021-11-21 04:04:57.622 Error transferring file 'D:\landcycle\Tiles\landregistry\16\32392\21852.pbf'. (XmlParseFailure)

So WinSCP then halts the upload process with a dialogue (attached)

If I click "Retry", it always works on this attempt and thousand more files are uploaded until Google throws the 503 again. This, of course, happens at 2am and I don't see the error until 9am. So those intervening hours have been lost.

Eventually I will be able to upload all these files. But what would be better if there was a setting and countdown. "Automatically retry failed uploads after X seconds" and WinSCP would continue and automatically close the dialogue and retry itself.

I've attached a short video of the clicking "Retry", and that the file that previously failed then works no problem on the reattempt.

Post a reply

Topic review

Re: Multi million file upload to Google Cloud Storage occasionally file fails stalls process.

Re: Multi million file upload to Google Cloud Storage occasionally file fails stalls process.

Multi million file upload to Google Cloud Storage occasionally file fails stalls process.

Documentation

Support

Associations

Follow Us