Topic "Queue stalled/halted with"cannot create file"errors[updated]"

Author Message
egt
[View user's profile]

Joined: 2012-03-20
Posts: 2
Location: New Zealand
Hi,
I am trying to download ~27GB spread over ~12,000 files overnight during my ISP's free off-peak time, 2-8am, while I am a sleep. However it appears that some of the files have a file-name path >~250 characters so that when winscp adds .filepart, to allow resuming on binary transferred files, the resulting in a file-name path is >260 characters normally supported by win32/NTFS. Naturally when that happens file creation fails and produces an error msgbox saying that it can't create the file. Sorry, didn't write down the error message this morning. [Update] see bottom for log sample and screen-shot.

When this error happens it stalls/blocks/halts that queue slot, causing two things to happen. 1) Total transfer speed drops as you now have n-1 active transfer queue slots and 2)When n-n occurs (all slots blocked) zero transfer speed. Which is the condition I see when I get up in the morning. Wasting hours of free download time and bandwidth capacity.

I have changed (ticked) 'continue on error (advanced users)'; to no effect and with no temp file. I also moved the download directory to the drive/volume root e.g. z:\ to reduce the number of effected files.

My current work-around is to download the problem files with an other program once winscp can't. So I know that the original file-name paths are short enough and that the files also pass integrity hash checks.

Tonight I am trying with resume disabled. (under Preferences->Endurance) and enable logging. Will update tomorrow with results.

Possible 'bugs'?:
1).filepart making the resulting filename path too long. e.g. >260 characters
2)These errors stalling the queue.

Possible solutions:
1)fallback to using \\?\ prefix to allow ~32,767 length paths. https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247.aspx#maxpath
2)disable resume for that file and continue. (auto or option in error message box; current and/or future files eg 'auto disable on error' for this queue/session ?)
3)shorten .filepart extension to maybe .fp or .!fp in the hope it is now short enough.
4)'suspend'/pause problem file(s) to allow the rest of the queue to continue. So they remain visible at the top of the queue and the user knows they had problems.
5)'move' them to the bottom of the queue to allow the other files to continue.

If any one knows of any work around(s), please let me know.

Regards
Peter

Server: 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------

Local: Window 7 Pro, Winscp 4.3.7, Interface: GUI-Commander

Session protocol = SSH-2
SSH implementation = OpenSSH_5.1p1 Debian-5
Encryption algorithm = aes
Compression = No
File transfer protocol = SFTP-3
------------------------------------------------------------
Server host key fingerprint
ssh-rsa 2048 ea:4a:9e:34:08:70:93:0f:14:98:25:4f:9d:63:e5:6b
------------------------------------------------------------
Can change permissions = Yes
Can change owner/group = Yes
Can execute arbitrary command = No
Can create symlink/hardlink = Yes/No
Can lookup user groups = No
Can duplicate remote files = No
Can check available space = No
Can calculate file checksum = No
Native text (ASCII) mode transfers = No
------------------------------------------------------------
Additional information
The server supports these SFTP extensions:
posix-rename@openssh.com="1"
statvfs@openssh.com="2"
fstatvfs@openssh.com="2"

Log sample: ~60,000 lines into the log of ~233,000 lines.
< 2012-03-21 03:33:17.928 [Background 1] Status code: 1
. 2012-03-21 03:33:17.928 [Background 1] 53 skipped SSH_FXP_WRITE, SSH_FXP_READ, SSH_FXP_DATA and SSH_FXP_STATUS packets.
> 2012-03-21 03:33:17.929 [Background 1] Type: SSH_FXP_CLOSE, Size: 13, Number: 20153860
< 2012-03-21 03:33:17.929 [Background 1] Type: SSH_FXP_STATUS, Size: 28, Number: 20148997
< 2012-03-21 03:33:17.929 [Background 1] Type: SSH_FXP_STATUS, Size: 28, Number: 20151557
< 2012-03-21 03:33:18.317 [Background 1] Type: SSH_FXP_STATUS, Size: 28, Number: 20153605
. 2012-03-21 03:33:18.318 [Background 1] Getting size of directory "Lynne Rossetto Kasper"
. 2012-03-21 03:33:18.319 [Background 1] Listing directory "/home/peter/EPUB Library (2012)/Lynne Rossetto Kasper".
> 2012-03-21 03:33:18.319 [Background 1] Type: SSH_FXP_OPENDIR, Size: 62, Number: 20156939
< 2012-03-21 03:33:18.319 [Background 1] Type: SSH_FXP_STATUS, Size: 24, Number: 20153860
. 2012-03-21 03:33:18.319 [Background 1] Discarding reserved response
< 2012-03-21 03:33:18.711 [Background 1] Type: SSH_FXP_HANDLE, Size: 13, Number: 20156939
> 2012-03-21 03:33:18.712 [Background 1] Type: SSH_FXP_READDIR, Size: 13, Number: 20160268
< 2012-03-21 03:33:19.111 [Background 1] Type: SSH_FXP_NAME, Size: 520, Number: 20160268
> 2012-03-21 03:33:19.111 [Background 1] Type: SSH_FXP_READDIR, Size: 13, Number: 20163084
< 2012-03-21 03:33:19.493 [Background 1] Type: SSH_FXP_STATUS, Size: 28, Number: 20163084
< 2012-03-21 03:33:19.493 [Background 1] Status code: 1
> 2012-03-21 03:33:19.493 [Background 1] Type: SSH_FXP_CLOSE, Size: 13, Number: 20165380
. 2012-03-21 03:33:19.493 [Background 1] Getting size of directory "/home/peter/EPUB Library (2012)/Lynne Rossetto Kasper/The Splendid Table's How to Eat Supper_ Recipes, Stories, and Opinions From Public Radio's Award-Win (13666)"
. 2012-03-21 03:33:19.493 [Background 1] Listing directory "/home/peter/EPUB Library (2012)/Lynne Rossetto Kasper/The Splendid Table's How to Eat Supper_ Recipes, Stories, and Opinions From Public Radio's Award-Win (13666)".
> 2012-03-21 03:33:19.493 [Background 1] Type: SSH_FXP_OPENDIR, Size: 171, Number: 20165643
< 2012-03-21 03:33:19.875 [Background 1] Type: SSH_FXP_STATUS, Size: 24, Number: 20165380
. 2012-03-21 03:33:19.875 [Background 1] Discarding reserved response
< 2012-03-21 03:33:20.260 [Background 1] Type: SSH_FXP_HANDLE, Size: 13, Number: 20165643
> 2012-03-21 03:33:20.260 [Background 1] Type: SSH_FXP_READDIR, Size: 13, Number: 20170508
< 2012-03-21 03:33:20.652 [Background 1] Type: SSH_FXP_NAME, Size: 561, Number: 20170508
> 2012-03-21 03:33:20.652 [Background 1] Type: SSH_FXP_READDIR, Size: 13, Number: 20173068
< 2012-03-21 03:33:21.038 [Background 1] Type: SSH_FXP_STATUS, Size: 28, Number: 20173068
< 2012-03-21 03:33:21.038 [Background 1] Status code: 1
> 2012-03-21 03:33:21.038 [Background 1] Type: SSH_FXP_CLOSE, Size: 13, Number: 20175876
. 2012-03-21 03:33:21.039 [Background 1] File: "/home/peter/EPUB Library (2012)/Lynne Rossetto Kasper"
. 2012-03-21 03:33:21.039 [Background 1] Listing directory "/home/peter/EPUB Library (2012)/Lynne Rossetto Kasper".
> 2012-03-21 03:33:21.039 [Background 1] Type: SSH_FXP_OPENDIR, Size: 62, Number: 20176139
< 2012-03-21 03:33:21.432 [Background 1] Type: SSH_FXP_STATUS, Size: 24, Number: 20175876
. 2012-03-21 03:33:21.432 [Background 1] Discarding reserved response
< 2012-03-21 03:33:21.827 [Background 1] Type: SSH_FXP_HANDLE, Size: 13, Number: 20176139
> 2012-03-21 03:33:21.827 [Background 1] Type: SSH_FXP_READDIR, Size: 13, Number: 20181004
< 2012-03-21 03:33:22.230 [Background 1] Type: SSH_FXP_NAME, Size: 520, Number: 20181004
> 2012-03-21 03:33:22.230 [Background 1] Type: SSH_FXP_READDIR, Size: 13, Number: 20183052
< 2012-03-21 03:33:22.614 [Background 1] Type: SSH_FXP_STATUS, Size: 28, Number: 20183052
< 2012-03-21 03:33:22.614 [Background 1] Status code: 1
> 2012-03-21 03:33:22.614 [Background 1] Type: SSH_FXP_CLOSE, Size: 13, Number: 20186116
. 2012-03-21 03:33:22.614 [Background 1] File: "/home/peter/EPUB Library (2012)/Lynne Rossetto Kasper/The Splendid Table's How to Eat Supper_ Recipes, Stories, and Opinions From Public Radio's Award-Win (13666)"
. 2012-03-21 03:33:22.614 [Background 1] Listing directory "/home/peter/EPUB Library (2012)/Lynne Rossetto Kasper/The Splendid Table's How to Eat Supper_ Recipes, Stories, and Opinions From Public Radio's Award-Win (13666)".
> 2012-03-21 03:33:22.614 [Background 1] Type: SSH_FXP_OPENDIR, Size: 171, Number: 20186379
< 2012-03-21 03:33:22.999 [Background 1] Type: SSH_FXP_STATUS, Size: 24, Number: 20186116
. 2012-03-21 03:33:22.999 [Background 1] Discarding reserved response
< 2012-03-21 03:33:23.389 [Background 1] Type: SSH_FXP_HANDLE, Size: 13, Number: 20186379
> 2012-03-21 03:33:23.389 [Background 1] Type: SSH_FXP_READDIR, Size: 13, Number: 20191244
< 2012-03-21 03:33:23.782 [Background 1] Type: SSH_FXP_NAME, Size: 561, Number: 20191244
> 2012-03-21 03:33:23.782 [Background 1] Type: SSH_FXP_READDIR, Size: 13, Number: 20195084
< 2012-03-21 03:33:24.167 [Background 1] Type: SSH_FXP_STATUS, Size: 28, Number: 20195084
< 2012-03-21 03:33:24.167 [Background 1] Status code: 1
> 2012-03-21 03:33:24.167 [Background 1] Type: SSH_FXP_CLOSE, Size: 13, Number: 20198404
. 2012-03-21 03:33:24.167 [Background 1] File: "/home/peter/EPUB Library (2012)/Lynne Rossetto Kasper/The Splendid Table's How to Eat Supper_ Recipes, Stories, and Opinions From Public Radio's Award-Win (13666)/The Splendid Table's How to Eat Supper_ Recipes, Stories, and Opinions From Public Radio's Award-Win - Lynne Rossetto Kasper.epub"
. 2012-03-21 03:33:24.167 [Background 1] Copying "/home/peter/EPUB Library (2012)/Lynne Rossetto Kasper/The Splendid Table's How to Eat Supper_ Recipes, Stories, and Opinions From Public Radio's Award-Win (13666)/The Splendid Table's How to Eat Supper_ Recipes, Stories, and Opinions From Public Radio's Award-Win - Lynne Rossetto Kasper.epub" to local directory started.
. 2012-03-21 03:33:24.167 [Background 1] Binary transfer mode selected.
* 2012-03-21 03:33:24.167 [Background 1] (EOSError) System Error. Code: 3.

* 2012-03-21 03:33:24.167 [Background 1] The system cannot find the path specified
. 2012-03-21 03:33:24.167 [Background 1] Asking user:
. 2012-03-21 03:33:24.168 [Background 1] Can't create file 'V:\Lynne Rossetto Kasper\The Splendid Table's How to Eat Supper_ Recipes, Stories, and Opinions From Public Radio's Award-Win (13666)\The Splendid Table's How to Eat Supper_ Recipes, Stories, and Opinions From Public Radio's Award-Win - Lynne Rossetto Kasper.epub'. ("System Error. Code: 3.

. 2012-03-21 03:33:24.168 [Background 1] The system cannot find the path specified")
Can't create file system error - code 3.png (19.12 KB) [Download]

Description: Consolidated screen-shot showing error msgbox & queue stall effect.

Advertisements
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 25015
Location: Prague, Czechia
Thanks for your post.
It looks like we will need to implement a support for long file paths.
Added to the tracker:
https://winscp.net/tracker/show_bug.cgi?id=821
_________________
Martin Prikryl
egt
[View user's profile]

Joined: 2012-03-20
Posts: 2
Location: New Zealand
prikryl wrote:
Thanks for your post.
It looks like we will need to implement a support for long file paths.
Added to the tracker:
https://winscp.net/tracker/show_bug.cgi?id=821


Hi Martin,
Thank you and your work-mates for working on this.
If there is any other info I can provide please ask, if you want it I can email you the full log . It is ~21MB text file (~1MB RARed).
Regards
Peter

PS. I am a computer programming student and the tutor covered, on Monday, how maintenance was often under appreciated and that it typically required wider and better skills than regular development as they do everything from; is it a real bug, analysis, design, coding, testing, deployment and maintenance; while others do just one or two.
fpabernard

Guest


Hello

I have the same problem. Folder synchronization failed, it seems to be because of local filenames too long.

Regards.

Fred
Advertisements

You can post new topics in this forum






Search Site

What is WinSCP?

It is award-winning SFTP client, SCP client, FTPS client and FTP client integrated into one software program for file transfer to FTP server or secure SFTP server. [More]

And it's free!

Donate

About donations

$9   $19   $49   $99

About donations

Recommend

WinSCP Privacy Policy

WinSCP License