Connections Fail Under "heavy" Load

Advertisement

radix
Guest

Connections Fail Under "heavy" Load

Hello, I requested help when I first began using WinSCP a few weeks ago here: https://winscp.net/forum/viewtopic.php?t=7224

That comment was very helpful, but I decided to use FileWatcherUtilities to watch a folder then trigger a simple WinSCP script when a change was recorded. This worked very well for the first week, but now we have a few more people scanning at their own workstations at the end of the day and the connection fails each time I try to connect, causing the users to either scan again to retrigger the script or manually execute it using a ftp_script.bat file I put on their desktops.

The command string I am using is 'START /MIN winSCP.exe /console /script=ftp_script.txt'

The ftp_script.txt I am using is:
"""
# Automatically answer all prompts negatively not to stall
# the script on errors (off), set to 'continue' to ignore all errors.
option batch on

# Attempt reconnect in 3 seconds if connection fails
#option reconnecttime 3

# Disable overwrite confirmations that conflict with the previous
option confirm off

# Open connection to server
open ftp://[User1]:[PASSWORD]@[DOMAIN].com:21

# We put all files in directory to server working directory, and
# delete the local file after sending.
put -delete "C:\ABSOLUTE\PATH\TO\FOLDER\*" "/SUB-FOLDER/"

# Synchronize directories so 'keepuptodate' won't cause problems.
#synchronize both "C:\ABSOLUTE\PATH\TO\FOLDER"

# With the sync we can use 'uptodate' to keep the folders synced
#keepuptodate -delete "C:\ABSOLUTE\PATH\TO\FOLDER"

exit
"""

I have three clues.

One, our aggregate connection speed is a little slow at ~138 kBps down ~142 kBps up.

Two, when it begins to fail, the connection will not be allowed to set for a period of minutes. During this time I tried to log in by the commander GUI interface and was also rejected. Then I tried using the Filezilla client, just-in-case, it connected but after switching back to WinSCP commander GUI it failed again.

Three, the scanner creates temp files (e.g. [date_string].pd~ or [numbers].dat. The problem is these temp files cause the script to be executed multiple times. This is a problem, but because I had the same problem with the commander gui I think this is likely unrelated.

That's everything I can remember. I will post a log when the problem appears this afternoon.

Oh, I don't think it is the firewall because we generally have no problem connecting, and the FTP server is FileZilla because that is what the remote admin usually uses.

Any suggestions?

Reply with quote

Advertisement

Radix
Guest

Log File for Issue

Here is the modified log file for this problem. It doesn't tell much more, I'm still not sure how to fix this.

. 2009-09-16 15:07:27.687 --------------------------------------------------------------------------
. 2009-09-16 15:07:27.687 WinSCP Version 4.1.9 (Build 416) (OS 5.1.2600 Service Pack 3)
. 2009-09-16 15:07:27.687 Login time: Wednesday, September 16, 2009 3:07:27 PM
. 2009-09-16 15:07:27.687 --------------------------------------------------------------------------
. 2009-09-16 15:07:27.687 Session name: [SESSION INFO]
. 2009-09-16 15:07:27.687 Host name: [HOST INFO] (Port: 21)
. 2009-09-16 15:07:27.687 User name: [USER NAME] (Password: Yes, Key file: No)
. 2009-09-16 15:07:27.703 Tunnel: No
. 2009-09-16 15:07:27.703 Transfer Protocol: FTP
. 2009-09-16 15:07:27.703 Ping type: C, Ping interval: 30 sec; Timeout: 15 sec
. 2009-09-16 15:07:27.703 Proxy: none
. 2009-09-16 15:07:27.703 FTP: Passive: No
. 2009-09-16 15:07:27.703 Local directory: default, Remote directory: home, Update: No, Cache: Yes
. 2009-09-16 15:07:27.703 Cache directory changes: Yes, Permanent: Yes
. 2009-09-16 15:07:27.703 DST mode: 1
. 2009-09-16 15:07:27.703 --------------------------------------------------------------------------
. 2009-09-16 15:07:27.859 Connecting to [DOMAIN NAME] ...
. 2009-09-16 15:07:27.875 Connected with [DOMAIN NAME]. Waiting for welcome message...
< 2009-09-16 15:07:27.906 220-FileZilla Server version 0.9.32 beta
< 2009-09-16 15:07:27.906 220-written by Tim Kosse (Tim.Kosse@gmx.de)
< 2009-09-16 15:07:27.906 220 Please visit http://sourceforge.net/projects/filezilla/
> 2009-09-16 15:07:27.906 USER [USER NAME]
< 2009-09-16 15:07:27.921 331 Password required for [USER NAME]
> 2009-09-16 15:07:27.921 PASS ******
< 2009-09-16 15:07:27.937 230 Logged on
> 2009-09-16 15:07:27.937 SYST
< 2009-09-16 15:07:27.953 215 UNIX emulated by FileZilla
> 2009-09-16 15:07:27.953 FEAT
< 2009-09-16 15:07:27.968 211-Features:
< 2009-09-16 15:07:27.968 MDTM
< 2009-09-16 15:07:27.968 REST STREAM
< 2009-09-16 15:07:27.968 SIZE
< 2009-09-16 15:07:27.968 MLST type*;size*;modify*;
< 2009-09-16 15:07:27.968 MLSD
< 2009-09-16 15:07:27.968 UTF8
< 2009-09-16 15:07:27.968 CLNT
< 2009-09-16 15:07:27.968 MFMT
< 2009-09-16 15:07:27.968 211 End
. 2009-09-16 15:07:27.968 Connected
. 2009-09-16 15:07:27.968 --------------------------------------------------------------------------
. 2009-09-16 15:07:27.968 Using FTP protocol.
. 2009-09-16 15:07:27.968 Doing startup conversation with host.
> 2009-09-16 15:07:27.968 PWD
< 2009-09-16 15:07:27.984 257 "/" is current directory.
. 2009-09-16 15:07:27.984 Getting current directory name.
. 2009-09-16 15:07:27.984 Startup conversation with host finished.
. 2009-09-16 15:07:27.984 Copying 1 files/directories to remote directory "/"
. 2009-09-16 15:07:27.984 PrTime: Yes; PrRO: No; Rght: rw-r--r--; PrR: No (No); FnCs: N; RIC: 01; Resume: S (102400); CalcS: No; Mask:
. 2009-09-16 15:07:27.984 TM: M; ClAr: No; CPS: 0; ExclM(No):
. 2009-09-16 15:07:27.984 AscM: *.*html; *.htm; *.txt; *.php; *.php3; *.cgi; *.c; *.cpp; *.h; *.pas; *.bas; *.tex; *.pl; .htaccess; *.xtml; *.css; *.cfg; *.ini; *.sh; *.xml
. 2009-09-16 15:07:27.984 File: "C:\Documents and Settings\Rockwall3.WS6A\My Documents\My ScanSnap\ScanSnap_FTP\2009_09_16_14_32_55.pdf"
. 2009-09-16 15:07:27.984 Copying "C:\Documents and Settings\Rockwall3.WS6A\My Documents\My ScanSnap\ScanSnap_FTP\2009_09_16_14_32_55.pdf" to remote directory started.
. 2009-09-16 15:07:27.984 Binary transfer mode selected.
. 2009-09-16 15:07:27.984 Starting upload of C:\Documents and Settings\Rockwall3.WS6A\My Documents\My ScanSnap\ScanSnap_FTP\2009_09_16_14_32_55.pdf
> 2009-09-16 15:07:27.984 TYPE A
< 2009-09-16 15:07:28.000 200 Type set to A
> 2009-09-16 15:07:28.000 PORT 10,10,2,80,13,150
< 2009-09-16 15:07:28.015 200 Port command successful
> 2009-09-16 15:07:28.015 LIST
< 2009-09-16 15:07:28.031 150 Opening data channel for directory list.
< 2009-09-16 15:07:28.062 226 Transfer OK
> 2009-09-16 15:07:28.078 TYPE I
< 2009-09-16 15:07:28.093 200 Type set to I
> 2009-09-16 15:07:28.093 PORT 10,10,2,80,13,152
< 2009-09-16 15:07:28.109 200 Port command successful
> 2009-09-16 15:07:28.109 STOR 2009_09_16_14_32_55.pdf
< 2009-09-16 15:07:28.125 150 Opening data channel for file transfer.
< 2009-09-16 15:07:28.125 425 Can't open data connection.
. 2009-09-16 15:07:28.125 Copying files to remote side failed.
* 2009-09-16 15:07:28.125 (ExtException) Copying files to remote side failed.
* 2009-09-16 15:07:28.125 Can't open data connection.
. 2009-09-16 15:07:28.125 Asking user:
. 2009-09-16 15:07:28.125 Error transferring file 'C:\Documents and Settings\Rockwall3.WS6A\My Documents\My ScanSnap\ScanSnap_FTP\2009_09_16_14_32_55.pdf'. ("Copying files to remote side failed.","Can't open data connection.")
* 2009-09-16 15:07:28.125 (EScpSkipFile) Error transferring file 'C:\Documents and Settings\Rockwall3.WS6A\My Documents\My ScanSnap\ScanSnap_FTP\2009_09_16_14_32_55.pdf'.
* 2009-09-16 15:07:28.125 Copying files to remote side failed.
* 2009-09-16 15:07:28.125 Can't open data connection.
. 2009-09-16 15:07:36.484 Disconnected from server

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
40,476
Location:
Prague, Czechia

Re: Connections Fail Under "heavy" Load

Can you post a log file both from WinSCP and Filezilla taken at the same time, showing situation where WinSCP does not transfer and Filezilla does?

Reply with quote

radix
Guest

Log of FileZilla

Command: USER [user]
Response: 331 Password required for [user]
Command: PASS ******
Response: 230 Logged on
Status: Connected
Status: Starting upload of C:\Documents and Settings\Amazing Charts\My Documents\My ScanSnap\ScanSnap_FTP\2009_10_08_15_40_52.pdf
Command: CWD /providers
Response: 250 CWD successful. "/providers" is current directory.
Command: PWD
Response: 257 "/providers" is current directory.
Command: TYPE I
Response: 200 Type set to I
Command: PASV
Response: 227 Entering Passive Mode (71,164,159,29,19,76)
Command: STOR 2009_10_08_15_40_52.pdf
Response: 150 Connection accepted
Response: 226 Transfer OK
Status: File transfer successful
Status: Retrieving directory listing...
Command: PASV
Response: 227 Entering Passive Mode (71,164,159,29,19,77)
Command: MLSD
Response: 150 Connection accepted
Response: 226 Transfer OK
Status: Directory listing successful


Sorry it took me a while to get a reading of the log file. The above is it. I note that it appears FileZilla Ques the file before sending it on its way. I do no know how to do this with the WinSCP script.

Also, is t incorrect to end a directory name with a / slash? I have my directory ending as '/providers/'.

Connection failed both with the console script and with the GUI interface.

Reply with quote

radix
Guest

Error Message

On the gui interface I'm getting this error message:

'''
Could not retrieve directory listing
Can't open data connection.
'''

Reply with quote

Advertisement

radix
Guest

last clue

Sorry, last post.

Given a connection to the server, now that it is working, I'm getting the following connection info.

C:2009_10_08_15_40_52.pdf | 170 KiB | 13.9 KiB/s | binary | 100%

According to speak easy's speed test I have ~170 KB/s upload speed. Shouldn't I be able to do closer to a 100KiB/s upload? The problem is doubtfully on the servers end. They claim to have plenty.

Reply with quote

martin
Site Admin
martin avatar

Re: Log of FileZilla

I wanted to see both WinSCP and Filezilla log from the same moment. Anyway, it seems that you use active mode with WinSCP. while passive with Filezilla. Can you try to switch to passive with WinSCP as well?

Reply with quote

Advertisement

You can post new topics in this forum