I've successfully run a single keepuptodate session on over 13,000 (sub) directories when those the directories are on my local machine that's running the WinSCP FTP client so it seems that the error code 56 issue is really between me and my local network fileserver setups.

Thanks for your assistance.

phillipdannward wrote:

Can you please provide a short explanation of what WinSCP does on the local machine/filesystem when it sets up a keepuptodate watch them so that I can assess what the limitations would be in my own machine/filesystem setup. Thanks.

WinSCP uses Windows API functions FindFirstChangeNotification and FindNextChangeNotification if that helps you.

I've found that the NETBIOS command limit exceeded (error code 56) is due to limits set on the client and server, see <invalid hyperlink removed by admin>

Since my 'local' directory for the FTP session is actually on a local fileserver I suspect that it's my PC and/or my fileserver settings that are causing the limit to be hit by WinSCP when it sets up the watch on all the directories on the fileserver.

Can you please provide a short explanation of what WinSCP does on the local machine/filesystem when it sets up a keepuptodate watch them so that I can assess what the limitations would be in my own machine/filesystem setup. Thanks.

For your information, I have also found that if I run parallel keepuptodate sessions each one on 10 (sub)directories they they are fine. However, if I run a fifth keepuptodate session also on 10 (sub)directories then I get the error code 56 on the fifth session although the original 4 sessions appear to be unaffected.

(This also means my back up option of multiple small-chunk keepuptodate sessions is not viable since the error code 56 is a system limitation not a session limitation :-( so I hope you can help me debug whatever I need to change in my system setup to remove/increase the network BIOS limitation that I appear to have.)

(P.S. With respect to my back up option, I found that I could call an INI file with the command line start and specify the SynchronizeOptions parameter as 0 in the INI file to prevent the command line keepuptodate session from looking in subdirectories.)

Actually WinSCP already does what you are trying to do. It splits the set of directories to chunks of ~60 (what is actual limit set by Windows). I do not know what's the another limit imposed by your system.
I'll try to prepare a debug version for you to trace this down.
Meanwhile, to exclude all subdirectories, try "option exclude ./".

I'm trying to use the keepuptodate function to keep a remote server up to date with items placed in a local working area which consists of 982 directories.

I get a error code 56 (see below) which (through trial and error testing) I have found is generated when I try to keep more than ~47 (sub)directories up to date.

Is there a way to remove or increase this limit?

If not, then given this limitation, I'm now trying to chop the keepuptodate into multiple keepuptodate chunks of less than ~45 (sub)directories but I still need to keep directories up to date which are higher in the structure than where the multiple keepuptodate chunks are starting from. Hence I need to exclude some or all subdirectories from these higher keepuptodate commands where they already have their own keepuptodate commands running on them. I'm able to do this from the GUI version of WinSCP by unticking the "Update subdirectories" Synchronize option in the "KeepUpToDate" window prior to starting it but I don't know how to set this condition from the script line. Can you tell me how to do this?

(PLEASE NOTE: I cannot use the synchronize command repeatedly instead since the remote directory is the temporary master for the items once the items are uploaded and before they are moved off the FTP server into the main master area on the other side of the FTP server (which is non-local). Hence it is not permitted to delete items from the remote directories by synchronising deletions in the local directories (synchdelete is turned off in the script) but deletions from the remote directories once they are moved off the FTP server (which can happen at any time) are eventually reflected back to the local directories by a seperate synchronization of existing files in the local directory with those same but now deleted files in the remote directory.)


P:\Q400_PEC\Dynamic\Transfer\ULTRAFTP\WinSCPScripts>"C:\Program Files\WinSCP\WinSCP.com" phillip.ward_silver-software.co
m@212.44.25.100 /privatekey="C:\Documents and Settings\phillip.dann.ward\My Documents\Other\phillip.ward.ppk" /log="P:\Q
400_PEC\Dynamic\Transfer\ULTRAFTP\WinSCPLogs\Pioneer_FromSilverToUltra.log" /script="P:\Q400_PEC\Dynamic\Transfer\ULTRAF
TP\WinSCPScripts\Script__Pioneer_FromSilverToUltra.txt"
Searching for host...
Connecting to host...
Authenticating...
Using username "phillip.ward_silver-software.com".
Authenticating with public key "rsa-key-20070809" from agent.
Authenticated.
Starting the session...
Reading remote directory...
Session started.
Active session: [1] phillip.ward_silver-software.com@212.44.25.100
batch continue
synchdelete off
Watching for changes, press Ctrl-C to abort...
Scanning 'P:\Q400_PEC\Dynamic\Transfer\ULTRAFTP\Pioneer\ToULTRA\Controlled Area\Project Configuration Management Area' f
or subdirectories...
Watching for changes in 982 directories...
Error watching for changes in directory 'P:\Q400_PEC\Dynamic\Transfer\ULTRAFTP\Pioneer\ToULTRA\Controlled Area\Project C
onfiguration Management Area\Software\003SWCM\01_DEVT\INS\T1'.
System Error. Code: 56.
The network BIOS command limit has been reached
winscp>


The following is the script file being used:

#
option batch continue
option synchdelete off
#
keepuptodate "P:\Q400_PEC\Dynamic\Transfer\ULTRAFTP\Pioneer\ToULTRA\Controlled Area\Project Configuration Management Area" "/pub/DASH8PEC/incoming/Pioneer/Controlled Area/Project Configuration Management Area"


The following is the log file generated:

. 2007-08-28 10:12:34.859 --------------------------------------------------------------------------
. 2007-08-28 10:12:34.859 WinSCP Version 4.0.3 (Build 345) (OS 5.1.2600 Service Pack 2)
. 2007-08-28 10:12:34.875 Login time: 28 August 2007 10:12:34
. 2007-08-28 10:12:34.875 --------------------------------------------------------------------------
. 2007-08-28 10:12:34.875 Session name: phillip.ward_silver-software.com@212.44.25.100
. 2007-08-28 10:12:34.875 Host name: 212.44.25.100 (Port: 22)
. 2007-08-28 10:12:34.875 User name: phillip.ward_silver-software.com (Password: No, Key file: No)
. 2007-08-28 10:12:34.875 Tunnel: No
. 2007-08-28 10:12:34.890 Transfer Protocol: SFTP (SCP)
. 2007-08-28 10:12:34.890 Ping type: -, Ping interval: 30 sec; Timeout: 15 sec
. 2007-08-28 10:12:34.890 Proxy: none
. 2007-08-28 10:12:34.890 SSH protocol version: 2; Compression: No
. 2007-08-28 10:12:34.906 Agent forwarding: No; TIS/CryptoCard: No; KI: Yes; GSSAPI: No
. 2007-08-28 10:12:34.906 Ciphers: aes,blowfish,3des,WARN,des; Ssh2DES: No
. 2007-08-28 10:12:34.906 SSH Bugs: -,-,-,-,-,-,-,-
. 2007-08-28 10:12:34.906 SFTP Bugs: -,-,-
. 2007-08-28 10:12:34.921 Return code variable: Autodetect; Lookup user groups: Yes
. 2007-08-28 10:12:34.921 Shell: default, EOL: 0
. 2007-08-28 10:12:34.921 Clear aliases: Yes, Unset nat.vars: Yes, Resolve symlinks: Yes
. 2007-08-28 10:12:34.937 Alias LS: No, Ign LS warn: Yes, Scp1 Comp: No
. 2007-08-28 10:12:34.937 Local directory: default, Remote directory: home, Update: No, Cache: Yes
. 2007-08-28 10:12:34.937 Cache directory changes: Yes, Permanent: Yes
. 2007-08-28 10:12:34.937 DST mode: 1
. 2007-08-28 10:12:34.953 --------------------------------------------------------------------------
. 2007-08-28 10:12:34.953 Looking up host "212.44.25.100"
. 2007-08-28 10:12:34.953 Connecting to 212.44.25.100 port 22
. 2007-08-28 10:12:35.187 Server version: SSH-1.99-OpenSSH_3.9p1
. 2007-08-28 10:12:35.187 We claim version: SSH-2.0-WinSCP_release_4.0.3
. 2007-08-28 10:12:35.203 Using SSH protocol version 2
. 2007-08-28 10:12:35.234 Doing Diffie-Hellman group exchange
. 2007-08-28 10:12:35.328 Doing Diffie-Hellman key exchange
. 2007-08-28 10:12:35.687 Host key fingerprint is:
. 2007-08-28 10:12:35.687 ssh-rsa 1024 8f:51:9d:8b:45:37:bc:d7:db:6b:41:c3:9f:a6:37:7c
. 2007-08-28 10:12:35.703 Initialised AES-256 client->server encryption
. 2007-08-28 10:12:35.703 Initialised HMAC-SHA1 client->server MAC algorithm
. 2007-08-28 10:12:35.703 Initialised AES-256 server->client encryption
. 2007-08-28 10:12:35.703 Initialised HMAC-SHA1 server->client MAC algorithm
! 2007-08-28 10:12:35.781 Using username "phillip.ward_silver-software.com".
. 2007-08-28 10:12:35.781 Reading private key file "C:\Documents and Settings\phillip.dann.ward\My Documents\Other\phillip.ward.ppk"
! 2007-08-28 10:12:35.828 WARNING!
! 2007-08-28 10:12:35.828 This is for unclassified information ONLY.
! 2007-08-28 10:12:35.828 If you are not sure about the classification of the
! 2007-08-28 10:12:35.843 information you are about to upload please query.
! 2007-08-28 10:12:35.843 You are responsible for the following:
! 2007-08-28 10:12:35.843 - export control approval of any information
! 2007-08-28 10:12:35.843 or documents placed here.
! 2007-08-28 10:12:35.859 - release of proprietary information, any
! 2007-08-28 10:12:35.859 company and/or F135 Program information
! 2007-08-28 10:12:35.859 UK citizens uploading documents!!
! 2007-08-28 10:12:35.859 - Using this site constitutes an EXPORT
! 2007-08-28 10:12:35.875 - Does this contain any Controlled Technology?
! 2007-08-28 10:12:35.875 - Have you registered this transmittal?
! 2007-08-28 10:12:35.875 - Exported governed by the UKG "OGEL"
! 2007-08-28 10:12:35.875 (Technology for Military Goods)
. 2007-08-28 10:12:35.890 Pageant is running. Requesting keys.
. 2007-08-28 10:12:35.890 Pageant has 1 SSH-2 keys
. 2007-08-28 10:12:35.890 Trying Pageant key #0
. 2007-08-28 10:12:35.890 This key matches configured key file
! 2007-08-28 10:12:35.937 Authenticating with public key "rsa-key-20070809" from agent
. 2007-08-28 10:12:36.046 Sending Pageant's response
. 2007-08-28 10:12:36.078 Access granted
. 2007-08-28 10:12:36.109 Opened channel for session
. 2007-08-28 10:12:36.156 Started a shell/command
. 2007-08-28 10:12:36.156 --------------------------------------------------------------------------
. 2007-08-28 10:12:36.171 Using SFTP protocol.
. 2007-08-28 10:12:36.171 Doing startup conversation with host.
> 2007-08-28 10:12:36.171 Type: SSH_FXP_INIT, Size: 5, Number: -1
< 2007-08-28 10:12:36.203 Type: SSH_FXP_VERSION, Size: 5, Number: -1
. 2007-08-28 10:12:36.203 SFTP version 3 negotiated.
. 2007-08-28 10:12:36.203 We believe the server has signed timestamps bug
. 2007-08-28 10:12:36.203 We will use UTF-8 strings for status messages only
. 2007-08-28 10:12:36.218 Limiting packet size to OpenSSH sftp-server limit of 262148 bytes
. 2007-08-28 10:12:36.218 Getting current directory name.
. 2007-08-28 10:12:36.218 Getting real path for '.'
> 2007-08-28 10:12:36.218 Type: SSH_FXP_REALPATH, Size: 10, Number: 16
< 2007-08-28 10:12:36.250 Type: SSH_FXP_NAME, Size: 23, Number: 16
. 2007-08-28 10:12:36.250 Real path is '/'
. 2007-08-28 10:12:36.250 Startup conversation with host finished.