Topic "Session.MoveFile: Recursive calls not allowed"

Author Message
ksain
[View user's profile]

Joined: 2015-04-14
Posts: 2
Location: Home
I receive approximately 150k files per day from an SFTP site. The files have to be archived in a folder on the SFTP after downloading. Iterating individual files in a loop after getting the RemoteDirectoryInfo is too slow for the volume. When the _session.FileTransferred event is fired, I call the _session.MoveFile to archive the file as they are downloaded. The error below is the stack trace.

_session.GetFiles(remoteFilePath + "/*.*", localFilePath, removeSource, _transferOptions);

Recursive calls not allowed
at WinSCP.Lock.Enter()
at WinSCP.CallstackAndLock..ctor(Logger logger, Lock alock)
at WinSCP.Logger.CreateCallstackAndLock()
at WinSCP.Session.MoveFile(String sourcePath, String targetPath)

I've a solution that works albeit temporarily whereby I create a second session just to move the files, but if the session is idle for too long even with the ReconnectTime set to TimeSpan.MaxValue, the remote server closes the connection. I've added the Session log and debug log option, but it hasn't been helpful since the app can't tell me the session has been closed until I try to archive a file.

Any help would be appreciated.
Advertisements
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24530
Location: Prague, Czechia
You can collect paths in Session.FileTransferred event and iterate the collected list after the transfer finishes.
_________________
Martin Prikryl
ksain
[View user's profile]

Joined: 2015-04-14
Posts: 2
Location: Home
Thanks for the response. Sorry for the late reply. Collecting the files in the Session.FileTransferred works when the volume is low, but since the task to download wakes up every 10 seconds, it attempts to pick up files which have already been downloaded and processed. Adding a second session that moved files in the FileTransferred event handler works until there is a large cache of files (over 1k) posted and then it slows down enough that we get a backlog of files which we have to process within seconds.

The old and current solution uses the parameko python library which can move (rename) the files in the same session. We are just trying to get away from the python script for a .net solution.

sftp.chdir(rpath)
files = sftp.listdir(rpath)
c = len(files) - DIRECTORY_COUNT
lblStatus['text'] = 'Downloading ' + str(c) + ' files'
i = 0
for rfile in files:
if not blnContinue:
break
if (rfile.endswith('txt')):
i = i + 1
fileCount.set(str(i))
sftp.get(rfile, lpath + rfile)
renFile = renamePath + rfile
sftp.rename(rfile, renFile)

I've also tried a similar path as the parameko solution getting the Session.ListDirectory and iterating files which average 3k in size, but it was too slow for the volume. Unfortunately, I've had to move back to the python script in the interim.

Thanks for you help.
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24530
Location: Prague, Czechia
ksain wrote:
I've also tried a similar path as the parameko solution getting the Session.ListDirectory and iterating files which average 3k in size, but it was too slow for the volume. Unfortunately, I've had to move back to the python script in the interim.

I'll test this scenario.
martin
[View user's profile]
Site Admin
Joined: 2002-12-10
Posts: 24530
Location: Prague, Czechia
Actually this change will improve the performance:
http://winscp.net/tracker/show_bug.cgi?id=1287

I'm sending you an email with a development version of WinSCP to address you have used to register on this forum.
Advertisements

You can post new topics in this forum






Search Site

What is WinSCP?

It is award-winning SFTP client, SCP client, FTPS client and FTP client integrated into one software program for file transfer to FTP server or secure SFTP server. [More]

And it's free!

Donate

About donations

$9   $19   $49   $99

About donations

Recommend

WinSCP Privacy Policy

WinSCP License