Topic "Can I use a partially copied file?"

Author Message
[View user's profile]

Joined: 2006-09-28
Posts: 9
The setup I have is WinSCP is used to transfer XML files from a windows server to a UNIX folder location. I am using SSH-2 with a private key for the SFTP operation.
At the destination server there is a 'job' which processes new XML files in the folder and then moves them to a processed folder.

My questions are:
- can the XML processing job attempt to read a new XML file while it is still being copied via SFTP to the destination folder?
- and therefore there is a danger it will process only part of the file?
- Does the SFTP put process invoke some UNIX file lock (pure guess) so that no other server file/user can do anything with the file?
Thanks very much


The behavior I have observed is that the file-fragment's name is appended with .filepart until the transfer is complete, at which time the .filepart is removed and the destination file has its final name. This may only occur for files larger that 2k (if I remember correctly, the default minimum size for allowing transfer resumes). So for example, xmlfile20070517.xml would be saved at the destination as xmlfile20070517.xml.filepart until the transfer was complete, then renamed to xmlfile20070517.xml. XML files are probably in some cases smaller than 2k, so it's possible your monitoring script could grab the incomplete file. The easiest workaround, I imagine, would be to have the monitoring application wait X seconds after detecting a new file before beginning to process it. (You would naturally choose X to be large enough to allow the transfer to complete in whatever worst-case scenario you imagine.)

OpenSSH doesn't do this to my knowledge, but other SSH servers offer "trigger" functionality that will perform an action on logout, or after the transfer of a file completes.

You can post new topics in this forum


What is WinSCP?

It is award-winning SFTP client, SCP client, FTPS client and FTP client integrated into one software program for file transfer to FTP server or secure SFTP server. [More]

And it's free!


About donations

$9   $19   $49   $99

About donations


WinSCP Privacy Policy

WinSCP License