Can I use a partially copied file?

Advertisement

dmalesd
Joined:
Posts:
9

Can I use a partially copied file?

Hi,
The setup I have is WinSCP is used to transfer XML files from a windows server to a UNIX folder location. I am using SSH-2 with a private key for the SFTP operation.
At the destination server there is a 'job' which processes new XML files in the folder and then moves them to a processed folder.

My questions are:
- can the XML processing job attempt to read a new XML file while it is still being copied via SFTP to the destination folder?
- and therefore there is a danger it will process only part of the file?
- Does the SFTP put process invoke some UNIX file lock (pure guess) so that no other server file/user can do anything with the file?
Thanks very much
David

Reply with quote E-mail

Advertisement

TS
Guest

The behavior I have observed is that the file-fragment's name is appended with .filepart until the transfer is complete, at which time the .filepart is removed and the destination file has its final name. This may only occur for files larger that 2k (if I remember correctly, the default minimum size for allowing transfer resumes). So for example, xmlfile20070517.xml would be saved at the destination as xmlfile20070517.xml.filepart until the transfer was complete, then renamed to xmlfile20070517.xml. XML files are probably in some cases smaller than 2k, so it's possible your monitoring script could grab the incomplete file. The easiest workaround, I imagine, would be to have the monitoring application wait X seconds after detecting a new file before beginning to process it. (You would naturally choose X to be large enough to allow the transfer to complete in whatever worst-case scenario you imagine.)

OpenSSH doesn't do this to my knowledge, but other SSH servers offer "trigger" functionality that will perform an action on logout, or after the transfer of a file completes.

Reply with quote

Advertisement

You can post new topics in this forum