Documentation » Features » Scripting » Useful Scripts »

Locking files while uploading / Upload to temporary file name

You may have an automated system monitoring a remote folder and you want to prevent it from accidentally picking a file that has not finished uploading yet. As majority of SFTP and FTP servers (WebDAV being an exception) do not support file locking, you need to prevent the automated system from picking the file otherwise.

Common workarounds are:

Upload “done” file once an upload of data files finishes and have the automated system wait for the “done” file before processing the data files. This is easy solution, but won’t work in multi-user environment.
Upload data files to temporary (“upload”) folder and move them atomically to target folder once the upload finishes.
Upload data files to distinct temporary name, e.g. with .filepart extension, and rename them atomically once the upload finishes. Have the automated system ignore the .filepart files.

- A gross hack is to periodically check for file attributes (size and time) and consider the upload finished, if the attributes has not changed for some time interval.

Here we focus on the third approach (although the second is very similar, implementation-wise).

Using Transfer to temporary filename feature
Moving/renaming uploaded files when upload finishes
Further Reading

Using Transfer to temporary filename feature

With SFTP protocol, you can use Transfer to temporary filename feature to have WinSCP handle the rename automatically for you.

In scripting, use:

put -resumesupport=on d:\toupload\*.txt /home/martin/upload/

With WinSCP .NET assembly, use:

$transferOptions = New-Object WinSCP.TransferOptions
$transferOptions.ResumeSupport.State = [WinSCP.TransferResumeSupportState]::On
$session.PutFiles("d:\toupload\*.txt", "/home/martin/upload/", $False, $transferOptions).Check()

Moving/renaming uploaded files when upload finishes

If you need to use a different name pattern or a protocol different from SFTP or you want to take the second approach, you need to code the rename/move.

Following example shows implementation of the third approach using WinSCP .NET assembly in PowerShell:

param (
    $localPath = "c:\toupload\",
    $remotePath = "/home/user/upload/"
)
 
try
{
    # Load WinSCP .NET assembly
    Add-Type -Path "WinSCPnet.dll"
 
    # Setup session options
    $sessionOptions = New-Object WinSCP.SessionOptions -Property @{
        Protocol = [WinSCP.Protocol]::Sftp
        HostName = "example.com"
        UserName = "user"
        Password = "mypassword"
        SshHostKeyFingerprint = "ssh-rsa 2048 xxxxxxxxxxx..."
    }
 
    $session = New-Object WinSCP.Session
 
    try
    {
        # Connect
        $session.Open($sessionOptions)
 
        # Deliberately using an underscore instead of a dot,
        # as the dot has specific meaning in operation mask
        $suffix = "_filepart"
 
        $transferOptions = New-Object WinSCP.TransferOptions
        # Particularly with SFTP protocol, prevent additional .filepart suffix
        # from being added to uploaded files larger than 100 KB
        $transferOptions.ResumeSupport.State = [WinSCP.TransferResumeSupportState]::Off
 
        # Upload all .txt files with temporary "_filepart" suffix
        $transferResult =
            $session.PutFiles(($localPath + "*.txt"), ($remotePath + "*.*" + $suffix),
                $False, $transferOptions)
 
        # Throw on any error
        $transferResult.Check()
 
        # Rename uploaded files
        foreach ($transfer in $transferResult.Transfers)
        {
            # Remove suffix
            $finalName =
                $transfer.Destination.SubString(
                    0, $transfer.Destination.Length - $suffix.Length)
            Write-Host "Renaming uploaded file $($transfer.Destination) to $finalName"
            # Rename uploaded file to its final name
            $session.MoveFile($transfer.Destination, $finalName)
        }
    }
    finally
    {
        # Disconnect, clean up
        $session.Dispose()
    }
 
    exit 0
}
catch
{
    Write-Host "Error: $($_.Exception.Message)"
    exit 1
}