Post a reply

Options
Add an Attachment

If you do not want to add an Attachment to your Post, please leave the Fields blank.

(maximum 10 MB; please compress large files; only common media, archive, text and programming file formats are allowed)

Options

Topic review

martin

Re: Exception calling "Open" with "1" argument(s): "WinSCP process terminated with exit code 3.

The system can be overloaded, so it refuses to start yet another process.
Did you consider preventing running many tasks in parallel by scheduling them differently?
blie

Exception calling "Open" with "1" argument(s): "WinSCP process terminated with exit code 3.

Hello,

First of all I want to describe our environment a little bit. In our environment we have a lot of external FTP sites where we get files from, most of them we poll the remote FTP servers every 5 / 10 minutes to see if there are new files available. I really hope anyone can help us out on this issue.

When we started this, we have created a couple of scripts in order to meet our demands (differs per interface). We now have around 80 scheduled tasks running (some start every 5 minutes and some start every hour) which polls remote servers to see if there are new files (mostly we are using the script that places all the filenames in a .txt to download only unique files).

Our scheduled tasks are run under the SYSTEM account and don't have highest privileges.

When we implemented around 20/30 scheduled tasks everything went perfect. Now we are at 80 we are noticing in our monitoring logs that we encounter some issue's:

This is a job that runs every 5 minutes. Sometimes when it runs it generates an error and ofcourse we cannot find the tempfile since it doesn't exist. Mostly after a couple of retry's it works, but we don't start these jobs for fun every 5 minutes.


2020-09-08-20-05-03,OK,0,JOB-72
2020-09-08-20-10-03,OK,0,JOB-72
2020-09-08-20-20-02,OK,0,JOB-72
2020-09-08-20-25-03,OK,0,JOB-72
2020-09-08-20-30-03,OK,0,JOB-72
2020-09-08-20-35-03,Error: Exception calling "Open" with "1" argument(s): "WinSCP process terminated with exit code 3. There was no output. Response log file C:\Windows\TEMP\wscp45D8.039CFF13.tmp was not created. This could indicate lack of write permissions to the log folder or problems starting WinSCP itself.",,JOB-72
2020-09-08-20-40-03,Error: Exception calling "Open" with "1" argument(s): "WinSCP process terminated with exit code 3. There was no output. Response log file C:\Windows\TEMP\wscp5124.01A453AA.tmp was not created. This could indicate lack of write permissions to the log folder or problems starting WinSCP itself.",,JOB-72
2020-09-08-20-45-04,Error: Exception calling "Open" with "1" argument(s): "WinSCP process terminated with exit code 3. There was no output. Response log file C:\Windows\TEMP\wscp2080.01A453AA.tmp was not created. This could indicate lack of write permissions to the log folder or problems starting WinSCP itself.",,JOB-72
2020-09-08-20-50-03,Error: Exception calling "Open" with "1" argument(s): "WinSCP process terminated with exit code 3. There was no output. Response log file C:\Windows\TEMP\wscp4D6C.01A453AA.tmp was not created. This could indicate lack of write permissions to the log folder or problems starting WinSCP itself.",,JOB-72
2020-09-08-20-55-04,OK,0,JOB-72
2020-09-08-21-05-04,Error: Exception calling "Open" with "1" argument(s): "WinSCP process terminated with exit code 3. There was no output. Response log file C:\Windows\TEMP\wscp3504.01A453AA.tmp was not created. This could indicate lack of write permissions to the log folder or problems starting WinSCP itself.",,JOB-72
2020-09-08-21-10-04,OK,0,JOB-72
2020-09-08-21-15-03,Error: Exception calling "Open" with "1" argument(s): "WinSCP process terminated with exit code 3. There was no output. Response log file C:\Windows\TEMP\wscp4CC4.01F4CB52.tmp was not created. This could indicate lack of write permissions to the log folder or problems starting WinSCP itself.",,JOB-72
2020-09-08-21-20-04,OK,0,JOB-72


Also we have implemented:
$sessionLogPath = "D:\Jobs\SessionLogs\JOB-72.log"
$session.SessionLogPath = $sessionLogPath


But when we receive an error in our monitoring logfile we don't see any new entry's in the sessionlogpath. So I don't have any more detailed logging.

The script is here below

param (
    # Use Generate Session URL function to obtain a value for -sessionUrl parameter.
    [Parameter(Mandatory = $True)]
    $sessionUrl,
    [Parameter(Mandatory = $True)]
    $localPath,
    [Parameter(Mandatory = $True)]
    $remotePath,
    [Parameter(Mandatory = $True)]
    $jobName,
    [Parameter(Mandatory = $True)]
    $FileMask,
    [Parameter(Mandatory = $True)]
    $wildcard
)
 
#Pre-script variables
$username = ""
$pwdTxt = Get-Content "*****" | Select -First 1
$securePwd = $pwdTxt | ConvertTo-SecureString
$credential = New-Object System.Management.Automation.PSCredential -ArgumentList $username, $securePwd
try
{
    #Set assemblypath
    $assemblyPath = if ($env:WINSCP_PATH) { $env:WINSCP_PATH } else { "D:\FTPSync\Scripts" }
    Add-Type -Path (Join-Path $assemblyPath "WinSCPnet.dll")
    #Session Opions
    $sessionOptions = New-Object WinSCP.SessionOptions
    $sessionOptions.ParseUrl($sessionUrl)
    $sessionOptions.GiveUpSecurityAndAcceptAnySshHostKey = "true"
 
    $session = New-Object WinSCP.Session
   
    $transferOptions = New-Object WinSCP.TransferOptions
    $transferOptions.ResumeSupport.State = [WinSCP.TransferResumeSupportState]::Off
    $transferOptions.TransferMode = [WinSCP.TransferMode]::Binary
    $transferOptions.FileMask = $FileMask
    $date = Get-Date
    $fulldate = $date.ToString("yyyy-MM-dd-HH-mm-ss")
    $listPath = "D:\FTPSync\Unique\$jobname.log"
    $monitoringPath = "D:\FTPSync\Monitoring\$jobname.log"
    $scriptresultPath = "D:\FTPSync\ScriptResults\$jobname.log"
    $sessionLogPath = "D:\FTPSync\SessionLogs\$jobname.log"
    try
    {
        New-PSDrive -Name fileserver -Root $localPath -Credential $credential -PSProvider FileSystem
        $session.SessionLogPath = $sessionLogPath
        $listPath = [Environment]::ExpandEnvironmentVariables($listPath)
        $listDir = (Split-Path -Parent $listPath)
        if (Test-Path $listPath)
        {
            Write-Host "$fulldate Loading list of already downloaded files from $listPath..."
            Add-Content -Path "$scriptresultPath" "$fulldate Loading list of already downloaded files from $listPath..."
            $downloaded = @(Get-Content $listPath)
        }
        else
        {
            Write-Host "File $listPath with list of already downloaded files doesn't exist yet."
            Add-Content -Path "$scriptresultPath" "$fulldate File $listPath with list of already downloaded files doesn't exist yet."
            $downloaded = @()
        }
 
        Write-Host "Connecting..."
        Add-Content -Path "$scriptresultPath" "$fulldate Connecting to $jobname"
        $session.Open($sessionOptions)
 
        Write-Host "Looking for new files..."
        Add-Content -Path "$scriptresultPath" "$fulldate Looking for new files for $jobname"
        $files =
            $session.EnumerateRemoteFiles(
                $remotePath, $wildcard, [WinSCP.EnumerationOptions]::None)
        $count = 0
        foreach ($fileInfo in $files)
        {
            $remoteFilePath = $fileInfo.FullName
            if ($downloaded -notcontains $remoteFilePath)
            {
                $remoteFileLen = $fileInfo.Length
                Write-Host `
                    "Found new file $remoteFilePath with size $remoteFileLen, downloading..."
                Add-Content -Path "$scriptresultPath" "$fulldate Found new file $remoteFilePath with size $remoteFileLen, downloading... for $jobname"
                New-PSDrive -Name fileserver9 -Root $localPath -Credential $credential -PSProvider FileSystem
                $localFilePath =
                    [WinSCP.RemotePath]::TranslateRemotePathToLocal(
                        $remoteFilePath, $remotePath, $localPath)
                $localFileDir = (Split-Path -Parent $localFilePath)
                $source = [WinSCP.RemotePath]::EscapeFileMask($remoteFilePath)
                $session.GetFiles($source, $localFilePath, $False, $transferOptions).Check()
                Add-Content $listPath $remoteFilePath
                $count++
                Write-Host "Downloaded."
                Add-Content -Path "$scriptresultPath" "$fulldate Downloaded. for $jobname"
                #}
            }
        }
 
        if ($count -gt 0)
        {
            Write-Host "Done, downloaded $count files."
            Add-Content -Path "$monitoringPath" "$fulldate,OK,$count,$jobname"
        }
        else
        {
            Write-Host "Done, no new files found."
            Add-Content -Path "$monitoringPath" "$fulldate,OK,$count,$jobname"
        }
    }
    finally
    {
        # Disconnect, clean up
        $session.Dispose()
    }
 
    $result = 0
}
catch
{
    Write-Host "Error: $($_.Exception.Message)"
    Add-Content -Path "$monitoringPath" "$fulldate,Error: $($_.Exception.Message),$count,$jobname"
    $result = 1
}
Exit $result