Post a reply

Options
Add an Attachment

If you do not want to add an Attachment to your Post, please leave the Fields blank.

(maximum 10 MB; please compress large files; only common media, archive, text and programming file formats are allowed)

Options

Topic review

Guest

Re: Provided successful and failed debug logs

Ah! I missed that there was another command. Below is the altered section of the powershell script.
# Set up session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property @{
    Protocol = [WinSCP.Protocol]::Sftp
    HostName = $Config.Configuration.HostIP
    UserName = $Config.Configuration.UserName
    Password = $Config.Configuration.password
    PrivateKeyPassphrase = $Config.Configuration.PassPhrase
    SshHostKeyFingerprint = $Config.Configuration.FingerPrint
    SshPrivateKeyPath = $Config.Configuration.SshPrivateKeyPath
    Timeout = New-TimeSpan -Minutes 10
}
 
$session = New-Object WinSCP.Session
$session.SessionLogPath = "$LogFilePath\${yyyyMMdd}_SFTP_MIS_Daily_Session.log"
$session.DebugLogLevel = 2
$session.DebugLogPath = "$LogFilePath\${yyyyMMdd}_SFTP_MIS_Daily_Debug.log"
$session.Timeout = New-TimeSpan -Minutes 10

Unless there is any other features to add resilience, I'm thinking I will have to keep an eye on it until we deal with the overloading issue.
Thank you for your help!
martin

Re: Provided successful and failed debug logs

In your script, you are setting SessionOption.Timeout.
I was suggesting you to set Session.Timeout:
$session.Timeout = New-TimeSpan -Minutes 10
Autocp

Re: Provided successful and failed debug logs

In the PowerShell script, I put
Timeout = New-TimeSpan -Minutes 10

This was to set the timeout to 10 minutes. May you confirm that I did that correctly? I am having trouble finding examples to make sure.

I suspect the failure is being caused by overload.
Any other tips to make this more resilient? Unfortunately, I am sharing the local server with another IT vendor so the processing may still be an issue going forward.
martin

Re: Provided successful and failed debug logs

Isn't the machine overloaded, when WinSCP fails?
Even the success log shows that it takes 10 seconds for WinSCP to start doing anything.
And the failure log shows much higher CPU use.
Did you try setting Session.Timeout to more than the default 60 seconds?
https://winscp.net/eng/docs/library_session#timeout
Autocp

Provided successful and failed debug logs

Thank you for your response!
I realized the debug file I uploaded before was for a slightly different process. So I have updated the debug logs, making sure it is consistent with the PowerShell code and transcript I have provided. I have also provided a successful debug log for comparison.
martin

Re: Exception: No handle of the given name exists.

Please post debug log for a run that does work.
Autocp

Exception: No handle of the given name exists.

This code runs successfully every day, except Saturdays:
param(
#### CONFIGURE FILEPATHS ####
    $xmlConfig = "C:\Batch\SFTP_Automation\Config.xml",
    $unrar_path = "C:\Users\USERNAME\AppData\Local\Programs\WinRAR\unrar",
    $RemoteFilePath = "/Daily",
    $LocalFilePath = "C:\mis",
    $LogFilePath = "C:\Batch\SFTP_Automation\Daily\Reports\Logs",
    $DailyExtracts = "C:\MIS\DailyExtracts",
#### DYNAMIC NAMING CONVENTIONS ####
    ${ddMMMyyyy} = (Get-Date).ToString("ddMMMyyyy"),
    ${yyyyMMdd} = (Get-Date).ToString("yyyyMMdd"),
    $FILENAME_Folder = (Get-Date).ToString("\MIS_DAILY_ddMMMyyyy"),
    $FILENAME_File = (Get-Date).adddays(-1).ToString("\MIS_dd")
)
 
#Record PowerShell Transcript
Start-Transcript -Path "$LogFilePath\${yyyyMMdd}_SFTP_MIS_Daily_Transcript.log"
[xml]$Config = Get-Content $xmlConfig
# Load WinSCP .NET assembly
Add-Type -Path "C:\Program Files (x86)\WinSCP\WinSCPnet.dll"
 
# Set up session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property @{
    Protocol = [WinSCP.Protocol]::Sftp
    HostName = $Config.Configuration.HostIP
    UserName = $Config.Configuration.UserName
    Password = $Config.Configuration.password
    PrivateKeyPassphrase = $Config.Configuration.PassPhrase
    SshHostKeyFingerprint = $Config.Configuration.FingerPrint
    SshPrivateKeyPath = $Config.Configuration.SshPrivateKeyPath
    Timeout = New-TimeSpan -Minutes 10
}
 
$session = New-Object WinSCP.Session
    $session.SessionLogPath = "$LogFilePath\${yyyyMMdd}_SFTP_MIS_Daily_Session.log"
    $session.DebugLogLevel = 2
    $session.DebugLogPath = "$LogFilePath\${yyyyMMdd}_SFTP_MIS_Daily_Debug.log"
 
try
{
    # Connect
    $session.Open($sessionOptions)
    $transferOptions = New-Object WinSCP.TransferOptions
    $session.GetFiles("$RemoteFilePath/MIS_*.exe","$LocalFilePath\MIS_DAILY_${ddMMMyyyy}\",$false,$transferOptions)
}
finally
{
    $session.Dispose()
}


Notice that I have set the timeout to 10 minutes in the $SessionOptions section. This was an attempt to address a timeout error that I am still getting from this transcript:

**********************

Windows PowerShell transcript start
Start time: 20231014051649
Username: DOMAIN_NAME\USERNAME
RunAs User: DOMAIN_NAME\USERNAME
Machine: A0208C03 (Microsoft Windows NT 10.0.14393.0)
Host Application: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe C:\Batch\SFTP_Automation\Daily\Reports\Daily_MIS.ps1
Process ID: 10136
PSVersion: 5.1.14393.5582
PSEdition: Desktop
PSCompatibleVersions: 1.0, 2.0, 3.0, 4.0, 5.0, 5.1.14393.5582
BuildVersion: 10.0.14393.5582
CLRVersion: 4.0.30319.42000
WSManStackVersion: 3.0
PSRemotingProtocolVersion: 2.3
SerializationVersion: 1.1.0.1
**********************
Transcript started, output file is C:\Batch\SFTP_Automation\Daily\Reports\Logs\20231014_SFTP_MIS_Daily_Transcript.log


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----        14Oct2023   5:18 AM                MIS_DAILY_14Oct2023
Exception calling "Open" with "1" argument(s): "Timeout waiting for WinSCP to respond - WinSCP has not responded in
time. There was no output. Response log file C:\Users\USERNAME\AppData\Local\Temp\wscp2798.03302225.tmp was not created.
This could indicate lack of write permissions to the log folder or problems starting WinSCP itself."
At C:\Batch\SFTP_Automation\Daily\Reports\Daily_MIS.ps1:45 char:5
+     $session.Open($sessionOptions)
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : TimeoutException
Exception calling "Open" with "1" argument(s): "Timeout waiting for WinSCP to respond - WinSCP
has not responded in time. There was no output. Response log file
C:\Users\USERNAME\AppData\Local\Temp\wscp2798.03302225.tmp was not created. This could indicate
lack of write permissions to the log folder or problems starting WinSCP itself."
At C:\Batch\SFTP_Automation\Daily\Reports\Daily_MIS.ps1:45 char:5
+     $session.Open($sessionOptions)
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : TimeoutException



PS>$global:?
True


I have attached a debug log. It looks like it is timing out when reaching out to my XML configuration file. The error shows:
[2023-10-14 09:09:15.071] [000d]   Timeout waiting for WinSCP to respond - asking for callstack

[2023-10-14 09:09:15.086] [000d]   ExeSessionProcess.RequestCallstack entering
[2023-10-14 09:09:41.150] [000d]     Exception: System.Threading.WaitHandleCannotBeOpenedException: No handle of the given name exists.

Exception: System.Threading.WaitHandleCannotBeOpenedException: No handle of the given name exists.

Has anyone had to deal with this before? I am not sure what to do next.

Thank you