EnumerateRemoteFiles hangs indefinitely
Hello,
I am using WinSCP .NET Assembly in Powershell to download files from an SFTP server. I am using $session.EnumerateRemoteFiles to get a list of files from a directory with about 12,000 very small files (1-2kb json files), and it appears to hang forever. Looking at the session log, I see the following.
In the debug log, around that time, it appears to dump the xml log and then continue logging out the following over and over repeatedly.
As far as I can tell from the session log, it appears to be listing all the files and then hanging before returning the results to Powershell.
Here's a snippet of my code where this is hanging.
I have attached the session log. The debug log is too large to upload here, even compressed (~12MB compressed).
Thanks!
I am using WinSCP .NET Assembly in Powershell to download files from an SFTP server. I am using $session.EnumerateRemoteFiles to get a list of files from a directory with about 12,000 very small files (1-2kb json files), and it appears to hang forever. Looking at the session log, I see the following.
. 2020-10-29 16:21:30.499 Session upkeep . 2020-10-29 16:21:30.499 Looking for network events . 2020-10-29 16:21:30.499 Timeout waiting for network events
[2020-10-29 16:24:19.861] [0011] Log contents has not changed [2020-10-29 16:24:19.862] [0011] SessionLogReader.Read leaving [2020-10-29 16:24:19.862] [0011] Session.ReadFile entering [2020-10-29 16:24:19.862] [0011] Session.ReadFile leaving [2020-10-29 16:24:19.863] [0011] SessionLogReader.Read entering [2020-10-29 16:24:19.863] [0011] Read node 1587: Element group [2020-10-29 16:24:19.982] [0011] Log contents has not changed [2020-10-29 16:24:19.983] [0011] SessionLogReader.Read leaving [2020-10-29 16:24:19.983] [0011] Session.ReadFile entering [2020-10-29 16:24:19.983] [0011] Session.ReadFile leaving [2020-10-29 16:24:19.984] [0011] SessionLogReader.Read entering [2020-10-29 16:24:19.984] [0011] Read node 1588: Whitespace
Here's a snippet of my code where this is hanging.
$winSCPLocation = Clean-FolderPath $winSCPLocation "\" -noStart Add-Type -Path $winSCPLocation try { # Setup session options $sessionOptions = New-Object WinSCP.SessionOptions -Property @{ Protocol = [WinSCP.Protocol]::"sftp" HostName = $sourceAddress UserName = $sourceUser PortNumber = $sourcePort GiveUpSecurityAndAcceptAnySshHostKey = 1 } $sessionOptions.SshPrivateKeyPath = "$sourceKeyDirectory" #Set Key Exchange Algorithm Selection Policy $sessionOptions.AddRawSettings("KEX", "ecdh,dh-gex-sha1,dh-group14-sha1,rsa,dh-group1-sha1,WARN") #Set Optimize Connection Buffer Size to off (0) and set SshSimple to 1 to completely disable optimzation of connection buffer size $sessionOptions.AddRawSettings("SendBuf",0) $sessionOptions.AddRawSettings("SshSimple",1) $session = New-Object WinSCP.Session if ($winSCPSessionLogEnabled) { $sessionLogPathWinSCP = "$logDirectory\WinSCPSession_$fileDate.log" $session.SessionLogPath = $sessionLogPathWinSCP $winSCPSessionLogLocation = "WinSCP Session Log Location: $sessionLogPathWinSCP" } else { $winSCPSessionLogLocation = "" } if ($winSCPDebugLogEnabled) { $debugLogPathWinSCP = "$logDirectory\WinSCPDebug_$fileDate.log" $session.DebugLogPath = $debugLogPathWinSCP $session.DebugLogLevel = 2 $winSCPDebugLogLocation = "WinSCP Debug Log Location: $debugLogPathWinSCP" } else { $winSCPSessionLogLocation = "" } } catch { log -statement "Error: $($_.Exception.Message)" RED exit 1 } log -statement "Attempting connection for sftp" GREEN $session.Open($sessionOptions) log -statement "sftp Connection Established - Still Connected" GREEN $fileList = $session.EnumerateRemoteFiles("/home/test","*.json", [WinSCP.EnumerationOptions]::None)
Thanks!
