Slow getting files from SFTP

Advertisement

glowe
Joined:
Posts:
1
Location:
East Islip, NY

Slow getting files from SFTP

When I put a file that is 319MB it takes 10 seconds however when I try to get the file using either GetFile or GetFileToDirectory it takes over 44 minutes for a file that is 197MB. I have tried disabling the Optimize connection buffer size. I have attached the logs and debug logs as well as the pertinent code to assist.

WinSCP Version 5.21.7

function Get_File_from_SFTP {
    SW 1
    try
    {
        # Load WinSCP .NET assembly
        Add-Type -Path (join-path ${env:ProgramFiles(x86)} "WinSCP\WinSCPnet.dll")
        $cfg=get-config $env
        # Setup session options
        $sessionOptions = New-Object WinSCP.SessionOptions -Property @{
            Protocol = [WinSCP.Protocol]::Sftp
            HostName = $cfg.srvprt
            UserName = $cfg.user
            Password = $cfg.token
        }
        $session = New-Object WinSCP.Session
        $sessionLogPathWinSCP =  "P:\Adstra_Appends\1477-AYR\130113\log.txt"
        $session.SessionLogPath = $sessionLogPathWinSCP
        $winSCPSessionLogLocation = "WinSCP Session Log Location: $sessionLogPathWinSCP"
 
        $debugLogPathWinSCP = "P:\Adstra_Appends\1477-AYR\130113\debug.txt"
        $session.DebugLogPath = $debugLogPathWinSCP
        $session.DebugLogLevel = 2
         
        $winSCPDebugLogLocation = "WinSCP Debug Log Location: $debugLogPathWinSCP"
 
        $sessionOptions.SshHostKeyFingerprint=$session.ScanFingerprint($sessionOptions, "SHA-256")
        $sessionOptions.AddRawSettings("SendBuf","0")
        $sessionOptions.AddRawSettings("SshSimple","0")
 
        $rmtdir="/ccftpDAVE/adstra/fromAdstra/"+$(split-path $OFName -Leaf)
        $lcldir=$OFName #split-path $OFName -Parent
        try
        {
            # Connect
            $session.Open($sessionOptions)
 
            $transferOptions = New-Object WinSCP.TransferOptions
            $transferOptions.TransferMode = [WinSCP.TransferMode]::ASCII
            # $transferOptions.SpeedLimit = 5120
 
            LogIt $OutputBox Black "GetFiles( $rmtdir, $lcldir, $false, $transferOptions )"
            Logit $OutputBox Black "Transfer Mode`t: $($transferOptions.Transfermode)"
            Logit $OutputBox Black "Overwrite Mode`t: $($transferOptions.Overwritemode)"
            $transferResult = $session.GetFiles($rmtdir, $lcldir, $False, $transferOptions)
            #$transferResult = $session.GetFileToDirectory( $rmtdir, $lcldir, $false. $transferOptions )
            #$session.GetFilesToDirectory( $rmtdir, $lcldir, "$pairName*.*", $False, $transferOptions)
            # Throw on any error
            LogIt $OutputBox Black "`nFile Name`t`t: $($transferResult.filename)"
            Logit $OutputBox Black "Length`t`t: $($transferResult.length)"
            Logit $OutputBox Black "Destination`t: $($transferResult.destination)"
            Logit $OutputBox Black "Error`t`t: $($transferResult.Error)"
 
            $transferResult.Check()
            # Print results
            if($transferResult.error -eq $null) {
                LogIt $OutputBox green "Upload succeeded"
            }
        }
        finally
        {
            # Disconnect, clean up
            $session.Dispose()
        }
    }
    catch
    {
        LogIt $OutputBox Red "Error: $($_.Exception.Message)"
        Return $false
    }
    SW 0
    Return $true
}
  • Log_Part1.ZIP (9.9 MB, Private file)
  • debug.ZIP (259.5 KB, Private file)
  • 1477_130113_AYR_FOR_ADSTRA_202304170122.txt (1.89 KB, Private file)

Reply with quote E-mail

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
41,518
Location:
Prague, Czechia

Re: Slow getting files from SFTP

That's likely because of
$session.DebugLogLevel = 2
https://winscp.net/eng/docs/faq_slow#logging

Side note: Make sure you understand the consequences of
$sessionOptions.SshHostKeyFingerprint =
    $session.ScanFingerprint($sessionOptions, "SHA-256")
That's basically an ineffective way to achieve the same insecure connection that you get with
$sessionOptions.SshHostKeyPolicy =
    [WinSCP.SshHostKeyPolicy]::GiveUpSecurityAndAcceptAny

Reply with quote

Advertisement

You can post new topics in this forum