Upload multiple files from source directory through PowerShell

Advertisement

jeroenbudding
Joined:
Posts:
2
Location:
Netherlands

Upload multiple files from source directory through PowerShell

So the problem is that I have 40 files in a directory. Add the moment if I use my Script which I will post below it will upload all the files but it takes ages.
So what I want is that the script downloads 6 or 8 files add the same time. Is this possible and if so how can I do it.
#Upload data to LivingStone SFTP
try
{
    # Load WinSCP .NET assembly Rember The folder must contain winscpnet.dl and WinSCP.exe
    Add-Type -Path "C:\Powershell\LivingStone_SFTP\WinSCPnet.dll"
 
    # Setup session options
    $sessionOptions = New-Object WinSCP.SessionOptions
    $sessionOptions.Protocol = [WinSCP.Protocol]::Sftp
    $sessionOptions.HostName = 
    $sessionOptions.UserName = 
    $sessionOptions.Password = 
    $sessionOptions.SshHostKeyFingerprint = 
 
    $session = New-Object WinSCP.Session
 
    try
    {
        # Connect
        $session.Open($sessionOptions)
 
        # Upload files
        $transferOptions = New-Object WinSCP.TransferOptions
        $transferOptions.TransferMode = [WinSCP.TransferMode]::Binary
        $TransferOptions.PreserveTimestamp = $false
        $TransferOptions.ResumeSupport.State = [WinSCP.TransferResumeSupportState]::on
        
        #here we define source and destination
        $transferResult = $session.PutFiles("D:\Backups\SCCM\*", "/inbox/", $False, $transferOptions) 
 
        # Throw on any error        
        $transferResult.Check()
 
        # Print results
        foreach ($transfer in $transferResult.Transfers)
        {
            Write-Host ("Upload of {0} succeeded" -f $transfer.FileName)
        }
    }
    finally
    {
        # Disconnect, clean up
        $session.Dispose()
    }
 
    exit 0
}
catch [Exception]
{
    Write-Host $_.Exception.Message
    exit 1
}

Reply with quote

Advertisement

jeroenbudding
Joined:
Posts:
2
Location:
Netherlands

Upload multiple files from source directory through Powershell

@martin: thanks for your help in the good direction only that script is downloading not uploading.
I adjusted the script to do:
$files = Get-ChildItem $localPath | Sort-Object Length -Descending
instead of
$files =
    $session.ListDirectory($remotePath).Files |
    Where-Object { -Not $_.IsDirectory } |
    Sort-Object Length -Descending
but still get the following error when I run the script:
Downloading C:\Powershell\LivingStone_SFTP\Test\4.exe to /inbox/4.exe in 2
Can't get attributes of file 'C:\Powershell\LivingStone_SFTP\Test\4.exe'.

Reply with quote

R1d3rBul
Joined:
Posts:
3
Location:
BG

Re: Upload multiple files from source directory through Powershell

Great script, but I can't get it to run properly because I'm getting one error:
Cannot bind argument to parameter 'Path' because it is an empty string.
Could you please help?

Reply with quote E-mail

Advertisement

R1d3rBul
Joined:
Posts:
3
Location:
BG

Re: Upload multiple files from source directory through Powershell

Hello Martin,

Thank you very much for the prompt reaction. Actually I managed to run the script. I'm using it for upload and just wonder if somehow I can set an email notification if the files are sent or not.
param ( 
    $sessionUrl = "ftp://user:password@ftp.example.com/",
    $remotePath = "/myremotepath/test", 
    $localPath = "D:\mylocalpath\test",
    $backuppath = "D:\myarchive\",
    $batches = 2
)
 
#Upload data to ftp.example.com
try 
{ 
    # Load WinSCP .NET assembly 
    $dllPath = (Join-Path -Path "C:\Program Files (x86)\WinSCP\" "WinSCPnet.dll")
    # Load WinSCP .NET assembly
    Add-Type -Path $dllPath
 
    # Setup session options
    $sessionOptions = New-Object WinSCP.SessionOptions
    $sessionOptions.ParseUrl($sessionUrl)
 
    $started = Get-Date
 
    try
    {
        # Connect
        Write-Output "$(Get-Date) Connecting..."
        $session = New-Object WinSCP.Session
        $session.Open($sessionOptions)
        
        # Retrieve list of files and sort them from larges to smallest
        $files = Get-ChildItem $localPath | Sort-Object Length -Descending | Where-Object Name -like 'myfile*'
       
        $listfiles = $files | Sort-Object Length,Name
        # Calculate total size of all files
        $total = ($files | Measure-Object -Property Length -Sum).Sum
        
        # And batch size
        $batch = [int]($total / $batches)
 
        Write-Output ("Will upload {0} files totaling {1} bytes in {2} parallel batches, {3} bytes on average in each" -f $files.Count, $total, $batches, $batch)
        
        $start = 0
        $sum = 0
        $no = 0
 
        for ($i = 0; $i -lt $files.Count; $i++)
        {
            $sum += $files[$i].Length
 
            # Found enough files for the next batch
            if (($sum -ge $batch) -or ($i -eq $files.Count - 1))
            {
                Write-Output ("Starting batch {0} to upload {1} files totaling {2}" -f $no, ($i - $start + 1), $sum)
                
                $fileList = $files[$start..$i] -join ";"
                
                # Start the background job for the batch
                Start-Job -Name "Batch $no" -ArgumentList $dllPath, $sessionUrl, $localPath, $remotePath, $backuppath, $no, $fileList {
                    param (
                        [Parameter(Position = 0)]
                        $dllPath,
                        [Parameter(Position = 1)]
                        $sessionUrl,
                        [Parameter(Position = 2)]
                        $localPath,
                        [Parameter(Position = 3)]
                        $remotePath,
                        [Parameter(Position = 4)]
                        $backuppath,
                        [Parameter(Position = 5)]
                        $no,
                        [Parameter(Position = 6)]
                        $fileList
                    )
 
                    try
                    {
                        Write-Output ("Starting batch {0}" -f $no)
 
                        # Load WinSCP .NET assembly.
                        # Need to use an absolute path as the Job is started from user's documents folder.
                        Add-Type -Path $dllPath
 
                        # Setup session options
                        $sessionOptions = New-Object WinSCP.SessionOptions
                        $sessionOptions.ParseUrl($sessionUrl)
                        
                        try
                        {
                            Write-Output ("Connecting batch {0}..." -f $no)
                            $session = New-Object WinSCP.Session
 
                            $session.Open($sessionOptions)
                            
                            $files = $fileList -split ";"
 
                            # Upload the files selected for this batch
                            foreach ($file in $files)
                            {
                                $localFilePath = "$localPath\$file"
                                $remoteFilePath = "$remotePath/$file"
                                Write-Output "Uploading $localFilePath to $remoteFilePath in $no and moving to archive"
 
                                $session.PutFiles($session.EscapeFileMask($localFilePath), $remoteFilePath).Check()
                                Move-Item "$localPath\$file" $backuppath
                            }
                        }
                        finally
                        {
                            # Disconnect, clean up
                            $session.Dispose()
                        }
                        
                        Write-Output ("Batch {0} done" -f $no)
                             
                    }
                    catch [Exception]
                    {
                        Write-Output $_.Exception.Message
                  #I want to send a notification if some error appears I recived two emails, because of the batches. I'm not sure where to put it to send only one. Can you assist?
                  $From = "mylaptop@mail.com"
                  $To = "R1d3rbul@mail.com"
                  $Subject = "Files were NOT sent!"
                  $Body = "<h2> Please check the log</h2><br>"
                  $Body += "$_"
                  $SMTPServer = "mail.examble.com"
                  $SMTPPort = "25"
                  Send-MailMessage -From $From -to $To -Subject $Subject -Body $Body -BodyAsHtml -SmtpServer $SMTPServer -Port $SMTPPort -UseSsl
                  #Email notification
                        exit 1
                    }
                } | Out-Null
 
                # Reset for the next batch
                $no++
                $sum = 0
                $start = $i + 1
            }
        }
 
        Write-Output "Waiting for batches to complete"
        Get-Job | Receive-Job -Wait
 
        Write-Output "Done"
 
        $ended = Get-Date
        Write-Output ("$(Get-Date) Took {0} to complete" -f (New-TimeSpan -Start $started -End $ended))
    }
    finally
    {
        # Disconnect, clean up
        $session.Dispose()
 
    }
      
    exit 0
      
}
catch [Exception]
{
    Write-Output $_.Exception.Message
    #if I put the email sending here its not working.
    exit 1                               
}

This is what I'm using and its working, but I don't have much experience with scripting so any help would be appreciated.
What I want to achieve is to send emails when the files are sent normally and another one if some errors appears. Can you assist? Also would like to add some checking if the files are transferred correctly.

Reply with quote E-mail

martin
Site Admin
martin avatar
Joined:
Posts:
40,476
Location:
Prague, Czechia

Re: Upload multiple files from source directory through Powershell

R1d3rBul wrote:

if I put the email sending here its not working
You didn't show us what you have tried, so we cannot tell you what you are doing wrong.
All we can do atm is to suggest you to use Send-MailMessage.
Also, sending an email from PowerShell is not really a WinSCP question.

Reply with quote

Advertisement

You can post new topics in this forum