Forum » Scripting / Automation »

Possible to Skip size calculating for SynchronizeDirectories

Advertisement

NottaSmartMan
Joined:
Posts:
43

Possible to Skip size calculating for SynchronizeDirectories

I have a PowerShell script that it taking a long time to calculate the size of a lot of file/folders. In waiting for the calculation to happen, my TCP connection times out. Below is a sample of when I was able to get it to work, however, when the calculation takes over 5 minutes.. it doesn't work: 
2024-08-26 12:31:03.035 Script: Synchronizing...
2024-08-26 12:35:58.076 Size of 7 local files/folders calculated as 1190960271608
WinSCP isn't timing out. I believe the timeout is happening at the firewall level, because WinSCP is not sending any data for 5 minutes, which I cannot control.

Is it possible to instruct WinSCP to not bother calculating the size of the files/folders?

Reply with quote

Advertisement

NottaSmartMan
Joined:
Posts:
43

Follow up: I tried adding $sessionOptions.AddRawSettings("Calculate", "0") to my script, but it didn't work.

It recognized the command (I think), but ignored it: 
> 2024-08-26 13:30:26.238 Script: open sftp://<removed> -hostkey="ssh-rsa 2048 <removed>" -timeout=15 -rawsettings Calculate="0"
> 2024-08-26 13:30:26.238 Script: Parameter: Calculate=0

Reply with quote

martin◆
Site Admin
martin avatar
Joined:
Posts:
41,517
Location:
Prague, Czechia

Re: Possible to Skip size calculating for SynchronizeDirectories

WinSCP should not calculate the size in [automated] scripting. What exactly are you doing? Please post your code and full log file.

Reply with quote

NottaSmartMan
Joined:
Posts:
43

This code is part of a much larger script, however I believe this line you are you interested in is this: 
$synchronizationResult = $session.SynchronizeDirectories([WinSCP.SynchronizationMode]::remote, $sourcedir, $remotedir , $true, $true, "time,size", $FoldersToExclude )
We are telling it to sync and use the time/size to determine if files need to be copied. The problem is that. It appears there's long moments of no tcp traffic between when the log says "Script: Synchronizing..." and anything after.

Often when the script errors out, the last line in the log is "Script: Synchronizing...".

Reply with quote

NottaSmartMan
Joined:
Posts:
43

If it helps, I set the debug level to 2 and here's the last few lines of the log (I removed the file names from here): 
. 2024-08-29 10:47:37.278 .;D;0;2024-07-12T13:29:10.000Z;4;"" [0];"" [0];rwxrwxrwx;1
. 2024-08-29 10:47:37.278 ..;D;0;2024-07-12T13:29:10.000Z;4;"" [0];"" [0];rwxrwxrwx;1
. 2024-08-29 10:47:37.278 Local file '\\<Removed>' [2024-08-27T14:20:00.013Z] [0] is new
. 2024-08-29 10:47:37.278 Local file '\\<Removed>' [2021-07-23T01:24:35.315Z] [0] is new
. 2024-08-29 10:47:37.278 Local file '\\<Removed>' [2021-04-18T03:20:21.934Z] [0] is new
. 2024-08-29 10:47:37.278 Local file '\\<Removed>' [2024-05-30T01:40:51.851Z] [0] is new
. 2024-08-29 10:47:37.278 Local file '\\<Removed>' [2024-06-21T11:05:51.870Z] [0] is new
. 2024-08-29 10:47:37.278 Local file '\\<Removed>' [2024-08-23T13:17:41.727Z] [0] is new
. 2024-08-29 10:47:37.278 Local file '\\<Removed>' [2021-09-02T10:30:31.776Z] [0] is new
. 2024-08-29 10:47:37.278 Local file '\\<Removed>' [2021-04-10T22:22:12.601Z] [0] is new
. 2024-08-29 10:47:37.278 Local file '\\<Removed>' [2024-08-29T11:00:08.512Z] [0] is new
< 2024-08-29 10:47:37.278 Script: Synchronizing...
Within the PowerShell window, this is the last output from my script:
Exception calling "SynchronizeDirectories" with "7" argument(s): "Timeout waiting for WinSCP to respond"
Session Closed - 08/29/2024 10:52:40

Reply with quote

Advertisement

NottaSmartMan
Joined:
Posts:
43

I've done more testing... I think WinSCP *may* be timing out because the session/process is not responding to... itself? I'm probably butchering terms here.

Here's an example of what I see in the logs after a recent change: 
< 2024-08-29 18:10:41.594 Script: Synchronizing...
. 2024-08-29 21:25:46.126 Synchronizing local directory '\\<removed>\' with remote directory '/<removed>/', params = 0x1403 (Delete, NoConfirmation, BySize, Mirror)
. 2024-08-29 21:25:46.126 Copying 1 files/directories to remote directory "/<removed>/" - total size: 1,734
You can see, it took 3 hours, 15 minutes and 5 seconds to go from 'Synchronizing...' the next line in the log.

I changed the $session.timeout line in my script to something crazy like 10 hours and it's working better. Is this the correct course of action or was there a better way to handle? Is there a max value for $session.timeout?

Reply with quote

martin◆
Site Admin
martin avatar
Joined:
Posts:
41,517
Location:
Prague, Czechia

Sorry, that's not enough information. I do not need your complete app code. I just need enough code to reproduce the problem. And the log. With your SynchronizeDirectories call, I'm not getting the "Size of ... local files/folders calculated as ..." log entry.

Btw, that entry is also in your last log snippet. Did you change anything? (log level?)

Just an idea: aren't you setting Session.FileTransferProgress?

Reply with quote

Advertisement

You can post new topics in this forum

Documentation

Support

Associations

Follow Us