Thanks for sharing your findings.
- martin
Post a reply
Before posting, please read how to report bug or request support effectively.
Bug reports without an attached log file are usually useless.
Topic review
- NottaSmartMan
I think I may have figured it out. I'm very very new to multithreading and didn't realize that threads couldn't see function defined outside the thread job. I didn't see that demo'd in the examples I ran across. Things worked once I placed the
This is a whole new world.
FileQuery function within a code block inside the thread-job.
This is a whole new world.
- NottaSmartMan
I've done more digging and created a similar and simple single instance and multithreaded version of the code. I tried adding the
After some testing, it seems the code is getting hung up around the
If I remove the
The last line of the log are:
add_QueryReceived event, for I think that's what I was truly missing.
After some testing, it seems the code is getting hung up around the
$syncSession.add_QueryReceived event. The event is triggered, but the script just hangs calling the FileQuery function. I've attach a sanitized copy of the script.
If I remove the
QueryReceived event, then the script doesn't hang... rather it just aborts. Neither are what I want... I want it to skip/continue.
The last line of the log are:
< 2024-11-06 13:28:42.265 Script: Upload of file '<removed>' was successful, but error occurred while setting the permissions and/or timestamp.
< 2024-11-06 13:28:42.265 If the problem persists, turn off setting permissions or preserving timestamp. Alternatively you can turn on 'Ignore permission errors' option.
< 2024-11-06 13:28:42.265 Script: General failure (server should provide error description).
< 2024-11-06 13:28:42.265 Error code: 4
< 2024-11-06 13:28:42.265 Error message from server (en-US):
< 2024-11-06 13:28:42.265 Common reasons for the Error code 4 are:
< 2024-11-06 13:28:42.265 - Renaming a file to a name of already existing file.
< 2024-11-06 13:28:42.265 - Creating a directory that already exists.
< 2024-11-06 13:28:42.265 - Moving a remote file to a different filesystem (HDD).
< 2024-11-06 13:28:42.265 - Uploading a file to a full filesystem (HDD).
< 2024-11-06 13:28:42.265 - Exceeding a user disk quota.
- martin
You are getting error 4 (failure), not 3 (permission denied). So the
If you believe that WinSCP behaviour is different with single-threaded code, please post log file showing that.
IgnorePermErrors does not apply.
If you believe that WinSCP behaviour is different with single-threaded code, please post log file showing that.
- NottaSmartMan
The server does support preserving timestamps. 99% of the time it works just fine. However, I have noticed issues at times where setting the timestamp will fail because the destination file is 'in use'. I suspect that there's an anti-virus scanner that sometimes gets in the way.
I use the
I don't understand why WinSCP is now choosing "Abort" rather than "Skip". Is something different with the
Example: Here's a sanitized excerpt from a file that succeeded, then then the next file that failed was skipped. This is from a different script that isn't multithread; it just uses the
I use the
IgnorePermErrors so that when these errors popup, that the error is skipped rather than aborted. It's worked fine up until I started playing with making my script threaded.
I don't understand why WinSCP is now choosing "Abort" rather than "Skip". Is something different with the
$session.CompareDirectories([WinSCP.SynchronizationMode] $difference.Resolve syntaxes? Or Am I barking up the wrong tree and there's actually a different command that instructs WinSCP to Skip rather than Abort?
Example: Here's a sanitized excerpt from a file that succeeded, then then the next file that failed was skipped. This is from a different script that isn't multithread; it just uses the
$session.SynchronizeDirectories and that's it.
. 2024-10-27 16:34:00.035 Preserving timestamp [2024-03-01T13:16:16.000Z]
. 2024-10-27 16:34:00.113 Transfer done: '\\<path removed>\SuccessfulFile.xlsx' => '/<Path Removed>/SuccessfulFile.xlsx' [301102]
. 2024-10-27 16:34:00.113 File: '\\<path removed>\FileThatHadIssues.xlsx' [2024-05-15T14:26:14.282Z] [301198]
. 2024-10-27 16:34:00.457 Copying "\\<path removed>\FileThatHadIssues.xlsx" to remote directory started.
. 2024-10-27 16:34:00.753 Preserving timestamp [2024-05-15T14:26:14.000Z]
* 2024-10-27 16:34:00.753 (ETerminal) General failure (server should provide error description).
* 2024-10-27 16:34:00.753 Error code: 4
* 2024-10-27 16:34:00.753 Error message from server (en-US):
* 2024-10-27 16:34:00.753
* 2024-10-27 16:34:00.753 Common reasons for the Error code 4 are:
* 2024-10-27 16:34:00.753 - Renaming a file to a name of already existing file.
* 2024-10-27 16:34:00.753 - Creating a directory that already exists.
* 2024-10-27 16:34:00.753 - Moving a remote file to a different filesystem (HDD).
* 2024-10-27 16:34:00.753 - Uploading a file to a full filesystem (HDD).
* 2024-10-27 16:34:00.753 - Exceeding a user disk quota.
. 2024-10-27 16:34:00.753 Asking user:
. 2024-10-27 16:34:00.753 **Upload of file 'FileThatHadIssues.xlsx' was successful, but error occurred while setting the permissions and/or timestamp.**
. 2024-10-27 16:34:00.753
. 2024-10-27 16:34:00.753 If the problem persists, turn off setting permissions or preserving timestamp. Alternatively you can turn on 'Ignore permission errors' option. ("General failure (server should provide error description).
. 2024-10-27 16:34:00.753 Error code: 4
. 2024-10-27 16:34:00.753 Error message from server (en-US):
. 2024-10-27 16:34:00.753
. 2024-10-27 16:34:00.753 Common reasons for the Error code 4 are:
. 2024-10-27 16:34:00.753 - Renaming a file to a name of already existing file.
. 2024-10-27 16:34:00.753 - Creating a directory that already exists.
. 2024-10-27 16:34:00.753 - Moving a remote file to a different filesystem (HDD).
. 2024-10-27 16:34:00.753 - Uploading a file to a full filesystem (HDD).
. 2024-10-27 16:34:00.753 - Exceeding a user disk quota.")
< 2024-10-27 16:34:00.753 Script: Upload of file 'FileThatHadIssues.xlsx' was successful, but error occurred while setting the permissions and/or timestamp.
< 2024-10-27 16:34:00.753 If the problem persists, turn off setting permissions or preserving timestamp. Alternatively you can turn on 'Ignore permission errors' option.
< 2024-10-27 16:34:00.753 Script: General failure (server should provide error description).
< 2024-10-27 16:34:00.753 Error code: 4
< 2024-10-27 16:34:00.753 Error message from server (en-US):
< 2024-10-27 16:34:00.753 Common reasons for the Error code 4 are:
< 2024-10-27 16:34:00.753 - Renaming a file to a name of already existing file.
< 2024-10-27 16:34:00.753 - Creating a directory that already exists.
< 2024-10-27 16:34:00.753 - Moving a remote file to a different filesystem (HDD).
< 2024-10-27 16:34:00.753 - Uploading a file to a full filesystem (HDD).
< 2024-10-27 16:34:00.753 - Exceeding a user disk quota.
. 2024-10-27 16:34:00.753 Answer: Skip
- martin
Re: Parallel Transfers + IgnorePermErrors
Why do you think it's
IgnorePermErrors issue? Isn't it about TransferOptions.PreserveTimestamp? Though if your server does not support preserving timestamps, then timestamp-based synchronization won't be working.
- NottaSmartMan
Here's my Console output in case it helps.
- NottaSmartMan
Parallel Transfers + IgnorePermErrors
I'm having an issue where the
I've tried setting the flag via
I will attach my code and logs in question.
The 'thread' log is the one that shows the abort.
IgnorePermErrors flag is being set within a thread, but doesn't appear to be taking affect. When a timestamp issue is encountered, the script Aborts rather than Skips.
I've tried setting the flag via
xxx.AddRawConfiguration("IgnorePermErrors","1") as well as xxx.AddRawSettings("IgnorePermErrors", "1"). Nothing works.
I will attach my code and logs in question.
The 'thread' log is the one that shows the abort.