Automated download – files missing

Advertisement

Cadross
Joined:
Posts:
3

Automated download – files missing

Hi,

I have an automated download copying and deleting files to a directory (Batch file trigger by Task scheduler). It's usually 1–5 files at a time, but about 10% of the files just seem to vanish into thin air. Files from the same batch are successfully downloaded, but 1 or 2 will be missing.

It seems completely sporadic and I haven't been able to find anything that the missing files have in common to start pulling at a thread.

Any help of where to start looking would be really helpful,

Thanks
  • RedactedLog20240207_125900.txt (105.54 KB, Private file)
Description: This is the log of a transfer I have confirmed missed a file. The Non_PO_* file was transferred, but the PO_* file was lost. Redacted URLs/IPs/Passwords so apologies if I've removed something necessary. Tried to leave as much in as possible. Seem to be ge

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
41,518
Location:
Prague, Czechia

Re: Automated download – files missing

To me it looks like you are logging to the same file from multiple parallel instances of WinSCP. So the logs contains intermingled records from multiple sessions. It is hard to follow. Avoid that.

Anyway, I see this:
* 2024-02-07 12:59:03.856 The process cannot access the file because it is being used by another process
. 2024-02-07 12:59:03.856 Asking user:
. 2024-02-07 12:59:03.856 Can't open file 'e:\DataProcesses\invoiceimport\oainvoices\PO_Invoices_Output_20240207_121920.csv'. ("System Error.  Code: 32.
And actually it's likely due to the same problem. The same file is being downloaded by two parallel WinSCP processes.
One started here:
. 2024-02-07 12:59:03.450 Copying "/PO_Invoices_Output_20240207_121920.csv" to local directory started.
The second started 0.1 second later:
. 2024-02-07 12:59:03.512 Copying "/PO_Invoices_Output_20240207_121920.csv" to local directory started.

Reply with quote

Cadross
Joined:
Posts:
3

Hi Martin,

Thank you for replying to this.
I've looked and I can't seem to find anything that is causing multiple sessions to run in parallel.

We have one task scheduled, that runs one file, that has a single command within it, yet still there seems to be multiple processes running at the same time.

Do you have any idea where I could potentially look to understand how and why this is happening?

Many thanks

Reply with quote

Cadross
Joined:
Posts:
3

Hi Martin,

Thanks for your help on this, I investigated Task Scheduler as from what you said it seemed the file was being triggered multiple times.

If you see something similar again – I've been able to resolve the issue by deleting the old tasks and creating new ones to do similar things.

I'm unsure what caused this, but that seems to have been the issue,

Thanks again for your help

Reply with quote

Advertisement

Advertisement

You can post new topics in this forum