Downloading only new files on server

Advertisement

Omega42
Joined:
Posts:
2
Location:
Ireland

Downloading only new files on server

Apologies if this has been asked and answered before, I did have a look through the fourm but couldn't see anything.

I'm looking to set up a script that logs on to a server every hour to check for new files and downloads them.
I don't think I can use sync as files will be moved on my local machine so I would end up re downloading the moved files.

Like wise the <1h would not work as the local PC may be switched off.

I'm thinking a external file on local PC with timestamp of last download then when it's next run.

Thanks in advance

Reply with quote

Advertisement

fairudyn
Joined:
Posts:
6
Location:
Singapore

Re: Downloading only new files on server

Omega42 wrote:

Apologies if this has been asked and answered before, I did have a look through the fourm but couldn't see anything.

I'm looking to set up a script that logs on to a server every hour to check for new files and downloads them.
I don't think I can use sync as files will be moved on my local machine so I would end up re downloading the moved files.

Like wise the <1h would not work as the local PC may be switched off.

I'm thinking a external file on local PC with timestamp of last download then when it's next run.

Thanks in advance

Hi you can use

put -neweronly

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
41,253
Location:
Prague, Czechia

Re: Downloading only new files on server

Omega42 wrote:

I'm thinking a external file on local PC with timestamp of last download then when it's next run.
That would indeed be the only solution, given your constraints. But WinSCP does not support it out of the box.
It should not be difficult to script this in PowerShell using WinSCP .NET assembly.
I'll try to write an example script in few days.

Reply with quote

Vann_the_Red
Guest

Piggybacking

Martin,

That script using file history is brilliant and almost solves my problem as well. I worry, though, that the list will become unwieldy quickly. I need to script several high traffic downloads. It seems like the simplest path would be to save the time stamp of the most recent file, get a directory listing, sort by recency, and download those added since the last download. Does anyone see a simpler solution? Are there pointers to doing it that way?

TIA,

VtR

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
41,253
Location:
Prague, Czechia

Re: Piggybacking

Vann_the_Red wrote:

That script using file history is brilliant and almost solves my problem as well. I worry, though, that the list will become unwieldy quickly. I need to script several high traffic downloads. It seems like the simplest path would be to save the time stamp of the most recent file, get a directory listing, sort by recency, and download those added since the last download. Does anyone see a simpler solution? Are there pointers to doing it that way?
If you can rely on the timestamps, than it's indeed a way to go.
For an example of downloading files newer than a given timestamp, see for example:
https://stackoverflow.com/q/37007929/850848#37010471

Reply with quote

Advertisement

You can post new topics in this forum