Getting Files Placed in Folders Two Days' Later

Advertisement

rsford31
Joined:
Posts:
19

Getting Files Placed in Folders Two Days' Later

Hi,

I'm downloading zipped files where the folder structure is yyyy/mm/dd. The download is a daily process where the files have the current date in the timestamp but the names have the previous days' date. For example, the current date is July 21st. The date/timestamp will be July 21st, but the files will have 2020-07-20 in the file name and will be found in the 2020/07/20. Currently, with PowerShell, I'm getting the current date-1 and using those values to build the source location to pull the files. I pull the files down locally, unzip the files, change the extension to .csv, and then move the whole 2020/07/20 folder to a file server. I recently found out that another file is placed in the folder two days later. So using the same example, a file with the 2020-07-20 in the file name will be placed in the 2020/07/20 folder on the 22nd and will have a date/timestamp of the 21st.

I orginally thought that syncing the files with the FTP server directly to the file server would work, but given the files will change (unzipped and the extension changed), there would always be two files in the folder: the zipped file and the CSV file. I don't want to keep a set of files on the local server just to sync with either. I then thought that using time constraint feature would work. set the FileMask property to "*>=1D", the source to /2020/07/*, /2020/07/19, 2020/07/19/*, and 2020/07/19/, but the files are not downloading.

Ideally, I would like to be able to sync files locally that are new, unzip and rename, then push them to the file server. This way any files that are placed in older folders will not be missed. I like that the sync functionality creates folders at destination if they do not exist. I'm not sure if this is possible or not with the functionality available in WinSCP.

Here is a code snippet:
$transferOptions = New-Object WinSCP.TransferOptions
$transferOptions.FileMask = "*>=1D"
$session.GetFiles("E:\temp\", "/2020/07/19/", $False, $transferOptions)
I'm using WinSCP v 5.15.9
Let me know! Thanks!

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
33,846
Location:
Prague, Czechia

Re: Getting Files Placed in Folders Two Days' Later

rsford31 wrote:

Ideally, I would like to be able to sync files locally that are new, unzip and rename, then push them to the file server. This way any files that are placed in older folders will not be missed. I like that the sync functionality creates folders at destination if they do not exist. I'm not sure if this is possible or not with the functionality available in WinSCP.
You can use Session.SynchronizeDirectories to fetch the new files. And then iterate the list of downloaded files in SynchronizationResult.Downloads – extracting/renaming/uploading them.
https://winscp.net/eng/docs/library_session_synchronizedirectories
https://winscp.net/eng/docs/library_synchronizationresult

Reply with quote

Advertisement

You can post new topics in this forum