Topic "Need input if WINSCP would be the best option"

Author Message
enohand
[View user's profile]

Joined: 2012-08-16
Posts: 2
Location: United States
I have a folder (with many sub-folders) & in some of those sub-folders there are hundreds of thousand, if not millions of small files (anywhere from 5kb-20kb hundreds of thousand document files in 1 folder, & millions of images ranging in size from 1mb-10mb in another folders which are also in sub folders dated per day)

I am trying to setup a way to sync the folders (1 way from source to target) & automate this to run every night to capture all the new files added to the source & then copy them to the traget.

I have initially done a baseline copy to a external nas, & shipped it to the location of the target machine. So right now the target machine has about 90% of the data, however it has been 2 months since this inital copy & the folders have grown approx. 40gb since then

the 2 sites are connected with a VPN.

My initial idea was to use something like Live Vault, however, it does not seem necessary to use it since there is a VPN connecting the 2 locations.

for smaller types of these migrations i use Richcopy & manually run it, or setup a schedule to do so, & then the night of the live cut over/migration i preform a initial sync of the 2 locations (1 way) & then restore their DB's

however, this client is 1 that plans on testing for a few months before going live on the new system & quite frankly i do not have the time to connect all the time & initiate the process.

any information would be greatly appreciated.



(also yes i know that hundred's of thousands of document files sitting in 1 directory is not the best thing on windows, but that is how their application does it.)
enohand
[View user's profile]

Joined: 2012-08-16
Posts: 2
Location: United States
enohand wrote:
I have a folder (with many sub-folders) & in some of those sub-folders there are hundreds of thousand, if not millions of small files (anywhere from 5kb-20kb hundreds of thousand document files in 1 folder, & millions of images ranging in size from 1mb-10mb in another folders which are also in sub folders dated per day)

I am trying to setup a way to sync the folders (1 way from source to target) & automate this to run every night to capture all the new files added to the source & then copy them to the traget.

I have initially done a baseline copy to a external nas, & shipped it to the location of the target machine. So right now the target machine has about 90% of the data, however it has been 2 months since this inital copy & the folders have grown approx. 40gb since then

the 2 sites are connected with a VPN.

My initial idea was to use something like Live Vault, however, it does not seem necessary to use it since there is a VPN connecting the 2 locations.

for smaller types of these migrations i use Richcopy & manually run it, or setup a schedule to do so, & then the night of the live cut over/migration i preform a initial sync of the 2 locations (1 way) & then restore their DB's

however, this client is 1 that plans on testing for a few months before going live on the new system & quite frankly i do not have the time to connect all the time & initiate the process.

any information would be greatly appreciated.



(also yes i know that hundred's of thousands of document files sitting in 1 directory is not the best thing on windows, but that is how their application does it.)


i dont typically like to bump threads but i was hoping for a comment by now.


i wonder if using like Rsync, or Robocopy would be a better solution.


I tried reading through the manuals & stuff, but haven't been able to find out the info i need.


also i dont think i mentioned, that i can hit a UNC path to the target system, so i have no need for FTP, or any other protocol (unless it would be faster then the UNC path)
Advertisements

You can post new topics in this forum






Search Site

What is WinSCP?

It is award-winning SFTP client, SCP client, FTPS client and FTP client integrated into one software program for file transfer to FTP server or secure SFTP server. [More]

And it's free!

Donate

About donations

$9   $19   $49   $99

About donations

Recommend

WinSCP Privacy Policy

WinSCP License