Slow When Syncing or Calculating many small directories

Advertisement

ontheritz
Joined:
Posts:
1
Location:
Los Angeles, California

Slow When Syncing or Calculating many small directories

Hey peeps,

I love love WinSCP but I'm wondering if there is a configuration tweak I can make to alleviate this problem I've been having.

Basically when syncing, keeping remote directory up to date, or calculating the size of a folder, if that folder has many many many sub directories, WinSCP is super duper slow. So slow in fact that it almost never completes its task by the time I have started punching myself in the face and running around the room with my underwear on top of my head.

The reason I have so many directories is I'm storing a large number of files in a sort of a leaf-like system. Lets say I'm using primary keys that are alphanumeric, and 12 digits long, like BDGSHE47YHSZ. To store the file, I'm using a path that would look like this:

/home/myfolder/blabla/files/B/D/G/S/H/E/BDGSHE47YHSZ

This system is great for storing a huge amount of files without all the hassles that come with directories that exceed 30,000 files. But this usually causes WinSCP to have to traverse a huge number of directories for common operations like Sync/Keep-Remote-Up-to-Date/Calculate-Size. So slow in fact, that I die a little inside each time I even think about it.

Is there any way that I can tweak my configuration somewhere, grab a plugin, or do something to WinSCP that would help speed this process up? Maybe could we get additional threads or something in the future?

Or maybe has WiSCP been proven to be fast in this situation, and maybe I need to find a new webhost?

~Mike

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
27,252
Location:
Prague, Czechia

Re: Slow When Syncing or Calculating many small directories

Sorry, answer is no.
_________________
Martin Prikryl

Reply with quote

Advertisement

You can post new topics in this forum