Re: Slow When Syncing or Calculating many small directories
Sorry, answer is no.
- martin
Post a reply
Before posting, please read how to report bug or request support effectively.
Bug reports without an attached log file are usually useless.
Topic review
- ontheritz
Slow When Syncing or Calculating many small directories
Hey peeps,
I love love WinSCP but I'm wondering if there is a configuration tweak I can make to alleviate this problem I've been having.
Basically when syncing, keeping remote directory up to date, or calculating the size of a folder, if that folder has many many many sub directories, WinSCP is super duper slow. So slow in fact that it almost never completes its task by the time I have started punching myself in the face and running around the room with my underwear on top of my head.
The reason I have so many directories is I'm storing a large number of files in a sort of a leaf-like system. Lets say I'm using primary keys that are alphanumeric, and 12 digits long, like BDGSHE47YHSZ. To store the file, I'm using a path that would look like this:
/home/myfolder/blabla/files/B/D/G/S/H/E/BDGSHE47YHSZ
This system is great for storing a huge amount of files without all the hassles that come with directories that exceed 30,000 files. But this usually causes WinSCP to have to traverse a huge number of directories for common operations like Sync/Keep-Remote-Up-to-Date/Calculate-Size. So slow in fact, that I die a little inside each time I even think about it.
Is there any way that I can tweak my configuration somewhere, grab a plugin, or do something to WinSCP that would help speed this process up? Maybe could we get additional threads or something in the future?
Or maybe has WiSCP been proven to be fast in this situation, and maybe I need to find a new webhost?
~Mike
I love love WinSCP but I'm wondering if there is a configuration tweak I can make to alleviate this problem I've been having.
Basically when syncing, keeping remote directory up to date, or calculating the size of a folder, if that folder has many many many sub directories, WinSCP is super duper slow. So slow in fact that it almost never completes its task by the time I have started punching myself in the face and running around the room with my underwear on top of my head.
The reason I have so many directories is I'm storing a large number of files in a sort of a leaf-like system. Lets say I'm using primary keys that are alphanumeric, and 12 digits long, like BDGSHE47YHSZ. To store the file, I'm using a path that would look like this:
/home/myfolder/blabla/files/B/D/G/S/H/E/BDGSHE47YHSZ
This system is great for storing a huge amount of files without all the hassles that come with directories that exceed 30,000 files. But this usually causes WinSCP to have to traverse a huge number of directories for common operations like Sync/Keep-Remote-Up-to-Date/Calculate-Size. So slow in fact, that I die a little inside each time I even think about it.
Is there any way that I can tweak my configuration somewhere, grab a plugin, or do something to WinSCP that would help speed this process up? Maybe could we get additional threads or something in the future?
Or maybe has WiSCP been proven to be fast in this situation, and maybe I need to find a new webhost?
~Mike