Calculating Directory Size

Hello, I have been using WinSCP for a while now and its been working quite well so far, except two things. (I'll just post one for now)

When trying to download a very large directory structures and huge amounts of files, it goes to 50% of calculating directory size and then it just keeps going through the directories one by one, which takes around 2-3 seconds per directory. This it taking forever as there are literally tens of thousands of directories. (can't tell how many there are, but its still running after having been on all night, still at 50% and still calculating directories)

Is there any way to skip that process? And is there any way to speed up downloading this huge amounts of files in WinSCP?

I have tried searching for an answer, I looked through the documentation and I searched the forum, but couldn't find an answer to this.

Is there no way to stop this? It's rendering WinSCP all but useless. :(

Please read documentation. If that does not help, come back.

I did again. Found the calculate size option, that turned it off thanks. Missed it the first time, must have been because I was looking for directory sizes instead.

For future reference:

Calculate Size can be found under menu Options/Preferences and then click Transfer and look to the Common Options-area.

I had the same as Antarus with tens of thousands of dirs.

Thank you! This setting drove me crazy. I was close to uninstall WinSCP ^^

This directory size feature should be turned off by default. It is annoying, not really wanted by most and the configuration option is hard to find. Ipswitch WSFTP has this feature but you can't turn it off! I am thankful it is possible with WINSCP.

@yereverluvinunclebert: I agree, at least for IIS FTPS.

yereverluvinunclebert wrote:

and the configuration option is hard to find.

Why? Have you tried typing "turn off calculate directory size" into search box on the right?

@adrian_vg: Thank you, that is MOUNTAINS more useful than "Please read documentation" in these search results.

However, by version 5.5.1 there was an extra step, to find and click "edit" once in the "Transfer" settings box. Then I see these "Common Options".

Finding it is one thing but the important issue is that the default value should be 'disabled'.

I second this.

In modern times in which we are flooded with apps and software to use for all kinds of stuff, there is no time to 'sit down' and outright study manual of anything anymore.

And i'm a developer who does database-heavy ecommerce coding – imagine the non-tech people.

yereverluvinunclebert wrote:

Finding it is one thing but the important issue is that the default value should be 'disabled'.

Every sensible application should show progress of an operation. And it cannot be properly calculated without knowing how much data we are going to transfer. I just do not think that many people want to turn this off really.

I see your point about wanting to show progress. However, I don't share your opinion on whether people would rather wait a whole day for this ability each time they want to FTP a directory structure by default.

@Guest: This is correct due the intrinsically slow nature of FTP communications. Performing a whole directory tree listing each operation in seconds instead of milliseconds is too much to bear. I abandoned WS-FTP because this option was there by default and it could not be turned off. When I look at disc space on a PC using a tool such as space sniffer then the operation can take minutes to complete and that is direct hard disc i/o access.

Have the configuration option easily available, perhaps an icon to "go and get folder size now" or "folder properties" through a right click that creates a spawned operation that does not interfere with current file transfer or operation. But please don't have it on by default. The first impression naive users will have of your product is how it locks up for ages before you can actually start using it.

OK, as a compromise:

Since 5.5.1 it already possible to cancel the calculation, without cancelling the actual transfer.

The next version will:
– Not require confirmation to cancel
– Say "Skip" instead of "Cancel" while calculating the size.

https://winscp.net/tracker/1296

I'm baffled that this is still a setting that's enabled by default. Every time I set up WinSCP on a new computer I have to google for this thread to find the instructions to turn this off. Clicking skip every time is not ideal, and if you set it to transfer in the background by default you never get prompted. And there's no way to skip it after you start, unless you cancel and restart it.

Every sensible application should show progress of an operation

I honestly think there is an exception for file transfers. You should calculate progress for something which is deterministic, absolutely. If you are transferring a single file it would be very annoying to have no idea of the progress. Remember how long it took browsers to show file upload progress natively, without using flash/java? It was awful.

But with FTP, in most cases you are dealing with a large amount of source files. Especially if you are working with many small files (pretty much any website).

You're easily spending HALF of the time just calculating the transfer!

It doesn't make sense! Not for remote transfer!

If this is scanning a local disk where the read operations take microseconds and much of the data is cached/indexed already, this isn't an issue. But FTP doesn't work like that.

IMO, this option should be opt-in. I love WinSCP, but this option alone embarrasses me. Trying to explain it to a co-worker is even worse.

Let me explain it in a fake chat with someone. I've actually had a discussion pretty similar a few days ago. Although not as eh... pessimistic?

them: "I'm still waiting for it to download the new files"
me: "How long is that going to take?"
them: "I don't know, it just says calculating. I don't think it has even started yet"
me: "So you can't tell me how long it's going to take, because it's still trying to figure out how long it's going to take?"
them: "Yeah, I guess so"
me: "Well at least we know it's going to take a long time. Longer since it has to calculate."
them: "Why does it have to calculate anyway?"
me: "Well, it's useful if you are copying a lot of files, I guess."
them: "But that's exactly when you don't want it to calculate, because it takes so long!"

OK, I'll try to find some better solution.

Hello,

I was wondering if you put a calculate button on top of the commander style list (next to other buttons) to calculate the file size of selected directories and then put the size of each directory on the size column.
Then show the total size of selected folders at the bottom of the list, as you do currently for the size of selected files.
Additionally, you could assign the keystroke Alt+Shift+Enter to calculate for all folders visible.

This will allow identifying quickly which folders contain the volume when looking for storage issues.

Thank you

@yannis_r: Thanks for your post.

This request is being tracked already:
https://winscp.net/tracker/41

adrian_vg wrote:

Calculate Size can be found under menu Options/Preferences and then click Transfer and look to the Common Options-area.

Has this feature been removed? I don't see it.

@johnywhy: You have to click Edit.
That will open this dialog:
https://winscp.net/eng/docs/ui_transfer_custom

Thx. It looks like I have "Calculate" already selected, but I don't see per-folder sizes.

Is there a way to show per-folder sizes?

@johnywhy: No. It seems that you have misunderstood the discussion above.
Showing of directory sizes in file panels was not implemented yet.
You can vote for it here:
https://winscp.net/tracker/41

I've disabled "Calculate total size" through Options, does not calculate but transfers files one by one. If I make the transfers in the background, it starts calculating again.

@modabasi: Yes, as documented:

The option cannot be disabled for background transfers, when using multiple connections. Note that when using multiple connections, the calculation does not add time to a transfer, as directories are not listed again for the transfer itself.