Calculating Directory Size

Advertisement

Antarus
Joined:
Posts:
4

Calculating Directory Size

Hello, I have been using WinSCP for a while now and its been working quite well so far, except two things. (I'll just post one for now)

When trying to download a very large directory structures and huge amounts of files, it goes to 50% of calculating directory size and then it just keeps going through the directories one by one, which takes around 2-3 seconds per directory. This it taking forever as there are literally tens of thousands of directories. (can't tell how many there are, but its still running after having been on all night, still at 50% and still calculating directories)

Is there any way to skip that process? And is there any way to speed up downloading this huge amounts of files in WinSCP?

I have tried searching for an answer, I looked through the documentation and I searched the forum, but couldn't find an answer to this.

Reply with quote

Advertisement

Antarus

I did again. Found the calculate size option, that turned it off thanks. Missed it the first time, must have been because I was looking for directory sizes instead.

Reply with quote

adrian_vg
Joined:
Posts:
20
Location:
Uppsala, Sweden

For future reference:

Calculate Size can be found under menu Options/Preferences and then click Transfer and look to the Common Options-area.

I had the same as Antarus with tens of thousands of dirs.

Reply with quote

Advertisement

yereverluvinunclebert
Donor
Joined:
Posts:
37
Location:
Brighton - The Beautiful South

This should be turned off by default

This directory size feature should be turned off by default. It is annoying, not really wanted by most and the configuration option is hard to find. Ipswitch WSFTP has this feature but you can't turn it off! I am thankful it is possible with WINSCP.

Reply with quote

martin
Site Admin
martin avatar

Re: This should be turned off by default

yereverluvinunclebert wrote:

and the configuration option is hard to find.
Why? Have you tried typing "turn off calculate directory size" into search box on the right?

Reply with quote

Advertisement

Guest

@adrian_vg: Thank you, that is MOUNTAINS more useful than "Please read documentation" in these search results.

However, by version 5.5.1 there was an extra step, to find and click "edit" once in the "Transfer" settings box. Then I see these "Common Options".

Reply with quote

Guest

I second this.

In modern times in which we are flooded with apps and software to use for all kinds of stuff, there is no time to 'sit down' and outright study manual of anything anymore.

And i'm a developer who does database-heavy ecommerce coding – imagine the non-tech people.

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
40,430
Location:
Prague, Czechia

Re: default value

yereverluvinunclebert wrote:

Finding it is one thing but the important issue is that the default value should be 'disabled'.
Every sensible application should show progress of an operation. And it cannot be properly calculated without knowing how much data we are going to transfer. I just do not think that many people want to turn this off really.

Reply with quote

Advertisement

Guest

I see your point about wanting to show progress. However, I don't share your opinion on whether people would rather wait a whole day for this ability each time they want to FTP a directory structure by default.

Reply with quote

Guest

no! to folder listings

@Guest: This is correct due the intrinsically slow nature of FTP communications. Performing a whole directory tree listing each operation in seconds instead of milliseconds is too much to bear. I abandoned WS-FTP because this option was there by default and it could not be turned off. When I look at disc space on a PC using a tool such as space sniffer then the operation can take minutes to complete and that is direct hard disc i/o access.

Have the configuration option easily available, perhaps an icon to "go and get folder size now" or "folder properties" through a right click that creates a spawned operation that does not interfere with current file transfer or operation. But please don't have it on by default. The first impression naive users will have of your product is how it locks up for ages before you can actually start using it.

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
40,430
Location:
Prague, Czechia

Re: Calculating Directory Size

OK, as a compromise:

Since 5.5.1 it already possible to cancel the calculation, without cancelling the actual transfer.

The next version will:
– Not require confirmation to cancel
– Say "Skip" instead of "Cancel" while calculating the size.

https://winscp.net/tracker/1296

Reply with quote

AnotherGuest
Guest

I'm baffled that this is still a setting that's enabled by default. Every time I set up WinSCP on a new computer I have to google for this thread to find the instructions to turn this off. Clicking skip every time is not ideal, and if you set it to transfer in the background by default you never get prompted. And there's no way to skip it after you start, unless you cancel and restart it.

Every sensible application should show progress of an operation
I honestly think there is an exception for file transfers. You should calculate progress for something which is deterministic, absolutely. If you are transferring a single file it would be very annoying to have no idea of the progress. Remember how long it took browsers to show file upload progress natively, without using flash/java? It was awful.

But with FTP, in most cases you are dealing with a large amount of source files. Especially if you are working with many small files (pretty much any website).

You're easily spending HALF of the time just calculating the transfer!

It doesn't make sense! Not for remote transfer!

If this is scanning a local disk where the read operations take microseconds and much of the data is cached/indexed already, this isn't an issue. But FTP doesn't work like that.

IMO, this option should be opt-in. I love WinSCP, but this option alone embarrasses me. Trying to explain it to a co-worker is even worse.

Let me explain it in a fake chat with someone. I've actually had a discussion pretty similar a few days ago. Although not as eh... pessimistic?

them: "I'm still waiting for it to download the new files"
me: "How long is that going to take?"
them: "I don't know, it just says calculating. I don't think it has even started yet"
me: "So you can't tell me how long it's going to take, because it's still trying to figure out how long it's going to take?"
them: "Yeah, I guess so"
me: "Well at least we know it's going to take a long time. Longer since it has to calculate."
them: "Why does it have to calculate anyway?"
me: "Well, it's useful if you are copying a lot of files, I guess."
them: "But that's exactly when you don't want it to calculate, because it takes so long!"

Reply with quote

Advertisement

yannis_r
Joined:
Posts:
1
Location:
Houston TX

Multiple directories file size

Hello,

I was wondering if you put a calculate button on top of the commander style list (next to other buttons) to calculate the file size of selected directories and then put the size of each directory on the size column.
Then show the total size of selected folders at the bottom of the list, as you do currently for the size of selected files.
Additionally, you could assign the keystroke Alt+Shift+Enter to calculate for all folders visible.

This will allow identifying quickly which folders contain the volume when looking for storage issues.

Thank you

Reply with quote

johnywhy
Joined:
Posts:
10
Location:
Bay Area

adrian_vg wrote:

Calculate Size can be found under menu Options/Preferences and then click Transfer and look to the Common Options-area.
Has this feature been removed? I don't see it.

image.png

Reply with quote

Advertisement

modabasi@...
Guest

Same problem here

I've disabled "Calculate total size" through Options, does not calculate but transfers files one by one. If I make the transfers in the background, it starts calculating again.

Reply with quote

Advertisement

Advertisement

You can post new topics in this forum