Calculate progress of parallel transfers

Advertisement

Jaoryx
Joined:
Posts:
5
Location:
Belgium

Calculate progress of parallel transfers

I am trying to parallel download a folder from a url (which works perfectly) but now i was wondering whether i can calculate the total progress into one progressbar. I used downloadProgressBar.Value = (int)(ee.OverallProgress * 100) before I used parallel.

Reply with quote E-mail

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
40,605
Location:
Prague, Czechia

Re: Calculate progress of parallel transfers

This is quite broad question.
Well, definitely you can. But we do not know how you do the "parallel transfers".

Reply with quote

Jaoryx
Joined:
Posts:
5
Location:
Belgium

I'm sorry, I didn't know exactly how to describe it. But I am using it as in the example here: Automating transfers or synchronization in parallel connections over SFTP/FTP protocol

This is my code:
private void Download(object sender, DoWorkEventArgs e)
{
    DisableDownloadButton();
 
    DownloadArguments args = (DownloadArguments) e.Argument;
    string url = args.Url;
    string localPath = args.Path;
    bool isDirectory = args.IsDirectory;
    string fileName = url.Substring(url.LastIndexOf("/") + 1);
 
    using (var session = new Session())
    {
        if (!isDirectory)
        {
            session.FileTransferProgress += (senderr, ee) =>
            {
                downloadProgressBar.Invoke((MethodInvoker)(() =>
                {
                    downloadProgressBar.Value = (int)(ee.OverallProgress * 100);
                }));
                lblLoading.Invoke((MethodInvoker)delegate { lblLoading.Text = "Download " + (ee.OverallProgress * 100).ToString() + "% completed"; });
            };
        }
 
        session.Open(ftpSessionOptions);
 
        lblLoading.Invoke((MethodInvoker)delegate { lblLoading.Text = "Enumerating files..."; });
 
        if (isDirectory)
        {
            localPath = $"{localPath}\\{fileName}";
            Directory.CreateDirectory(localPath);
            var opts = EnumerationOptions.AllDirectories;
            IEnumerable<RemoteFileInfo> files = session.EnumerateRemoteFiles(url, null, opts);
            IEnumerator<RemoteFileInfo> filesEnumerator = files.GetEnumerator();
 
            var tasks = new List<Task>();
 
            for (int i = 1; i <= 10; i++)
            {
                var task = new Task(() => {
                    using (var downloadSession = new Session())
                    {
                        downloadSession.Open(ftpSessionOptions);
 
                        while (true)
                        {
                            string remoteFilePath;
 
                            lock (filesEnumerator)
                            {
                                if (!filesEnumerator.MoveNext()) break;
 
                                RemoteFileInfo file = filesEnumerator.Current;
                                remoteFilePath = file.FullName;
                            }
 
                            string localFilePath = RemotePath.TranslateRemotePathToLocal(remoteFilePath, url, localPath);
                            string localFileDir = Path.GetDirectoryName(localFilePath);
                            Directory.CreateDirectory(localFileDir);
                            downloadSession.GetFileToDirectory(remoteFilePath, localFileDir);
                        }
                    }
                });
 
                tasks.Add(task);
                task.Start();
            }
            Task.WaitAll(tasks.ToArray());
        }
        else
        {
            currentDownload = fileName;
            session.GetFiles(url, localPath + "\\" + fileName).Check();
        }
 
        session.Close();
    }
}

Reply with quote E-mail

Jaoryx
Joined:
Posts:
5
Location:
Belgium

Still haven't figured it out

What I have tried now:
I've created 2 values, one for CurrentProgress and one for maxProgress, maxProgress is the total of all enumerated filesizes. I tried creating a class which holds each files full name and size so i can get the current filesize from the FileTransferProgressEventArgs.

This is my code per session
downloadSession.FileTransferProgress += (ds, ef) =>
{
    ServerFile currentFile = serverFiles.First(f => f.FullName == ef.FileName); //Collection i made
    CurrentProgress += (ef.FileProgress * currentFile.Lenght);
};

and this is the class of CurrentProgress
public double CurrentProgress
{
    get { return currentProgress; }
    set {
        currentProgress = value;
        lblLoading.Invoke((MethodInvoker)delegate { lblLoading.Text = "Download " + ((currentProgress / maxProgress) * 100).ToString() + "% completed"; });
    }
}
The thing is: now it goes beyond 100% and I don't know how or what is happening.

Reply with quote E-mail

martin
Site Admin
martin avatar
Joined:
Posts:
40,605
Location:
Prague, Czechia

Re: Still haven't figured it out

The FileTransferProgress is called many times for each file transfer.
So you keep adding all the already transferred file size repeatedly.
If you intended to count size of each file only once, the original code already does that. See the bytes variable.
If you really want to display real time progress for individual files, you need to remember the previous value of ef.FileProgress * currentFile.Lenght to be able to calculate/add only the newly transferred bytes.

Reply with quote

Advertisement

You can post new topics in this forum