Average progress of all the NSURLSessionTasks in a NSURLSession

Question

An NSURLSession will allow you to add to it a large number of NSURLSessionTask to download in the background.

If you want to check the progress of a single NSURLSessionTask, it’s as easy as

double taskProgress = (double)task.countOfBytesReceived / (double)task.countOfBytesExpectedToReceive;

But what is the best way to check the average progress of all the NSURLSessionTasks in a NSURLSession?

I thought I’d try averaging the progress of all tasks:

[[self backgroundSession] getTasksWithCompletionHandler:^(NSArray *dataTasks, NSArray *uploadTasks, NSArray *allDownloadTasks) {

    double totalProgress = 0.0;

    for (NSURLSessionDownloadTask *task in allDownloadTasks) {

        double taskProgress = (double)task.countOfBytesReceived / (double)task.countOfBytesExpectedToReceive;

        if (task.countOfBytesExpectedToReceive > 0) {
            totalProgress = totalProgress + taskProgress;
        }
        NSLog(@"task %d: %.0f/%.0f - %.2f%%", task.taskIdentifier, (double)task.countOfBytesReceived, (double)task.countOfBytesExpectedToReceive, taskProgress*100);
    }

    double averageProgress = totalProgress / (double)allDownloadTasks.count;

    NSLog(@"total progress: %.2f, average progress: %f", totalProgress, averageProgress);
    NSLog(@" ");

}];

But the logic here is wrong: Suppose you have 50 tasks expecting to download 1MB and 3 tasks expecting to download 100MB. If the 50 small tasks complete before the 3 large tasks, averageProgress will be much higher than the actual average progress.

So you have to calculate average progress according to the TOTAL countOfBytesReceived divided by the TOTAL countOfBytesExpectedToReceive. But the problem is that a NSURLSessionTask figures out those values only once it starts, and it might not start until another task finishes.

So how do you check the average progress of all the NSURLSessionTasks in a NSURLSession?

Answer 1

Ah yes. I remember dealing with this back in 1994, when I wrote OmniWeb. We tried a number of solutions, including just having the progress bar spin instead of show progress (not popular), or having it grow as new tasks figured out how big they would be / got added to the queue (made users upset because they saw reverse progress sometimes).

In the end what most programs have decided to use (including Messages in iOS 5, and Safari) is a kind of cheat: for instance, in Messages they knew the average time to send a message was about 1.5 seconds (example numbers only), so they animated the progress bar to finish at about 1.5 seconds, and would just delay at 1.4 seconds if the message hadn’t actually gotten sent yet.

Modern browsers (like Safari) vary this approach by dividing the task into sections, and showing a progress bar for each section. Like (example only), Safari might figure that looking up a URL in DNS will usually take 0.2 seconds, so they’ll animate the first 1/10th (or whatever) of the progress bar over 0.2 seconds, but of course they’ll skip ahead (or wait at the 1/10th mark) if the DNS lookup takes shorter or longer respectively.

In your case I don’t know how predictable your task is, but there should be a similar cheat. Like, is there an average size for most files? If so, you should be able to figure out about how long 50 will take. Or you just divide your progress bar into 50 segments and fill in a segment each time a file completes, and animate based on the current number of bytes / second you’re getting or based on the number of files / second you’ve gotten so far or any other metric you like.

One trick is to use Zeno’s paradox if you have to start or stop the progress bar—don’t just halt or jump to the next mark, instead just slow down (and keep slowing down) or speed up (and keep speeding up) until you’ve gotten to where the bar needs to be.

Good luck!

Answer 2

Before you start downloading the files you can send tiny small HEAD requests to get the file-sizes. You simply add their expectedContentLength and have your final download size.

- (void)sizeTaskForURL:(NSURL *)url
{

    NSMutableURLRequest *request = [[NSMutableURLRequest alloc] initWithURL:url];
    [request setHTTPMethod:@"HEAD"];

    NSURLSessionDataTask *sizeTask =
    [[[self class] dataSession]
     dataTaskWithRequest:request
     completionHandler: ^(NSData *data, NSURLResponse *response, NSError *error)
     {
         if (error == nil)
         {
             NSHTTPURLResponse *httpResponse = (NSHTTPURLResponse *)response;

             if ([httpResponse statusCode] == 200) {

                 totalBytesExpectedToReceive += (double)[httpResponse expectedContentLength];

                 numberOfFileSizesReceived++;

                 NSLog(@"%lu/%lu files found. file size: %f", (unsigned long)numberOfFileSizesReceived, (unsigned long)numberOfTasks, totalBytesExpectedToReceive);

                 if (numberOfFileSizesReceived == numberOfTasks){
                     NSLog(@"%lu/%lu files found. total file size: %f", (unsigned long)numberOfFileSizesReceived, (unsigned long)numberOfTasks, totalBytesExpectedToReceive);
                 }
             }
             else {
                 NSLog(@"Bad status code (%ld) for size task at URL: %@", (long)[httpResponse statusCode], [[response URL] absoluteString]);
             }
         }
         else
         {
             NSLog(@"Size task finished with error: %@", error.localizedDescription);
         }
     }];

    [sizeTask resume];

}

Download the files afterwards

This is how it looks like:

On the left you can see that when it sends the HEAD requests it does not yet start the UIProgressView. When it has finished it downloads the files.

So if you download large files it might be useful to "waste" those seconds making HEAD requests and henceforth show the user the correct progress instead of some wrong progress.

During the downloads, you (definitely) want to use the delegate methods to get smaller subdivisions of new data (otherwise the progress view will "jump").

Answer 3

Yours is a specific case of the "progress bar" problem. Confer, e.g. this reddit thread.

It's been around forever, and as Wil seemed to be saying, it can be bedevilingly hard even for Megacorp, Inc.™ to get "just so". E.g. even with completely accurate MB values, you can easily hang with no "progress", just due to network problems. Your app is still working, but the perception may be that your program is hung.

And the quest to provide extremely "accurate" values can overcomplicate the progressing task. Tread carefully.

Answer 4

I have two possible solutions for you. Neither get exactly what you are looking for, but both give the user an understanding of what is going on.

First solution you have multiple progress bars. One large bar that indicates file number progress (finished 50 out of 200). Then you have multiple progress bars beneath that (number of them equal to the amount of concurrent downloads possible, in my case this is 4, in your it may be unwieldy). So the user knows both fine grain detail about the downloads and gets an overall total progress (that does not move with download bytes, but with download completion).

Second solution is a multi-file progress bar. This one can be deceiving because the files are of differing sizes, but you could create a progress bar that is cut into a number of chunks equal to your file download count. Then each chunk of the bar goes from 0% to 100% based on single file's download. So you could have middle sections of the progress bar filled while the beginning and ending are empty (files not downloaded).

Again, neither of these are a solution to the question of how to get total bytes to download from multiple files, but they are alternative UI so the user understands everything going on at any given time. (I like option 1 best)