I have an app that has been out in the wild for many years.
This app, in order to be 100% functional while offline, needs to download hundreds of thousands of images (1 for each object) one time only (delta updates are processed as needed).
The object data itself comes down without issue.
However, recently, our app has started crashing while downloading just the images, but only on newer iPads (3rd gen iPad Pros with plenty of storage).
The image download process uses NSURLSession download tasks inside an NSOperationQueue.
We were starting to see Energy Logs stating that CPU usage was too high, so we modified our parameters to add a break between each image, as well as between each batch of image, using
[NSThread sleepForTimeInterval:someTime];
This reduced our CPU usage from well above 95% (which, fair enough) to down below 18%!
Unfortunately, the app would still crash on newer iPads after only a couple of hours. However, on our 2016 iPad Pro 1st Gen, the app does not crash at all, even after 24 hours of downloading.
When pulling crash logs from the devices, all we see is that CPU usage was over 50% for more than 3 minutes. No other crash logs come up.
These devices are all plugged in to power, and have their lock time set to never in order to allow the iPad to remain awake and with our app in the foreground.
In an effort to solve this issue, we turned our performance way down, basically waiting 30 seconds in between each image, and 2 full minutes between each batch of images. This worked and the crashing stopped, however, this would take days to download all of our images.
We are trying to find a happy medium where the performance is reasonable, and the app does not crash.
However, what is haunting me, is that no matter the setting, and even at full-bore performance, the app never crashes on the older devices, it only crashes on the newer devices.
Conventional wisdom would suggest that should not be possible.
What am I missing here?
When I profile using Instruments, I see the app sitting at a comfortable 13% average while downloading, and there is a 20 second gap in between batches, so the iPad should have plenty of time to do any cleanup.
Anyone have any ideas? Feel free to request additional information, I'm not sure what else would be helpful.
EDIT 1: Downloader Code Below:
//Assume the following instance variables are set up:
self.operationQueue = NSOperationQueue to download the images.
self.urlSession = NSURLSession with ephemeralSessionConfiguration, 60 second timeoutIntervalForRequest
self.conditions = NSMutableArray to house the NSConditions used below.
self.countRemaining = NSUInteger which keeps track of how many images are left to be downloaded.
//Starts the downloading process by setting up the variables needed for downloading.
-(void)startDownloading
{
//If the operation queue doesn't exist, re-create it here.
if(!self.operationQueue)
{
self.operationQueue = [[NSOperationQueue alloc] init];
[self.operationQueue addObserver:self forKeyPath:KEY_PATH options:0 context:nil];
[self.operationQueue setName:QUEUE_NAME];
[self.operationQueue setMaxConcurrentOperationCount:2];
}
//If the session is nil, re-create it here.
if(!self.urlSession)
{
self.urlSession = [NSURLSession sessionWithConfiguration:[NSURLSessionConfiguration ephemeralSessionConfiguration]
delegate:self
delegateQueue:nil];
}
if([self.countRemaining count] == 0)
{
[self performSelectorInBackground:@selector(startDownloadForNextBatch:) withObject:nil];
self.countRemaining = 1;
}
}
//Starts each batch. Called again on observance of the operation queue's task count being 0.
-(void)startDownloadForNextBatch:
{
[NSThread sleepForTimeInterval:20.0]; // 20 second gap between batches
self.countRemaining = //Go get the count remaining from the database.
if (countRemaining > 0)
{
NSArray *imageRecordsToDownload = //Go get the next batch of URLs for the images to download from the database.
[imageRecordsToDownload enumerateObjectsUsingBlock:^(NSDictionary *imageRecord,
NSUInteger index,
BOOL *stop)
{
NSInvocationOperation *invokeOp = [[NSInvocationOperation alloc] initWithTarget:self
selector:@selector(downloadImageForRecord:)
object:imageRecord];
[self.operationQueue addOperation:invokeOp];
}];
}
}
//Performs one image download.
-(void)downloadImageForRecord:(NSDictionary *)imageRecord
{
NSCondition downloadCondition = [[NSCondition alloc] init];
[self.conditions addObject:downloadCondition];
[[self.urlSession downloadTaskWithURL:imageURL
completionHandler:^(NSURL *location,
NSURLResponse *response,
NSError *error)
{
if(error)
{
//Record error below.
}
else
{
//Move the downloaded image to the correct directory.
NSError *moveError;
[[NSFileManager defaultManager] moveItemAtURL:location toURL:finalURL error:&moveError];
//Create a thumbnail version of the image for use in a search grid.
}
//Record the final outcome for this record by updating the database with either an error code, or the file path to where the image was saved.
//Sleep for some time to allow the CPU to rest.
[NSThread sleepForTimeInterval:0.05]; // 0.05 second gap between images.
//Finally, signal our condition.
[downloadCondition signal];
}]
resume];
[downloadCondition lock];
[downloadCondition wait];
[downloadCondition unlock];
}
//If the downloads need to be stopped, for whatever reason (i.e. the user logs out), this function is called to stop the process entirely:
-(void)stopDownloading
{
//Immediately suspend the queue.
[self.operationQueue setSuspended:YES];
//If any conditions remain, signal them, then remove them. This was added to avoid deadlock issues with the user logging out and then logging back in in rapid succession.
[self.conditions enumerateObjectsUsingBlock:^(NSCondition *condition,
NSUInteger idx,
BOOL * _Nonnull stop)
{
[condition signal];
}];
[self setConditions:nil];
[self setConditions:[NSMutableArray array]];
[self.urlSession invalidateAndCancel];
[self setImagesRemaining:0];
[self.operationQueue cancelAllOperations];
[self setOperationQueue:nil];
}
EDIT 2: CPU usage screenshot from Instruments. Peaks are ~50%, valleys are ~13% CPU usage.
EDIT 3: Running the app until failure in Console, observed memory issue
Alright! Finally observed the crash on my iPhone 11 Pro after over an hour downloading images, which matches the scenario reported by my other testers.
The Console reports my app was killed specifically for using too much memory. If I am reading this report correctly, my apps used over 2 GB of RAM. I'm assuming that this has to do more with the internal management of NSURLSESSIOND, since it is not showing this leak during debugging with either Xcode or Instruments.
Console reports: "kernel 232912.788 memorystatus: killing_specific_process pid 7075 [PharosSales] (per-process-limit 10) 2148353KB - memorystatus_available_pages: 38718"
Thankfully, I start receiving memory warnings around the 1 hour mark. I should be able to pause (suspend) my operation queue for some time (let's say 30 seconds) in order to let the system clear its memory.
Alternatively, I could call stop, with a gcd dispatch after call to start again.
What do you guys think about this solution? Is there a more elegant way to respond to memory warnings?
Where do you think this memory usage is coming from?
