iOS app exhausting memory during download of very large file

Question

I know variations of my question have been asked in one form or another many times on here - I've been searching these posts extensively for days and I think my problem is unique. Allow me to describe:

I have an iOS app on iPad 4 that downloads a large (260MB to 650MB) file to disk. Memory usage is increasing during the download proportional to the amount of data downloaded. At around 500MB, the app receives low memory warnings, and around 650MB it is killed for low memory.

I've repeatedly used Instruments to try to track down where the Allocation is coming from - I've ran Allocations, VM Allocations, Leaks, Memory monitor, and Activity monitor multiple times. I've also taken many heapshots as the memory increases. Instruments has never shown large amounts of allocations coming from my app! Memory remains flat. Leaks also doesn't report anything.

Instead I've been relying on the set of functions for asking the kernel about memory usage, that I found on another thread here:

   vm_size_t usedMemory(void) {
    struct task_basic_info info;
    mach_msg_type_number_t size = sizeof(info);
    kern_return_t kerr = task_info(mach_task_self(), TASK_BASIC_INFO, (task_info_t)&info, &size);
    return (kerr == KERN_SUCCESS) ? info.resident_size : 0; // size in bytes
}

vm_size_t freeMemory(void) {
    mach_port_t host_port = mach_host_self();
    mach_msg_type_number_t host_size = sizeof(vm_statistics_data_t) / sizeof(integer_t);
    vm_size_t pagesize;
    vm_statistics_data_t vm_stat;

    host_page_size(host_port, &pagesize);
    (void) host_statistics(host_port, HOST_VM_INFO, (host_info_t)&vm_stat, &host_size);
    return vm_stat.free_count * pagesize;
}

void logMemUsage(void) {
    // compute memory usage and log if different by >= 100k
    static long prevMemUsage = 0;
    long curMemUsage = usedMemory();
    long memUsageDiff = curMemUsage - prevMemUsage;

    // if (memUsageDiff > 100000 || memUsageDiff < -100000) {
    prevMemUsage = curMemUsage;
    //NSLog(@"Memory used %7.1f (%+5.0f), free %7.1f kb", curMemUsage/1000.0f, memUsageDiff///1000.0f, freeMemory()/1000.0f);
    printf("Memory used %7.1f (%+5.0f), free %7.1f kb\n", curMemUsage/1000.0f, memUsageDiff/1000.0f, freeMemory()/1000.0f);

    //}
}

Using this has allowed me to finally see that something is consuming memory in the download loop. I am using RequestQueue for the download along with the following method:

- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data
{
    if (_filePath && _filePath.length)
    {
        if (_fileHandle == nil)
        {
            [[NSFileManager defaultManager] createFileAtPath:_filePath contents:nil attributes:nil];
            self.fileHandle = [NSFileHandle fileHandleForUpdatingAtPath:_filePath];
            _bytesWritten = _rangeStart;
        }

        [_fileHandle writeData:data];
        _bytesWritten += [data length];
        NSInteger totalBytes = MAX(0, _responseReceived.expectedContentLength) + _rangeStart;

        if (_downloadProgressHandler)
        {
            _downloadProgressHandler(_userData, _userData2, (float)_bytesWritten / (float)totalBytes, _bytesWritten, totalBytes);
        }
    }
}

Even if I comment out the [_fileHandle writeData:data] call, I still observe memory increasing! I've double-checked that I'm closing the file handle and setting it to nil when done, but the memory growth is happening during the download.

Note: this project is using ARC.

I'm out of things to try at this point. Not writing the data at all to file still causes memory to grow. Disabling the UIStatusBar that shows the download progress doesn't help. I can't see anything in Instruments. I was going to try using an NSOutputStream to write to file instead, but I'm doubtful that will help as not writing the data at all still causes memory growth.

I tried using an NSURLCache and clearing the cache when the low memory warning is received. I tried assigning nil to data in the didReceiveData method, but that didn't change anything. I also experimented with changing as many strong pointers as possible to weak, but that just resulted in selectors being called on deallocated instances.

Any suggestions are very welcome at this point.

Answer 1

The fact that you're not seeing it in Instruments suggests that you have a difference between your Release and Debug configurations that is significant. Do you receive memory warnings under Instruments?

Edit your Scheme, select the "Profile" options and select Build Configuration "Debug." See if that matches what you're seeing when you use "Run". If so, then from there you can use Instruments to track this down.

It's a long-shot, but one thing to verify is that you haven't turned on NSZombies by accident. If you did, you would see this kind of behavior. Go to your Scheme and check the "Run" options. Look on the Diagnostics pane and make sure nothing is selected.