CoreData - delete all entities consumes RAM and take a long time

Question

I have a CoreData configuration where I stored measured data. The data is grouped by Session, so each Session entity has a one-to-many relationship with SensorData. The Delete Rule is set to Cascade. I also have a Delete All button. When it is pressed I run the following code.

// All sessions (cascading)
[self.context performBlockAndWait:^{

    // Fetch only managedObjectID to reduce memory impact
    NSFetchRequest *sessionsRequest = [NSFetchRequest fetchRequestWithEntityName:@"Session"];
    [sessionsRequest setIncludesPropertyValues:NO];

    NSError *sessionsError;
    NSArray *allSessions = [self.context executeFetchRequest:sessionsRequest error:&sessionsError];

    if (sessionsError) {
        NSLog(@"Failed to fetch all sessions. %@", sessionsError);
    }

    // Release fetch request
    sessionsRequest = nil;

    // Loop and delete
    int i = 0;
    for (NSManagedObject *session in allSessions) {
        NSLog(@"Deleting session (%d / %d)", ++i, allSessions.count);
        if (i != allSessions.count) {
            [self.context deleteObject:session];
        }
    }


    NSLog(@"All session deleted.");
}];

NSLog(@"Saving.");
[self.context performBlock:^{
    [self.document saveToURL:self.documentPath
        forSaveOperation:UIDocumentSaveForOverwriting
       completionHandler:^(BOOL success){
           NSLog(@"Document saved %@.", success ? @"successfully" : @"unsuccessfully");
       }];
}];
NSLog(@"Done saving.");

What happens is that I get the log output as below, so that executes fairly quickly. But the UI then freezes and RAM usage rockets. There are ~60 sessions and ~1M measurements (3 float values per measurement) and eventually RAM usage is too large and app crashes after 20 min or so, with all entries still there.

Obviously the saving is at play here, but what am I doing wrong? Grateful for any pointers.

Log output:

2015-08-13 15:16:56.825 MyApp[4201:1660697] Deleting session (1 / 61)
2015-08-13 15:16:56.826 MyApp[4201:1660697] Deleting session (2 / 61)
.
.
.
2015-08-13 15:16:56.862 MyApp[4201:1660697] Deleting session (60 / 61)
2015-08-13 15:16:56.863 MyApp[4201:1660697] Deleting session (61 / 61)
2015-08-13 15:16:56.863 MyApp[4201:1660697] All session deleted.
2015-08-13 15:16:56.864 MyApp[4201:1660697] Saving.

2015-08-13 15:16:56.864 MyApp[4201:1660697] Done saving.

Update

I edited the procedure to first delete the measurement data and by setting a fetch limit to keep RAM down (as suggested). However, if I delete 200 of SensorData, saving takes around 3s, except for the first ~1000 entries. Deletion is fast though. See trace below.

I would like to fix this without deleting the document. Feels like a hack (albeit probably a good one).

Trace

How are you setting up your CoreData stack? Are you using UIManagedDocument? Is self.context the main thread NSManagedObjectContext? What is your OS Deployment Target Version? — GayleDDS, Aug 13 '15 at 14:39
It looks like you're using `UIDocument`, in which case it's probably easier to delete the current document. — Tom Harrington, Aug 13 '15 at 16:20
Yes, I am using an UIDocument. `self.context` is the NSManagedObjectContext of the UIDocument (i.e. Deployment target is iOS8 (as vaguely specified in tag). I also tried changing the `performBlock` to `performBlockAndWait`, which executes quickly. — T'n'E, Aug 14 '15 at 07:14

score 2 · Accepted Answer · answered Aug 13 '15 at 16:00

the thing that takes a lot of memory is that you are loading all sessions :

NSArray *allSessions = [self.context executeFetchRequest:sessionsRequest error:&sessionsError];

try to load for example 100 by 100 like this :

NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];
[self.context setUndoManager:nil];
NSEntityDescription *entity = [NSEntityDescription entityForName:@"Session" inManagedObjectContext:self.context];
[fetchRequest setEntity:entity];
[fetchRequest setIncludesPropertyValues:NO];
[fetchRequest setFetchLimit:100];
NSError *error;
NSArray *items = [self.context executeFetchRequest:fetchRequest error:&error];
while ([items count] > 0) {
    @autoreleasepool {
        for (NSManagedObject *item in items) {
            [self.context deleteObject:item];
        }
        if (![self.context save:&error]) {
            NSLog(@"Error deleting %@ - error:%@",self.entityName, error);
        }
    }
    items = [self.context executeFetchRequest:fetchRequest error:&error];
}

This wouldn't work right of the bat, as most of the data is not contained in `Session`, but in `SensorData`. So even though I set the limit to 1, it could still fetch in 100k `SensorData`. But it worked very well if I used this method to directly delete `SensorData` first, then delete the `Session`. However, after deleting 200 `SensorData`, saving can take up to 3 seconds. Any clues why? — T'n'E, Aug 14 '15 at 09:05

score 1 · Answer 2 · answered Aug 13 '15 at 14:45

As a quick fix without knowing the answers to the above questions. I would delete all the measurement data first then delete the session data. But most importantly you should always set a batch size on your fetch request.

// Fetch only managedObjectID to reduce memory impact
NSFetchRequest *sessionsRequest = [NSFetchRequest fetchRequestWithEntityName:@"Session"];
[sessionsRequest setIncludesPropertyValues:NO];
[sessionsRequest setFetchBatchSize:20];

The batch size of the receiver.

The default value is 0. A batch size of 0 is treated as infinite, which disables the batch faulting behavior.

If you set a non-zero batch size, the collection of objects returned when the fetch is executed is broken into batches. When the fetch is executed, the entire request is evaluated and the identities of all matching objects recorded, but no more than batchSize objects’ data will be fetched from the persistent store at a time. The array returned from executing the request will be a proxy object that transparently faults batches on demand. (In database terms, this is an in-memory cursor.)

You can use this feature to restrict the working set of data in your application. In combination with fetchLimit, you can create a subrange of an arbitrary result set.

For purposes of thread safety, you should consider the array proxy returned when the fetch is executed as being owned by the managed object context the request is executed against, and treat it as if it were a managed object registered with that context.

score 1 · Answer 3 · edited May 23 '17 at 11:43

As Tom said, it may be easier to delete the entire document, and start with a fresh document.

iOS9 introduces NSBatchDeleteRequest which should help you. You can see the WWDC video here: https://developer.apple.com/videos/wwdc/2015/?id=220, and that part is discussed starting 15 minutes into the video.

Here is a link to a similar question/answer: Core Data: delete all objects of an entity type, ie clear a table

CoreData - delete all entities consumes RAM and take a long time

3 Answers3