Making Core Data Thread-safe

Question

Long story short, I'm tired of the absurd concurrency rules associated with NSManagedObjectContext (or rather, its complete lack of support for concurrency and tendency to explode or do other incorrect things if you attempt to share an NSManagedObjectContext across threads), and am trying to implement a thread-safe variant.

Basically what I've done is created a subclass that tracks the thread that it was created on, and then maps all method invocations back to that thread. The mechanism for doing this is slightly convoluted, but the crux of it is that I've got some helper methods like:

- (NSInvocation*) invocationWithSelector:(SEL)selector {
    //creates an NSInvocation for the given selector
    NSMethodSignature* sig = [self methodSignatureForSelector:selector];    
    NSInvocation* call = [NSInvocation invocationWithMethodSignature:sig];
    [call retainArguments];
    call.target = self;

    call.selector = selector;

    return call;
}

- (void) runInvocationOnContextThread:(NSInvocation*)invocation {
    //performs an NSInvocation on the thread associated with this context
    NSThread* currentThread = [NSThread currentThread];
    if (currentThread != myThread) {
        //call over to the correct thread
        [self performSelector:@selector(runInvocationOnContextThread:) onThread:myThread withObject:invocation waitUntilDone:YES];
    }
    else {
        //we're okay to invoke the target now
        [invocation invoke];
    }
}


- (id) runInvocationReturningObject:(NSInvocation*) call {
    //returns object types only
    [self runInvocationOnContextThread:call];

    //now grab the return value
    __unsafe_unretained id result = nil;
    [call getReturnValue:&result];
    return result;
}

...and then the subclass implements the NSManagedContext interface following a pattern like:

- (NSArray*) executeFetchRequest:(NSFetchRequest *)request error:(NSError *__autoreleasing *)error {
    //if we're on the context thread, we can directly call the superclass
    if ([NSThread currentThread] == myThread) {
        return [super executeFetchRequest:request error:error];
    }

    //if we get here, we need to remap the invocation back to the context thread
    @synchronized(self) {
        //execute the call on the correct thread for this context
        NSInvocation* call = [self invocationWithSelector:@selector(executeFetchRequest:error:) andArg:request];
        [call setArgument:&error atIndex:3];
        return [self runInvocationReturningObject:call];
    }
}

...and then I'm testing it with some code that goes like:

- (void) testContext:(NSManagedObjectContext*) context {
    while (true) {
        if (arc4random() % 2 == 0) {
            //insert
            MyEntity* obj = [NSEntityDescription insertNewObjectForEntityForName:@"MyEntity" inManagedObjectContext:context];
            obj.someNumber = [NSNumber numberWithDouble:1.0];
            obj.anotherNumber = [NSNumber numberWithDouble:1.0];
            obj.aString = [NSString stringWithFormat:@"%d", arc4random()];

            [context refreshObject:obj mergeChanges:YES];
            [context save:nil];
        }
        else {
            //delete
            NSArray* others = [context fetchObjectsForEntityName:@"MyEntity"];
            if ([others lastObject]) {
                MyEntity* target = [others lastObject];
                [context deleteObject:target];
                [context save:nil];
            }
        }
        [NSThread sleepForTimeInterval:0.1];
    }
}

So essentially, I spin up some threads targeting the above entry point, and they randomly create and delete entities. This almost works the way it should.

The problem is that every so often one of the threads will get an EXC_BAD_ACCESS when calling obj.<field> = <value>;. It's not clear to me what the problem is, because if I print obj in the debugger everything looks good. Any suggestions on what the problem might be (other than the fact that Apple recommends against subclassing NSManagedObjectContext) and how to fix it?

P.S. I'm aware of GCD and NSOperationQueue and other techniques typically used to "solve" this problem. None of those offer what I want. What I'm looking for is an NSManagedObjectContext that can be freely, safely, and directly used by any number of threads to view and change application state without requiring any external synchronization.

Answer 1

As noa rightly pointed out, the problem was that although I had made the NSManagedObjectContext thread-safe, I had not instrumented the NSManagedObject instances themselves to be thread-safe. Interactions between the thread-safe context and the non-thread-safe entities were responsible for my periodic crashes.

In case anyone is interested, I created a thread-safe NSManagedObject subclass by injecting my own setter methods in lieu of (some of) the ones that Core Data would normally generate. This is accomplished using code like:

//implement these so that we know what thread our associated context is on
- (void) awakeFromInsert {
    myThread = [NSThread currentThread];
}
- (void) awakeFromFetch {
    myThread = [NSThread currentThread];
}

//helper for re-invoking the dynamic setter method, because the NSInvocation requires a @selector and dynamicSetter() isn't one
- (void) recallDynamicSetter:(SEL)sel withObject:(id)obj {
    dynamicSetter(self, sel, obj);
}

//mapping invocations back to the context thread
- (void) runInvocationOnCorrectThread:(NSInvocation*)call {
    if (! [self myThread] || [NSThread currentThread] == [self myThread]) {
        //okay to invoke
        [call invoke];
    }
    else {
        //remap to the correct thread
        [self performSelector:@selector(runInvocationOnCorrectThread:) onThread:myThread withObject:call waitUntilDone:YES];
    }
}

//magic!  perform the same operations that the Core Data generated setter would, but only after ensuring we are on the correct thread
void dynamicSetter(id self, SEL _cmd, id obj) {
    if (! [self myThread] || [NSThread currentThread] == [self myThread]) {
        //okay to execute
        //XXX:  clunky way to get the property name, but meh...
        NSString* targetSel = NSStringFromSelector(_cmd);
        NSString* propertyNameUpper = [targetSel substringFromIndex:3];  //remove the 'set'
        NSString* firstLetter = [[propertyNameUpper substringToIndex:1] lowercaseString];
        NSString* propertyName = [NSString stringWithFormat:@"%@%@", firstLetter, [propertyNameUpper substringFromIndex:1]];
        propertyName = [propertyName substringToIndex:[propertyName length] - 1];

        //NSLog(@"Setting property:  name=%@", propertyName);

        [self willChangeValueForKey:propertyName];
        [self setPrimitiveValue:obj forKey:propertyName];
        [self didChangeValueForKey:propertyName];

    }
    else {
        //call back on the correct thread
        NSMethodSignature* sig = [self methodSignatureForSelector:@selector(recallDynamicSetter:withObject:)];
        NSInvocation* call = [NSInvocation invocationWithMethodSignature:sig];
        [call retainArguments];
        call.target = self;
        call.selector = @selector(recallDynamicSetter:withObject:);
        [call setArgument:&_cmd atIndex:2];
        [call setArgument:&obj atIndex:3];

        [self runInvocationOnCorrectThread:call];
    }
}

//bootstrapping the magic; watch for setters and override each one we see
+ (BOOL) resolveInstanceMethod:(SEL)sel {
    NSString* targetSel = NSStringFromSelector(sel);
    if ([targetSel startsWith:@"set"] && ! [targetSel contains:@"Primitive"]) {
        NSLog(@"Overriding selector:  %@", targetSel);
        class_addMethod([self class], sel, (IMP)dynamicSetter, "v@:@");
        return YES;
    }

    return [super resolveInstanceMethod:sel];
}

This, in conjunction with my thread-safe context implementation, solved the problem and got me what I wanted; a thread-safe context that I can pass around to whomever I want without having to worry about the consequences.

Of course this is not a bulletproof solution, as I have identified at least the following limitations:

/* Also note that using this tool carries several small caveats:
 *
 *      1.  All entities in the data model MUST inherit from 'ThreadSafeManagedObject'.  Inheriting directly from 
 *          NSManagedObject is not acceptable and WILL crash the app.  Either every entity is thread-safe, or none 
 *          of them are.
 *
 *      2.  You MUST use 'ThreadSafeContext' instead of 'NSManagedObjectContext'.  If you don't do this then there 
 *          is no point in using 'ThreadSafeManagedObject' (and vice-versa).  You need to use the two classes together, 
 *          or not at all.  Note that to "use" ThreadSafeContext, all you have to do is replace every [[NSManagedObjectContext alloc] init]
 *          with an [[ThreadSafeContext alloc] init].
 *
 *      3.  You SHOULD NOT give any 'ThreadSafeManagedObject' a custom setter implementation.  If you implement a custom 
 *          setter, then ThreadSafeManagedObject will not be able to synchronize it, and the data model will no longer 
 *          be thread-safe.  Note that it is technically possible to work around this, by replicating the synchronization
 *          logic on a one-off basis for each custom setter added.
 *
 *      4.  You SHOULD NOT add any additional @dynamic properties to your object, or any additional custom methods named
 *          like 'set...'.  If you do the 'ThreadSafeManagedObject' superclass may attempt to override and synchronize 
 *          your implementation.
 *
 *      5.  If you implement 'awakeFromInsert' or 'awakeFromFetch' in your data model class(es), thne you MUST call 
 *          the superclass implementation of these methods before you do anything else.
 *
 *      6.  You SHOULD NOT directly invoke 'setPrimitiveValue:forKey:' or any variant thereof.  
 *
 */

However, for most typical small to medium-sized projects I think the benefits of a thread-safe data layer significantly outweigh these limitations.

Answer 2

Why not just instantiate your context using one of the provided concurrency types, and leverage performBlock / performBlockAndWait?

That implements the necessary thread confinement with having to mangle with the implementation of Core Data's accessor methods. Which, as you will soon find out will be either very painful to get right or end quite badly for your users.

Answer 3

A great tutorial by Bart Jacobs entitled: Core Data from Scratch: Concurrency for those that need an elegant solution for iOS 5.0 or later and/or Lion or later. Two approaches are described in detail, the more elegant solution involves parent/child managed object contexts.