Core Data saving in multiple threads

Question

I'm little bit confused about Core Data multithreading saving.

I have following NSManagedObjectContext setup (same as MagicalRecord):

SavingContext (NSPrivateQueueConcurrencyType) has child DefaultContext(NSMainQueueConcurrencyType)

Each saving thread has own context (NSPrivateQueueConcurrencyType) with DefaultContext as parent.

So the question is: how can I rely on saving same type on different threads if I need to guarantee uniqueness?

Here is small test example (Test is subclass of NSManagedObject):

@implementation Test

+ (instancetype) testWithValue:(NSString *) str {
    [NSThread sleepForTimeInterval:3];
    Test *t = [Test MR_findFirstByAttribute:@"uniqueField" withValue:str];

    if (!t) {
        NSLog(@"No test found!");
        t = [Test MR_createEntity];
    }

    t.uniqueField = str;

    return t;
}
@end

It first checks if there is a Test in newly created thread context (which has parent DefaultContext) and if no - create it in current thread context.

And here is the test code:

NSOperationQueue *queue = [[NSOperationQueue alloc] init];
queue.maxConcurrentOperationCount = 2;

[queue addOperationWithBlock:^{
    [Test operationWithValue:@"1"];
    [[NSManagedObjectContext MR_contextForCurrentThread] MR_saveToPersistentStoreAndWait];
}];

[queue addOperationWithBlock:^{
    [Test operationWithValue:@"1"];
    [[NSManagedObjectContext MR_contextForCurrentThread] MR_saveToPersistentStoreAndWait];
}];

dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(5 * NSEC_PER_SEC)), dispatch_get_main_queue(), ^{
    NSLog(@"Total tests: %lu", (unsigned long)[Test MR_countOfEntities]);
    [Test MR_truncateAll];
    [[NSManagedObjectContext MR_defaultContext] MR_saveToPersistentStoreAndWait];
});

It just run two operations, and trying to save same data. After creating Test I save all contexts (current thread, default context and root saving context). Most of the time there will be 2 tests. You can modify and add semaphore to ensure both threads reach checking at the same time.

Update 20.09.2014 I've added the code, provided by mitrenegade (see below) and now my Test.m has a function:

-(BOOL)validateUniqueField:(id *)ioValue error:(NSError **)outError {

    // The property being validated must not already exist

    NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:NSStringFromClass([self class])];
    fetchRequest.predicate = [NSPredicate predicateWithFormat:@"uniqueField == %@ AND self != %@", *ioValue, self];

    int count = [self.managedObjectContext countForFetchRequest:fetchRequest error:nil];
    if (count > 0) {
        NSLog(@"Thread: %@ (isMain: %hhd), Validation failed!", [NSThread currentThread], [NSThread isMainThread]);
        return NO;
    }

    NSLog(@"Thread: %@ (isMain: %hhd), Validation succeeded!", [NSThread currentThread], [NSThread isMainThread]);
    return YES;
}

With two threads creating the same value (test sample is in the beginning of post) I have the following output:

2014-09-20 11:48:53.824 coreDataTest[892:289814] Thread: <NSThread: 0x15d38940>{number = 3, name = (null)} (isMain: 0), Validation succeeded!
2014-09-20 11:48:53.826 coreDataTest[892:289815] Thread: <NSThread: 0x15e434a0>{number = 2, name = (null)} (isMain: 0), Validation succeeded!
2014-09-20 11:48:53.830 coreDataTest[892:289750] Thread: <NSThread: 0x15e172c0>{number = 1, name = main} (isMain: 1), Validation failed!
2014-09-20 11:48:53.833 coreDataTest[892:289750] Thread: <NSThread: 0x15e172c0>{number = 1, name = main} (isMain: 1), Validation failed!
2014-09-20 11:48:53.837 coreDataTest[892:289750] Thread: <NSThread: 0x15e172c0>{number = 1, name = main} (isMain: 1), Validation failed!
2014-09-20 11:48:53.839 coreDataTest[892:289750] Thread: <NSThread: 0x15e172c0>{number = 1, name = main} (isMain: 1), Validation failed!
2014-09-20 11:48:56.251 coreDataTest[892:289750] Total tests: 2

But if I look at underlying sqlite file there is no records at all (that means they stuck in Main Context)

Answer 1

There doesn't seem to be any sort of validation checking for your actual objects, so the fact that two objects with the "uniqueField" attribute set to "1" doesn't mean they can't exist at the same time, according to the model you've provided.

While both threads are operating, each inserts a new object with some value ("1") associated with some attribute ("uniqueField"). When Core Data merges the contexts, there's no rules saying that this is prohibited, so there will be two objects in the main context. They are unique objects with unique objectIDs. The same thing would happen if you created two "Person" objects with "name" = "John".

Core data automatically calls certain validation methods for each field if you format the signature correctly, as seen here.

https://developer.apple.com/library/ios/documentation/Cocoa/Conceptual/KeyValueCoding/Articles/Validation.html.

In your NSManagedObject subclass (Test.m), you need to have a method with the signature

-(BOOL)valide<YourFieldName>:error:

So try adding this code to your Test.m, and put a break point on it. This method should get called when the context is saved.

-(BOOL)validateUniqueField:(id *)ioValue error:(NSError * __autoreleasing *)outError{

    // The property being validated must not already exist

    NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:NSStringFromClass([self class])];
    fetchRequest.predicate = [NSPredicate predicateWithFormat:@"uniqueField == %@", *ioValue];

    int count = [self.managedObjectContext countForFetchRequest:fetchRequest error:nil];
    if (count > 0) {
        if (outError != NULL) {
            NSString *errorString = NSLocalizedString(
                                                          @"Object must have unique value for property",
                                                          @"validation: nonunique property");
                NSDictionary *userInfoDict = @{ NSLocalizedDescriptionKey : errorString };
                *outError = [[NSError alloc] initWithDomain:nil
                                                       code:0
                                                   userInfo:userInfoDict];
        }
        return NO;
    }
    return YES;
}

When the context saves, this validation is automatically called by core data. you can do whatever you want inside here; i'm adding logic that does a fetch and compares the count.

Edit: I posed this question shortly after this topic, and received some answers but nothing super definitive. So I want to put it out there that my answer works for our current situation but is apparently not a good one for efficiency. However, i have not yet found a solution that works for multiple threads without doing stuff in validateForInsert. As far as I can tell, there's no way to just set a parameter to be unique in the database.

Is doing a fetch request in validateForInsert overly expensive

Answer 2

MagicalRecord already implements much of the work required to perform saves on the background queue. Have a look at +[MagicalRecord saveWithBlock] (or [MagicalRecordStack saveWithBlock] in MR 3.0). This method will dispatch save operations onto a background queue for you. However, in order for this to work properly, you must do your data updates on the background context as to not cross threading boundaries. Generally, use the following pattern:

Test *testObj = ///....
[MagicalRecord saveWithBlock:^(NSManagedObjectContext *localContext){

   Test *localTest = [testObj MR_inContext:localContext];
   //localTest.property = newValue
}]; ///Exiting the block will save the context and your changes.

Answer 3

Take a look at this link for optimum concurrency setup.

The best way I have found in coordinating saves between multiple threads is through notifications (NSManagedObjectContextDidSaveNotification). When the one context saves the system sends notifications to other contexts with an object containing IDs of the objects that were affected. Those context can then use those objects and merge the saves.

For a detailed explanation for the setup take a look at this post.