0

I need to process a large amount of data from a NSDictionary into NSManagedObjects (3k-10k objects).

With synchronous processing, 3k takes about 60 seconds:

for (NSString *key in myDictionary) {
    //process data
}

When I use GCD and create a new thread for each key, it takes about 20 seconds and used a bunch of threads:

for (NSString *key in myDictionary) {
    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
        //process data
    }
}

Then I tried using NSOperationQueue with NSBLockOperation and for each key, adding a block execution which took about 30 seconds and only used 2 threads:

NSOperationQueue *queue = [[NSOperationQueue alloc] init];
NSBlockOperation *operation = [[NSBlockOperation alloc] init];

for (NSString *key in myDictionary) {
    [operation addExecutionBlock:^{
    // processs data
    }
}

[operation setCompletionBlock:^{
    DLog(@"Block completed | time: %f", [[NSDate date] timeIntervalSinceDate:startTime]);
}];

[queue addOperation:operation];

This gives the advantage of having a completion block.

In terms of speed, a thread per key seems to be the quickest.

All times where done on a device, not simulator.

I've read when considering NSOperation vs GCD, they suggest using the higher level API which is the NSOperation, but the GCD gives better performance.

Is there any advantage to using the NSOperationQueue and NSBlockOperation other then the completion block? Or am I missing something that makes the queue and blocks inherently better then GCD?

Padin215
  • 7,444
  • 13
  • 65
  • 103
  • Related: http://stackoverflow.com/questions/18632271/what-is-the-difference-between-nsinvocationoperation-and-nsblockoperation – JRG-Developer Feb 18 '14 at 20:28
  • 3
    There are about a half-dozen different ways to do threading in Objective-C. Unless there is a particular reason for picking one over the others, use the technique you understand the best and are most comfortable with. Threading is confusing enough as is and using a unfamiliar interface in a complex situation simply adds to the confusion. – Hot Licks Feb 18 '14 at 20:36

2 Answers2

2

Or am I missing something that makes the queue and blocks inherently better then GCD?

You're not missing anything -- operations are implemented using GCD. As you've seen, it can be nice to have a higher-level API than just straight GCD, but in the end operations aren't inherently different from using GCD.

Caleb
  • 124,013
  • 19
  • 183
  • 272
  • I'm curious why the operations only used 2 threads while the GCD used 30+ (thread ID's where in the 30s, not 100% sure there were 30+ threads)? – Padin215 Feb 18 '14 at 20:28
  • 1
    I do not think that more than one thread is used. There is a thread pool so the tread# you get is "random". GCD dispatches blocks on different thread from that pool, but AFAIK will never execute one block on different threads. (Your dispatch is outside the for loop.) Every thread has its own stack, so moving from one thread to another would be quite expensive. – Amin Negm-Awad Feb 18 '14 at 20:41
  • Then how would you explain that GCD is 10 seconds faster? I log [NSThread currentThread] and GCD, I see a lot of thread IDs while operation only uses 2 thread IDs? – Padin215 Feb 18 '14 at 20:47
  • It's hard to say for sure from here. I'd profile your code using Instruments to see what's going on. Also consider manually setting the operation queue's max concurrent operations yourself rather than relying on the default, just to see if that makes a difference. – Caleb Feb 18 '14 at 20:53
  • 1
    Okay, after your fix your Q is partially answered by yourself. Anyway if you do not set the concurrent operation count of an OQ, it will choose one. Maybe your task is done slower, but the systems thinks that it is better. – Amin Negm-Awad Feb 18 '14 at 20:54
  • I set the max concurrent operations to 20 and no speed improvement and still only seeing 2 thread IDs. – Padin215 Feb 18 '14 at 21:04
  • There are many possible explanations for the speed difference. One is that your `process data` blocks are relatively small, so that the overhead associated with each operation is a significant part of the execution time; you're not paying that price using GCD directly. But that's just speculation -- profiling is the only way to know for sure. Also, even an A7 processor has only two cores, so there's not much point in using more than two simultaneous threads. Your GCD code may just result in more short-lived threads being created and destroyed. – Caleb Feb 18 '14 at 21:09
  • 1
    Even you set the max count higher OQs selects a concrete number of threads up to this value. How do your iPhone react on having this GCD load? – Amin Negm-Awad Feb 18 '14 at 21:15
1

As mentioned operations and operations queues are built on top of GCD. But there are some differences:

  • Operation queues can have a width. GCD knows serial or concurrent queues only.
  • Operation queues can handle dependencies.
  • Operations can have a priority (added)
  • GCD can handle "run loop stuff" (system events)

As mentioned in my comment your GCD code uses only one thread:

dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
    for (NSString *key in myDictionary) {
        //process data
    }  
}

Try this and report the result, please:

for (NSString *key in myDictionary) {
    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
        //process data
    }  
}
Amin Negm-Awad
  • 16,582
  • 3
  • 35
  • 50