Obj-C design pattern : parallel task launcher

Question

I currently have a shell script that process many images one after the other, with the help of GraphicsMagick. It works fine, all calculations are correct, everything works. (that's not a "simple" script, it involves reading dimensions from a JSON file, converting a bunch of images with respect to many constraints).

As we're working with dual-core or quad-core computer, I'd like to parallelize it. And as I'm an iPhone developer liking to introduce myself to Mac development, I'd like to create it with XCode and Objective-C using the "command-line tool" template.

So far so good, but now I'm face with the design of the "task dispatcher" object. I'm fairly lost between running NSTasks in a run loop, in separate threads, using blocks, with or without GCD, with or without ARC.

How would one achieve this? I was thinking of using simple threads to spawn NSTasks, having them report when they're done, and notify my dispatcher's delegate so that it can upgrade its progress bar. But I'd really like to get in touch with Grand Central Dispatch. Does anyone have any thoughts, ideas, advice about what to do and what not?

Edit: I'm reading Apple's docs, and have found the NSOperationQueue class. Could it be that this is precisely what I'm needing here?

score 4 · Answer 1 · edited Sep 07 '11 at 17:19

A good class to use to launch independant processes including parameters and environment variables is NSTask. See the documentation for the gory details. Here is a little commandline tool that starts 10 concurrent processes and waits for them to finish. NSOperationQueue would be redundant here because the tasks are already launched concurrently.

-- Edit: Improved Version With Limited Concurrency --

int main (int argc, const char * argv[])
{
   @autoreleasepool {

       // Let's not have more than 5 parallel processes
       dispatch_semaphore_t limit = dispatch_semaphore_create(5);
       dispatch_semaphore_t done  = dispatch_semaphore_create(0);

       for (int i=0;  i<10;  i++) {
           // Setup the taks as you see fit including the environment variables.
           // See docs on NSTask for more on how to use this object.
           NSTask *task = [[NSTask alloc] init];
           task.launchPath = @"/bin/ls";
           task.arguments = [NSArray arrayWithObject:@"-la"];
           task.terminationHandler = ^(NSTask *task) {
               dispatch_semaphore_signal(limit);
               if (i==9) dispatch_semaphore_signal(done);
           };

           dispatch_semaphore_wait(limit, DISPATCH_TIME_FOREVER);
           [task launch];
       }
       dispatch_semaphore_wait(done, DISPATCH_TIME_FOREVER);
       dispatch_release(limit);
       dispatch_release(done);
   }
   return 0;

}

-- Original Version --

int main (int argc, const char * argv[])
{
    @autoreleasepool {

        NSObject *lock = [[NSObject alloc] init];
        int __block counter = 10;

        for (int i=0;  i<10;  i++) {
            // Setup the taks as you see fit including the environment variables.
            // See docs on NSTask for more on how to use this object.
            NSTask *task = [[NSTask alloc] init];
            task.launchPath = @"/bin/ls";
            task.arguments = [NSArray arrayWithObject:@"-la"];
            task.terminationHandler = ^(NSTask *task) {
                @synchronized(lock) { counter--; }
            };
            [task launch];
        }

        while (counter)
            usleep(50);

        [lock release];
    }
    return 0;
}

In your case you might want to hold the NSTask objects in an array for easier management.

That's fine if I have a limited number of processes to launch, but I really need to *not* launch something like 500 processes in parallel (that's the average, it can rise to thousands). Interesting for simple operations, but in this case I'll need something more granular, and NSOperationQueue seems the best workhorse here. — Cyrille, Sep 07 '11 at 09:58
Wow, semaphores... been wondering how they work since I first discovered the win32 API something like 10 years ago! Actually that seems quite simple when reading it. I'll try and have a look at this, too. Thanks again! — Cyrille, Sep 08 '11 at 08:43
IIRC, the `dispatch_*` functions are the ones provided by GCD, right? — Cyrille, Sep 08 '11 at 08:45

score 1 · Accepted Answer · answered Sep 06 '11 at 19:15

yes - NSOperation/NSOperationQueue are good for this task.

i'd start with something like this:

@protocol MONTaskRequestDelegate

- (void)taskRequestDidComplete:(MONTaskRequest *)taskRequest;

@end

@interface MONTaskRequest : NSOperation
{
@private
    NSTask * task;
    NSObject<MONTaskRequestDelegate>* delegate; /* strong reference. cleared on cancellation and completion, */
}

- (id)initWithTask:(NSTask *)task delegate:(NSObject<MONTaskRequestDelegate>*)delegate;

// interface to access the data from the task you are interested in, whether the task completed, etc.

@end

@implementation MONTaskRequest

// ...

- (void)performDelegateCallback
{
    [self.delegate taskRequestDidComplete:self];
    self.delegate = nil;
}

- (void)main
{
    NSAutoreleasePool * pool = [NSAutoreleasePool new];

    [self runTheTask];
    // grab what is needed and handle errors
    [self performDelegateCallback];

    [pool release];
}

- (void)cancel
{
    [super cancel];
    [self stopTaskIfPossible];
    [self performDelegateCallback];
}

@end

then you can use NSOperationQueue to limit the number of active tasks to a reasonable number.

That's more or less what I did accomplish meanwhile. Indeed, NSOperationQueue was what I was looking for (specifically, to limit the number of active tasks). Thanks for confirming! — Cyrille, Sep 07 '11 at 09:56
This is a fine solution and has but one drawback: it launches both a thread an an independant process for each concurrent task, which increases the resource requirements. — aLevelOfIndirection, Sep 07 '11 at 17:16
@aLevelOfIndirection the independent process is a requirement. also, NSOperationQueue *does* reuse threads -- its implementation (10.6+) utilizes libdispatch (aka GCD). — justin, Sep 07 '11 at 19:53

score -1 · Answer 3 · answered Sep 06 '11 at 11:36

-1

I use for this purpose the [myObj performSelectorInBackground:@selector(doSomething) withObject:nil]; functionality of an NSObject.

The idea is quite simple: you write a method that does the work, call it from the main thread using the aforementioned method, then call some callback selector if you need to somehow process the results from different threads.

answered Sep 06 '11 at 11:36

Ibolit

9,218
7
52
96

I'm aware of this method, but would like to use some more advanced form of threading, like GCD or NSOperationQueue. Thanks anyway. – Cyrille Sep 06 '11 at 11:43

Obj-C design pattern : parallel task launcher

3 Answers3