Passing result of NSOperation to another NSOperation

Question

I have two NSOperation that are responsible for Downloading, and Parsing. After the downloading operation is successful and I receive some NSData I want to then set that data as the data to be used by the parsing operation:

init(context: NSManagedObjectContext, completionHandler: Void -> Void) {

    downloadOperation = DownloadActivitiesOperation() { data in
        self.parseOperation.data = data
    }
    parseOperation = ParseActivitiesOperation(context: context)

    let finishOperation = NSBlockOperation(block: completionHandler)

    parseOperation.addDependency(downloadOperation)
    finishOperation.addDependency(parseOperation)

    super.init(operations: [downloadOperation, parseOperation, finishOperation])

    name = "Get Activities"
}

This however doesn't work as I'm attempting to use self inside of my download completion block before calling super.init.

My question is what would be the best approach when attempting to pass the result of one operation to the next one in the chain?

Answer 1

Each instance of NSOperation contains array of dependencies. Operations are not removed from this array after finishing. You can use those objects to access the data:

class DownloadActivitiesOperation: NSOperation {
   var data: NSData?
   ...
   // set self.data when it has downloaded
}

class ParseActivitiesOperation: NSOperation {
    func main() {
      if let downloadOp = self.dependencies.last as? DownloadActivitiesOperation {
          let dataToParse = downloadOp.data
          ...
      }
    }
 }

And so on.

Answer 2

Use a file cache

It looks like you are using some kind of implementation of GroupOperation from the WWDC2015 talk on Advanced NSOperations. In the sample code from the talk they use a cache file to pass data between the downloader and the parser.

Following snippet from the GetEarthquakesOperation class:

    
        let cachesFolder = try! NSFileManager.defaultManager().URLForDirectory(.CachesDirectory, inDomain: .UserDomainMask, appropriateForURL: nil, create: true)

        let cacheFile = cachesFolder.URLByAppendingPathComponent("earthquakes.json")
        
        /*
            This operation is made of three child operations:
            1. The operation to download the JSON feed
            2. The operation to parse the JSON feed and insert the elements into the Core Data store
            3. The operation to invoke the completion handler
        */
        downloadOperation = DownloadEarthquakesOperation(cacheFile: cacheFile)
        parseOperation = ParseEarthquakesOperation(cacheFile: cacheFile, context: context)

Use a memory cache

My current solution in one of my projects is to wrap the results in a class and pass it to both operations:

class OperationResultWrapper<T> {
    var result: T?
}

    let wrapper = OperationResultWrapper<NSData>()
    downloadOperation = DownloadEarthquakesOperation(resultWrapper: wrapper)
    parseOperation = ParseEarthquakesOperation(dataWrapper: wrapper, context: context)

Answer 3

I have just finished moving a large part of production code to use NSOperation and NSOperationQueues. The typical solutions to share results (notifications, delegates) seemed cumbersome and unwieldy so here is my solution.

1. Subclass NSOperationQueue to include a thread safe mutable dictionary instance property. I adapted the thread safe mutable dictionary code published here into my subclass JobOperationQueue: https://www.guiguan.net/ggmutabledictionary-thread-safe-nsmutabledictionary/

2. Subclass NSOperation to include a reference back to it's owning/initial JobOperationQueue. This ensures the operation can always find it's owner, even when code has to get run on different queues (which happens more than I thought!). Add subclass methods to JobOperationQueue to set this value whenever an operation is added to the queue via addOperation: or addOperations:

3. As operations process, they can add values into the queue's dictionary, and access values placed there by earlier processes.

I have been very happy with this approach and it has solved a lot of issues.

Be careful regarding race conditions -- if one operation MUST HAVE a value from another operation, ensure there is a dependency explicitly added to ensure the order of operations.

Here are my two main classes. I also added two-way dependency information, which I found useful in the situation where an operation has to spawn child operations, but wants to maintain the dependency network. In that situation you have to know who is depending on the original operation, so that you can propagate dependencies onto the spawned operations.

//
//  JobOperation.h
//  
//
//  Created by Terry Grossman on 9/17/15.
//

#import <Foundation/Foundation.h>
#import "JobOperationQueue.h"
#import "ThreadSafeMutableDictionary.h"
#import "ThreadSafeMutableArray.h"

@interface JobOperation : NSOperation

@property (strong, atomic) ThreadSafeMutableArray *dependents;    
@property (strong, atomic) NSDate *enqueueDate;
@property (weak, atomic) JobOperationQueue *homeJobQueue;

-(ThreadSafeMutableDictionary *)getJobDict;

@end

//
//  JobOperation.m
// 
//
//  Created by Terry Grossman on 9/17/15.
//

#import "JobOperation.h"

@implementation JobOperation


- (id)init
{
    if((self = [super init])) {
        _dependents = [[ThreadSafeMutableArray alloc] init];
    }

    return self;
}


-(ThreadSafeMutableDictionary *)getJobDict
{
    id owningQueue = self.homeJobQueue;
    if (owningQueue && [owningQueue isKindOfClass:[JobOperationQueue class]])
    {
        return ((JobOperationQueue *)owningQueue).jobDictionary;
    }


    // try to be robust -- handle weird situations
    owningQueue = [NSOperationQueue currentQueue];
    if (owningQueue && [owningQueue isKindOfClass:[JobOperationQueue class]])
    {
        return ((JobOperationQueue *)owningQueue).jobDictionary;
    }

    return nil;
}

-(void) addDependency:(NSOperation *)op
{
    [super addDependency:op];  // this adds op into our list of dependencies

    if ([op isKindOfClass:[JobOperation class]])
    {
        [((JobOperation *)op).dependents addObject:self];  // let the other job op know we are depending on them
    }
}

@end


//
//  JobOperationQueue.h
// 
//
//  Created by Terry Grossman on 9/17/15.
//

#import <Foundation/Foundation.h>
#import "ThreadSafeMutableDictionary.h"

// A subclass of NSOperationQueue
// Adds a thread-safe dictionary that queue operations can read/write
// in order to share operation results with other operations.

@interface JobOperationQueue : NSOperationQueue

// If data needs to be passed to or between job operations
@property (strong, atomic) ThreadSafeMutableDictionary *jobDictionary;


-(void)addOperation:(NSOperation *)operation;
-(void)addOperations:(NSArray<NSOperation *> *)ops waitUntilFinished:(BOOL)wait;

+(BOOL) checkQueue:(JobOperationQueue *)queue hasOpsOlderThan:(NSInteger)secondsThreshold cancelStaleOps:(BOOL)cancelOps;



@end

//
//  JobOperationQueue.m
// 
//
//  Created by Terry Grossman on 9/17/15.
//

#import "JobOperationQueue.h"
#import "JobOperation.h"


@implementation JobOperationQueue


// if this method returns NO, should set the queue to nil and alloc a new one
+(BOOL) checkQueue:(JobOperationQueue *)queue hasOpsOlderThan:(NSInteger)secondsThreshold cancelStaleOps:(BOOL)cancelOps
{
    if (queue == nil) 
    {
        return NO;
    }

    if ([queue operationCount] > 0) 
    {
        NSLog(@"previous share still processing!");

        // recently started or stale?  Check the enqueue date of the first op.
        JobOperation *oldOp = [[queue operations] objectAtIndex:0];

        NSTimeInterval sourceSeconds = [[NSDate date] timeIntervalSinceReferenceDate];
        NSTimeInterval destinationSeconds = [oldOp.enqueueDate timeIntervalSinceReferenceDate];    
        double diff =  fabs( destinationSeconds - sourceSeconds );        

        if (diff > secondsThreshold) 
        {
            // more than three minutes old!  Let's cancel them and tell caller to proceed
            [queue cancelAllOperations];
            return NO;
        }
        else
        {
            return YES;
        }

    }
    return NO;
}


-(id) init;
{
    if((self = [super init])) {
        _jobDictionary = [[ThreadSafeMutableDictionary alloc] initWithCapacity:12];
    }

    return self;
}

-(void)addOperation:(NSOperation *)operation;
{
    if (operation == nil) 
    {
        return;
    }

    if ([operation isKindOfClass:[JobOperation class]]) 
    {
        ((JobOperation *)operation).enqueueDate = [NSDate date];
        //((JobOperation *)operation).homeQueueT = self.underlyingQueue; // dispatch_get_current_queue();
        ((JobOperation *)operation).homeJobQueue = self;
    }

    [super addOperation:operation];    
}

-(void)addOperations:(NSArray<NSOperation *> *)ops waitUntilFinished:(BOOL)wait;
{
    for (NSOperation *operation  in ops) 
    {
        if ([operation isKindOfClass:[JobOperation class]]) 
        {
            ((JobOperation *)operation).enqueueDate = [NSDate date];
            //((JobOperation *)operation).homeQueueT = self.underlyingQueue; //dispatch_get_current_queue();
            ((JobOperation *)operation).homeJobQueue = self;
        }
    }

    [super addOperations:ops waitUntilFinished:wait];
}

@end  

Answer 4

You could first create the chain of dependent NSOperations without using self, then initialize your properties and call super.init last.

init(context: NSManagedObjectContext, completionHandler: () -> Void) {
    let finishOperation = NSBlockOperation(block: completionHandler)

    let parseOperation = ParseActivitiesOperation(context: context)
    finishOperation.addDependency(parseOperation)

    let downloadOperation = DownloadActivitiesOperation() { data in
        parseOperation.data = data
    }
    parseOperation.addDependency(downloadOperation)

    self.parseOperation = parseOperation
    self.downloadOperation = downloadOperation
    self.name = "Get Activities"

    super.init(operations: [downloadOperation, parseOperation, finishOperation])
}

Note: It seems strange to me that you store downloadOperation and parseOperation in properties and pass them in an array to the super class, but I don't know the rest of your code.

Answer 5

You can create a weak variable of self before starting the block.

Try adding this line before your block begins:

__weak __typeof(self) weakSelf = self;

Answer 6

Thanks to ilya for the answer, I noticed I accessed other operation through my Operation subclasses through their dependencies array.

In the end I came up with this extension :

extension Operation{

    func getOperationFromDependancies<T:Operation>(withType operationType:T.Type) -> T?{
        for dependency in self.dependencies {
            if let operation = dependency as? T{
                return operation
            }
        }
        return nil
    }

}

Then lets say you use two operations, operation one downloads and operation two processes the downloaded file, your main() would contain something like:

override func main(){
    let downloadOperation = 
    self.getOperationFromDependencies(withType:DownloadOperation.self)
    let downloadedFile = downloadedOperation?.downloadedFile
    //Process file here
}

Answer 7

Try this

` In Swift language it is not allowed to call designated initializer from other designated initializer so you can mark initializer as convenience.

For more information, please check Swift Programming Language's Initialization in Language Guide Section `